Document of The World Bank Report No: 63766-PE PROJECT APPRAISAL DOCUMENT ON A PROPOSED LOAN IN THE AMOUNT OF US$25 MILLION TO THE REPUBLIC OF PERU FOR A HIGHER EDUCATION QUALITY IMPROVEMENT PROJECT NOVEMBER 1, 2012 Human Development Department Bolivia, Chile, Ecuador, Peru and Venezuela Country Management Unit Latin America and the Caribbean Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange rate effective September 7, 2012) Currency Unit = Peruvian Nuevos Soles (PEN) 2.61 PEN = US$1 1 PEN = US$0.38 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ANR Asamblea Nacional de Rectores (National Assembly of Rectors) CAE Comité Asesor Externo (External Advisory Council) CP Comités de Pares, (Committees of Peers) CDP Comité Directivo del Proyecto (Project Steering Comittee) CONEACES Consejo de Evaluación, Acreditación y Certificación de la Calidad de la Educación Superior No Universitaria (Council for the Evaluation, Accreditation and Certification of Non-University Higher Education) CONEAU Consejo de Evaluación, Acreditación y Certificación de la Calidad de la Educación Superior Universitaria (Council for the Evaluation, Accreditation and Certification of University Higher Education) COSUSINEACE Consejo Superior del Sistema Nacional de Evaluación, Acreditación y Certificación de la Calidad Educativa (Higher Council of the National System for Evaluation, Accreditation and Certification of the Quality of Education) CPS Country Partnership Strategy CTF Comisión Técnica del Fondo (Fund Technical Commission) DA Designated Account DGPP Dirección General de Presupuesto Público (General Directorate of Public Budget) EEFAs Entidades Evaluadora con Fines de Acreditación (Evaluation Entities for Accreditation) FEC Fondo de Estímulo de la Calidad (Fund for Quality Enhancement) FM Financial Management GDP Gross Domestic Product GoP Government of Peru HEIs Higher Education Institutions IBRD International Bank for Reconstruction and Development IDF Institutional Development Fund IEESs Institutos y Escuelas de Educación Superior (Higher Education Institutes and Schools) IPP Indigenous Peoples Plan IPPF Indigenous Peoples Planning Framework LAC Latin America and the Caribbean MEF Ministerio de Economía y Finanzas (Ministry of Economy and Finance) MINEDU Ministerio de Educación (Ministry of Education) PMI Plan de Mejora Institucional (Institutional Improvement Plan) ii SEPA Sistema de Ejecución de Planes de Adquisiciones (Procurement Plans Execution System) SIAC Sistema de Información del Aseguramiento de la Calidad (Quality Assurance Information System) SIAF Sistema Integrado de Administración Financiera (Integrated System of Financial Management) SINEACE Sistema Nacional de Evaluación, Acreditación y Certificación de la Calidad Educativa (National System for Evaluation, Accreditation and Certification of the Quality of Education) SNIP Sistema Nacional de Inversión Pública (National System of Public Investment) STF Secretaría Técnica del Fondo (Fund Technical Secretariat) UCP Unidad Coordinadora del Proyecto (Project Coordination Unit) Regional Vice President: Hasan A. Tuluy Country Director: Susan G. Goldmark Sector Director: Keith Hansen Sector Manager: Reema Nayar Task Team Leader: Marcelo Becerra iii PERU HIGHER EDUCATION QUALITY IMPROVEMENT PROJECT CONTENTS Page I. STRATEGIC CONTEXT .................................................................................................1 A. Country Context ............................................................................................................ 1 B. Sector and Institutional Context.................................................................................... 1 C. Higher Level Objectives to which the Project Contributes .......................................... 5 II. PROJECT DEVELOPMENT OBJECTIVES ................................................................5 A. PDO............................................................................................................................... 5 Project Beneficiaries ........................................................................................................... 5 PDO Level Results Indicators ............................................................................................. 5 III. PROJECT DESCRIPTION ..............................................................................................6 A. Project Components ...................................................................................................... 6 B. Project Financing .......................................................................................................... 7 Lending Instrument ............................................................................................................. 7 Project Cost and Financing ................................................................................................. 7 C. Lessons Learned and Reflected in the Project Design .................................................. 8 IV. IMPLEMENTATION .......................................................................................................9 A. Institutional and Implementation Arrangements .......................................................... 9 B. Results Monitoring and Evaluation ............................................................................ 10 C. Sustainability............................................................................................................... 11 V. KEY RISKS AND MITIGATION MEASURES ..........................................................11 A. Risk Ratings Summary ............................................................................................... 11 B. Overall Risk Rating Explanation ................................................................................ 11 VI. APPRAISAL SUMMARY ..............................................................................................12 A. Economic and Financial Analyses .............................................................................. 12 B. Technical ..................................................................................................................... 12 C. Financial Management ................................................................................................ 13 D. Procurement ................................................................................................................ 14 E. Social (including Safeguards) ..................................................................................... 14 F. Environment (including Safeguards) .......................................................................... 15 iv Annex 1: Results Framework and Monitoring .........................................................................16 Annex 2: Detailed Project Description .......................................................................................25 Annex 3: Implementation Arrangements ..................................................................................32 Annex 4: Operational Risk Assessment Framework (ORAF) .................................................52 Annex 5: Implementation Support Plan ....................................................................................55 Annex 6: Economic Analysis .......................................................................................................57 v PAD DATA SHEET Republic of Peru Higher Education Quality Improvement Project PROJECT APPRAISAL DOCUMENT Latin-American and the Caribbean Region LCSHE . Basic Information Date: November 1, 2012 Sectors: Tertiary Education (100%) Country Director: Susan G. Goldmark Themes: Education for the knowledge economy (100%) Sector Reema Nayar / EA C Manager/Director: Keith Hansen Category: Project ID: P122194 Lending Specific Investment Instrument: Lending Team Leader(s): Marcelo Becerra Does the project include any CDD component? NO Joint IFC: NO . Borrower: Republic of Peru, Ministry of Economy and Finance (MEF, Ministerio de Economía y Finanzas) Responsible Agency: Higher Council of the National System for Evaluation, Accreditation and Certification of the Quality of Education (COSUSINEACE, Consejo Superior del Sistema Nacional de Evaluación, Acreditación y Certificación de la Calidad Educativa) Contact: Alexis Dueñas Dávila Title: Head of the Project’s Formulating Unit Telephone No. 51-1-262-2641 Email: aduenas@coneau.gob.pe . Project Implementation Period: Start Date: 01/01/2013 End Date: 12/31/2017 Expected Effectiveness Date: 01/01/2013 Expected Closing Date: 04/01/2018 . Project Financing Data(US$M) [ X ] Loan [ ] Grant [X ] Other [] Credit [ ] Guarantee The Loan is an IBRD Flexible Loan with a variable-spread and has a final maturity of 18 years, including a grace period of 17.5 years. vi For Loans/Credits/Others Total Project Cost:US$52.17 million Total Bank Financing: US$25 million Total Cofinancing: US$27.17 million Financing Gap: . Financing Source Amount(US$M) BORROWER/RECIPIENT: 27.17 IBRD 25.00 IDA: New IDA: Recommitted Others Financing Gap Total 52.17 . Expected Disbursements (in USD Million) Fiscal Year 2013 2014 2015 2016 2017 2018 Annual 1.66 4.40 4.76 5.69 5.51 3.0 Cumulative 1.66 6.04 10.80 16.49 22.0 25.00 . Project Development Objective(s) The objective of the Project is to improve Peru’s higher education quality assurance system through the promotion of self and external evaluations, the financing of improvement plans, and the provision of information. . Components Component Name Cost (USD Millions) Component 1. Development of Methods, Instruments, Norms and 3.80 Capacity for Evaluation and Accreditation Component 2. Development and Consolidation of a Higher Education 2.49 Quality Assurance Information System Component 3. Fund for Quality Enhancement 18.71 . Compliance Policy Does the project depart from the CAS in content or in other significant respects? Yes [ ] No [ X ] . Does the project require any exceptions from Bank policies? Yes [ ] No [ X ] Have these been approved by Bank management? Yes [ ] No [ ] Is approval for any policy exception sought from the Board? Yes [ ] No [ X ] Does the project meet the Regional criteria for readiness for implementation? Yes [ X ] No [ ] . vii Safeguard Policies Triggered by the Project Yes No Environmental Assessment OP/BP 4.01 X Natural Habitats OP/BP 4.04 X Forests OP/BP 4.36 X Pest Management OP 4.09 X Physical Cultural Resources OP/BP 4.11 X Indigenous Peoples OP/BP 4.10 X Involuntary Resettlement OP/BP 4.12 X Safety of Dams OP/BP 4.37 X Projects on International Waters OP/BP 7.50 X Projects in Disputed Areas OP/BP 7.60 X . Legal Covenants Name Recurrent Due Date Frequency Conditions of Effectiveness, Article V, 5.01 No Effectiveness N/A Description of Covenant The UCP has been created within COSUSINEACE as provided in Section I.A.1 of Schedule 2 to the Loan Agreement. Name Recurrent Due Date Frequency Institutional Arrangements, Yes N/A Annual Schedule 2, Section I.A.1 (a) Description of Covenant The Borrower, through COSUSINEACE, shall implement the Project through an independent unit within COSUSINEACE (the UCP), responsible for the management, coordination, supervision, monitoring and evaluation (except the evaluations referred to in Section I.A.3 of this Schedule), and fiduciary management of the Project. Name Recurrent Due Date Frequency Institutional Arrangements, Yes 3 months after the Annual Schedule 2, Section I.A.1 (b) Effective Date, N/A thereafter Description of Covenant The Borrower, through COSUSINEACE, shall ensure that not later than 3 months after the Effective Date the UCP have a structure, responsibilities, the necessary budget and key staff assigned with functions, experience, responsibilities and qualifications acceptable to the Bank, as described in the Operational Manual. viii Name Recurrent Due Date Frequency Institutional Arrangements, Yes N/A Annual Schedule 2, Section I.A.1 (c)(i) Description of Covenant The Borrower, through COSUSINEACE, shall, for the purposes of Part 3 of the Project, establish and maintain a committee (the CTF) responsible, inter alia, for the selection and approval of Improvement Plans under Part 3 of the Project, and composed by seven members, including a representative from MEF, as described in the Operational Manual Name Recurrent Due Date Frequency Institutional Arrangements, Yes N/A Annual Schedule 2, Section I.A.1 (c)(ii) Description of Covenant The Borrower, through COSUSINEACE, shall, for the purposes of Part 3 of the Project, designate, within the UCP, a technical secretariat (the STF) responsible, inter alia, for: (A) carrying out of pre- evaluation processes for Improvement Plan proposals: (B) supporting HEIs in the elaboration of final Improvement Plans; (C) supporting HEIs on the monitoring and implementation of Improvement Plans, as described in the Operational Manual. Name Recurrent Due Date Frequency Institutional Arrangements, Yes 90 days after the Annual Schedule 2, Section I.A.2 Effective Date, N/A thereafter Description of Covenant Not later than 90 days after the Effective Date, the Borrower, through COSUSINEACE-UCP, shall establish, and thereafter maintain during the implementation of the Project, a steering committee (CDP) composed by representatives from COSUSINEACE (CONEAU and CONEACES), MINEDU and MEF, with functions and responsibilities acceptable to the Bank, included, inter alia, the responsibility of providing general Project oversight and coordination, as set forth in the Operational Manual. Name Recurrent Due Date Frequency Institutional Arrangements, Yes 6 months after the Annual Schedule 2, Section I.A.3 Effective Date, N/A thereafter Description of Covenant Not later than nine months after the Effective Date, the Borrower, through COSUSINEACE-UCP, shall create an external advisory council (the CAE) composed by a group of six recognized international and local authorities in their respective fields, with functions and responsibilities acceptable to the Bank, including the carrying out of yearly evaluations of the Project, as well as a mid-term and a final evaluation of Project implementation, as described in the Operational Manual. ix Name Recurrent Due Date Frequency Safeguards and Operational Manual, Yes N/A Annual Schedule 2, Section I.D Description of Covenant The Borrower shall, and shall cause COSUSINEACE to carry out the Project in accordance with the IPPF and the Operational Manual. Name Recurrent Due Date Frequency Withdrawal Conditions, No N/A N/A Schedule 2, Section IV.B. 1(c) Description of Covenant Notwithstanding the provisions of Part A of this Section, no withdrawal shall be made for payments under Category (2), unless: (i) the CTF and STF units referred in Section I.A.1 (ii) of this Agreement are in place and staffed in a manner acceptable to Bank; and (ii) the Project information system referred to in Section IV.C of Schedule 2 to this Agreement is implemented in a manner satisfactory to the Bank. Name Recurrent Due Date Frequency Other Undertakings, No 3 months after the N/A Schedule 2, Section IV.C Effectiveness Date Description of Covenant The Borrower, through COSUSINEACE-UCP shall design and implement a Project information system, acceptable to the Bank, for the purposes of recording, controlling, reporting and monitoring Project transactions and issuance of the financial reports required for the Project. x . Team Composition Bank Staff Name Title Specialization Marcelo Becerra Team Leader Economist / Higher Education Michael Crawford Senior Education Specialist Higher Education Omar Arias Sector Leader Economist / Labor Economics Janet Entwistle Senior Operations Officer Operations Maria Elena Paz Gutzalenko ACS Program Assistant Guillermo Toral Junior Professional Associate Operations Mariana Montiel Senior Counsel Lawyer Francisco Rodriguez Procurement Specialist Procurement Selene del Rocio La Vera Procurement Specialist Procurement Nelly Ikeda Financial Management Analyst Financial Management Kristyna Bishop Senior Social Development Specialist Safeguards Patricia de la Fuente Hoyes Senior Finance Officer Finance Tatiana de Abreu Souza Finance Analyst Finance Non Bank Staff Name Title City Liz Reisberg Higher Education Specialist Boston Maria Elizabeth Dasso Social Development Specialist Lima Lucy Violeta Castro Consultant Lima Locations Country First Administrative Location Planned Actual Comments Division Peru xi I. STRATEGIC CONTEXT A. Country Context 1. Peru is an upper middle income country with high economic growth over the last decade. Peru has been one of the best-performing economies in Latin America, as a result of prudent macroeconomic policies and a favorable external environment. Peru’s GDP growth has been significant from a regional and historical perspective, averaging 6.8 percent between 2002 and 2011. This strong economic performance enabled a continuous improvement of Peru’s income per capita, which increased by more than 50 percent during the decade, after almost 30 years of stagnation. Poverty declined as a percent of the population during this period (4 million people exited poverty between 2002 and 2011) but overall development outcomes remain far below expected, given Peru’s per-capita income, and gaps remain high (Peru’s Gini coefficient stands at 0.48). 2. The goal of the Government is to preserve a sustained economic growth of 6 percent annually while fostering inclusiveness. The Government recognizes that in order to sustain growth, firms need to move into higher value-added production and foster innovation, which requires that companies have access to highly skilled labor. In recent years, employers have substantially increased the demand for educated workers, especially those with higher education. 1 However, the tertiary education system is fragmented and offers divergent paths of access and quality to the population. An important priority for the new administration is to address the country’s significant gaps in human capital (especially in education) and infrastructure development. As indicated by the National Education Project 2021 (Proyecto Educativo Nacional al 2021), the Government’s top strategic priorities in education include enhancing the accreditation system for tertiary education institutions through the National System for Evaluation, Accreditation and Certification of the Quality of Education (SINEACE, Sistema Nacional de Evaluación, Acreditación y Certificación de la Calidad Educativa). B. Sector and Institutional Context 3. With a higher education enrollment rate of around 36 percent, Peru has already achieved mass tertiary education coverage in line with regional standards. Most of the expansion in recent years has taken place in private institutions. The adoption in 1996 of Legislative Decree No. 882, which authorized the creation of for-profit institutions, led to significant growth in the private provision of higher education. Currently, 35 public universities (accounting for 55 percent of the total university enrollment), 65 private universities (45 percent of the total enrollment), and more than 1,000 non-university higher education institutions (53 percent of which are public) operate as higher education providers. 4. Higher education policy making in Peru is hindered by weak governance. On the one hand, the National Assembly of Rectors (ANR, Asamblea Nacional de Rectores) both represents and regulates universities , which leads to a conflict of interest, especially with regards to quality assurance and the authorization of universities existence or expansion. On the other hand, the 1 World Bank. 2011. Strengthening Skills and Employability in Peru (Report No. 61699-PE). 1 Ministry of Education (MINEDU, Ministerio de Educación) does not have sufficient capacity either for analysis or for the design of comprehensive public policies of higher education. 5. As for the financing of higher education, Peruvian universities and the higher education system as a whole is characterized by insufficient funds and limited accountability. Peru invests about 1.3 percent of its GDP in higher education, of which 0.7 percent is provided by the public sector 2. Regression analyses comparing Peru to other countries of the Latin America and the Caribbean region show that the level of both public and private investment in higher education was below the level corresponding to Peru’s income, considered both in aggregate and per capita terms. 3 Limited investments in higher education have led to a gradual worsening of universities’ infrastructure, as well as to shortages in teaching material and equipment. The combination of insufficient public financing and growing demand has led public universities to an increasing dependency on self-generated funds. 4 Furthermore, Peruvian Higher Education Institutions (HEIs) are not sufficiently accountable for the funds they receive. Their annual budget is assigned through a process based on historical priorities and political bargaining, in which the ANR has significant leverage as per the 1983 University Act. The lack of accountability mechanisms and performance-based funding criteria has perpetuated significant differences in subsidies between HEIs and led to insufficient quality improvements. 6. Generally low levels and large variance of quality in the higher education system constitute a major obstacle to the creation of human capital. Although the Peruvian system has a traditional group of prestigious institutions of relatively good quality, the growing number of providers has also increased the range of program quality and relevance, thus diminishing the lower bound to the system’s quality. 5 The main determinants of low quality include: outdated programs, weak institutional capacity, insufficient teaching equipment and laboratories, and the low qualification of professors (only 7 percent have a PhD and only 40 percent have a post- graduate degrees). An estimated one-third of higher education graduates who are employed do not work in their line of specialization. Peru still lacks an integrated information system for higher education 6 that allows the market (students and their families, policy makers, employers) to differentiate programs and HEIs on the basis of their quality and thus provides incentives to HEIs for improving their quality. 7. A critical tool for addressing all of these weaknesses in a higher education system characterized by a large number and diversity of education providers is a well-functioning higher education quality assurance system. Such a system would both raise the minimum standards and foster quality improvements by promoting self and external evaluations, promoting quality 2 OECD. 2002. Education at a glance: OECD indicators 2002. Paris: OECD. 3 World Bank. 2007. World Development Report 2007: Development and the next generation. Washington DC: The World Bank. 4 This income is collected from a variety of sources, including fees related to university entrance examinations, consultancy services, and tuition fees at the postgraduate level. 5 Castro, Juan F., Gustavo Yamada, and Omar Arias. 2010. “Analysis of School Trajectories, Career Choice, and Insertion into the Peruvian Labor Market�. Mimeo. Lima; and Yamada, Gustavo, and Juan F. Castro. 2010. “Educación superior e ingresos laborales: Estimaciones paramétricas y no paramétricas de la rentabilidad por niveles y carreras en el Perú.� Documento de Discusión DD/10/06, Centro de Investigación de la Universidad del Pacífico. 6 Currently only a set of fragmented and incomplete higher education information systems exist, which are maintained by different institutions and are not interconnected. 2 enhancements based on these evaluations, providing a system for accreditation of quality HEIs, and providing information to stakeholders. By increasing the quality of non-top institutions 7 and strengthening systemic mechanisms (like a sector-wide information system), a strengthened quality assurance system would indirectly reinforce equity aspects of higher education. 8 These are being addressed primarily through the recently created National Program of Scholarships and Educational Loans (PRONABEC, Programa Nacional de Becas y Crédito Educativo). 8. In this context, the Government of Peru (GoP) adopted a bold strategy, reflected in the National Education Project 2021, to increase the quality and relevance of tertiary education. This strategy resulted in the legal creation of a quality assurance system, SINEACE, in 2006. The SINEACE Act established a framework for an intertwined quality assurance system across both basic and higher education levels. For this purpose, the Higher Council of the National System for Evaluation, Accreditation and Certification of the Quality of Education (COSUSINEACE, Consejo Superior del Sistema Nacional de Evaluación, Acreditación y Certificación de la Calidad Educativa) was created in 2011 to steer, develop and supervise SINEACE. COSUSINEACE includes three agencies: the Peruvian Institute for Evaluation, Accreditation and Certification of Basic Education (IPEBA, Instituto Peruano de Evaluación, Acreditación y Certificación de la Calidad de la Educación Básica); the Council for the Evaluation, Accreditation and Certification of Non-University Higher Education Quality (CONEACES, Consejo de Evaluación, Acreditación y Certificación de la Calidad de la Educación Superior No Universitaria); and the Council for the Evaluation, Accreditation and Certification of University Higher Education Quality (CONEAU, Consejo de Evaluación, Acreditación y Certificación de la Calidad de la Educación Superior Universitaria). These agencies are responsible for the development and supervision of SINEACE in the fields of basic education, higher education institutes and schools (IEES, Institutos y Escuelas de Educación Superior) and universities, respectively. 9. The higher education quality assurance system is composed of four sets of actors. First, public entities that set standards and foster evaluation, take accreditation decisions based on these, and steer the system, namely CONEAU and CONEACES. Secondly, HEIs that periodically submit themselves and their programs to evaluation and accreditation decisions. Thirdly, Evaluation Entities for Accreditation (EEFAs, Entidades Evaluadoras con Fines de Acreditación) that carry out external evaluations for HEIs. Finally, a higher education information system that fuels the quality assurance system and allows for better decision making by policy makers, HEIs, students and other stakeholders. This system’s functioning follows three sequential steps: self-evaluation, external evaluation, and accreditation. Self-evaluation is the responsibility of HEIs. External evaluation is to be carried out by EEFAs, private entities authorized by CONEAU (for universities) or CONEACES (for IEESs) that employ peer 7 COSUSINEACE has recently commissioned a qualitative study on the determinants of higher education quality. Through interviews with authorities and employers, and focus groups with graduates and students, the study has identified two sets of HEIs (labeled as top and non-top, for those of high and low quality respectively). Moreover, the study identified the factors driving the quality of non-top institutions to significantly lower levels, as related to the incoming students, the transformation process at HEIs, and their outcomes in the labor market, including the quality of students and teachers, the level of demand, and the HEI’s management. 8 Access to higher education in Peru is not equal for all income levels. While only 10 percent of the young population in the lowest quintile attends university, the proportion goes up to 35 percent for those in the highest quintile. Source: INEI, 2010, National Households Survey (ENAHO, Encuesta Nacional de Hogares). 3 reviewers to conduct the evaluation of those HEIs that hire them. External evaluation reports must include recommendations for improvement and a recommendation on whether to proceed to apply for accreditation or not. Both self-evaluation and external evaluation are conducted on the basis of standards developed and approved by CONEAU and CONEACES. Accreditation decisions are taken by CONEAU or CONEACES on the basis of those standards. 10. The objective of the quality assurance system for higher education is threefold. First, to classify HEIs and programs into two categories: acceptable to minimum standards (accredited) or low-quality (not accredited). Second, to make that information available for prospective students and their families, the education community, and other stakeholders. And finally, to promote quality improvements in individual HEIs and throughout the system. At the current stage of development in Peru’s higher education system, it is critical to ensure that quality standards are approved and legitimized, that a majority of HEIs participate in the quality assurance system, and that a systematic and reliable source of higher education information is established. Despite some capacity limitations, COSUSINEACE has fully developed standards and guides for all key disciplines, 9 most of which are obliged to reach accreditation and are therefore the first programs to be evaluated. 11. As of the end of 2011, 56 programs had completed self-evaluation and 19 of these had requested external evaluation towards accreditation. Concerning the EEFAs, 4 were already authorized and are currently functioning, while a total of 8 are envisaged to be operating by 2014. 10 This number is considered sufficient to carry out the expected evaluation workload. 12. By supporting the strengthening of SINEACE and thus one of the key policy priorities identified in the National Education Project 2021, this Project would respond to a demand from the GoP to assist in the improvement of the quality of its higher education system, particularly in its institutional, informational, and financial dimensions. The Government has identified this as a critical first step in improving Peru’s higher education system, and to eventually addressing other governance issues. This Project would provide continuity to previous Bank support to the GoP with regards to addressing the various challenges of the higher education system. The recent Technical Assistance program on skill sets of the Peruvian workforce (‘Strengthening Skills and Employability in Peru’, Report No. 61699-PE) provided valuable findings about the constraints on workers’ entry into the labor market, and policy advice to address them. The program not only underlined the need for a higher education information system, but also advanced the strengthening of COSUSINEACE as a strategy for enhancing quality in the higher education system. A recent Policy Note 11 focusing on the challenges Peru faces in the field of human capital makes a similar argument. Finally, an Institutional Development Fund (IDF) Grant (TF 096586) is being implemented with the objective of strengthening CONEAU and CONEACES. 9 Key disciplines have been selected on the basis of their impact on public welfare. They are the following: (i) at university level: Education, Medicine, Other Health Sciences, and Engineering; (ii) at non-university level: Education, Other Health, and Technology (mechanics, construction, electrical, computer/informatics, etc.). These disciplines include those for which the accreditation process is legally compulsory, except Law (for which accreditation was not compulsory when the Project was first registered at SNIP) and Engineering, which is considered of strategic value for the country. 10 Efforts are being made by COSUSINEACE to provide information about this emerging market and thus promote the creation of more EEFAs by both Peruvian and foreign companies. 11 Michael Crawford and Kristian Thorn. 2011. Hacia una educación superior para la competitividad. 4 C. Higher Level Objectives to which the Project Contributes 13. The proposed Project is consistent with the World Bank Group’s Country Partnership Strategy (CPS) FY201216 (Report No. 66187-PE), discussed by the Board of Executive Directors on March 13, 2012, which gives a prominent role to quality assurance in higher education as a way to enhance market-relevant skills and foster social mobility, productivity, and growth. It also helps strengthen the role of the state as a guarantor of quality in the provision of higher education. It is fully aligned with three GoP strategic, long-term development goals, as reflected on the CPS: (i) economic growth; (ii) social development; and (iii) modernization of institutions. II. PROJECT DEVELOPMENT OBJECTIVES A. PDO 14. The objective of the Project is to improve Peru’s higher education quality assurance system through the promotion of self and external evaluations, the financing of improvement plans, and the provision of information. Project Beneficiaries 15. The main direct Project beneficiaries would be: (i) HEIs, through the establishment of a culture and practice of continuous quality improvement and evaluation. It is envisaged that at least 250 HEIs would benefit directly from the Project’s financing (Component 3) and about 600 HEIs would benefit from the Project’s capacity building activities (Component 1). (ii) Students who graduate from accredited HEIs and/or from HEIs or programs that implement improvement plans as well as prospective students. Since it is expected that programs of key disciplines would undertake evaluation and accreditation processes in the next 5 years, the potential number of beneficiary students, in a 10 years horizon, would be up to 1,135,433. It is important to note that HEIs would be the Project’s main direct beneficiaries. 16. Indirectly, other beneficiaries include employers, faculty who work at participating HEIs, students’ families and society at large (through the information system), and other stakeholders such as research institutions. PDO Level Results Indicators 17. Progress toward the PDO would be measured through the following four PDO-level results indicators: a. Number of programs in the key disciplines and HEIs that have completed self- evaluation on the basis of CONEAU or CONEACES standards. b. Number of programs in the key disciplines and HEIs that have completed external evaluation on the basis of CONEAU or CONEACES standards. 5 c. Number of program improvement plans and institutional improvement plans that are implemented satisfactorily. d. Number of users registered in the website of the Quality Assurance Information System (SIAC). III. PROJECT DESCRIPTION A. Project Components 18. Component 1: Development of Methods, Instruments, Norms and Capacity for Evaluation and Accreditation (estimated total cost: US$7.92 million; Bank: US$3.80 million). Strengthening the capacity of CONEAU, CONEACES, HEIs and EEFAs to handle their respective responsibilities within the quality assurance system, through: (a) Subcomponent 1.1: Development of management, planning and evaluation capacity of CONEAU and CONEACES (estimated total cost: US$1.41 million; Bank: US$0.68 million). Enhancing of CONEAU and CONEACE’s management, planning and evaluation capacity by developing standards, norms and procedures for evaluation and accreditation, including those for the supervision of the EEFAs. (b) Subcomponent 1.2: Development of self-evaluation and external evaluation capacity (estimated total cost: US$1.91 million; Bank: US$0.91 million). Strengthening of the capacity of (i) HEIs to engage in both internal and external evaluation processes; and (ii) EEFAs to undertake external evaluations, including learning from other experiences at the regional and international levels. (c) Subcomponent 1.3: Support for Project implementation (estimated total cost: US$4.61 million; Bank: US$2.21 million). Provision of support for the technical and administrative management of the Project. 19. Component 2: Development and Consolidation of a Higher Education Quality Assurance Information System (estimated total cost: US$5.19 million; Bank: US$2.49 million). Developing and consolidating a higher education quality assurance information system for the systematic gathering, processing and dissemination of reliable data regarding higher education quality assurance as well as graduates’ performance in the labor market, through: (a) Subcomponent 2.1: Development and consolidation of an observatory of the accreditation of Peru’s higher education (Acredita Perú) (estimated total cost: US$1.65 million; Bank: US$0.79 million). Development and consolidation of an observatory of the accreditation process of Peru’s higher education (Acredita Perú), responsible for the organization and dissemination of information on standards, criteria, indicators, EEFAs, as well as the evaluation and accreditation procedures by degree programs and HEIs. (b) Subcomponent 2.2: Establishment of an observatory of higher education graduates (Futuro Profesional) (estimated total cost: US$3.13 million; Bank: US$1.50 million). Establishment of an observatory of higher education graduates (Futuro Profesional), 6 for systematically collecting, processing and disseminating statistical information about the performance of higher education graduates in the labor market, and providing information regarding employability and salaries of graduates from different programs and HEIs. (c) Subcomponent 2.3: Elaboration of sector studies (estimated total cost: US$0.41 million; Bank: US$0.19 million). Carrying out of several studies aiming at a better understanding of Peru’s higher education, including inter alia: (i) the development of a methodology and initial data collection to carry out an evaluation regarding the long term effects of the introduction of a quality assurance system; (ii) in depth socio- cultural assessment, in the framework of the Project’s Indigenous Peoples Planning Framework (IPPF); (iii) assessing the role of information in the choice of post- secondary education; and (iv) the effect of accreditation in Peru’s higher education system. 20. Component 3: Fund for Quality Enhancement (estimated total cost: US$39.06 million; Bank: US$18.71 million). Promoting the improvement of the quality and relevance of academic programs and HEIs, through the supporting of the establishment of the FEC and its two windows of financing, namely: (a) Subcomponent 3.1: External Evaluation of IEESs (estimated total cost: US$0.59 million; Bank: US$0.28 million). External evaluation of IEESs (institution-wide and for technical programs in health and education) by the EEFA of their choice, after completion of self-evaluation. (b) Subcomponent 3.2: Support to Improvement Plans (estimated total cost: US$38.46 million; Bank: US$18.43 million). Implementation of Improvement Plans for both IEESs and universities (including programs within HEIs), based on the results of the self and external evaluation. B. Project Financing Lending Instrument 21. The proposed Project would be financed by a Specific Investment Loan (SIL). This is appropriate given that both the Project’s activities and the implementing agency are new, the flexible nature of the instrument and the specificity of the investment. Project Cost and Financing 22. Total Project financing requirements are estimated at US$52.17 million, US$27.17 million of which (52.08 percent) constitute the GoP’s contribution. 7 Table 1: Project cost and financing IBRD Project cost Project Components Financing % Financing (US$ million) (US$ million) 1. Development of Methods, 7.92 3.80 47.98 Instruments, Norms and Capacity for Evaluation and Accreditation 2. Development and Consolidation 5.19 2.49 47.98 of a Higher Education Quality Assurance Information System 3. Fund for Quality Enhancement 39.06 18.71 47.93 Total Baseline Costs Physical contingencies Price contingencies Total Project Costs Interest During Implementation Front-End Fees Total Financing Required 52.17 25.00 47.92 C. Lessons Learned and Reflected in the Project Design 23. The Project has been designed taking into consideration and incorporating lessons learned from other similar Bank operations in the region and from other national experiences in the fields of higher education and quality assurance. 24. The design of Component 1 is inspired by the experience of the Argentinean Program for the Reform of Higher Education (PRES, Programa de Reforma de la Educación Superior), which included (i) capacity building at the Ministry of Education and its Secretariat for University Policy, and (ii) support to Argentina’s CONEAU, with institutional strengthening activities focused on its first few years of functioning. 25. The design of Component 2 has greatly benefited from previous experiences in Chile and Colombia. In 2006, both of these countries established comprehensive academic information systems with the objective of strengthening the quality of their higher education systems. In Chile, the Higher Education Information System (SIES) made clear the importance of ensuring a wide coverage for data collection efforts, and the usefulness of funding conditionality for ensuring that HEIs share information., The Colombian Higher Education National Information System (SNIES, Sistema Nacional de Información de la Educación Superior) illustrated the importance of providing information not only in relation to the supply of skills, but also regarding demand, through surveys conducted with both graduates and employers. In general, both experiences demonstrated the importance of building a national information system that maximizes both coverage and reliability. The main lesson learned, however, is that the data must 8 include a critical proportion of institutions and be of sufficiently high quality in order to ensure that the higher education information system is useful and effective for the decision-making process of prospective students, employers, HEI managers, and policy makers. 26. Component 3 has been designed after a worldwide review of similar quality enhancement funds, and particularly with inspiration from the Chilean and Argentinean experiences in establishing incentive funds for higher education. Chile’s Program for the Improvement of Quality and Equity in Higher Education (MECESUP, Programa de Mejora de la Calidad y la Equidad en la Educación Superior) and Argentina’s Fund for the Improvement of University Quality (FOMEC, Fondo para la Mejora de la Calidad Universitaria) offered valuable insights regarding the relevance of institutional design for maximizing the effect of incentives in diversified systems of higher education. IV. IMPLEMENTATION A. Institutional and Implementation Arrangements 27. COSUSINEACE would be the responsible agency. Institutional arrangements for Project implementation have been designed in order to maximize ownership, efficiency and accountability while including suggestions and requirements from the Ministry of Economy and Finance (MEF, Ministerio de Economía y Finanzas). (a) The Project Steering Committee (CDP, Comité Directivo del Proyecto) would be the highest body for Project management. The CDP would have a tripartite composition with representatives of COSUSINEACE (one from CONEAU and one from CONEACES), MINEDU and MEF and would be responsible for overall Project oversight and coordination. (b) The Project Coordination Unit (UCP, Unidad Coordinadora de Proyecto), which would work as an independent unit within COSUSINEACE, would be responsible for the management, coordination, supervision, monitoring and evaluation (except the evaluations of proposed improvement plans under Component 3), and fiduciary management of the Project. To this end, the UCP would be led a Project Executive Directorate, composed of a Project Executive Director and an assistant, and include two units: (i) Management and Finance Unit (including Procurement, Accounting, and Treasury areas); and (ii) Planning, Budget, Monitoring and Evaluation Unit. In addition, the UCP would include three Coordinators, one for each component, each of whom may have some administrative support staff. The Coordination Unit for Component 3 would be the STF and would include one coordinator (the Fund’s Technical Secretariat) as well as two sub-coordinators (one for universities and one for IEESs) as well as the corresponding committees of peers. The UCP would be formally created by the General Directorate of Public Budget (DGPP, Dirección General de Presupuesto Público), and incorporate an existing core team composed of FM, Procurement, and monitoring specialists. (c) The FEC requires specific arrangements due to the number of transactions and to the involvement of universities and IEESs in implementation. The Fund Technical 9 Commission (CTF, Comisión Técnica del Fondo), in charge of the selection and approval of the improvement plans proposals, would be separate from the UCP to enhance transparency and responsiveness vis-à-vis HEIs. However, the Fund Technical Secretariat (STF, Secretaría Técnica del Fondo), which would be part of the UCP, would be responsible for the technical management and coordination of Component 3. The CTF would be the highest decision-making body for approving improvement plans and for fund allocation. The STF would be responsible for administering the calls for proposals for PMIs and PMCs, assess the proposals received, and assist HEIs to improve the proposals following the CTF’s guidelines. While HEIs would be responsible for the technical implementation of the corresponding improvement plans, the UCP would keep responsibility for the fiduciary management of all activities. The FEC would also count on two coordinators, one for the support of universities and another one for the support of IEESs. The evaluation of HEIs’ improvement plan proposals would be done by Committees of Peers. 28. The Project’s Operational Manual includes detailed descriptions of institutional arrangements, the activity flow, financial management, accounting, and audit, as well as standards and templates to ensure that Project implementation complies with standard Bank practice and is as professional and transparent as possible. B. Results Monitoring and Evaluation 29. Sufficient Monitoring and Evaluation (M&E) capacity has been identified within COSUSINEACE, both with regard to its own activities (as evidenced by the successful implementation of the IDF grant by CONEAU and CONEACES) and with regard to HEIs and the higher education system at large. 30. The UCP would be responsible for gathering, processing, and analyzing data on the progress toward PDO and Intermediate Results Indicators targets. Specifically, the UCP’s Planning, Budget, Monitoring and Evaluation Unit would ensure that those responsible for data collection and data management do so in a systematic manner and make it available when necessary. Ultimate responsibility for results monitoring and evaluation, especially regarding semi-annual Project Reports, would lie within the CDP. The Bank would work jointly with the CDP during regular supervision missions in order to assess Project implementation progress. 31. The External Advisory Council (CAE, Comité Asesor Externo), composed of an international group of six recognized authorities in their respective fields of scientific, professional, and management activity, would perform a yearly evaluation of the Project, including one at mid-term and at the end. Special attention would be paid to the FEC and its improvement plans. CAE would also assess the quality of a sample of self and external evaluation through the selection and assessment of a sample. Composed of both Peruvians and foreigners, the CAE would report directly to the CDP. 10 C. Sustainability 32. Sustainability of the Project would ultimately be determined by the appropriate working of institutional design, material incentives, and increased flow of information in the field of higher education quality assurance. Specifically, how sustainable the Project is in the long term would largely depend on: (i) the professionalization of CONEACES, CONEAU, EEFAs, and evaluation teams within HEIs, with views to developing a culture of evaluation in the higher education system; (ii) continuity in the understanding currently shared by educational stakeholders, employers, and policy makers, that efforts must be made to improve quality in the higher education system; (iii) growth and continuity of the information system, with high levels of recognition and ownership by a wide range of actors including prospective students and their families, HEI managers, and employers; and (iv) continuity in the legal, organizational and economic resources allocated to COSUSINEACE. The Project incorporates a number of measures to enhance its sustainability, particularly as it relates to points (i) and (iii) above, through Components 1 and 2 respectively. V. KEY RISKS AND MITIGATION MEASURES A. Risk Ratings Summary Table 2: Risk Ratings Summary Stakeholder Risk Moderate Implementing Agency Risk - Capacity Substantial - Governance Low Project Risk - Design Moderate - Social and Environmental Low - Program and Donor N/A - Delivery Monitoring and Sustainability Low Overall Implementation Risk Moderate B. Overall Risk Rating Explanation 33. The overall risk rating of the Project is considered to be moderate. Nevertheless, capacity risk is significant given that COSUSINEACE is not a fully operational unit yet. A number of measures are being taken to mitigate against this risk. A Project Coordination Unit (UCP) would be established within COSUSINEACE, including the required fiduciary staff. The UCP would be staffed with qualified and experienced staff, referring to both Bank and GoP requirements. Staff needs in terms of number and qualifications, and their financing, have been defined. The Project’s Component 1 would implement capacity-building activities including training, technical assistance and equipment for COSUSINEACE. Based on an FM capacity assessment, specific processes and procedures for the operation of the UCP and FEC have been designed – including clear roles and responsibilities, internal controls, the design and implementation of a management information system, and recruitment of FM staff (with terms of reference approved 11 by the Bank) no later than 3 months after effectiveness. The Operational Manual includes clear FM and Procurement processes and procedures. VI. APPRAISAL SUMMARY A. Economic and Financial Analysis 34. An in-depth economic and financial analysis of the Project has been carried out, including a cost-benefit analysis, a social evaluation and sensibility analysis. The economic analysis has been done within a 20 year time horizon, since the benefits of accreditation can only be measured on a long-term basis following the cumulative impact on the labor market of successive cohorts of graduates from accredited programs. Calculating economic benefits involves looking at four elements: (i) assuming the positive impact of quality improvements on wages; (ii) assessing the magnitude of these quality improvements on average wages increases and its dispersion, (iii) taking into consideration the dynamics of these effects along the economic analysis’ time horizon of 20 years, and (iv) considering the beneficiaries as those graduates from accredited programs. 35. In this framework, the Project’s benefits would be determined by the salary increase that is associated with the higher productivity stemming from quality improvements in higher education. Each year, an increasing number of graduates would benefit from the wage premium. The actual flow of beneficiaries would be determined by the percentage of institutions that have been accredited: additional beneficiaries each year are a percentage of the student population equal to the percentage of HEIs that have been accredited. The stock of beneficiaries is determined by all those who graduated from an accredited HEI five years after accreditation or later 12. Finally, the wage premium corresponding to each beneficiary would depend on the income quintile to which they would belong had the accreditation process not taken place, as well as on the number of years of accumulated professional experience. 36. The social assessment was carried out considering both private and social costs. For the evaluation considering private costs the procedure is as follows: the social cash flow is estimated on the basis of wage premia after Project implementation (as compared to the non-Project scenario), minus investment and minus the costs of operation and maintenance at current prices. Since these are real figures, quantities are expressed on the basis of the prices corresponding to year 0. The flow is discounted with the real social discount rate (11 percent), as calculated by MEF. Assuming that disbursements would take place from year 3 onwards (i.e the period in which improvements due to activities financed by Component 3 would begin), the estimates for Project cost-effectiveness when considering different alternatives offer IRRs ranging from 30 to 42 percent, both with private and social costs. This demonstrates the high cost-effectiveness of the Project for any of the scenarios being considered. B. Technical 37. The rationale for the technical approach selected is based on both local and international experience and evidence of its effectiveness. Locally, the Project has benefited from the lessons 12 It is assumed that the average program lasts for 5 years. 12 of experience in the fields of accreditation and quality assurance, as well as from the thorough review and technical inputs from MEF’s National System of Public Investment (SNIP, Sistema Nacional de Inversión Pública). Internationally, the Project has benefited from the increasing move towards improvement-oriented financing in the higher education sector. 38. The Project is highly relevant for the development and better use of human capital in Peru. In a scenario characterized by both massive access to higher education and by a marked proliferation of HEIs, it becomes vital that institutional, policy and economic efforts are focused on the quality of higher education. Quality of the provision of higher education in Peru is considered to be below regional standards, at least partly due to a lack of updating in both programs and institutions, a lack of investment in HEIs’ human resources, and a severe disconnection between the supply of graduates and the demands of the labor market. This Project’s threefold strategy (establishing incentives for the promotion of a culture of evaluation in the higher education system; fostering efforts by both government and HEIs to increase quality in tertiary education; and promoting the collection and dissemination of relevant data about higher education and its connection to the labor market) encompasses the necessary next steps for increasing Peru’s human capital and its use in and out of HEIs. 39. In this context, the Bank is in a privileged position to offer both the financial and the technical support needed for the development of a quality assurance system for Peru’s higher education system. Given the Bank experience in other countries in the region and the high level of commitment among Peruvian officials, the proposed Project design seems the most appropriate strategy within the current constraints for fostering synergies and achieving the PDO. C. Financial Management 40. A financial management (FM) assessment was performed to determine the adequacy and mechanism of COSUSINEACE’s FM arrangements to support Project implementation. 41. COSUSINEACE has not become a completely operational entity yet, although the necessary steps to become fully operational have been identified and an action plan designed. The Project’s design requires management of a large number of small transactions and interaction with HEIs throughout the country for the financing and implementation of FEC improvement plans. Those features require sound operational and FM arrangements, as well as mitigation measures, which have been substantially designed but are not fully in place yet. On this basis, the FM risk is considered substantial. 42. In order to complete effective implementation and later operation of FM arrangements to adequately support Project execution, the following aspects need to be addressed, as per the Loan Agreement: (i) complete the actions required for the UCP to become a fully operational unit with its subsequent budgetary allocation in the 2013 National Budget; (ii) design and implement a Project information system to allow the recording, controlling, reporting and monitoring of Project transactions, including FEC improvement plans and issuance of required financial reports, the readiness of which is a condition of disbursement for Component 3; and (iii) recruitment of key financial management staff not later than three months from effectiveness. Subject to the successful implementation of these actions, the proposed financial management arrangements – as designed – are considered acceptable to the Bank. 13 D. Procurement 43. An assessment of the implementing agency’s capacity to implement procurement actions for the Project was conducted. Based on the information available at the time of the procurement capacity assessment, the Bank has assessed the preliminary overall risk as substantial. The Procurement area of the UCP would be composed of a Director who would be assisted by a number of specialists to be determined on the basis of the number and frequency of transactions. As part of Project preparation, the UCP has prepared an Operational Manual which provides detailed procurement information for Project implementation, including: (i) the particular methods for the procurement of goods, non consulting services and consultants, (ii) a clear definition of responsibilities that would apply to each kind of procurement, (iii) filing procedures and, (iv) management of the Procurement Plan. The Borrower, through the UCP, would prepare an acceptable Procurement Plan which would be made available through the Procurement Plans Execution System (SEPA). Procurement for the proposed Project would be carried out in accordance with the Bank’s "Guidelines: Procurement Under IBRD Loans and IDA Credits" dated January 2011; "Guidelines: Selection and Employment of Consultants by World Bank Borrowers" dated January 2011, and the provisions stipulated in the Loan Agreement. E. Social (including Safeguards) 44. The overall social impact of the Project is expected to be positive. Peruvian stakeholders and society at large seem to share a common understanding and interest regarding the need to foster quality assurance in higher education. Moreover, the Project concentrates a good proportion of its funds (at least 50 percent of the resources of Component 3) for the benefit of non-top HEIs. In this sense, it is expected that this investment would greatly contribute to improved equity among all HEIs. Since the student body of these HEIs is generally from the lower end of the national income distribution, the Project would directly benefit those from lower socioeconomic status. 45. OP/BP 4.10 Indigenous Peoples is triggered as the Project has a nationwide scope and would affect HEIs that have indigenous students. Linguistic, cultural, ethnic and socioeconomic barriers have historically undermined indigenous peoples’ access to and success in relation to higher education. The Indigenous Peoples Planning Framework (IPPF) that has been developed for the Project provides important guidance and a menu of possible activities for HEIs implementing a Institutional Improvement Plan (PMI) where more than 5 percent of the student body is indigenous as a way to address these barriers and improve both the quality as well as effectiveness of higher education for indigenous students. 46. As per OP/BP 4.10 Indigenous Peoples, an IPPF has been prepared and has been consulted with a representative group of stakeholders, including indigenous student unions, individual students, national indigenous organizations and associations, and academic inclusion experts. The final version of the IPPF was disclosed on the Bank’s and CONEAU’s websites on September 21 and September 12, 2012, respectively. A draft had been posted on both websites on August 30, 2012, prior to Appraisal. The IPPF includes the following: (a) a summary of the legal and institutional framework regarding indigenous peoples and education in Peru; (b) a summary of the relevant demographic, social, cultural, and political characteristics of indigenous 14 peoples in Peru; (c) an analysis of the current barriers to access or gaps in information that impact participation in the labor market; d) the results of the consultation with indigenous students and representatives mentioned above; and e) a matrix of recommendations based on the consultations and the social assessment that are intended to improve access and quality of higher education for indigenous peoples in Peru. 47. For those HEIs that have more than 5 percent of indigenous peoples within their student body and that are selected for the implementation of a PMI, an Indigenous Peoples Plan (IPP) shall be prepared and appropriately consulted based on the guidance contained in the IPPF. In terms of consultation, the IPPs must undergo a process of free, prior and informed consultations with: (i) the indigenous student body at the institution, (ii) representatives of indigenous student unions related to the institution, and/or (iii) representatives of the local indigenous community where appropriate. F. Environment (including Safeguards) 48. The Project is not expected to have any major environmental impact. The Project will not be financing any new construction or civil works. The improvement plans for the various HEIs are expected to include some routine maintenance and refurbishment of existing infrastructure that will be subject to local legislation concerning worker health and safety and disposal of materials such as paint. For these reasons, this Project is a Category C. 15 Annex 1: Results Framework and Monitoring PERU: Higher Education Quality Improvement Project Results Framework Project Development Objective (PDO): The objective of the Project is to improve Peru’s higher education quality assurance system through the promotion of self and external evaluations, the financing of improvement plans, and the provision of information. Responsibility Description Core Unit of Cumulative Target Values Freque Data Source/ Indicator Baseline for Data (indicator Measure ncy Methodology Collection definition etc.) 2013 2014 2015 2016 2017 PDO LEVEL RESULTS INDICATORS Indicator One: Number of # University University University University University University Quar- CONEAU UCP in Self-evaluation programs in the key disciplines programs programs programs programs programs programs terly and collaboration is conducted on and HEIs that have completed 31 122 182 286 381 470 CONEACES with the basis of self-evaluation on the basis of Registries CONEACES standards CONEAU or CONEACES IEES IEES IEES IEES IEES IEES and CONEAU approved by standards. programs programs programs programs programs programs CONEAU (for 1 100 185 262 333 401 universities) and CONEACES Univer- Univer- Univer- Univer- Univer- Univer- (for IEESs). Self sities sities sities sities sities sities evaluation is 0 0 7 13 19 24 completed when the HEI’s legal IEESs IEESs IEESs IEESs IEESs IEESs representative 0 0 49 100 152 205 reports to CONEAU / CONEACES. The key disciplines are: (a) at University level: Education; Medicine; Other Health Sciences; engineering; (b) At IEES level: Health; Education; and Technology. 16 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) Baseline data correspond to July 2012. Indicator Two: Number of # University University University University University University Quart- CONEAU UCP in External programs in the key disciplines programs programs programs programs programs programs erly and collaboration evaluation is and HEIs that have completed 0 7 17 30 45 63 CONEACES with conducted on external evaluation on the basis Registries CONEACES the basis of of CONEAU or CONEACES IEES IEES IEES IEES IEES IEES and CONEAU standards standards. programs programs programs programs programs programs approved by 1 9 24 43 65 89 CONEAU (for universities) and Univer- Univer- Univer- Univer- Univer- Univer- CONEACES sities sities sities sities sities sities (for IEESs). 0 0 1 2 3 5 External evaluation is IEESs IEESs IEESs IEESs IEESs IEESs completed once 0 0 1 3 6 10 the corresponding EEFA sends the final report to CONEAU / CONEACES. The key disciplines are: (a) at University level: Education; Medicine; Other Health Sciences; engineering; (b) At IEES level: Health; Education; Technology. Baseline data correspond to July 2012. Indicator Three: Number of # PMIs PMIs PMIs PMIs PMIs PMIs Annual STF UCP in Institutional program improvement plans and 0 0 0 0 2 6 collaboration improvement 17 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) institutional improvement plans with plans (PMI) and that are implemented PMCs PMCs PMCs PMCs PMCs PMCs CONEACES program satisfactorily. 0 0 0 5 22 60 and CONEAU improvement plans (PMCs) would be judged satisfactory if the following two conditions are met: (i) all activities are implemented, and (ii) a minimum level of targets are met, as specified in the Operational Manual. Indicator Four: Number of # 0 0 10,000 30,000 60,000 100,000 Annual The website’s UCP, in SIAC would users registered in the website of users registry. collaboration have a website the Quality Assurance with the granting access Information System (SIAC). website’s to Acredita Perú manager. and Futuro Profesional to all registered users, which would allow to count them (and, for other monitoring, evaluation and research purposes, to track how much time is spent looking for information in the portal). Registration for regular users 18 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) would be made simple to avoid deterring use. Expected users include secondary and higher education students and their families, the university community at large, policy makers and other stakeholders. INTERMEDIATE RESULTS Component One: Development of Methods, Instruments, Norms and Capacity for Evaluation and Accreditation Intermediate Result indicator % 52 64 76 88 100 100 Annual CONEAU UCP, in The Directions One: Percentage of higher and collaboration for Evaluation education student enrolment in CONEACES with CONEAU and programs covered by finished Registries and Accreditation and validated standards. CONEACES. (DEAs) at CONEAU and CONEACES define standards for each discipline according to their own procedures. These standards and then presented to CONEAU / CONEACES, which subject them to a process of 19 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) technical review and consultation. After the corresponding modifications have been made, the directors of CONEAU / CONEACES approve the standards. Standards are validated when COSUSI- NEACE issues a resolution in the Official Journal El Peruano. Intermediate Result indicator # 0 391 846 1,133 1,413 1,611 Annual CONEAU UCP, in Each evaluation Two: Number of evaluation and collaboration team at either teams coached and/or trained by CONEACES with CONEAU IEESs or CONEAU or CONEACES. Registries and universities CONEACES. benefiting from CONEAU’s or CONEACES’ coaching and/or training. If a team receives both coaching and training it would not be double counted. Intermediate Result indicator # 151 308 502 709 901 1,084 Annual CONEAU UCP, in Academic peers Three: Number of academic and collaboration would be trained peers trained by CONEAU or CONEACES with CONEAU by CONEAU CONEACES. Registries and and/or CONEACES. CONEACES for HEIs’ self- 20 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) evaluation and/or for their external evaluation to be conducted by EEFAs. If an academic peer participates in more than one training session it would not be double counted. Intermediate Result indicator # 4 6 8 8 8 8 Annual CONEAU UCP, in EEFAs are Four: Number of EEFAs and collaboration private entities authorized and supervised by CONEACES with CONEAU that require the CONEAU or CONEACES Registries and authorization of CONEACES. CONEAU (if they operate with universities) or CONEACES (if they do so with IEESs) to carry out valid external evaluation of HEIs. EEFAs are considered supervised as per the supervision guidelines, the development of which the Project would finance. Component Two: Development and Consolidation of a Higher Education Quality Assurance Information System 21 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) Intermediate Result indicator Text Acredita The The system The system 30% of the Annual COSUSI- UCP, in Acredita Perú is Five: Development of Acredita Perú has technical is fully is well degree NEACE collaboration an online Perú some design is functional dissemi- programs with the SIAC platform that preliminary finalized and nated are Technical contains data design responsive included Commission entered and organized on higher education quality assurance, including programs, HEIs and EEFAs. Intermediate Result indicator Text Futuro The Mega The Mega The system The system An Annual COSUSI- UCP, in The Futuro Six: Development and use of the Profesional Graduates Graduates is fully is well evaluation NEACE collaboration Profesional Futuro Profesional Observatory has some Survey is Survey is functional dissemi- of the use with the SIAC Observatory is preliminary started finalized and nated of the Technical an online design and the IT responsive. Observa- Commission platform that system for tory is contains data the finished. collected, Observator organized and y is analyzed designed, regarding constructed graduates’ and performance in validated. the labor market. Intermediate Result indicator # 0 0 1 2 3 4 Annual COSUSI- UCP, in Expected areas Seven: Number of sectoral NEACE collaboration of the four studies published and with the SIAC studies: (i) disseminated Technical methodology Commission and initial data collection to carry out an impact evaluation on the long term effects of the introduction of a quality 22 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) assurance system; (ii) an in-depth socio- cultural assessment; (iii) the role of information in the choice of post-secondary education; and (iv) accreditation in the higher education system. Component Three: Fund for Quality Enhancement Intermediate Result indicator # At the At the At the At the At the At the Annual FEC activities UCP, in Absolute Eight: Number of external program program program program program program database collaboration number of evaluations financed by the level level level level level level with the STF external FEC. 0 7 17 28 38 48 evaluations (either of IEESs At the At the At the At the At the At the or of non- IEES level IEES level IEES level IEES level IEES level IEES level university 0 0 1 2 4 6 programs) funded by the FEC. Only technical programs in education and health are eligible for external evaluation financed by the FEC. Intermediate Result indicator # PMIs PMIs PMIs PMIs PMIs PMIs Annual FEC activities UCP, in Absolute Nine: Number of improvement 0 0 2 5 8 13 database collaboration number of plans financed by the FEC. with the STF improvement 23 Responsibility Description Core Unit of Freque Data Source/ Indicator Baseline Cumulative Target Values for Data (indicator Measure ncy Methodology Collection definition etc.) PMCs PMCs PMCs PMCs PMCs PMCs plans, either for 0 17 35 58 82 106 HEIs (PMIs) or for programs (PMCs) funded by the FEC. Intermediate Result indicator #, % 759,607 853,564 947,520 1,041,477 1,135,433 1,135,433 Annual COSUSI- UCP, in Direct Ten: Direct Project beneficiaries 49 49 49 49 49 49 NEACE collaboration beneficiaries are (number), of which female Registry with CONEAU defined as (percentage). and students CONEACES enrolled at key disciplines, which would participate first in the quality assurance process by engaging in self- and external evaluation. The percentage of female beneficiaries is an approximation based on the percentage of female students in 2011. 24 Annex 2: Detailed Project Description PERU: Higher Education Quality Improvement Project 1. Component 1: Development of Methods, Instruments, Norms and Capacity for Evaluation and Accreditation (estimated total cost: US$7.92 million; Bank: US$3.80 million). Strengthening the capacity of CONEAU, CONEACES, HEIs and EEFAs to handle their respective responsibilities within the quality assurance system. This Component would finance the development of technical, methodological, and procedural capacity in the quality assurance system. CONEAU and CONEACES would be responsible for the implementation of Component 1. By reinforcing the foundations of the quality assurance system and strengthening the capacity and legitimacy of CONEAU and CONEACES as the central player in it, Component 1 plays a key role in the achievement of the PDO. This Component includes three subcomponents: (a) Subcomponent 1.1: Development of management, planning and evaluation capacity of CONEAU and CONEACES (estimated total cost: US$1.41 million; Bank: US$0.68 million). Enhancing of CONEAU and CONEACE’s management, planning and evaluation capacity by developing standards, norms and procedures for evaluation and accreditation, including those for the supervision of the EEFAs. Expected outputs would include the production and dissemination of methodological guides, technical protocols, and other instruments as needed. It would provide targeted technical assistance from national, regional, and international experts in the field of evaluation and accreditation of higher education institutions and programs. (b) Subcomponent 1.2: Development of self-evaluation and external evaluation capacity (estimated total cost: US$1.91 million; Bank: US$0.91 million). Strengthening of the capacity of (i) HEIs to engage in both internal and external evaluation processes; and (ii) EEFAs to undertake external evaluations, including learning from other experiences at the regional and international levels. This would include regional networks such as RIACES (Red Iberoamericana para la Acreditación de la Calidad en la Educación Superior, i.e. Iberoamerican Network for Quality Accreditation in Higher Education) and ENQA (European Association for Quality Assurance in Higher Education). This subcomponent would finance: (i) The development of coaching capacities at CONEAU and CONEACES to encourage and incubate the establishment of evaluation teams within HEIs; (ii) The training of evaluation teams within HEIs, so that their capacity to engage with, and take advantage from, evaluation processes can be maximized; (iii)The development of regional and national knowledge exchange, including workshops and other networking platforms, so that inter- institutional synergies and learning from evaluation are encouraged; (iv) The development of groups of experts within CONEACES and CONEAU for providing ad hoc, demand-driven technical assistance to 25 HEI evaluation teams as they encounter issues throughout the evaluation process; (v) The provision of technical assistance to EEFAs, including the training of academic peers and other specialists in evaluation; and (vi) The provision of training and technical assistance to CONEACES and CONEAU on the supervision of EEFAs, so that EEFAs are held accountable and thus gain and retain the legitimacy necessary for a sustainable quality assurance system. (c) Subcomponent 1.3: Support for Project implementation (estimated total cost: US$4.61 million; Bank: US$2.21 million). Provision of support for the technical and administrative management of the Project. This subcomponent would enhance COSUSINEACE’s human, technical, and organizational capacity for the implementation of the Project. It would finance human resources, training, and technical assistance for a number of units dedicated to Project implementation, namely the Project Steering Committee, the Project Executive Directorate, and the Project Coordination Unit, and the Technical Commission and the Technical Secretariat of the Fund for Quality Enhancement. Human resources would be financed chiefly by counterpart funds. Subcomponent 1.3 would also finance equipment and audit. 2. Component 2: Development and Consolidation of a Higher Education Quality Assurance Information System (estimated total cost: US$5.19 million; Bank: US$2.49 million). Developing and consolidating a higher education quality assurance information system for the systematic gathering, processing and dissemination of reliable data regarding higher education quality assurance as well as graduates’ performance in the labor market. This Component would develop and consolidate a higher education Quality Assurance Information System (SIAC, Sistema de Información del Aseguramiento de la Calidad). Specifically, it would provide both resources and technical assistance for the systematic gathering, processing, and dissemination of reliable data regarding higher education quality assurance, and graduates’ performance in the labor market. By establishing, developing, and consolidating a centralized, comprehensive information system on higher education quality assurance, this component would provide all stakeholders with relevant, high-quality information about quality of higher education and its links to the labor market. The development and consolidation of SIAC is expected to yield the following benefits: (i) enhance CONEAU’s and CONEACES’s management and supervision capacity; (ii) improve decision making by HEIs in terms of management and investment, thus increasing the quality and relevance of programs offered; (iii) improve decision making by parents and prospective students in terms of career decisions, leading to an improvement in employment opportunities and in the external efficiency of the higher education system; and (iv) allow better public scrutiny of higher education quality by civil society, policy makers, and society at large. In order to incentivize HEIs’ participation in SIAC and to increase its coverage, the allocation of Component 3 funds would be linked to the release of key data by HEIs into the SIAC, on top of the standard data from the evaluation process that would feed the SIAC databases. Component 2 plays a central role in the pursuit of the PDO, since reliable and well-communicated information constitutes a fundamental component of a quality assurance system. This system has three subcomponents. 26 (a) Subcomponent 2.1: Development and consolidation of an observatory of the accreditation of Peru’s higher education (Acredita Perú) (estimated total cost: US$1.65 million; Bank: US$0.79 million). Development and consolidation of an observatory of the accreditation process of Peru’s higher education (Acredita Perú), responsible for the organization and dissemination of information on standards, criteria, indicators, EEFAs, as well as the evaluation and accreditation procedures by degree programs and HEIs. This Subcomponent would finance data entry and data management regarding the accreditation of universities and IEESs, on the basis of the information collected by CONEAU and CONEACES, including information received through evaluation. It would also finance a communications campaign and an internet resource in order to make this information widely accessible. (b) Subcomponent 2.2: Establishment of an observatory of higher education graduates (Futuro Profesional) (estimated total cost: US$3.13 million; Bank: US$1.50 million). Establishment of an observatory of higher education graduates (Futuro Profesional), for systematically collecting, processing and disseminating statistical information about the performance of higher education graduates in the labor market, and providing information regarding employability and salaries of graduates from different programs and HEIs. Futuro Profesional would develop and disseminate its analyses of the supply of skills in the labor market. This Subcomponent would finance two major activities: (i) a Mega Graduates Survey, 13 which would measure graduates’ performance in the labor market by program and institution; 14 and (ii) the observatory of graduates, which would systematically process and disseminate such statistical information. (c) Subcomponent 2.3: Elaboration of sector studies (estimated total cost: US$0.41 million; Bank: US$0.19 million). Carrying out of several studies aiming at a better understanding of Peru’s higher education, including inter alia: (i) the development of a methodology and initial data collection to carry out an evaluation regarding the long term effects of the introduction of a quality assurance system; (ii) in depth socio- cultural assessment, in the framework of the Project’s Indigenous Peoples Planning Framework (IPPF); (iii) assessing the role of information in the choice of post- secondary education; and (iv) the effect of accreditation in Peru’s higher education system. This Subcomponent would support a number of studies that lead to a better understanding of Peru’s higher education, thus building more reliable information and stronger analysis. 13 A sampling strategy has been designed by the National Institute of Statistics and Computing (INEI), which would execute the Mega Graduates Survey. This strategy identifies variation across HEI, program, and region, and costs some estimated US$2,500,000. 14 Alternative options to a graduates survey have been considered (namely through the income tax administration, via the collection of data on taxpayers’ education by the National Superintendency for Tax Administration (SUNAT, Superintencia Nacional de Administración Tributaria); or through the electronic report that companies send monthly to the Ministry of Labor) but rejected due to the high degree of informality in Peru’s labor market and to the difficulty of reaching the necessary institutional arrangements. 27 3. Component 3: Fund for Quality Enhancement (estimated total cost: US$39.06 million; Bank: US$18.71 million). Promoting the improvement of the quality and relevance of academic programs and HEIs, through the supporting of the establishment of the FEC and its two windows of financing. This component would promote external evaluation and accreditation by supporting the establishment of a Fund for Quality Enhancement (FEC, Fondo para el Estímulo de la Calidad), designed to promote the improvement of the quality and relevance of academic programs and higher education institutions. The FEC would finance two types of activities: (i) external evaluation for IEESs, given their capacity is significantly more limited than that of universities; 15 and (ii) improvement plans at the program or HEI level for both IEESs and universities, based on the results of previous self- and external evaluation financed outside of the Project (except for those IEESs that receive funding for their external evaluation). The FEC would be a critical tool for strengthening the quality assurance system for higher education, particularly by encouraging HEIs to conduct self- and external evaluation so they can access its funds. Its specific goals would be: (i) to foster the quality and relevance of higher education in Peru, encouraging processes of evaluation, accreditation, and quality improvement of HEIs; (ii) to support improvement plans that contribute to capacity building in HEIs, including teaching and management; (iii) to promote a holistic approach to the activity of HEIs, encouraging them to develop links with their national and international context; (iv) to stimulate the improvement of human resources, particularly the human capital of staff and quality; and (v) to support the effective investment in equipment for the modernization and improvement of teaching. The FEC would create financial incentives to support quality improvement through two Subcomponents: (a) Subcomponent 3.1: External Evaluation of IEESs (estimated total cost: US$0.59 million; Bank: US$0.28 million). External evaluation of IEESs (institution-wide and for technical programs in health and education) by the EEFA of their choice, after completion of self-evaluation. This Subcomponent would finance external evaluation 16 of IEESs, institution-wide and for programs (technical programs in education and health only 17) by the EEFA of their choice among those authorized by CONEACES. To access this fund, the IEESs would first need to complete their self- evaluation. (b) Subcomponent 3.2: Support to Improvement Plans (estimated total cost: US$38.46 million; Bank: US$18.43 million). Implementation of Improvement Plans for both IEESs and universities (including programs within HEIs), based on the results of the self and external evaluation. Only institutions or programs that have completed an external evaluation process would be eligible to access FEC funding. Institutional Improvement Plans (PMI, Planes de Mejora Institucional) must have established objectives, activities and indicators for mid-term improvements, and they must address the weaknesses identified in the self-evaluation and external evaluation. Program Improvement Plans (PMC, Planes de Mejora de Carreras) would select 15 IEESs are regulated by Act nº 29394, which does not allow them to establish income and makes them almost exclusively dependent on government transfers. These restrictions make IEESs particularly vulnerable. 16 While Subcomponent 1.2 finances capacity building, advice and coaching for the evaluation process, Subcomponent 3.1 finances external evaluation itself, particularly the costs of hiring an EEFA’s services. 17 Technology programs at the IEES level were excluded from eligibility for FEC funds under Subcomponent 3.1, following a request by MEF, due to their small number. 28 improvement plans for strengthening or reforming existing programs or establishing new programs, but only in the key disciplines defined above. In order to avoid that high-quality institutions crowd out those where improvement is most needed, Subcomponent 3.2 would have two sorts of calls for proposals, one for “top� and one for “non-top� HEIs. (i) Calls for proposals for the promotion of institutional excellence, which would be addressed at HEIs and/or programs that have received a recommendation of accreditation from the corresponding EEFA. These calls for proposals would operate on a competitive basis. They would concentrate between 40 and 50 percent of the FEC funds for any given fiscal year. (ii) Calls for proposals for initiating institutional change, which would be targeted at HEIs and/or programs that, in their external evaluation, have not received the recommendation for accreditation. These would operate not only on a competitive basis but also with a view to engage the low- quality HEIs in the quality assurance system, as per criteria specified in the Operational Manual. These calls for proposals would represent between 60 and 50 percent of FEC funds for any given fiscal year. 4. Eligible expenses to be financed under Subcomponent 3.2 are expected to be: (i) stimuli for training (externships, professors training, workshops); (ii) technical assistance to improve programs, curricula, institutional strengthening, capacity building, development of learning materials, etc. (through consulting services, visiting professorships, in-house training); (iii) equipment for teaching (such as equipment for workshops, laboratories and libraries; multimedia equipment; goods for the improvement of management, such as software and computing equipment) and small improvements in existing educational infrastructure; and (iv) institutional strengthening (goods and services necessary to developing academic and management improvement processes, such as standardizations, certifications, auditing and implementation of academic improvements). 5. Eligibility criteria for accessing FEC funds. Duly authorized HEIs that fulfill the ownership criterion established in the Operational Manual 18 would be eligible to access FEC funds. Any proposal presented to the FEC would have to be expressly authorized by the highest authority of the applying HEI. 19 Only IEESs authorized by the Ministry of Education would be eligible for funds under Subcomponent 3.1. Universities are expected to self-finance their external evaluation and would thus be eligible only for Subcomponent 3.2. In any case, the following eligibility conditions would have to be fulfilled for HEIs that present proposals for improvement plans (Subcomponent 3.2): 18 The Operational Manual defines the ownership criterion to be fulfilled by HEIs to access FEC funds. At first, only public HEIs will be eligible, as per the Operational Manual, to receive FEC support. The ineligibility of private HEIs will be re-assessed by the Government and the Bank during implementation, at which point a change to the Operational Manual may be agreed upon to include non-profit and/or for-profit private higher education institutions. 19 In the case of universities, this authorization would have to be formalized through a resolution from the Rector. 29 (a) The HEI should fulfill the ownership criterion established in the Operational Manual. (b) The HEI should have completed its self-evaluation and its external evaluation processes. (c) The corresponding report from the external evaluation should have been accepted by the CONEAU or the CONEACES (d) The improvement plan’s goals should be aligned with the recommendations stemming from the self-evaluation and external evaluation processes. (e) The HEI must present its improvement plan under the corresponding call for proposals, according to whether they are considered a top or a non-top institution. i. If the external evaluation recommended the HEI to apply for accreditation, it is considered a top institution and is therefore eligible for improvement plans funding under the calls for proposals for the promotion of institutional excellence. ii. If the external evaluation recommended the HEI not to apply for accreditation, it is considered a non-top institution and is therefore eligible for improvement plans funding under the calls for proposals for initiating institutional change. (f) In order for the selected improvement plans to become effective, HEIs must: i. Authorize access to the information required for the monitoring and the financial and physical auditing of the plans financed by the FEC. ii. Sign an agreement with the FEC, which would stipulate the legal and financial obligations in the implementation of the corresponding plan. These obligations would include, inter alia: compliance with the procedures specified in the Operational Manual, particularly with regards to the procurement of goods and services; having an accounting system adequate for the implementation of the plan; and, in the case of universities, having a team for the implementation of FEC plans. Expected allocation of FEC funds 6. The funding for Component 3 (FEC) would be distributed following the criteria presented above. Significant efforts have been made to plan the quantity (q) and unit cost (p) of improvement plans, on the basis of the estimated characteristics of the accreditation process at the program and at the HEI level. Efforts have been made to maximize q while ensuring that p is large enough to permit substantial changes at the HEI or program level. The planned distribution presented in the tables below also aims at balancing the two dimensions in the FEC’s goals (promoting excellence and initiating low-quality HEIs to institutional change). 30 Table 3: Estimated number and unit costs for external evaluations Number of external By level and discipline Unit cost (US$) Total cost (US$) evaluations Education programs at IEESs 26 10,100 262,600 Health programs at IEESs 22 10,100 222,200 Technology IEESs 6 18,364 110,184 Total 54 - 594,984 Table 4: Estimated number and unit costs for PMIs By institution By the result of the Number Unit cost (US$) Total cost (US$) type external evaluation of PMIs Promotion of excellence 4 1,310,960 5,243,840 Universities Initiation to change 4 725,212 2,900,848 Promotion of excellence 1 446,284 446,284 IEESs Initiation to change 5 254,545 1,272,725 Total Both 14 - 9,863,697 Table 5: Estimated number and unit costs for PMCs By By the result Number Unit cost Total cost institution of the external By program of (US$) (US$) type evaluation PMCs Education 6 334,713 2,008,278 Promotion of Medicine 5 436,634 2,183,170 excellence Other health sciences 6 363,636 2,181,816 Engineering 13 436,634 5,676,242 Univer- Education 6 167,272 1,003,632 sities Initiation to Medicine 5 363,636 1,818,180 change Other health sciences 5 327,273 1,636,365 Engineering 14 363,636 5,090,904 Total universities 60 - 21,598,587 Promotion of Education 5 251,035 1,255,175 excellence Health and Technology 5 236,364 1,181,820 IEESs Initiation to Education 19 127,273 2,418,187 change Health and Technology 17 127,273 2,163,641 Total IEESs 46 - 7,018,823 Total all HEIs 106 - 28,617,410 31 Annex 3: Implementation Arrangements PERU: Higher Education Quality Improvement Project Project Institutional and Implementation Arrangements Project administration mechanisms 1. COSUSINEACE would be the responsible agency. Institutional arrangements for Project implementation have been designed in order to maximize ownership, efficiency and accountability within the requirements of the Ministry of Economy and Finance (MEF). 2. The Project’s institutional structure would be organized as follows: (i) a Project Steering Committee (CDP), as higher decision-making body; (ii) a Project Coordination Unit (UCP), which would be responsible for general implementation and for relations with the Bank; (iii) an ad hoc structure for evaluating improvement plan proposals under the FEC; and (iv) implementation teams in each HEI for executing FEC funds. Figure 1 graphically presents the main organizational arrangements for Project management. Figure 1: Project organizational chart 32 3. The following paragraphs succinctly describe the roles of each institution and the relationships between them. (a) Project Steering Committee (CDP). MEF requires that all projects have a CDP. The responsibilities of the CDP would include: (i) dealing with and approving the annual operating plans and their associated budgets; (ii) setting the dates and quantities for the FEC’s calls or rounds for proposals; and (iii) monitoring objectives, results and indicators for the overall Project. The CDP would include four full members, each representing CONEAU, CONEACES, MINEDU and MEF. Furthermore, the CDP would include the Executive Project Director with voice, but not vote. The CDP would be created in the framework of the Loan Agreement. (b) Project Coordination Unit (UCP). The management, coordination, supervision, monitoring and evaluation (except the evaluations of improvement plan proposals under Component 3), and fiduciary management of the Project would fall under responsibility of the UCP. Regarding fiduciary management, and following MEF’s recommendations, the UCP would work as an independent unit, with no administrative link to COSUSINEACE’s General Office of Administration (OGA, Oficina General de Administración). Therefore, it would have to be created by the DGPP, which would grant the UCP its own allocation in the national budget. The UCP would thus work as an additional administrative structure within COSUSINEACE, specific for the Project. The internal organization of the UCP and the distribution of responsibilities would be as follows: (i) Project Executive Directorate. The Project Executive Director (DEP) would lead the UCP and report to the CDP. Moreover, the DEP would be responsible for overall Project management, including management, supervision, and evaluation of all Project activities as well as the relation with the Bank. The DEP would be supported by an assistant. (ii) Planning, Budget, Monitoring and Evaluation Unit. This area would include a coordinator, a specialist in programming and budget, and a specialist in monitoring and evaluation. This area would support DEP in the evaluation of Project implementation and monitoring of indicators and results. (iii)Management and Finance Unit. This area would deal with matters related to financial management, procurement and accounting in all three components. It would be divided in the following three areas and led by a coordinator who would be supported by a human resources specialist and an assistant. 1. Procurement area, which would deal with the procurement of goods and services. 2. Accounting area, which would deal with financial management for the Project. 33 3. Treasury area, which would deal with treasury matters. (iv) Coordinator for Component 1. The UCP would include one person in charge of coordinating Component 1. This person would play the role of technical facilitator for the implementation of Component 1, under the responsibility of CONEAU and CONEACES. (v) Coordinator for Component 2. The UCP would include one person in charge of coordinating Component 2. This person would play the role of technical facilitator for the implementation of Component 2, under the responsibility of the SIAC Technical Commission. (vi) Coordination Unit for Component 3 (Fund’s Technical Secretariat). The Coordination Unit for Component 3 would be the Fund’s Technical Secretariat, which would fall under the UCP. Nonetheless, the evaluation of improvement plan proposals would be responsibility of the Fund Technical Commission (CTF) due to the specificities of the FEC and the need to preserve transparency. On the basis of a careful analysis of international experiences with similar funding tools, it has been agreed that the FEC would: (i) have an institutional design favoring collegiate governance, based in pluralism and prestige; (ii) be guided by an efficient, professional, and austere style of management; (iii) work with flexibility and transparency when designing calls for proposals and their evaluation; and (iv) set up activities for providing technical assistance to HEIs regarding requirements, design, and procedures for improvement plans. 1. Functions of the STF. The STF’s main responsibility would be to provide all necessary administrative and technical support for the fulfillment of the FEC’s mission, with special attention to the technical assistance to HEIs, advice to CTF and monitoring. The STF’s functions would be: (i) to carry out secretariat tasks for the CTF; (ii) to manage the calls for proposals, as decided by the CTF; (iii) to manage the resources assigned to FEC in order to meet operational costs derived from the functioning of the institutional structure per se; (iv) to process all improvement plan proposals verifying compliance with formal and feasibility requirements including the preparation of the IPP if required; (v) to set up and keep updated the bank of approved peer evaluators and to convene their evaluation meetings; (vi) to circulate eligible improvement plans to Committees of Peers (CP) for their evaluation; (vii) to elaborate and submit to CTF the proposal for the funding of improvement plans and programs; (viii) to provide support to Committees of Peers in the fulfillment of their functions; (ix) to implement all decisions adopted by the CTF, organizing and executing the necessary administrative, information, and diffusion tasks; (x) to provide advice and support to the HEIs that require so in the preparation of both improvement plan proposals and improvement plan final versions addressed to the FEC; and (xi) to elaborate reports for the follow up of the FEC by the CTF. 34 Although the CTF would hold the highest decision-making powers, the STF would be the body maintaining the technical relationship with the UCP and the Executive Director. 2. Membership of the STF. The STF would be composed of an Executive Secretary, who would be supported by a technical team. The Secretariat would count on a structure within the sphere of COSUSINEACE, and supported by the posts and resources necessary for the fulfillment of its tasks. The basic structure would consist of the Executive Secretary and two Coordination Units, one of them for universities and another one for IEESs. Those Coordination Units composed by a core professional team would be responsible for managing the evaluation of improvement plans presented as well as training, advising and assisting HEIs to prepare improvement plan proposals and final improvement plans by each type of HEI. Moreover, Coordination Units would be in charge of convening their Committees of Peers, on an ad hoc basis and for each corresponding call for proposals from the FEC, for the evaluation of improvement plan proposals (PMIs and PMCs). Finally, the Coordination Units would take care of the follow up of those proposals that are granted funding. Additionally, the STF would have the following entities: 1. Sub-Coordinators for universities and IEESs. In order to increase FEC’s responsiveness to HEIs applying for and benefiting from its funding, there would be two coordinators, one per type of institutions. These two coordinators would facilitate the relationship between the STF and HEIs. 2. Committees of Peers (CPs, Comités de Pares). Committees of Peers would ensure the transparency and professionalism in the evaluation of proposals of PMIs and PMCs. They would meet ad hoc to assess the quality, expected benefits and viability of proposals presented by HEIs. a. Functions. Only during the evaluation period for each call for proposals, the CPs would: (i) write evaluation decisions on the basis of the evaluation criteria established for the corresponding call for proposals; and (ii) consolidate the results of the evaluation in a comprehensive vision for each knowledge area or discipline, proposing a list of merits for the evaluated plans. CP members would not be allowed to assess those plans for which they present a conflict of interest (i.e. those presented by the HEI to which they belong, or those related to people to whom they are relatives of up to third degree). b. Membership. CP members would be designated by the CTF, with the No Objection of the Bank, on the basis of proposals done by the STF. The Bank would support the identification of 35 international peers, since each CP would have at least one international member. National and international peers would be people with proven experience in the formulation and/or evaluation of improvement plans in higher education. The composition of the CPs would vary in relation to the improvement plans to be evaluated. CPs would be composed of a variable number of experts, depending on the knowledge area and on the number of plans to be assessed. When multidisciplinary CPs are formed, efforts would be made for them to have at least one representative of each of the main disciplines covered. (c) Fund Technical Commission (CTF). The CTF would be separate from the UCP to enhance transparency and responsiveness vis-à-vis HEIs, since the FEC requires specific arrangements due to the number of transactions and to the involvement of universities and IEESs in implementation. The CTF would be the highest decision-making body of the FEC. It would therefore be responsible for improvement plan approval, funds allocation, improvement plan control and follow up, as well as improvement plan changes and terminations. (i) Functions. The main responsibility of the CTF would be to approve and/or reject improvement plans presented in response to FEC’s calls for proposals. The CTF’s functions would be: (i) to ensure the achievement of FEC’s objectives; (ii) to guarantee transparency in all phases of the evaluation process; (iii) to establish the relative weight of each evaluation criterion previously to each call for proposals; (iv) to name the members of the Committees of Peers; (v) to decide about the allocation of FEC funds to eligible improvement plans that are given priority following the established principles; (vi) to request from the STF diagnosis reports about each of the FEC components in order to improve their functioning; and (vii) to inform the CDP and through it COSUSINEACE and the Bank on the development of FEC activities. (ii) Membership. The CTF would be composed of seven members, including a representative of MEF, who would seat the Commission in their individual capacity, with independence, and without representing any institution (with the exception of the representative of MEF). Four of them would have strong academic standing, preferably one from each of the following knowledge areas: education, health sciences, and engineering and technology. Another member would have ample experience and prestige in the management of HEIs (universities and/or IEESs). Another member would come from the Ministry of Economy and Finance and have ample experience in public management. The Project Executive Director would chair the CTF. Additionally, the Technical Secretary of the FEC would act as secretary for the CTF, with voice but no vote. CTF members would have a four-year mandate. 1. Member selection mechanisms. CTF members would be appointed by the CDP after consultations with a wide range of institutions representative of activities related to higher education or professional associations, who 36 would responsible for proposing a short list to CDP, under the basis of a profile agreed with the Bank. Once the decision is taken but before the person is appointed, the CDP would submit it to the Bank for its No Objection. 4. Management of Component 1. Since COSUSINEACE would be the direct beneficiary of Component 1, the responsibility for the technical management of the activities attached to this component would fall on CONEAU and CONEACES. Technically, Component 1 would be implemented by the Evaluation and Accreditation Directorates at CONEAU and CONEACES, respectively, facilitated by the Coordinator for Component 1. The UCP would maintain the following functions in relation to Component 1, in addition to fiduciary management: (i) registering the programming of activities carried out by the teams of consultants and experts that offer technical assistance to HEIs; and (ii) providing the human, material, and other type of resources that CONEAU and CONEACES need for the implementation of Component 1. On the other hand, CONEAU and CONEACES would be responsible for: (i) coordinating the formation of teams of consultants and experts in each of the Directorates for Evaluation and Accreditation at the operating bodies in the field of higher education within COSUSINEACE (i.e., CONEAU and CONEACES); (ii) proposing the consultants who would join the teams of national experts and thus offer technical assistance; (iii) demanding the hiring of specialized technicians for the follow up of the external evaluation phase; (iv) proposing the national or foreign experts who they wish to hire for the validation of their technical and normative tools; (v) ensuring that the national experts, technicians, and consultants demanded by them fulfill their tasks; and (vi) designing, proposing, and implementing programs for the promotion and training regarding the process of accreditation, in its phases of self-evaluation and external evaluation. 5. Management of Component 2. Component 2, focused on the development of a comprehensive information system for quality assurance, would be centrally managed by the UCP, facilitated by the Coordinator for Component 2 and under the steering role of the SIAC Technical Commission. The SIAC Technical Commission would be composed of representatives from CONEACES, CONEAU, MINEDUC and ANR, as well as the head of the Professional Future Observatory. The UCP would have the following functions in relation to the implementation of Component 2, in addition to fiduciary management: (i) establishing the information management team in charge of implementing the information system and labor observatory; (ii) elaborating and disseminating the reports stemming from the information system; (iii) making suggestions to the CDP regarding the necessary covenants and/or agreements to be pursued in order to guarantee and define the participation of those institutions that make the Technical Commission. CONEAU and CONEACES would be responsible for: (i) feeding and updating constant and permanent flows of data and information into the information system; (ii) requesting, with the favorable opinion of the Technical Commission, the specialists and/or consultants as well as the computer equipment that they may need; and (iii) programming and complying with goals and activities to be developed in the framework of Component 2. 6. Management of Component 3. The responsibility for the management and coordination of Component 3 would be shared between the CTF and STF, with responsibilities of UCP and HEIs in the implementation of the improvements plans: 37 (i) The CTF would be the highest decision-making body of the FEC. It would therefore be responsible for the approval improvement plan proposals, funds allocation, improvement plan control and follow up, as well as approving improvement plan changes and terminations. More details are provided in the paragraphs below. (ii) The STF would be responsible for: (i) carrying out a pre-evaluation process for all proposals for both Windows 1 and 2 of the FEC; (ii) supporting HEIs in the elaboration of final improvement plans; and (iii) contributing to monitor the implementation of improvement plans. (iii) The UCP, as indicated above, would be responsible for the fiduciary management of all improvement plan activities, including contract management (for consultancies and good acquisitions) and financial management (e.g. payments to consultants, interns, and suppliers). (iv) HEIs would be responsible for implementing their improvement plans at a technical level, i.e. elaborating terms of reference and performing the selection process for individual consultants, interns and scholars; establishing technical specifications for the acquisition of lab equipment; approval of activities, reports (and therefore payment authorizations); follow up of indicators, etc. 7. Implementation arrangements for FEC-funded improvement plans. Improvement plans corresponding to Component 3 (FEC) would follow different management routes depending on the Subcomponent to which they belong. (a) Subcomponent 3.1 improvement plans would use a simplified procedure for allocating funds for external evaluation. Only IEESs that have carried out self-evaluation are eligible for funding under this Subcomponent: universities would have to fund their own external evaluation 20. The external evaluation would be carried out through Peer Evaluators through the authorized EEFAs. IEESs and the EEFAs would formalize an agreement of External Evaluation, which would have to be implemented in 3-4 months. IEESs would be allowed to choose the EEFA responsible for its external evaluation among those authorized by CONEACES. Payments to EEFAs would be executed by the UCP in two installments: the first one, for the value of 50 percent, once the subsidy is approved by the CTF and the contract is signed; and a second one, for the remaining 50 percent, after the external evaluation is finished and its report is assessed. (b) Subcomponent 3.2 improvement plans (PMIs and PMCs), for which both IEESs and universities are eligible, would follow a separate procedure, which may be separated in four phases, as represented in Figure 2. 20 IEESs are regulated by Act nº 29394, which does not allow them to establish income and makes them almost exclusively dependent on government transfers. 38 Figure 2: Management arrangements for the implementation of FEC improvement plans corresponding to Window 2: Phases 1 and 2 (i) First phase. Preparation and approval of improvement plan proposal. HEIs would submit a improvement plan proposal (either a PMC or a PMI) for evaluation in response to a specific FEC call for proposals. This improvement plan proposal would be based on a template distributed by the FEC, and include as many details as possible regarding: specific objectives, summary diagnosis of the situation to be overcome and of the recommendations of the external evaluation, activities, beneficiaries, follow up indicators, timing, budget, preliminary terms of reference (ToRs), preliminary technical specifications, and preliminary proposals for consultants, interns, visiting professorships, etc. Before submitting the improvement plan proposal for consideration by the CTF, HEIs would be able to count on technical assistance from the STF, especially small IEESs and other HEIs with reduced administrative capacity. Improvement plan proposals would be subjected to evaluation from the corresponding Committee of Peers (CP), which would be convened ad hoc and submit its reports for consideration of the CTF. Considering the substantive quality of the improvement plan, its adjustment to the pre-established criteria of the call for proposals, and the assessment by the corresponding CP, the CTF would decide if the proposal is approved or rejected. The pre-established criteria would 39 importantly include two aspects, among others: (i) value added of the improvement plan (in terms of its impact on quality improvement) and; (ii) regional and social impact. (ii) Second phase. Preparation and approval of final improvement plan. Once a improvement plan proposal has been approved by the CTF, the HEI, with the technical support from STF, would prepare the final improvement plan, including an IPP if required. The final (or operational) improvement plan would detail, besides all items included in the improvement plan proposal, the final terms of reference and technical specifications corresponding to the proposed activities. In the 45 days following the approval of a improvement plan by the CTF, the STF would analyze the final improvement plan and propose the necessary changes if necessary, in order to agree with the HEI upon the implementation of the improvement plan. To that end, the STF and the HEI would sign a covenant or program contract that would detail all the agreements reached. (iii)Third phase. Implementation. Once final improvement plans are approved, implementation responsibilities would be shared between the corresponding HEI and the STF in the following manner: 1. HEI responsibilities. First of all, HEIs would have to establish an Implementation Team for each FEC improvement plan. Implementation Teams would include at least with one responsible person who would act as technical liaison with the FEC. The responsibilities of each HEI, through its Implementation Team, would be: (i) to carry out the technical implementation of improvement plans, i.e. basically monitoring the implementation of activities (consultants, interns, visiting professors, etc.); and (ii) to participate in the management of a number of processes, including the selection of individual consultants, of visiting professors, and of interns. The acquisition of goods and/or firm consultancies would be carried out exclusively by the UCP. 2. UCP responsibilities. The UCP would be responsible for fiduciary and procurement management. Regarding individual consultants, interns and visiting professors it would be responsible for verifying the selection processes and for administering contracts, including payments. As for firm consultancies and/or the acquisition of equipment, procurement would be carried out exclusively by the UCP. (iv) Fourth phase. Follow up, termination, and evaluation of improvement plans. The STF would carry out the academic and operational follow up of improvement plans and would report periodically and systematically to the CTF about their development. Moreover, the STF would be responsible for producing an assessment report about the achievements / results of each improvement plan. 40 8. Both the current IDF grant (TF096586, “Strengthening the National System for Evaluation and Quality Assurance of Education�) and the proposed Project address the capacity risk identified within the responsible agency. While significant improvements have already been achieved, the Project’s Component 1 would implement capacity-building activities including training, technical assistance and equipment for the instances described above. Financial Management, Disbursements and Procurement Financial Management 9. A financial management assessment was performed to determine the adequacy of COSUSINEACE’s financial management arrangements to support Project implementation. 10. COSUSINEACE has not become a completely operational entity yet, although the necessary steps to become fully operational have been identified and an action plan designed. Additionally, although the Project’s design does not include transfer of funds to HEIs for implementation of the FEC improvement plans, it requires coordination among different actors and HEIs as well as management of a large number of small transactions under Subcomponent 3.2. Those features represent some complexities and call for sound operational arrangements, including financial management. The design of these arrangements is almost finalized, but they are not in place yet. Completion of the design of financial management arrangements still requires some additional work by COSUSINEACE and close follow-up from the Bank side. On such basis, the FM risk is considered substantial. 11. In order to complete effective implementation and later operation of FM arrangements to adequately support Project execution, the following aspects need to be addressed, as per the Loan Agreement: (i) complete the actions required for the UCP to become a fully operational unit with its subsequent budgetary allocation in the 2013 National Budget; (ii) design and implement a Project information system to allow the recording, controlling, reporting and monitoring of Project transactions, including FEC improvement plans and issuance of required financial reports, the readiness of which is a condition of disbursement for Component 3; and (iii) recruitment of FM key staff not later than three months from effectiveness. Subject to the successful implementation of these actions, the proposed financial management arrangements – as designed – can be considered acceptable to the Bank. Summary of Financial Management Arrangements 12. As it relates to financial management, so far, it has been agreed that the UCP would include: (a) a Management and Finance Unit led by an Administrative and Financial Management Coordinator with the support of three areas: i) procurement, ii) accounting; and iii) treasury; and (b) a Planning, Budget, Monitoring and Evaluation Unit led by a Coordinator, and composed of a Program and Budgeting Specialist and a Monitoring and Evaluation Specialist. Staff needs and profiles have been defined taking into consideration the processes and procedures, functions and responsibilities to be assigned to different levels, size and volume of operations, and dynamics with other units. 41 13. Programming and Budget. The preparation of the annual program and budget would follow local regulations established by MEF through the Direccion General de Presupuesto Público. However, those procedures have been complemented by specific procedures in the Operational Manual. Based on the Project’s design, the UCP would be in charge of preparing the Project annual program and budget, in coordination with the FEC for Component 3. In order to allow adequate budget control, there is need to provide for: (i) timely preparation of programming, budget and procurement plan; (ii) proper recording of the approved budget in the respective information systems (following Government required programmatic structure and a classification by Project component/cost category; and FEC improvement plans); and (iii) timely recording of commitments, payments and accruals to allow an adequate budget monitoring, provide accurate information on Project commitments for programming purposes. 14. Accounting – Information system. COSUSINEACE has to comply with Peru’s laws governing budget and financial management, including the use of the GoP’s Financial Management Integrated System (SIAF, Sistema Integrado de Administración Financiera) and the General Chart of Accounts established in SIAF. The Project would benefit from the use of SIAF for budget execution, accounting and payment processing. However, SIAF only allows the recording of Project expenditures in general budget lines (objeto de gasto –genérica y específica de gasto), but it does not allow for distinguishing expenditures by improvement plan. Therefore, SIAF would be complemented with a management information system for the recording, controlling, reporting, and monitoring of Project expenditures by Project component/cost category and by improvement plan for Component 3 for further issuance of financial reports and statements of expenditures. The completion of the development of the mentioned information system has been considered a condition of disbursement for Component 3. 15. Processes and procedures (including internal controls). Overall, COSUSINEACE has to comply with local requirements related to financial management, including internal controls and internal procedures. In addition, COSUSINEACE has designed specific and detailed processes and procedures, mainly as it relates to the implementation of Component 3, including internal controls, adequate segregation of duties in terms of authorization, recording and approval of payments, as well as for monitoring purposes. However, the adequacy of those processes and procedures, and availability of reliable information for monitoring purposes, still needs to be reviewed together with the operation of the information system. Processes and procedures designed for the Project have been set up in the financial management chapter of the Operational Manual. 16. Although HEIs would not manage Project funds, they would monitor improvement plan activities and initial approval of services/goods received for payment approvals. Thus, for this purpose, a document with guidelines for HEIs (Manual del Fondo) has been designed and included in the Project’s Operational Manual. 17. Financial reporting. Taking into account the considerations made in the Accounting- Information System section, the interim financial reports (IFRs) would be directly issued from the management information system. Those interim financial reports would specify sources and uses of funds, reconciling items (as needed) and cash balances, with expenditures classified by Project component/subcomponent; and a statement of investments, reporting the current semester 42 and the accumulated operations against ongoing plans and footnotes explaining the important variances. The reports would include loan proceeds, and local funds (RO or “recursos ordinarios�). Those reports would be prepared and submitted to the Bank on a semi-annual basis no later than 45 days after the end of each calendar semester. The reports would be prepared in local currency and US dollars. The IFRs would also include a monitoring report of Component 3, by improvement plan, which would be issued by the financial information system. 18. On an annual basis, the UCP would also prepare Project financial statements including cumulative figures, for the year and as of the end of the year. Those financial statements would include explanatory notes in accordance with International Public Sector Accounting Standards. Those financial statements, duly audited in accordance with the Bank’s requirements, would be submitted to the Bank within the six months after the end of the Government’s fiscal year (December 31). 19. Audit. Annual audit reports on Project financial statements, including management letter, would be submitted to the Bank within six months of the end of the Borrower’s fiscal year (December 31). The audit would be conducted by an independent private audit firm acceptable to the Bank and under terms of reference approved by the Bank. Audit cost can be financed out of loan proceeds and appointment would follow regulations established by the CGR. The scope of the audit would be defined by the UCP in agreement with the Bank. Audit requirements would include the following: Table 6: Audit reports Audit type Due date Project financial statements June 30 Special Opinions - SOE June 30 FLOW OF FUNDS AND DISBURSEMENT ARRANGEMENTS 20. Following the general practice of the current portfolio, the following disbursement methods may be used: (a) reimbursement, (b) advance, and (c) direct payment. Under the advance method, a Designated Account (DA) in US dollars would be opened and maintained in the Banco de la Nacion by COSUSINEACE, who would have direct access to funds advanced by the Bank to the DA. Funds deposited into the DA as advances, would follow Bank’s disbursement policies and procedures, to be described in the Loan agreement and Disbursement Letter. 21. Funds deposited in the DA as advances would follow Bank’s disbursement policies and procedures, as described in the Disbursement Letter and Disbursement Guidelines. To process payments, COSUSINEACE would be able to withdraw the required amount to a local currency bank account (Operating bank account) from where payments to consultant and suppliers would be made. To process payments through bank transfers (cartas orden) to suppliers, consultants and service providers located out of Lima, a second Operating bank account would be opened and maintained at Banco de la Nacion. Specific arrangements for the operation of all Project 43 bank accounts would be reflected in the Operational Manual, and balances would be clearly reflected in the DA reconciliation. 22. The ceiling for advances to be made into the DA would be US$2,000,000. The minimum value for applications for direct payments and reimbursements would be US$500,000. Supporting documentation for documenting Project expenditures under advances and reimbursement methods would be specified in the Disbursement Letter. 23. A customized SOE would be used for Component 3. Format and content of customized SOEs would be fully based on the information provided by the information system. All supporting documentation would be maintained for post-review and audit purposes. 24. Allocation of loan proceeds. Loan proceeds would be disbursed against the following expenditures categories: Table 7: Allocation of loan proceeds Category Amount of the Loan Percentage of Expenditures Allocated to be financed (expressed in US$) (1) Goods, Minor Works, non- consulting services, consultants’ services, Training 3,796,891 100% of Eligible Expenditures and Operational Costs for Component 1 of the Project (2) Goods, Minor Works, non- consulting services, consultants’ services, Training 2,485,572 100% of Eligible Expenditures and Operational Costs for Component 2 of the Project (3) Goods, Minor Works, non- consulting services, consultants’ services, Training 18,717,537 100% of Eligible Expenditures and Operational Costs for Component 3 of the Project TOTAL AMOUNT 25,000,000 25. Supervision strategy. On a preliminary basis, the FM team plans to perform at least two supervision missions per year, and review the annual audit reports and the IFRs. Procurement 26. Procurement for the proposed Project would be carried out in accordance with the Bank’s “Guidelines Procurement of Goods, Works and Non-Consulting Services under IBRD Loans and IDA Credits & Grants by World Bank Borrowers�, dated January 2011; the Bank’s “Guidelines Selection and Employment of Consultants under IBRD Loans and IDA Credits & Grants by World Bank Borrowers�, dated January 2011; and the provisions stipulated in the Loan 44 Agreement. For each contract to be financed by the Loan, the different procurement methods or consultant selection methods, the need for pre-qualification, estimated costs, prior review requirements, and time frame are agreed between the Borrower and the Bank in the Procurement Plan. Goods and non-consulting services (including contracts under improvement plans) shall be procured under contracts awarded on the basis of International Competitive Bidding, National Competitive Bidding, Shopping and Direct Contracting. Consultants’ services (including contracts under improvement plans) shall be procured under contracts awarded on the basis of Quality and Cost-based Selection, Quality-Based Selection, Selection under a Fixed Budget, Least Cost Selection, Selection Based on the Consultants’ Qualifications, Single Source Selection and, Procedures set forth in Section V of the Consultant Guidelines for the Selection of Individual Consultants, including single-source Selection for Individual Consultants. A. Assessment of the agency’s capacity to implement procurement 27. Country. The national procurement system of Peru has made significant progress in the last few years, particularly with regard to access to information and adoption of standard bidding documents. However, some serious challenges remain which affect the efficiency of the procurement system. 28. Agency. In accordance with the implementation arrangements, the UCP would be responsible for the general management of Components 1 and 2, as well as fiduciary management of all three Components (i.e. financial management and management of procurement and contracting). The UCP would have to be created by the DGPP and would work as an ad hoc, autonomous unit, with no administrative link to COSUNIEACE’s General Office of Administration (OGA). An assessment of the implementation agency’s capacity to implement procurement actions for the Project was conducted. The capacity assessment looked into COSUSINEACE’s: (a) organizational structure, (b) facilities and support capacity, (c) qualifications and experience of the staff that are working in procurement, (d) record-keeping and filing systems, (e) procurement planning and monitoring/control systems used, and (f) capacity to meet the Bank’s procurement contract reporting requirements. The UCP would count on three specialized, internal areas: Project Executive Directorate (DEP), Management and Finance Unit, and Planning, Budget, Monitoring and Evaluation Unit. The Procurement area, included in the Management and Finance Unit, would be composed of a Director who would be assisted by a number of specialists to be determined on the basis of the number and frequency of transactions. Based on the information available at the time of the procurement capacity assessment, the Bank has assessed the risk as substantial. Procurement procedures are established in the Operational Manual. B. Procurement Special Provisions. 29. In addition and without limitation or restriction to any other provisions set forth in this Section or the Procurement Guidelines, the following provisions shall govern the procurement of goods, and works with National Competitive Bidding procedures: (a) Foreign Bidders shall not be required to be locally registered as a condition of participation in the selection process. 45 (b) No reference value shall be required for publication in the bidding documents or used for the purpose of evaluation. (c) Award of contracts shall be based exclusively on price and, whenever appropriate, shall take into account factors that can be quantified objectively, and the procedure for such quantification shall be disclosed in the invitation to bid. (d) Foreign Bidders shall be allowed to participate in without restrictions and shall not be subject to any unjustified requirement which will affect their ability to bid such as, but not limited to, the requirement to authenticate their bidding documents or any documentation related to such bidding documents with either Peruvian Consulates, the Ministry of Foreign Affairs, or any Peruvian authorities as a prerequisite of bidding. (e) The Borrower, through the UCP, shall use standard bidding documents and standard evaluation formats, all satisfactory to the Bank. 30. In addition and without limitation or restriction to any provision set forth in this Section or the Consultant Guidelines, the following provisions shall govern all employment of consultants referred to in the Loan Agreement: (a) Foreign consultants shall not be required to be locally registered as a condition of participation in the selection process. (b) Foreign consultants shall not be required to authenticate any documentation related to their participation in the selection process with either Peruvian Consulates, the Ministry of Foreign Affairs, or any Peruvian authorities as a condition of participation in said selection process. (c) The Borrower shall use standard requests for proposals and standard evaluation formats, all satisfactory to the Bank. (d) Foreign consultants, either individuals or firms, shall not be required to pay fees to the Peruvian Consultants’ Association that are different from those required for Peruvian consultants. (e) No consultant hired for the Project, at the time he or she is carrying out his or her contractual obligations as a consultant, may hold civil service office or any other position in any agency of the Borrower, nor shall such consultant have any right to re-entry into any such office or position upon the conclusion of his or her consulting services. 31. Annual Procurement External Reviews. The Borrower, through the UCP, shall (i) have all the procurement records and documentation for each fiscal year of the Project reviewed, in accordance with appropriate procurement review principles, by independent reviewers, contracted by the UPE following the procedures of the Bank’s Consultants Guidelines, acceptable to the Bank; (ii) furnish to the Bank as soon as available, but in any case not later than six months after the end of each such fiscal year, the procurement external review report of such review by said reviewers of such scope and in such detail as the Bank shall reasonably request; and (iii) furnish to the Bank such other information concerning said procurement records and documentation and the procurement review thereof as the Bank shall from time to time reasonably request. 46 32. Twenty percent of all contracts would by post-reviewed by the Bank. Based on the finding of the Post Procurement Reviews and the proposed ratings, the Bank may determine the revision of the prior review requirements. C. Procurement Plan General 33. The Borrower, through the UCP, has prepared a simplified Preliminary Procurement Plan (October 5, 2012) based on an indicative list of eligible activities to be implemented. The Procurement Plan would be updated semi-annually or as required to reflect the actual Project implementation needs and improvements in institutional capacity. The Procurement Plan would be available at the Procurement Plans Execution System (SEPA). Procurement of Works: No major civil works are expected as part of the Project. Goods and non-consulting services 34. Prior Review Threshold: Procurement Decisions subject to Prior Review by the Bank as stated in Appendix 1 to the Guidelines for Procurement: Table 8: Thresholds for procurement methods and prior review (thousands of US$) Contract Value Contracts Expenditure (Thresholds) US$ Procurement Method Subject to Category thousands Prior Review 1. Works <50 Shopping First 2. Goods >250 ICB >350 50 - 250 NCB First <50 Shopping First Regardless of value DC All Note: ICB = International Competitive Bidding NCB = National Competitive Bidding DC = Direct Contracting Reference to the Project’s Operational Manual 35. The Borrower, as part of Project’s preparation, has prepared an Operational Manual which provides detailed procurement information for the Project’s implementation, including: (i) the particular methods for the procurement of goods, non consulting services and consultants, (ii) a clear definition of responsibilities that would apply to each kind of procurement, (iii) filing procedures, (iv) management of the Procurement Plan and, (v) templates for each procurement method. 47 I. Selection of Consultants 36. Prior Review Threshold: Selection decisions subject to Prior Review by the Bank as stated in Appendix 1 to the Guidelines Selection and Employment of Consultants: Table 9: Thresholds for methods and prior review (thousands of US$) Contract Value Consulting Contracts Subject to (Thresholds) US$ Procurement Method Services Prior Review thousands >200; >100 QCBS, QBS, FBS, LCS <200 Terms of 3.a Firms Reference only QCBS, QBS, FBS, LCS, Terms of Reference <100 CQS only Regardless of value SSS All Comparison of 3 CVs in >100; accordance with Chapter <100 Terms of 3.b Individuals V of the Guidelines Reference only Note: QCBS = Quality- and Cost-Based Selection QBS = Quality-Based Selection FBS = Fixed Budget Selection LCS = Least-Cost Selection CQS = Selection Based on Consultants' Qualifications SSS: Single Source Selection 37. Short list comprising entirely of national consultants: Short list of consultants for services, estimated to cost less than US$350,000 equivalent per contract, may comprise entirely of national consultants in accordance with the provisions of paragraph 2.7 of the Consultant Guidelines. 38. In addition to the prior review supervision to be carried out from Bank offices, the capacity assessment of the UCP has supervision missions to visit the field to carry out post review of procurement actions. The size of the sample for post-review would be not less than 1 in 10 contracts. Environmental and Social (including safeguards) 39. The overall social impact of the Project is expected to be positive. Peruvian stakeholders and society at large seem to share a common understanding and interest regarding the need to foster quality assurance in higher education. Moreover, the Project concentrates a good proportion of its funds (at least 50 percent of the resources of Component 3) for the benefit of non-top HEIs. In this sense, it is expected that this investment would greatly contribute to improved equity among all HEIs. Since the student body of these HEIs is generally from the lower end of the national income distribution, the Project would directly benefit those from lower socioeconomic status. 48 40. The Project is not expected to have any major environmental impact, as it would not finance any civil works involving new construction. The improvement plans for the various HEIs are expected to include some routine maintenance and refurbishment of existing infrastructure that will be subject to local legislation concerning worker health and safety and disposal of materials such as paint. For these reasons, this Project is a Category C as per OP/BP 4.01 Environmental Assessment. 41. OP/BP 4.10 Indigenous Peoples is triggered as the Project has a nationwide scope and would affect HEIs that have indigenous students. Linguistic, cultural, ethnic and socioeconomic barriers have historically undermined indigenous peoples’ access to and success in relation to higher education. The Indigenous Peoples Planning Framework (IPPF) that has been developed for the Project provides important guidance and a menu of possible activities for HEIs implementing a PMI where more than 5 percent of the student body is indigenous as a way to address these barriers and improve both the quality as well as effectiveness of higher education for indigenous students. 42. As per OP/BP 4.10 Indigenous Peoples, an IPPF has been prepared and has been consulted with a representative group of stakeholders, including indigenous student unions, individual students, national indigenous organizations and associations, and academic inclusion experts. The final version of the IPPF was disclosed on the Bank’s and CONEAU’s websites on September 21 and on September 12, 2012, respectively. A draft had been posted on both websites on August 30, 2012, prior to Appraisal. The IPPF includes the following: (a) a summary of the legal and institutional framework regarding indigenous peoples and education in Peru; (b) a summary of the relevant demographic, social, cultural, and political characteristics of indigenous peoples in Peru; (c) an analysis of the current barriers to access or gaps in information that impact participation in the labor market; d) the results of the consultation with indigenous students and representatives mentioned above; and e) a matrix of recommendations based on the consultations and the social assessment that are intended to improve access and quality of higher education for indigenous peoples in Peru (see Table 10). 43. For those HEIs that have more than 5 percent of indigenous peoples within their student body and that are selected for the implementation of a PMI, an Indigenous Peoples Plan (IPP) shall be prepared and appropriately consulted based on the guidance contained in the IPPF. In terms of consultation, the IPPs must undergo a process of free, prior and informed consultations with: (i) the indigenous student body at the institution, (ii) representatives of indigenous student unions related to the institution, and/or (iii) representatives of the local indigenous community where appropriate. 44. IPPs developed by those HEIs above the threshold that implement a PMI would be required to: (i) include a social assessment identifying the potentially affected indigenous peoples; (ii) the legal/regulatory framework in relation to indigenous peoples and the efforts undertaken by the HEI in the past in favor of indigenous peoples; (iii) all agreements reached in the framework of previous, free, and informed consultations; (iv) the mitigation mechanism(s) that the HEI chose in favor of indigenous students; and (v) the monitoring mechanism(s). 49 Table 10: Matrix of recommendations for IPPs Theme Social assessment Consultation Intervention Monitoring Assess the existence of potential Monitor (i) the differences between indigenous Offer remediation clases participation of and non-indigenous first-year and tutoring in issues of indigenous students in students. Some examples of self-esteem and motivation the remediation classes Improve- áreas that require special for indigenous students in and/or activities for ment of attention are: (i) Spanish those areas where the cultural integration; and 1 higher language skills, (ii) information evaluation has identified (ii) the academic education and communication technology significant differences, progress of those quality skills, (iii) training and and/or other activities that indigenous students consulting in research, (iv) promote the cultural who participate, as academic remediation programs, integration of indigenous opposed to those who and (v) advising on issues of and other students. Develop a do not. self-esteem and motivation. free, prior and informed Offer activities that foster consultation of the insertion of current the draft indigenous students into the Indigenous labor market, once they Peoples Plan graduate. Examples of Assess graduates’ employability with (i) the Monitor the Improve- activities along these lines and insertion into the labor institution’s employability and ment of are (i) a program of market (salary, relationship indigenous salaries of the the professional tutoring between degree program studied students; (ii) institution’s graduates, 2 insertion specific to indigenous and professional sector) for representative disaggregating those into the students; (ii) a program for indigenous students who s of who participated in the labor special support to graduated from the institution indigenous activities and those who market indigenous students to find during recent years. student did not. internships, or (iii) organizations; agreements with and/or (iii) organizations specialized in representative insertion into the labor s of the local market. indigenous community. Monitor the success of including intercultural Inclusion matters in the curricula of Inclusion of issues relevant Assess the incidence of of certain degree intercultu to Peru’s interculturality interculturality issues in programs (for instance, -rality into the curricula of one or curricula of one or several number of class hours 3 issues several degree programs in degree programs in the areas of dedicated to these into the areas of Education, Education, Health Sciences matters, number of higher Health Sciences, and/or and/or Social Sciences. bibliography education Social Sciences. references, or number curricula of pedagogic materials generated). 50 45. IPPs would be produced parallel to the final PMI. Once these documents are ready, the highest authority in the HEI would send them to the CDP for their review. Once approved by the CDP, the IPP would be sent to the Bank for its No Objection and, unless objected, its disclosure at Info Shop. The IPP would also be disclosed on the website of the corresponding HEI and of the Project. The approval and disclosure process of the IPP would be finalized before the agreement between the FEC and the HEI is signed. 46. HEIs would be fully responsible for the implementation and monitoring of the IPP. HEIs would report on the implementation of the IPP together with their general PMI progress reports. Monitoring & Evaluation 47. The Project Executive Directorate would have overall responsibility for tracking progress towards the Project outcomes. However, the CDP would count on technical support from the UCP and STF, especially regarding data collection and processing. To this end, the UCP would have a Planning, Budget, Monitoring and Evaluation Unit, which would assist the CDP in the follow up of Project activities and in the regular monitoring of indicators. 48. Advancement toward the PDO and intermediate result indicators would be measured for all three components at least annually and would be publicly disclosed. Data would be obtained from the administrative files of the UCP and STF, which would keep track of improvement plans and disbursements. In the case of FEC improvement plans, HEIs would be responsible for a mid- term and a final Results Report that would be done on the basis of the indicators agreed upon in their PMI or PMC. The STF would consolidate this data and send it to the UCP’s Planning, Budget, Monitoring and Evaluation Unit. Furthermore, when a qualitative assessment and/or surveys are necessary (e.g. for evaluating the quality of the higher education information system), consultancies would be contracted. 49. The Bank would work closely with the CDP to evaluate Project implementation progress during regular implementation support missions. 50. The External Advisory Council (CAE, Comité Asesor Externo), composed of an international group of six recognized authorities in their respective fields of scientific, professional, and management activity, would perform an annual evaluation of the Project, including one at mid-term and one at the end. Special attention would be paid to the FEC and its improvement plans. CAE would also assess the quality of a sample of self and external evaluation through the selection and assessment of a sample. Composed of both Peruvians and foreigners, the CAE would report directly to the CDP. 51 Annex 4: Operational Risk Assessment Framework (ORAF) PERU: Higher Education Quality Improvement Project Stage: Board Project Stakeholder Risks Rating Moderate Description: Risk Management: There is a risk that the Government does not maintain its support The Project received the final approval (Viabilidad) from the Ministry of Finance’s evaluation for quality assurance and therefore of SINEACE’s objectives. system, SNIP, and the sectorial entity at the Ministry of Education (OPI) after a long, in-depth joint evaluation. Resp: Government, Stage: Preparation Due Date: April 1, Status: In Bank and Implementation 2018 progress Implementing Agency Risks (including fiduciary) Capacity Rating: Substantial Description: Risk Management: COSUSINEACE has insufficient capacity to design and A comprehensive capacity assessment of COSUSINEACE was recently conducted, leading to a implement the Project, as evidenced by a recent capacity detailed report with in-depth analysis and recommendations. COSUSINEACE directors stand fully assessment. The weaknesses identified can be summarized as behind these conclusions and are ready to address the weaknesses with the support of the GoP and follows: (i) despite important progress, the scope of tasks ahead the Bank. is large and requires new processes, prospective thinking and significant management skills; (ii) key elements of the quality Five major mitigation measures have been put into place: assurance scheme outlined by the law are missing; (iii) COSUSINEACE’s systems for monitoring and coordinating i. A Project Coordination Unit (UCP) would be established within COSUSINEACE, activities need improvement; (iv) COSUSINEACE lacks a including the required fiduciary staff. The UCP would include two units: (i) a Management modern and appropriate management information system; (v) a and Finance Unit with the following areas: Procurement, Accounting, and Treasury; and (ii) a broader program for public communication is required; and (vi) Planning, Budget, Monitoring and Evaluation Unit. more training and support is needed at all levels–for the professional staff of COSUSINEACE as well as for HEIs. ii. The Project’s Component 1 would implement capacity-building activities including training, technical assistance and equipment for COSUSINEACE. iii. The UCP would be staffed with qualified and experienced staff, not only as it relates to Bank requirements, but also as it relates to local ones. Staff needs in terms of number and qualifications, and their financing, have been defined. iv. Based on FM capacity assessment, specific processes and procedures for the operation of the UCP and FEC have been designed –including clear roles and responsibilities, internal controls, the design and implementation of a management information system, and 52 recruitment of FM staff (with terms of reference approved by the Bank) no later than 3 months after effectiveness. v. The Operational Manual includes clear FM and Procurement processes and procedures. Resp: Stage: Preparation, Due Date: April 1, Status: In COSUSINEACE, implementation 2018 progress UCM, Bank Governance Rating: Low Description: Risk Management: The relationship between the Ministry of Education and The respective responsibilities of the Ministry of Education and COSUSINEACE are clearly COSUSINEACE is a new one and carries the risk of confusion spelled out in Peru’s legal framework. COSUSINEACE would be in charge of Project over roles and responsibilities. implementation, including fiduciary management. The specific roles of each agency are being clearly defined in the PAD, the Operational Manual and the Loan Agreement. Stage: Preparation Due Date: April 1, Status: In Resp: Government and implementation 2018 progress Project Risks Design Rating: Moderate Description: Risk Management: HEIs, in particular non-university institutions, may find it The Project design incorporates lessons from other countries on the need to keep it simple and challenging to prepare proposals to access the FEC due to the produce a clear Operational Manual for HEIs. The Project design also factors in the need to finance lack of experience in preparing improvement plans. training and other institutional strengthening measures for HEIs, as well as the need for increased implementation support during the first year of the Project. Experience with similar competitive funds in other countries in Latin America and the Caribbean shows that HEIs rapidly adapted and built capacity. Resp: HEIs, Stage: Preparation, Due Date: April 1, Status: In COSUSINEACE, Implementation 2018 progress Bank Social & Environmental Rating: Low Description: Risk Management: There may be some resistance to address the specific needs of The GoP, and in particular COSUSINEACE, is sensitive to the need to provide specific measures indigenous communities, given their minority status. that address issues related to minority indigenous populations. An Indigenous Peoples Planning Framework (IPPF) for the Project has been developed and consulted. The IPPF includes a preliminary assessment of the current situation and includes incentives for fostering the Project’s positive impacts on indigenous peoples and maximizing its positive ones on them. In addition, the Project would finance an in-depth socio-cultural assessment. Since the Project is financing minor civil works there may be a The Project will not be financing any new construction or civil works. The Improvement Plans for negative impact on the environment. the various institutions are expected to include some routine maintenance and refurbishment of 53 existing infrastructure that will be subject to local legislation concerning worker health and safety and disposal of materials such as paint, a summary of which is included in the Operational Manual. Resp: Government, Stage: Preparation, Due Date: April 1, Status: Ongoing Bank implementation 2018 Program & Donor Rating: N/A Description: Risk Management: N/A N/A Resp: N/A Stage: N/A Due Date: N/A Status: N/A Delivery Monitoring & Sustainability Rating: Low Description: Risk Management: There may be difficulty in influencing quality assurance in the This risk is mitigated by making participation external evaluation an eligibility criterion for access university sector through a voluntary accreditation process. to FEC financing. In addition, the Project would finance measures to support COSUSINEACE to pro-actively mentor university staff to promote a culture of quality improvement. The SIAC would extract data from the information provided by HEIs for the self-evaluation, There may be lack of sufficient data about HEIs and programs to external evaluation, and accreditation process. Moreover, HEIs that participate in the FEC would build and sustain a higher education quality assurance be required to send semiannual Results Reports to the STF. information system. Resp: Government, Due Date: April 1, Status: Not yet Stage: Implementation COSUSINEACE 2018 started Implementation Risk Rating: Moderate The overall risk rating of the Project is considered to be moderate. Nevertheless, capacity risk is significant, and a number of measures are being taken to mitigate it. 54 Annex 5: Implementation Support Plan PERU: Higher Education Quality Improvement Project Strategy and Approach for Implementation Support 1. The strategy for implementation support has been developed based on the nature of the Project and its risk profile. It aims at making implementation support more flexible and efficient, and focuses on implementation of the risk mitigation measures defined in the ORAF, especially with regards to capacity risk (which is rated as significant), as well as the traditional supervision focus areas including safeguards and fiduciary aspects. Implementation Support Plan 2. Formal supervision and field visits would be carried out semi-annually, and would focus on: (a) Technical inputs. An international expert on higher education, specialized in evaluation and accreditation, would be a key member of the Bank’s team. Their support would focus on following up activities under Component 1, giving advice on evaluation and accreditation processes based on best international practices. In particular, a close supervision and recommendations would be given regarding standards, norms and procedures as well as capacity building at HEIs. (b) Fiduciary requirements and inputs. Training would be provided by the Bank’s financial management specialist and the procurement specialist before the commencement of Project implementation. The Bank would support the UCP in its FM capacity and to improve procurement management efficiency. The FM specialist and the procurement specialist would both be based in the country office to provide timely support. Supervision of FM arrangements would be carried out semi-annually as part of the Project supervision plan and support would be provided on a timely basis to respond to client needs. Procurement supervision would be carried out on a timely basis as required by the client. (c) Safeguards. The social development specialist would ensure that training is provided to relevant counterpart staff. On the social side, supervision would focus on the implementation of the IPPF. Field visits would be made on a semi-annual basis. (d) Client Relations. The Task Team Leader would coordinate the Bank team to ensure Project implementation is consistent with Bank requirements, as specified in the legal documents. They would meet with senior officials on a regular basis to keep them apprised of Project progress and issues requiring resolution at their level. 55 Table 11: Main focus in terms of support to implementation Resource Partner Time Focus Skills Needed Estimate Role Financial Management Supervision and training Specialist; in fiduciary matters First 12 Procurement Specialist US$96,000 months Monitoring of Project Education Economist; implementation and Education Specialist results Monitoring of Financial Management compliance with Specialist; N/A fiduciary guidelines Procurement Specialist Support to preparation 12-60 Social Development of Indigenous Peoples US$96,000/year months Specialist Plan and Supervision Monitoring of Project Education Economist; implementation and Education Specialist results Table 12: Staff skills mix required Skills Needed Number of Staff Weeks Number of Comments Trips Higher Education Specialist 6 SWs first year, then 4 Three first year SWs annually in the then two each following years year. Procurement 4 SWs annually Fields trips as Country required. office-based Social Development 4 SWs annually Fields trips as Specialist required. Financial Management 2 SWs annually Fields trips as Country Specialist required. office-based Task Team Leader 10 SWs annually (15 SWs Three first year the first year) then two each year 56 Annex 6: Economic Analysis PERU: Higher Education Quality Improvement Project 1. The economic analysis has been done within a 20 year time horizon, since the benefits of accreditation can be only measured on a long terms basis, when it take place the effects on the labor market of successive cohorts of graduates from accredited programs. Calculating economic benefits involves looking at four elements,: (i) assuming the positive impact of quality improvements on wages; (ii) assessing the magnitude of these quality improvements on average wages increases and its dispersion, (iii) taking into consideration the dynamics of these effects along the economic analysis’ time horizon of 20 years, and (iv) considering the beneficiaries as those graduates from accredited programs. (i) Regarding the impact of quality improvements, it is assumed that they would translate into a raise of wages for graduates of those HEIs that have obtained the accreditation. This is based on the premise that a competitive labor market exists, where differences in productivity are reflected in wage differentials. (ii) As for the magnitude of these quality improvements, it is assumed that wages dispersion for graduates of a given program is mostly due to the heterogeneity in higher education, so that accreditation would reduce this dispersion and increase the average quality. (iii) The dynamics of the effects along the 20 years horizon are closely linked to the progress in the percentage of HEIs that achieve accreditation. This is determined by the speed in the Project’s implementation and the effectiveness of the incentives for HEIs to finish the accreditation process. (iv) The Project’s beneficiaries are defined as those graduates from a program in a HEI that has been accredited. Conservatively, it is assumed that only the cohort starting a program once the HEI has been accredited and those starting in the following years are beneficiaries. In order to quantify the benefits they receive, it is assumed that those who, in a non-Project scenario, would have been in income quintiles 1 to 4, receive a wage premium equivalent to stepping up one quintile 21. 2. The goal is then to estimate the long term Project’s effect in reducing Peru’s wage dispersion in the lifelong wage profile of higher education graduates, as expressed in Figure 3. The flow of beneficiaries would be determined by the percentage of accredited institutions. The wage premium for each beneficiary would depend on (i) the income quintile where they would have been if the Project were not implemented, and (ii) on the number of years of work experience. 3. The Chilean economy is taken as the reference, since it is a regional leader in higher education quality (with about 90 percent of its enrolment focusing in accredited HEIs). For 21 Conservatively, the model assumes that those in the upper quintile do not receive any wage premium. 57 university programs, the rate of average wage differential between the first and the last decile, up to 5 years after graduation, is 3.9 in Chile but it goes up to 11.4 in Peru (for non-university programs it is 3.7 and 10.6, respectively). If Peru “removed� its lowest quintile of the income distribution, the rate of wage differentials would equal 3.4 and 3.1 for university and non- university programs, respectively. Figure 3: Effects of higher education quality improvement on the dispersion of wages as they relate to professional experience Ingreso laboral Dispersión salarial en economía de referencia Dispersión salarial en Perú Experiencia Relationship between income (vertical axis) and experience (horizontal axes): Peru would show a larger dispersion of income trajectories than the reference economy. 4. The programs considered for the economic analysis are those that are legally compelled to become accredited, plus engineering 22. Income profiles for graduates of these programs have been estimated using data about different cohorts from the National Household Survey, as shown in Figures 4 and 5. The population of potential beneficiaries is defined according to (i) the flow of entrants into higher education (represented in Figure 6) and how they distribute among programs, and (ii) the probability that they finish their studies. This is calculated using projections of the 17-18 year old population from INEI, projections of the percentage that goes into higher education, and the current distribution of higher education students among programs. This leads to an estimation that, in 20 years time, 30 percent of youth goes into university education and 15 percent into non-university higher education. The probability of these students finishing their studies is assumed to be constant and consistent with ANR data for the mid- 2000’s: 43.7 percent graduate from universities, and 56.3 percent graduate from IEESs. 22 Medicine, other health sciences, education, engineering and law at the university level; health sciences, education and technology at the non-university level. 58 Figure 4 – Income profiles for graduates of certain university programs Medicine 60,000 50,000 Q1 40,000 Q2 30,000 Q3 Q4 20,000 Q5 10,000 0 1 3 5 7 9 11 13 15 17 19 Obstetrics 35,000 30,000 25,000 Q1 20,000 Q2 15,000 Q3 Q4 10,000 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 Nursing 35,000 30,000 25,000 Q1 20,000 Q2 Q3 15,000 Q4 10,000 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 59 Odontology 45,000 40,000 35,000 30,000 Q1 25,000 Q2 20,000 Q3 15,000 Q4 Q5 10,000 5,000 0 1 3 5 7 9 11 13 15 17 19 Psychology 25,000 20,000 Q1 15,000 Q2 Q3 10,000 Q4 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 Other Health Sciences 40,000 35,000 30,000 Q1 25,000 Q2 20,000 Q3 15,000 Q4 10,000 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 60 Education 25,000 20,000 Q1 15,000 Q2 Q3 10,000 Q4 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 Engineering 80,000 70,000 60,000 Q1 50,000 Q2 40,000 Q3 30,000 Q4 20,000 Q5 10,000 0 1 3 5 7 9 11 13 15 17 19 Law 60,000 50,000 40,000 Q1 Q2 30,000 Q3 20,000 Q4 Q5 10,000 0 1 3 5 7 9 11 13 15 17 19 61 Figure 5 – Income profiles for graduates of certain non-university programs Technical Nursing 25,000 20,000 Q1 15,000 Q2 Q3 10,000 Q4 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 Technician in Clinic Laboratory 30,000 25,000 Q1 20,000 Q2 15,000 Q3 10,000 Q4 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 Other Health Sciences 30,000 25,000 20,000 Q1 Q2 15,000 Q3 10,000 Q4 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 62 Teachers Training 25,000 20,000 Q1 15,000 Q2 Q3 10,000 Q4 Q5 5,000 0 1 3 5 7 9 11 13 15 17 19 Technology 60,000 50,000 40,000 Q1 Q2 30,000 Q3 20,000 Q4 Q5 10,000 0 1 3 5 7 9 11 13 15 17 19 63 Figure 6 – Projected flow of entrants into higher education: 17-18-year-olds enrolled in higher education 600,000 500,000 400,000 University 300,000 Non University Total 200,000 100,000 0 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 5. According to the Project’s estimates of the speed of the accreditation process, it is assumed that the percentage of HEIs that reach accreditation would behave as shown in Table 13. This allows us to estimate the size and distribution of wage premiums over time. 64 Table 13: Percentage of HEIs projected to have completed the accreditation process, by discipline % at year 5 % at year 20 University education Education 100 100 Medicine 63 Obstetrics Nursing Odontology 36 90 Psycology Other health sciences Law 38 Engineering 58 70 Non-university education Technical nursing Clinical Lab Technician 39 90 Other health sciences Teachers training 30 Technology degrees 26 50 6. For the social assessment with private prices the social cash flow is calculated on the basis of wage premiums for the implementation of the Project as compared to its non- implementation, deducing investment and the costs of operation and maintenance at current prices. MEF’s discount rate of 10 percent is used. The costs of operation and maintenance are divided between direct costs (those related with COSUSINEACE’s operation) and indirect (those related with the additional budget in the education sector for financing operation and maintenance of HEIs’ improvements). This leads to an NPV of PEN3,091.7 million, and an IRR of 35.9 percent. 7. For the social assessment with social prices, it is assumed that private income equals social income to the extent that the former refer to income of qualified labor force who works within the country. However, the calculation of costs needs to be adjusted in order to estimate social prices. First, costs are divided in two components: consulting and goods and non- consulting services, which represent 30 percent and 70 percent of the Project’s investment respectively. Consulting services are assumed to be only national, where as goods and non- consulting services are assumed to be 75 percent national and 25 percent international. Different adjustments are done to each group in order to reach social prices, assuming a correction factor for the social exchange rate for foreign goods and services of 1.08, an income tax of 10 percent, a VAT of 18 percent, and a tariff of 5 percent. This leads to an NPV of PEN3,183.9 million, and an IRR of 38.2 percent. 8. Sensitivity analyses. High NPV and IRR values are robust to adjustments of -10 percent to +30 percent changes in (i) costs of operation and maintenance, (ii) investment flows, and (iii) 65 wage differentials with and without Project. In the most pessimistic scenario the NPV stays at PEN2,618.6 million (Figure 7) and the IRR stays at 32.1 percent (Figure 8). Figure 7: NPV sensitivity S/. 6,000 Millions S/. 5,500 S/. 5,000 S/. 4,500 S/. 4,000 S/. 3,500 S/. 3,000 -10% -5% 0% 5% 10% 15% 20% 25% 30% Operation and maintenance Investments Income Figure 8: IRR sensitivity 48.0% 47.0% 46.0% 45.0% 44.0% 43.0% 42.0% 41.0% 40.0% 39.0% 38.0% -10% -5% 0% 5% 10% 15% 20% 25% 30% Operation and maintenance Investments Income 9. On the basis of the economic analyses presented above and other technical considerations, SNIP declared the Project viable on July 23, 2012. 66