Document of The World Bank Report No: ICR2932 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-77080 IBRD-79680) ON LOANS IN THE AMOUNT OF US$1,503.76 MILLION AND US$1,250 MILLION OF ADDITIONAL FINANCING TO THE UNITED MEXICAN STATES FOR A SUPPORT TO OPORTUNIDADES PROJECT June 26, 2014 Human Development Sector Management Unit Colombia and Mexico Country Management Unit Latin America and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 4, 2014) Currency Unit = Mexican Peso MXN 12.9= US$1.00 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AGE School Management Support Program (Apoyo a la Gestión Escolar) BANSEFI National Savings and Financial Services Bank CCT Conditional Cash Transfer CN National Coordination (Coordinación Nacional) CONAFE National Council for Educational Development (Consejo Nacional de Fomento Educativo) CONEVAL National Council for Evaluation of Social Development Policy (Consejo Nacional de Evaluación de la Política de Desarrollo Social) COP Communities of Practice CPC Community Promotion Committees (Comites de Promoción Comunitaria) CPS Country Partnership Strategy DICONSA Primary Commodity Supplier (Distribuidora Conasupo, S.A.) ECD Early Childhood Development ENSANUT National Health and Nutrition Survey FM Financial Management GDP Gross Domestic Product IADB Inter-American Development Bank IBRD International Bank for Reconstruction for International Development IEG Independent Evaluation Group INALI National Institute of Indigenous Languages IPP Indigenous Peoples Plan MIC Middle Income Countries MIR Program’s Results Matrix (Matriz de Indicadores para Resultados) MIS Management Information System NAFIN National Financing Agency (Nacional Financiera) PAL Food Program (Programa de Apoyo Alimentario) PEC Quality Schools Program (Programa Escuelas de Calidad) PEI Initial Education Program (Programa Educación Inicial) PRONABES National Scholarship Program for Higher Education SEDESOL Social Development Ministry (Secretaría de Desarrollo Social) SEP Public Education Ministry (Secretaria de Educación Pública) SHCP Ministry of Finance and Public Credit (Secretaria de Hacienda y Crédito Público) Vice President: Jorge Familiar Country Director: Gloria Grandolini Sector Manager: Mansoora Rashid Project Team Leader: Theresa Jones ICR Team Leader: Francesca Lamanna MEXICO Support to Oportunidades Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes .......................................................................................... 10 4. Assessment of Risk to Development Outcome......................................................... 15 5. Assessment of Bank and Borrower Performance ..................................................... 15 6. Lessons Learned ....................................................................................................... 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 18 Annex 1. Project Costs and Financing .......................................................................... 19 Annex 2. Outputs by Component ................................................................................. 20 Annex 3. Economic and Financial Analysis ................................................................. 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34 Annex 5. Beneficiary Survey Results ........................................................................... 35 Annex 6. Stakeholder Workshop Report and Results................................................... 36 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 37 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 53 Annex 9. List of Supporting Documents ...................................................................... 54 MAP i A. Basic Information Support to Oportunidades Country: Mexico Project Name: Project IBRD-77080,IBRD- Project ID: P115067 L/C/TF Number(s): 79680 ICR Date: 06/26/2014 ICR Type: Core ICR Lending Instrument: SIL Borrower: MEXICO Original Total USD 1,503.76M Disbursed Amount: USD 2,753.76M Commitment: Revised Amount: USD 2,753.76M Environmental Category: C Implementing Agencies: SEDESOL/Oportunidades National Coordination Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 11/25/2008 Effectiveness: 09/01/2009 09/01/2009 07/13/2010 Appraisal: 02/23/2009 Restructuring(s): 11/22/2011 10/08/2012 Approval: 04/09/2009 Mid-term Review: Closing: 12/31/2010 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Highly Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision No None (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 100 Theme Code (as % of total Bank financing) Social safety nets 100 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Gloria M. Grandolini Axel van Trotsenburg Sector Manager: Mansoora Rashid Helena G. Ribe Project Team Leader: Theresa Jones Theresa Jones ICR Team Leader: Francesca Lamanna ICR Primary Author: Sara Francesca Giannozzi F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The proposed Project would support Mexico's conditional cash transfer (CCT) program, Oportunidades, which serves slightly more than 5 million poor families. About 70 percent of these families are in rural areas, 16 percent live in semi-urban areas, with the remaining 14 percent in urban areas. The principal Project Development Objectives are: -Increase capacities in health, nutrition and education of poor families through human capital investment by promoting regular health check-ups, improving health status, and raising school enrollment and attendance rates. -Build sustainable connections between Oportunidades and other social programs of the Government of Mexico in order to improve health and education outcomes for Program participants. ii Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years % of families complying with health conditions [by age (w. breakdown for pregnant Indicator 1 : women and children <5 and for indigenous and non-indigenous)] 98.0% 96% Value 98.0% Children <5: 99.1% (breakdown no quantitative or Children <5: 98.9% 96.39% Pregnant women: longer Qualitative) Pregnant women: 98.99% 99% available) Date achieved 12/30/2006 12/31/2010 12/31/2013 12/31/2013 Comments Achieved. In 2011, baseline was revised to 96.8% for 2007 for consistency with (incl. % Program's MIS data for that year, as documented via email, but not revised formally. achievement) Targets revised in AF to include only families, detailed breakdown no longer available. Indicator 2 : % of children complying with education conditions by age Primary: 98% Primary: 98.4% Secondary: Primary: 99.50% Value Primary: 98.0% Secondary: 94.7% 98.4% Secondary: 99.49% quantitative or Secondary: 94.3% Upper Secondary: Upper Upper Secondary: Qualitative) Upper Secondary: 95.6% 97.3% Secondary: 99.59% 98.46% Date achieved 12/30/2005 12/31/2010 12/31/2013 12/31/2013 Comments Surpassed. Target revised in AF to account for Project extension. High rates of (incl. % compliance due to the fact that compliance is only measured for enrolled children for achievement) whom compliance is verified. % of students who transition from primary to lower secondary education Indicator 3 : **Defined as: Primary school Oportunidades students enrolled in next level/Oportunidades students enrolled in last grade of primary in previous cycle Value quantitative or 82.3% 85.36% 85.7% 90.06% Qualitative) Date achieved 12/30/2006 12/31/2010 12/31/2013 12/31/2013 Comments (incl. % Surpassed. Target revised to 85.7 % in AF to account for extension in closing date. achievement) % of students who transition from lower to upper secondary school Indicator 4 : **Defined as: Oportunidades students enrolled in upper secondary/Oportunidades students enrolled in last year of lower secondary in previous cycle Value quantitative or 55.82% 56.93% 57.2% 68.94% Qualitative) iii Date achieved 12/30/2006 12/31/2010 12/31/2013 12/31/2013 Comments (incl. % Surpassed. Target revised to 57.2% in AF to account for extension in closing date. achievement) Number (and %) of Oportunidades schools participating in the Programa Escuelas de Calidad (PEC) Indicator 5 : **Defined as: Number of PEC-Oportunidades schools/number of Oportunidades schools Value 24,955 30,824 40,000 28,944 quantitative or 23% 24% 22% 24.56% Qualitative) Date achieved 12/30/2006 12/31/2010 12/31/2013 12/31/2013 Achieved. Surpassing % target demonstrates improvement in targeting towards Comments Oportunidades beneficiaries. Thus the indicator is considered achieved even though (incl. % absolute number of schools falls short of target. Target revised in AF to account for achievement) extension. Number (and %) of Oportunidades beneficiaries with PRONABES scholarships Indicator 6 : **Defined as: Number of Oportunidades beneficiaries with PRONABES scholarships/Number of PRONABES scholarships Value 36,941 90,000 68,181 quantitative or 70,000 15.4% 20% 20.20 % Qualitative) Date achieved 12/30/2007 12/31/2010 12/31/2013 12/31/2013 Achieved. Surpassing % target demonstrates improvements in targeting of Comments Oportunidades beneficiaries. Thus, indicator is considered achieved even though (incl. % absolute number of beneficiaries falls short of target. Target revised in AF to account achievement) for extension. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : Number of municipalities, localities, families and children in Oportunidades 2,445 municipalities 2,451 municipalities Value 95,819 localities, Maintain at 109,852 localities Maintain at least at (quantitative 5,049,206 families, least at current 5,922,246 families current level or Qualitative) 7,222,855 children (age 0- level 12,402,864 children 19) (age 0-19) Date achieved 12/30/2008 12/31/2010 12/31/2013 12/31/2013 Comments Surpassed. Program revised in 2011 the baseline for children to 10,511,273 for 2008 as (incl. % reported by MIS. With this correction, the changes in coverage are consistent (873,040 achievement) additional families and 1,891,591 additional children). Indicator 2 : Number of exits of participants, by type of exit Value 45,011 exiting families n/a DROPPED iv (quantitative (23,266 exiting temporarily, or Qualitative) 21,745 exiting permanently) Date achieved 12/30/2008 07/30/2010 10/06/2010 Comments Dropped with AF. Indicator was meant to provide information regarding the quality of (incl. % the registry by monitoring how it was updated to reflect changes in family status. achievement) Indicator was considered too volatile as a measure of quality of registry management. Indicator 3 : Implementation of PEI-CONAFE pilot with indigenous and non-indigenous families Pilot started. Proportion of Pilot study completed Value Analyze results of Oportunidades beneficiaries in 2012 and analysis (quantitative pilot impact participating in PEI- of impact evaluation or Qualitative) evaluation CONAFE identified completed June 2012 Date achieved 12/30/2008 12/30/2013 12/31/2012 Comments Achieved. Findings suggest that adding PEI-CONAFE provides additional benefits to (incl. % the nutritional status and development of Oportunidades children. Based on pilot results achievement) the program is considering ECD as a key pillar of its redesign strategy. Indicator 4 : Implement madrinas pilot Value 2 municipalities in 30 municipalities (quantitative Guerrero, 438 pregnant DROPPED included or Qualitative) women Date achieved 12/30/2008 05/13/2009 10/06/2010 Comments Dropped with AF. Budgetary difficulties experienced by the Ministry of Health (incl. % prevented implementing the pilot as originally planned and it was therefore dropped. achievement) Indicator 5 : Implement AGE with Telesecundaria pilot Value Pilot started. Baseline Act on Pilot completed and (quantitative survey conducted in 250 recommendations of results of follow up or Qualitative) Community Committees pilot evaluation. survey analyzed Date achieved 12/30/2008 12/30/2010 12/31/2012 Comments Mostly achieved. The pilot was implemented in a year in which many changes in the (incl. % Program's community committees occurred. Because the pilot's lack of results may be achievement) due to implementation challenges, the target is considered not fully achieved. Indicator 6 : Increase number of beneficiaries with Seguro Popular Increase proportion Value Identify proportion of of Oportunidades (quantitative Oportunidades beneficiaries beneficiaries DROPPED or Qualitative) enrolled in Seguro Popular enrolled in Seguro Popular by 20% Date achieved 12/30/2008 10/06/2010 10/06/2010 Comments Dropped with AF. The indicator was incorporated into the WBG Mexico Social (incl. % Protection in Health Project and by 2013 had increased by 40 % points. achievement) v G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 06/18/2009 Satisfactory Satisfactory 0.00 2 12/14/2009 Satisfactory Satisfactory 898.76 3 06/28/2010 Satisfactory Satisfactory 1498.77 4 02/04/2011 Satisfactory Satisfactory 1750.00 5 07/12/2011 Satisfactory Satisfactory 1750.06 6 02/12/2012 Satisfactory Satisfactory 1848.70 7 09/21/2012 Satisfactory Satisfactory 2025.53 8 04/13/2013 Satisfactory Satisfactory 2154.94 9 11/10/2013 Satisfactory Satisfactory 2718.28 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions The Government requested a reallocation of resources (US$360,000) from consulting 07/13/2010 N S S 1498.77 services towards the financing of grants in order to complete disbursements by the closing date. The Government requested a reallocation of resources allocated to cover operating costs for salaries of the Program staff 11/22/2011 N S S 1847.39 (US$530,000) towards the financing of beneficiaries cash grants (i.e., a change in expenditure category). The restructuring addressed three issues which required amending the Loan Agreement: a. The reference to the cash grant “energy support” (apoyo energetico) was dropped because 10/08/2012 N S S 2025.53 as of January 2012 the grant was consolidated into the nutrition grant. b. With the objective to improve efficiency, avoid delays and possible errors and strengthen capacity in the transmission of vi ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions information, the Program consolidated the arrangements for paying participants under one financial institution, the Banco de Ahorro Nacional y Servicios Financieros, S.N.C. (BANSEFI). c. The Education Sector decided that, starting in January 2012, resources for the education grants would be drawn from the budget of the Public Education Ministry (Secretaria de Educación Públic – SEP) instead of from the National Council for Educational Development (Consejo Nacional de Fomento Educativo– CONAFE). I. Disbursement Profile vii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal After a period of macroeconomic stability and moderate growth (3.8 percent average annual GDP growth between 2004 and 2007), coupled with public sector deficit reduction and debt-to- GDP ratio declines which contributed to a healthy domestic credit expansion and growth of domestic demand, the global financial crisis of 2008 led to a sharp tightening of financing conditions in Mexico. The Mexican economy was hit through contractions in income and employment, and a shortfall in fiscal revenue. Estimates at the time were that the recession would cause the elimination of about half a million jobs in 2009 alone, and the number of poor was expected to increase by 3.6 million. Previous experience had suggested that social impacts of such a crisis could be significant: during the 1994-95 economic crisis, household welfare of the poor declined not only from a fall in real income, but also from the weak fiscal and social protection response by Government. Therefore in 2008, Government took important steps to mitigate the impact of a global slowdown on growth and employment generation, and protect human capital accumulation of the poorest. In contrast to the previous crisis periods, Mexico had a more developed social protection system with better mechanisms to protect the poor. One of the main instruments of social policy, and central to the domestic policy response of the Government to the 2008 crisis, was the Conditional Cash Transfer Program Oportunidades, designed in response to the crisis of 1994-95. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The Project Development Objectives were to 1: (i) Increase capacities in health, nutrition and education of poor families through human capital investment by promoting regular health check-ups, improving health status, and raising school enrollment and attendance rates; and (ii) Build sustainable connections between Oportunidades and other social programs of the Government of Mexico in order to improve health and education outcomes for Program participants. Outcome indicators for tracking Project progress towards these objectives were the following 2: 1 In the PAD, the following sentence was included as part of the PDO although it refers to the Project beneficiaries: “The Project’s objective was to support Mexico's Conditional Cash Transfer (CCT) Program, Oportunidades, which served slightly more than 5 million poor families at the time of appraisal. About 70 percent of these families were in rural areas, 16 percent lived in semi-urban areas, with the remaining 14 percent in urban areas.” 2 As indicated in results monitoring matrix, PAD, p. 70-72. 1 Outcome Indicators • % of families complying with health conditions [by age (with breakdown for pregnant women and children <5 and for indigenous and non-indigenous groups)] • % of children complying with education conditions by age • % of students who transition from primary to lower secondary education • % of students who transition from lower to upper secondary education • Number (and %) of Oportunidades schools participating in the Programa Escuelas de Calidad (PEC) • Number (and %) of Oportunidades beneficiaries with PRONABES scholarships Intermediate Results Indicators • Number of municipalities, localities, families and children in Oportunidades • Number of exits of participants by type of exits • Implementation of PEI-CONAFE pilot with indigenous and non-indigenous families • Implementation of madrinas y parteras pilot • Implementation of AGE-CPC pilot in telesecundarias • Increase Number of beneficiaries with Seguro Popular 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The Project’s PDOs were not revised. 1.4 Main Beneficiaries, The main beneficiaries were poor families participating in the Conditional Cash Transfer Program Oportunidades. In particular, the Program targeted poor households in marginalized rural, semi-urban and urban areas (areas are classified according to their social marginalization index, and individual households are then targeted based on their socioeconomic characteristics and the likelihood of being extremely poor). 1.5 Original Components (as approved) Component 1: Support to the Continuation of the Oportunidades Program (US$1,498.5 million of loan financing). Resources of the Bank loan would be used to finance some of the cash transfers to the participants in Oportunidades. It would not be used to finance in-kind support to families, nor the provision of education or health services to them. Under the first Component, the Project planned on financing the bi-monthly cash payments for five transfers (nutrition, education, school supplies, compensation for international food price increases, and compensation for spending on energy) in the full structure of cash transfers to families. It was expected that the Project would finance the equivalent of 2.5 million beneficiary families through 2010. Component 2: Technical Assistance to Oportunidades (US$1.5 million of loan financing). The activities to be carried out under this Component aimed to: (a) increase the quality and accessibility of the health and education services used by Oportunidades families. In particular, 2 this Component was supposed to support efforts to make better use of several on-going public initiatives, including in the areas of early childhood development (PEI-CONAFE), school quality (Programa Escuelas de Calidad - PEC) and Support to School Management Program (Apoyo a la Gestión Escolar-AGE), community health, and health insurance for the poor (the non- contributory scheme of the Seguro Popular); (b) develop strategies to increase the level of self- sufficiency of Oportunidades families so that they would be able to increase their well-being, including, (i) strategies for graduating youth, (ii) facilitate access to scholarships for higher education and/or labor orientation and training courses and, particularly for families with many years in Oportunidades, (iii) facilitate access to other social programs, housing, and employment; (c) further improve the design and operation of Oportunidades, including, a review of the reasons for the incomplete take-up of the education grants by school-age children, and a study on possible community-level activities to support the Program; and (d) implement the Indigenous Peoples Plan (IPP). Under this Component, the Project financed consultancy services which were required to carry out studies, systematization of best practices and experience in implementation of pilot programs, diagnosis of the requirements to facilitate better coordination with other social programs (for example, development of standards for sharing data, identifying changes in the operating rules of various programs to improve coordination, etc.), and design and implementation of dissemination and communication strategies. 1.6 Revised Components The Project’s components were not revised during implementation. 1.7 Other significant changes The main significant change to the scope and scale of the Project came through an Additional Financing (AF) of US$1,250 million approved in November 2010 (Component 1 US$1,244.5, and Component 2 US$2.3, excluding front-end fee). The AF responded to two objectives. First, it allowed the Government to cover an unanticipated financing gap for 2010-2011, since Mexico’s fiscal situation was more severely affected by the international economic and financial crisis than expected at the time the original Project was appraised. Secondly, the additional resources contributed to Government’s efforts to strengthen the social response, expanding Oportunidades’ coverage and benefits in an attempt to mitigate the impact of the 2008 economic crisis on the poor. The AF contributed to financing the expanded coverage of beneficiaries (600,000 more beneficiaries, in addition to the 5.2 million originally supported by the Project, for a total of 5.8 million). While the development objective and the components of the Project did not change with the AF, the indicators were revised to reflect minor streamlining and to adjust targets to the extended closing date of December 31, 2013. One minor adjustment relative to the original Project was the provision for financing of operating costs to cover the salary costs of program staff responsible for administering the World Bank Group (WBG) Loan. In addition to the AF, the Project underwent three Level Two restructurings described below: The first restructuring responded to a request by Government for a small reallocation (US$360,000) from the Technical Assistance (TA) Component towards the financing of cash grants to complete disbursements by the closing date of December 31, 2010. While the full amount of the Grants Component had been disbursed, some delays in the TA Component of the 3 Project allowed the availability of additional resources to be reallocated from that component. The reallocation did not affect the Project’s satisfactory performance and achievement of PDOs. The second restructuring was necessary because the Ministry of Finance and Public Credit withheld the authorization necessary for the National Coordination of Oportunidades to use Loan resources to cover operating costs for salaries of the Program staff, as provided for in the AF. The Government requested a reallocation of these resources (US$530,000) towards the financing of cash grants (i.e., a change in expenditure category). The operating costs which were to have been covered by the Loan, were instead covered by national resources, and no significant changes or fiduciary issues arose as a result of the reallocation. The third restructuring addressed three issues which required amending the Loan Agreement: a. The reference to the cash grant “energy support” (apoyo energetico) was dropped because as of January 2012 the grant was consolidated into the nutrition grant. The energy grant was the smallest and was already linked to the same co-responsibilities as the nutrition grant, so there were no substantive implications of this change for Program participants. b. With the objective to improve efficiency, avoid delays and possible errors, and strengthen capacity in the transmission of information, the Program consolidated the arrangements for paying participants under only one financial institution, the Banco de Ahorro Nacional y Servicios Financieros, S.N.C. (BANSEFI), thereby eliminating the need for agreements with two other distribution channels (DICONSA and Telecomunicaciones de Mexico). This change implied a slight revision in the Project’s overall financial management (FM) arrangements, but did not affect the FM risks assessed. c. The Education Sector decided that, starting in January 2012, resources for the education grants would be drawn from the budget of the Public Education Ministry (Secretaria de Educación Pública – SEP) instead of from the National Council for Educational Development (Consejo Nacional de Fomento Educativo– CONAFE). There were no substantive implications to this change, as the control of the budget for the education grants remained under the responsibility of the education sector. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Soundness of the background analysis. The Project represented the largest social protection loan ever made by the Bank and one of the largest in the country. As reflected by the Independent Evaluation Group (IEG) crisis response analysis, 3 the large amount of lending to Mexico was a reflection of country demand, the existence of a mature social protection program, and the capacity of the Government to absorb the funds. At the time the Project was requested, the Bank had no lending operation in social protection in the country. The WBG had, however, 3 IEG, The World Bank Groups’ Response to the Global Economic Crisis, Phase II, (2012). 4 been closely engaged with the country since 2006 through the CCT Community of Practice (COP). 4 The dialogue fostered by the COP, including through a two-day workshop organized in Mexico in early 2008, contributed to Mexico’s decision to turn to the WBG to support Oportunidades when the global economic slowdown began. The crisis required a rapid response to protect the poor and most vulnerable, and the choice to support a well-functioning Program with proven impact (see Box 1), was appropriate and made Project preparation possible in a short timeframe with the knowledge and trust acquired through the close engagement in the COP. Box 1: Oportunidades - Impact Evaluation and Key Results Extensive qualitative and quantitative data has been collected on Oportunidades since 1997. The most recent surveys were in 2007 for rural areas and 2009 for urban areas. All external evaluations have been made public, often posted on the Program website and/or published in respected academic journals. In the area of education: • Important increases in the grades of schooling achieved for young male and female adults (0.85 and 0.65 additional years respectively, between the ages of 17-22). • Significant and positive impacts on indigenous people, as high as or higher than for non-indigenous youth (0.64 (women), 0.84 (men) additional years), • Some encouraging positive tendencies in achievement (Increase of 10 percent in the scores on the Woodcock- Johnson exams in mathematics for men and women between 19 and 22 years) – though lag in outcomes is persistent and possibly tied to quality of services • Quality of education services is strongly correlated with education gaps. Controlling for variables of context, there is no difference between test scores of Oportunidades students and other students. • Schools attended by Oportunidades students are disadvantaged in their access to important inputs and services. In the area of health: • Increases in prenatal care, less likelihood of adolescents to engage in risky behavior, and reductions in obesity and chronic illness among Program participants. • Increases in both overall and food consumption, sustained over time. • In the case of nutrition, reduction by 22.2 percentage points in the prevalence of stunting and of 11.8 percentage points in the prevalence of anemia in children below the age of two • Also, reduction by 5.4 (urban areas) and 14.2 (rural areas) percentage points in the prevalence of anemia in pregnant women between 17 and 22 years. Assessment of Project design. The Project design responded to the need of the Government to support its flagship social protection program in a time of financial difficulties, which would in turn meet the objective of protecting the income and human capital investments of the poorest. On the one hand, the Project supported a rapid financial response targeted to the most vulnerable groups of Mexico (Component 1-Cash Grants) while relying entirely on existing systems, which had proven to be well-suited for the stated objective of protecting human capital, and which allowed the Project to be designed and implemented quickly. At the same time, through its 4 The CCT COP is a virtual learning community managed by the WBG, in which 5 Latin American countries (Brazil, Mexico, Colombia, Chile, and El Salvador) engage and discuss topics important for their CCT programs, such as management, local and Government institutional strengthening, monitoring and evaluation, and the future role of CCT programs. The CCT COP had fostered a strong, tight-knit network of senior practitioners who found a forum for continuous communication, where they could learn from each other on CCT implementation challenges and provided essential inputs in the design of the operation. 5 technical component (Component 2), the Project provided key evidence-based inputs to the Oportunidades Program in a flexible way, responding to emerging needs in the agenda to strengthen the Program’s operation. As requested by the Government, the TA Component focused on areas within the Program’s existing evaluation agenda where it was expected that the WBG could bring the greatest value added. In fact, TA support was more extensive than expected when the Loan was approved, both through supervision itself and through related Bank- financed analytical and advisory services. 5 The framing of Project objectives was fully aligned to Oportunidades’ goals. The first PDO was articulated based on what could be measured and attributed to the Oportunidades Program. Outcome indicators focused on maintaining Program activities at a desirable level, under the assumption that those would lead to improved results in consumption, health and education outcomes, as had been shown by the Program’s impact evaluations. For the second PDO, the indicators chosen to monitor progress in building sustainable connections with other social programs were ambitious in that they partially depended on decisions made within the education and health sectors regarding the operation and coverage of their programs. These indicators were chosen to signal the direction in which the Program had to move to start addressing the issue of access to services and quality of outcomes for its beneficiaries. With this in mind, the targets were articulated both in absolute and percentage terms to track improvements in coordination even if programs’ expansion per se took longer to materialize Incorporating Lessons Learned. The design of the TA Component and TA support during implementation took into account the lessons learned from Bank experience with CCT programs internationally. These lessons focused on two areas: (i) key elements that have to be in place and monitored carefully to guarantee the success of CCT programs; and (ii) issues related to second- generation CCT programs. Table 1. Lesson learned that informed Project design Lessons learned which informed project design Key elements An accurate system to identify and select the target population for success of Selection of co-responsibilities that take into account specific human capital development CCT programs shortcomings to be addressed CCT programs cannot be thought of in isolation from other social policies and programs, and Issues related achieving human capital accumulation goals will require adaptation of the supply of social to second- services (coverage, quality) generation CCTs have the potential to evolve from their effective, but isolated, role in poverty reduction CCT programs strategies to become part of a broader social protection system Assessment of Government’s commitment. Oportunidades is Mexico’s flagship social program and from the beginning the Government showed strong commitment to the Project, both 5 In particular: Programmatic Knowledge Services: Social Protection for the Poor (116169), NLTA: Improvements in CCT Governance SFLAC (P125712) and Programmatic Knowledge Services: Strengthening Social Protection System (P129698). Details can be found in Annex 2. 6 from the Ministry of Finance and the National Financing Agency 6 (Nacional Financiera, NAFIN), as well as from the implementing agency, the National Coordination of Oportunidades (which is part of the Ministry of Social Development - SEDESOL). Project design was well aligned with the needs of the Program, and the Government more broadly. The National Coordination was committed to working with all stakeholders involved to ensure results in the context of the Project. No substantial problems arose during implementation and Government commitment was sustained throughout implementation and beyond, as evidenced by the request for a new Bank-financed operation. Assessment of risk. During Project preparation a number of risks were identified that could affect implementation. One concern was related to international experiences with CCTs that had mixed results; however Oportunidades had had a solid track record (see also Box 1), strong organizations and well-established institutions. These risks were low and did not materialize. A second concern was related to difficulties in achieving coordination with ministries and institutions to implement activities that would improve the Program’s operational model (linkages with other social programs and activities included in the Indigenous Peoples Plan - IPP). These risks were also rated as low after accounting for the mitigation measures considered within the Project, and they did not affect overall achievement of Project outcomes. However, it is worth mentioning that this was an area that proved much more challenging than expected and to which significant Program and Bank staff time and resources were devoted to ensure success. 2.2 Implementation Project implementation was satisfactory. At no time was the Project considered a potential problem or actual problem Project and no significant external factors emerged that affected implementation. In particular, Component 1 (Cash Transfers) disbursed regularly, and Component 2 was also on track with only minor delays in contracting some technical activities. All major outputs were delivered on time and Project implementation rating was satisfactory throughout implementation. While at appraisal Oportunidades was covering five million families, its coverage was increased to 5.8 million largely in an attempt to mitigate the impact of the crises. By Project’s end, the Program was covering over 5.9 million families. Two-thirds of the new households came from semi-urban and urban areas where there had been under-coverage of the program; there was also an important expansion to incorporate additional small, isolated rural areas that had been harder to reach in previous years. Despite the fact that it was the first project with the World Bank for the implementing agency, the National Coordination was overall very effective. The agency and the Bank developed a joint work program for Component 2, which supported the Program’s longstanding evaluation agenda and could be adapted flexibly according to emerging needs of Oportunidades. In fact, the scope of the Bank’s involvement on TA was greater than expected, and this increased demand for it was addressed both through supervision activities and additional financing. This flexible approach was instrumental in achieving results in areas that were unplanned but important to 6 The National Financing Agency (NAFIN) acts as financial administrator and financial agent of the Borrower with regard to external loans. In that capacity, NAFIN manages loan disbursement processes and provides other implementation support and oversight. 7 improve the Program’s effectiveness (i.e., targeting). More details are provided in Annex 2. All TA work was carried out in close collaboration with the Inter-American Development Bank (IADB), which was also providing financing and TA to Oportunidades. The two organizations coordinated joint supervision missions to ensure full alignment and maximize the level of technical assistance provided to the Government. The timely restructurings of the Project showed pro-activeness on the side of the Bank team in responding to changing needs, and resulted in a smooth implementation. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Given the nature of the Project, which was financing US$2.7 billion out of a total of US$22 billion in cash transfers provided by the well-established Conditional Cash Transfer Program that demonstrated its positive impact in rigorous evaluations in many important areas, the Project’s Monitoring and Evaluation framework was designed to build on the Program’s established monitoring system. Project indicators were chosen among those established by Oportunidades itself to reflect Program results. 7 The methods identified to monitor progress toward the PDO were timely and adequate and could be constantly measured to inform the operation. At the time of Project design Oportunidades had an ongoing evaluation agenda (including needs, process, and impact evaluations), which was part of a national effort to promote management for results and evidence-based policy making through a systematic evaluation of public programs. 8 The Program had also developed a comprehensive monitoring structure supported by sophisticated mechanisms to monitor program processes and indicators to follow day-to-day information on program operation, providing useful and real-time information and feedback to program staff at different levels. 2.4 Safeguard and Fiduciary Compliance Safeguards – Indigenous People’s Plan. The Project triggered OP 4.10 (Indigenous Peoples). An Indigenous People’s Plan (IPP) was prepared at the time of the original Project. A social assessment was conducted, and updated at the time of the Additional Financing. The social assessment highlighted the high proportion of indigenous participants in the Program since its inception in 1998 as Progresa. In 2009, about 26 percent of the 5 million beneficiary families were indigenous and the proportion remained constant after program expansion (26 percent of 5.8 million beneficiaries in 2013). 9 Given the high proportion of indigenous beneficiaries, and the nature of the Program which aims at facilitating the accumulation of human capital in participant families through incentives to behavior change, the main activities envisaged in the 7 The Program’s results matrix (Matriz de Indicadores para Resultados, MIR) is revised every year and approved by the Ministry of Finance in advance of the budget cycle. The one for 2013 can be found at: http://www.oportunidades.gob.mx/EVALUACION/es/wersd53465sdg1/matriz/pal_resumen_narrativo_2013.pdf 8 The Federal Government has established a system of performance evaluations for all social programs in the framework of results-based management, against a Matrix of Indicators of Results agreed for each program, including Oportunidades. The results of this evaluation is reported to the Congress and linked to the budget process. 9 The Program estimates the number of indigenous beneficiaries based on number of beneficiaries in localities classified by the Government as indigenous (where population is majority indigenous). 8 IPP were directed at further adapting Program communication/rules to improve results for indigenous beneficiaries. In particular, several evaluations indicated larger gaps in outcomes, such as reproductive and maternal health, nutrition and transition from lower to upper secondary education and beyond, for indigenous beneficiaries. The Program therefore focused on making improvements in these areas. The results of some of the activities implemented were greater than expected and led to improvements in Program design, while others did not prove as successful and were discontinued (see Annex 2). Overall, the Project successfully implemented the IPP and contributed to improve Program results for indigenous groups through: (i) increased coverage in indigenous areas/localities, and (ii) improved capacity to reach out/communicate and provide services to indigenous beneficiaries by strengthening the indigenous people/s communication plan and the bilingual program staff initiative. A significant achievement in the latter was that in 2009 Oportunidades embarked on a process to improve the ratio of frontline program staff able to serve indigenous beneficiary populations in their respective language. This required reassigning some staff to different regions, training, and hiring new staff. In the process, Oportunidades partnered with the National Institute of Indigenous Languages (INALI from the Spanish acronym) and developed a Program to formally certify staff as bilingual, as well as to further strengthen the capacity of these staff to perform their functions. 10 Fiduciary – Financial Management. Financial management performance was always rated satisfactory. As is usual with CCT programs, the large number of payments and their geographical spread pose significant FM implementation challenges and an inherent FM risk. Despite an initial classification as high risk, however, the residual FM risk for the Project at appraisal was considered modest after accounting for the existing control system. All financial management aspects were coordinated by the National Coordination of Oportunidades, using Mexico’s existing systems and organizational arrangements. FM processes and information systems were reviewed, often jointly with the IADB as both development agencies had ongoing projects for Oportunidades and were consistently considered effective in maintaining a sound control environment and mitigating the major inherent risks associated with the execution of a conditional cash transfer program. The interim financial reports and annual external audits were delivered on time and considered acceptable by the Bank. In the 2010 audit, auditors had initially issued observations related to weaknesses in the Project’s internal controls. However, following a Bank supervision mission these findings were considered not adequately formulated and the result of a misunderstanding of the auditors on the functioning of the Program. Fiduciary – Procurement. Procurement performance was always rated satisfactory. The Project financed mainly cash transfers for which no procurement was involved. Under Component 2, only procurement of consultant services was required. The procurement risk assessment at appraisal was low. The National Coordination had had no experience with procurement under Bank-Financed operations, but could count on two professionals familiar with procurement 10 The certification process established specific standards and requirements (two options, one for “Attention in Native Language” and one for “Evaluation”), and is open to all Program staff that speaks an indigenous language and wishes to become certified. By the time the Project closed in December 2013, 55 people had been certified in one or both of the possible standards (Specifically, 47 people were certified for “Service in Native Language” and 8 for “Evaluation.” 15 of these were certified in both). 9 procedures of multilateral agencies and experience in the implementation of IADB-financed operations. The most relevant procurement issues were a series of requests for large single source selection contracts with specialized institutions and a few delays in the implementation of some technical assistance activities. The single source contracts were required for the Project and justified according to Bank policies, 11 while delays in contract effectiveness affected some studies. 2.5 Post-completion Operation/Next Phase The Program is well institutionalized at the core of Mexico’s social protection system. At Project completion, no impact on the Program’s operation was expected due to the closing of the Loan. Oportunidades has demonstrated ability to reach a large share of the poor and positively impact their welfare and human capital accumulation. The Program remains highly relevant within the Government’s priorities for social policy. Important challenges remain to consolidate the Program, particularly related to strengthening health and education outcomes and developing stronger links to social and productive interventions to improve families’ wellbeing. The Government assigns high priority to scaling up and consolidating the strategy to link Oportunidades beneficiaries with social and productive programs that can help them develop skills and generate opportunities for income generation to increase employability and productivity of beneficiaries. In 2013, the Government formalized its objective to connect Oportunidades beneficiaries with other interventions, by modifying the operating rules of twenty-nine federal productive and social programs to ensure that Oportunidades beneficiaries receive priority attention. While the Government’s political and financial commitment to Oportunidades is guaranteed, in 2013 the Mexican Government requested a new World Bank Loan to support the redesign of the Oportunidades Program, with the objective of further improving its impact by linking beneficiaries to additional social and productive programs. The objectives and engagement remain highly relevant both to country priorities and the World Bank twin goals of eradicating extreme poverty and increasing shared prosperity. For the new Loan the Government requested a stronger program of technical assistance which will also include support to the operational part of the Program. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: High The objectives, design and implementation of the Project were highly relevant at the time of appraisal. Project objectives remain consistent with Mexico’s efforts to reduce poverty and with the World Bank vision to reduce extreme poverty and increase shared prosperity. The 2008-2013 11 Procurement Guidelines under IBRD Loans and IDA Credits, May 2004 (revised October 2006 and May 2010). 10 Country Partnership Strategy (CPS) 12 for Mexico identified flexibility, fast response, and selectivity, driven by the key long term development challenges of Mexico, as drivers for engagement. The Loan reflected the Bank group’s flexibility and fast response to the financial requirements of the Mexican Government while maintaining close alignment to one of the long- term development challenges identified as a priority for Bank support: promoting social inclusion and reducing poverty. The Government’s decision to expand Oportunidades, as well as recent efforts to increase its role by strengthening linkages with other social and productive programs, are a demonstration of the continued relevance of this engagement. The Project’s objectives remain consistent with the Government’s 2013-2018 National Development Plan and its goal to reduce poverty and inequality while fostering economic development. Project objectives are also fully aligned with the World Bank Group twin goals of ending extreme poverty and promoting shared prosperity in a sustainable manner and, more recently, with the goal of the recently approved 2014-2019 Country Partnership Strategy 13 to increase social prosperity. The design of the Project combined the financial support that ensured sustained coverage of the Program with an analytical and evaluation agenda that provided the necessary elements to strengthen Program design. In fact, the Government valued the international perspective and technical advice that the Bank brought to support its effort to strengthen the Program. This is demonstrated by the fact that the WBG’s involvement in the operational aspects of the Program grew over time, expanding through complementary analytical and advisory services first and then evolving into a request for a new operation to support the redesign of the Program that relies heavily on analytical work conducted under the Project. 3.2 Achievement of Project Development Objectives Rating: High Given that the Project was designed to support a well-established Government program with a strong monitoring framework, the framing of the PDO was driven largely by the Program’s expected outcomes and targets. The outcome and intermediate indicators chosen were appropriate to track progress in achieving Project objectives, and were linked to the Project’s activities. At the time of completion, all the outcome targets, as well as the intermediate indicators, were largely achieved. The full results matrix with more details can be found in the Data Sheet. Indicator targets to track progress in increasing capacities in health, nutrition and education of poor families as described in the program’s Operational Manual were realistic and based on ensuring that Oportunidades could maintain its effectiveness and coverage during a time of economic recession and during which the Government was under fiscal constraint. The number of Oportunidades’ beneficiaries increased above initial plans, reaching beyond expected 12 Country Partnership Strategy with the United States of Mexico, 2004, Report No.28141-ME. 11 coverage in urban, semi-urban and rural areas and all targets for compliance with health, nutrition and education co-responsibilities indicators were fully achieved or surpassed. The indicators focused on tracking the success of the program in building sustained linkages between Oportunidades and other social programs were the first attempt at broadening and strengthening the links between Oportunidades and the two sectors to further improve health and education outcomes for beneficiaries. Those indicators focused on where the WBG would be well positioned to have a positive impact, both through its work with Oportunidades and through its health and education projects, 14 given that achieving results in this area depended crucially on decisions made in coordination with these sectors at the state and national levels. Since those indicators were not entirely under the control of Oportunidades itself, the choice of targets was intentionally conservative and reflected in percentage as well as absolute terms. The percentage targets aimed to monitor whether the health (Seguro Popular) and education (PEC and PRONABES) programs were improving their targeting by focusing more on Oportunidades beneficiaries, and were fully achieved. 15 3.3 Efficiency Rating: High At appraisal, no ad hoc economic and financial analyses of the Project were undertaken. The following factors – which remain relevant today – were considered in determining the efficiency of the Project (see also Annex 3): • The body of evidence on Oportunidades, based on the numerous external evaluations conducted since the program started in 1997, shows significant positive impacts of the Program on household consumption, school enrollment and attendance, health prevention/access to health and nutrition – all of which are part of the Project’s development objectives (Box 1). • The most recent cost-effectiveness analysis 16 was conducted monetizing the results of these impact evaluations, but was limited to returns to schooling. These simulations show mostly positive results, with positive cost/benefit ratios ranging from 1.4 – 7.1, except in some cases using a 10 percent discount rate. These results can be considered a lower bound on the expected benefits since they do not consider the impacts on health and nutrition (or consumption more generally). 14 Support to the Social Protection System in Health (P116226), School Based Management (P115347) and Tertiary Education Student Assistance APL (P085593). 15 The absolute numbers depended on a combination of targeting and expansion of these programs, which was less certain. For example, according to the PAD of the School Based Management (P115347) Project, targets for the PEC program as a whole for end of 2012 were of 41,605 participating schools. Reaching the 40,000 target set in the results framework would have implied that almost 100 percent of PEC schools were Oportunidades schools, which was not the intent of the target. In the case of PRONABES the absolute target set of 90,000 scholarships to Oportunidades beneficiaries would have represented 28 percent of PRONABES scholarships offered by SEP in 2013 while the percentage set as a target was 20. 16 Parker, S., Behrman, J. (2008) “Seguimiento de adultos jóvenes en hogares incorporados desde 1998 a Oportunidades: Impactos en educación, pruebas de desempeño.” 12 • The Project supported improvements in targeting that should lead to further improvements in Program effectiveness since studies have shown larger impact of CCTs among households that are poorer at the baseline. 17 Moreover, the Project supported increases in coordination of social policy which are expected to further improve efficiency and returns from investment: evaluations have shown that an important factor behind the persistent gaps in outcomes for Oportunidades beneficiaries is the lower quality of services received. Greater coordination with education and health sectors to improve the coverage and quality of services – as evidenced by the efforts to increase coverage of PEC and PRONABES – for the Oportunidades population is thus expected to lead to greater impacts on final outcomes for the target population. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory The Project as a whole achieved its objectives, and the Overall Outcome Rating is satisfactory. Oportunidades was critical in protecting human capital during the crisis and has, over time, been successful in improving living conditions of the extreme poor, as evidenced by the impact evaluations. While WBG support was small as a share of the overall budget for the Program, its resources and technical assistance contributed to ensure continued appropriate investment in the Program in a difficult financial situation, and to strengthen Oportunidades and its effectiveness (targeting, coverage, quality and links with other services). The TA directed at forging stronger ties with other sectors went beyond immediate response during crisis, to fostering deeper support for the long-term quality of the program and thus greater impact on the well-being of its beneficiaries Project activities were appropriate to achieve objectives, which remain highly relevant both to the Government and the WBG strategy for Mexico. The Program remains a key element of the Government’s social protection and poverty reduction strategy and its role is expanding, particularly in the more challenging area of coordination with other programs, which was the most innovative – and challenging – part supported by the Project. Supporting Oportunidades remains an important element of the new Country Partnership Strategy as the program constitutes a key building block of the social protection system in Mexico. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development While poverty reduction was not stated explicitly as an objective of the Project, the Bank and the Government were clearly concerned with the impact the global financial crisis was going to have on poverty levels in Mexico, as well as with the consequences for human capital accumulation. According to the analysis conducted by CONEVAL and simulations by World Bank staff, Oportunidades played a key role in moderating losses among the poor during the crisis. 17 Conditional Cash Transfers: Reducing Present and future Poverty (World Bank, 2009), p. 136. 13 Estimates suggested that social programs contained the growth of poverty rates by around 1 percentage point between 2006 and 2008. Moreover, simulations by the World Bank and IEG18 suggested that expanding the coverage of Oportunidades and Programa de Apoyo Alimentario (PAL) 19 programs by a total of 1 million households (as was planned in 2010) would reduce the depth of poverty by 0.4 percentage point (or 2 percent) or the poverty headcount rate by 1.8 percentage points (or 3.6 percent), compared to a scenario with no expansion, depending on whether the expansion targeted the poorest or those close to the poverty line. More recent studies indicate that without the expansion of Oportunidades’ coverage, which was supported by the Project, the income of those in the 20th percentile would have fallen by just over 8 percent, but with the expansion the drop was less than six percent. For those in the tenth percentile, incomes fell only 3 percent. 20 The evidence clearly points to the critical role played by the transfers in mitigating this impact on the poor and, based on previous experience in Mexico, it can be argued that the cash transfers allowed families to devote resources to their human capital development which they would not have had otherwise. Moreover, the review of the targeting instrument shows that the program is effectively targeted to the very poorest households. (b) Institutional Change/Strengthening. The technical assistance provided in the context of supervision missions, supported synergies created within the social sector by supporting the Government’s efforts to strengthen social policy evaluation and harmonize social policy tools. In particular, technical assistance supported existing synergies between CONEVAL, SEDESOL and Oportunidades in the areas of poverty identification and beneficiary selection through the dialogue on targeting, as well as more generally in the area of evidence-based policy formulation through coordination of the evaluation agenda. Additionally, Oportunidades significantly strengthened and internalized its operational strategy and mechanisms for improving services provided to indigenous beneficiaries. (c) Other Unintended Outcomes and Impacts (positive or negative) The Project has contributed to the public good through a wealth of information on the operation and evaluation of CCTs. This has resulted in better design and implementation of several CCT programs and is evidenced by the number of Bank coordinated South-South exchanges with countries interested in Mexico’s experience. 18 See World Bank, Poverty Reduction and Economic Management Sector Unit Latin America and Caribbean Region, Recent Trends and Forecasts of Poverty in Mexico (October 2010) and IEG, The World Bank Groups’ Response to the Global Economic Crisis, Phase II, (2012). 19 As of 2010 the food support program (Programa de Apoyo Alimentario - PAL) was moved under the control of Oportunidades. It targets the same population as Oportunidades, but beneficiaries are selected in areas where the supply of services is not running in full capacity, meaning that Oportunidades co-responsibilities are not implementable. 20 Understanding the Poverty Impact of the Global Financial Crisis in Latin America and the Caribbean, edited by Margaret Grosh, Maurizio Bussolo, Samuel Freije, World Bank (forthcoming) 14 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A 4. Assessment of Risk to Development Outcome Rating: Negligible The country has enjoyed macroeconomic stability over the last two decades and prudent policies have prevented the occurrence of major fiscal and financial disruptions. Even after the 2008- 2009 global crises, Mexico’s economy rebounded quickly and the fiscal challenges were overcome. While weaker economic activity (externally or internally driven) could potentially pose downside risks to Mexico’s growth and public sector revenue perspectives, and thus affect Program progress, the current Government is undertaking crucial structural reforms to unleash growth and raise productivity levels. Oportunidades continues to be a key social program in Mexico, and to be highly valued for its contribution to reducing poverty and extreme poverty, ensuring a better income distribution, and fostering continued investment in human capital for poor families. As a result, the Program has withstood several Government transitions and has expanded significantly over time. It counts on widespread political support, Government commitment and sustained budget allocations. The Government recently announced that the program will now undertake new initiatives to promote productive inclusion, further expanding the role and scope of Oportunidades within Mexico’s poverty reduction agenda. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Highly Satisfactory The Bank’s performance in ensuring quality at entry is highly satisfactory. Thanks to the strong ties that the WBG had forged with the Oportunidades Program and other country stakeholders through the CCT COP, the Bank was able to respond quickly and effectively to the Government’s request for assistance. The Project was prepared and presented to the Board in record time, with identification to Board occurring in the space of four months. Project design supported the Borrower’s priority of ensuring funding of its flagship social program during a time of crisis, while positioning the program beyond the crisis by creating space for a strong evidence-based technical agenda that would further strengthen Oportunidades’ effectiveness. The WBG provided key technical inputs to ensure that the lessons learned and experience from other countries and Mexico would inform the design and implementation of the Program. Risks were candidly identified during preparation and appraisal, and the Project’s design incorporated mitigation measures with the result that those risks never materialized or, in the case of the coordination challenges identified, did not affect Project outcomes. Safeguards were taken into consideration appropriately and measures were built into Project that significantly strengthened Program’s outreach to Indigenous Populations. During preparation, a 15 comprehensive Indigenous People Plan was prepared to address the needs of the Indigenous population in the Program, which were highly represented among the target beneficiary groups. (b) Quality of Supervision Rating: Satisfactory The Bank’s performance during supervision was satisfactory. The Bank team maintained a strong dialogue with the counterpart, ensuring that its technical advice and support maintained relevance throughout implementation. The Bank effectively mobilized complementary programmatic resources (non-lending technical assistance-NLTA/Programmatic Knowledge Service-PKS) to support key elements of Oportunidades’ design when it became apparent that the technical assistance agenda was larger than expected and required timely support. The WBG team also leveraged its supervision budget to carry out timely technical assistance, adding experts in targeting, graduation and other areas as needed to respond to evolving Government needs. The large volume of analytical work conducted as part of the Project’s TA and the complementary programmatic resources required substantive implementation support effort and resources on the part of the Bank, often in coordination with the IADB. Quality of supervision remained satisfactory throughout the Project. No mid-term review was conducted due to the short implementation time of the original Project and the timing of the AF. To respond to changing needs under the Program, the Project was restructured three times as described earlier. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory For the reasons described in the previous sections above, the Overall Bank Performance is considered satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory The Government’s performance is considered satisfactory. Throughout the Project, the Government showed high commitment at all levels and responded in a timely manner to any need for adjustments and changes identified by studies or best practice. The new administration is now revisiting the Program to ensure productive inclusion of beneficiaries, demonstrating that the Program remains a priority. (b) Implementing Agency or Agencies Performance Rating: Satisfactory The Implementation Agency’s performance is considered satisfactory. Since the beginning of the Project, the National Coordination for Oportunidades showed strong leadership and ownership 16 of Project objectives. It established a solid partnership with the Bank to support its efforts to strengthen the program, and was crucial in solidifying a culture of strong monitoring and evaluation that effectively contributed to increase Program impact. The National Coordination and the States Delegations also made important efforts to strengthen their partnership with agencies that could contribute to improve outcomes for Oportunidades beneficiaries, as aspired to in the PDO. In particular, the National Coordination sought to coordinate with the health and education sectors both at the central and State levels. While it did not fully succeed in achieving all proposed goals in terms of cross-sectoral coordination, significant progress in bringing stakeholders together was made. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory For the reasons described in the previous sections, the Overall Borrower Performance is considered Satisfactory. 6. Lessons Learned This operation was the largest World Bank operation in Social Protection in the region and was prepared in a very short time. It offered a distinct opportunity for the Bank to learn from, and contribute to, the implementation of a Program that is internationally recognized and has been a model for conditional cash transfer programs in many other countries. The implementation experience offers a wealth of lessons, which are summarized below as: (i) lessons learned for Oportunidades as a CCT; and (ii) lessons learned with respect to the Bank’s involvement. All are important in the context of the preparation of a new operation. Lessons learned with respect to Conditional Cash Transfer Programs: • In a time of crisis CCTs can be easily scaled up in coverage or generosity ensuring a pro- poor targeted support in a difficult time. • Households can remain beneficiaries of CCTs for a very long time and may need additional support to transition out of poverty, particularly given the high economic and social marginalization of the areas in which beneficiaries tend to live. • CCT programs cannot be thought of in isolation from other social policies, and achieving human capital accumulation goals will require adaptation of the supply of social services (coverage but also quality). • Cross-sectoral coordination might be more challenging and move slower than expected. Effective collaboration and articulation across sectors requires clears rules and well-defined tracking mechanisms, as well as participation of all stakeholders from the design stage and constant communication. It remains a major challenge for the new generation of CCTs which seek stronger linkages with social and productive programs as part of a social protection system. • A shift in vision from a Program to a systemic approach requires strong institutional leadership, and commitment from all actors involved. • A rigorous evaluation agenda provides legitimacy and support for CCT programs as well as the evidence base to introduce changes in program operation that can improve outcomes. 17 This evidence base is especially important in the context of the need to build stronger links with other social programs. Lessons learned with respect to the Bank’s assistance: • Maintaining dialogue in the absence of lending is key to keeping a place at the table, and to being able to respond rapidly. The CCT COP was instrumental in for this. • Being engaged matters. Having an operation allowed the Bank to understand program operation and demands and provide better technical assistance through different vehicles. • Having a suite of services (lending and non-lending) that can respond in a flexible way to demands generated by the Government is important. Team proactivity and availability of complementary financial resources to be able to respond to emerging technical assistance demand are essential. • Timely, focused, technical assistance can lead a project from responsive to transformative. The direct support to Oportunidades was the main purpose of the Project at the outset. However, the high-quality technical assistance rendered under the Project led to improvements in operational design and a deeper engagement during the redesign phase for the second generation of one of the largest CCT programs in the world. • Being able to rapidly mobilize WBG specialists is key to successful TA and maintaining the relevance of WBG assistance to Middle Income Countries (MICs). • Working closely with development partners and institutions, such as the IADB, while challenging and time-consuming for the teams, strengthened the Program. • Effective coordination among World Bank operations for better health and education outcomes for Oportunidades beneficiaries proved to be less successful than anticipated. • The inter-institutional collaboration with health and education, while crucial for the success of the Program proved challenging and advanced at a slow pace. In light of the new phase for the Program these efforts to continuously improve articulation are even more critical. The lessons learned in this experience should inform the design of the new operation in ensuring the right balance between innovative and ambitious vision and direct results attributable to the operation. • The choice to use a Sector Investment Loan (SIL) instead of Development Policy Lending (DPL) made it possible for the team to be more involved in the operational details of the Program and, as a consequence, to provide timely TA that was well aligned with emerging program demands. • The large amount of lending and fast disbursement was possible because of the existence of mature country programs and systems for payment in Mexico. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies In addition to preparing a completion report (See Annex 7), the Borrower made comments on the document drafted by Bank staff. These comments have been reflected in the document. (b) Cofinanciers: N/A (c) Other partners and stakeholders; (e.g. NGOs/private sector/civil society) N/A 18 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Support to the continuation of 1,498.50 1,498.90 Oportunidades Program (original) Technical Assistance to 1.50 1.10 Oportunidades (original) Total Project Cost (before AF) 1,500.00 1,500.00 Support to the continuation of Oportunidades Program (including 2,743.00 2,744.00 100% AF) Technical Assistance to 3.80 2.80 74% Oportunidades (including AF) Total Project Costs (with AF) 2,746.80 2,746.80 100% Front-end fee IBRD 3.76 3.76 Front-end fee IBRD (AF) 3.20 3.20 Total Financing Required 2,753.76 2,753.76 100% (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (USD millions) (USD millions) Borrower 13,255.00 20,084.65 151% International Bank for Reconstruction 2753.76 2753.76 100% and Development 19 Annex 2. Outputs by Component Component 1: Support to the Continuation of the Oportunidades Program (US$2,744 million of loan financing) Resources of the Bank Loan were used to finance cash transfers to Oportunidades beneficiaries. The Project financed the bi-monthly cash payments for five cash transfers (nutrition, transfer to compensate for the international food price increases, transfer to compensate for spending on energy, education and school supplies) within the full structure of cash transfers to families. This first part of the annex contains additional information on Component 1 outputs as they relate to expansion of the Program, number of beneficiaries receiving transfers, and changes in the structure of payments that happened during the time the Project was supporting the Program. Box 1. Changes in structure of payments Over time, the Program introduced some changes to the structure of payments to adjust the Program to emerging needs as well as attempts to streamline the operations. Between 2009 and 2013 two changes were introduced: • The introduction in 2010 of an additional cash transfer for families with very young children (Apoyo Infantil) to protect food consumption for children during the crisis, with no changes in co-responsibilities for beneficiaries • The merging in 2012 of the energy transfer into the health and nutrition transfer (with no changes in co-responsibilities for beneficiaries since the co-responsibility for the energy transfer was already tied to health and nutrition) Structure of Oportunidades Cash Transfers (US$/month), 2009 and 2014 Amount Transfer 2009 2014 Education $9.2-58.4 $13-81* Health and Nutrition** $13.8 $24 Energy** $3.9 - Nutrition complement (Alimentario Complementario)*** $8.5 $10 Children (Infantil)**** - $9 Youth with Oportunidades $254.7 (one time) $350 (one time) Elderly $19.4 $26 Maximum amount educational transfers $96/$176* Maximum amount transfers $130/$210* *Depending on gender, age, grade level;** As of January 2012 the energy grant was consolidated into the health and nutrition grant; *** previously: Vivir Mejor (Living better); **** previously: Vivir Mejor Infantil (Living better – children) Source: Project PAD and Oportunidades Operating Rules, 2014 20 Expansion and coverage. Since the Project started in 2009, coverage of Oportunidades increased from 5.2 million families to 5.9 million when the Project closed in December 2013, for a total of almost 26 million beneficiaries. The largest expansion came at the time of the additional financing in 2010 when the Program incorporated 600,000 additional families within the first three months of the year. Although two-thirds of the new households came from semi- urban and urban areas, there was also an important expansion to incorporate additional small, isolated, and vulnerable rural areas. During this expansion the Bank, together with the IADB, maintained close dialogue with the program supporting key operational areas important for maintaining program effectiveness. For example, the Bank supported improvements in targeting and the strategy to move as many beneficiaries of PAL as possible into Oportunidades as health services were strengthened to respond to demands generated by the program. 21 Table 1. Oportunidades Cash Transfers (US$ Million) 2008 2009 2010 2011 2012 2013 (retroactive) Program Total 2,656 3,046 3,897 4,171 4,053 4,201 World Bank 450 692 607 97 280 618 World Bank (as 17% 23% 16% 2% 7% 15% % of total) Table 2. Coverage of Oportunidades (2009-2013) Beneficiaries 2008 2009 2010 2011 2012 2013 Program Families 5,049,206 5,209,359 5,818,954 5,827,318 5,845,056 5,922,246 Individuals 23,360,971 23,565,952 26,012,418 25,701,574 25,631,340 25,718,586 21 The target population of Oportunidades and PAL are households classified as extremely poor and those with insufficient resources to invest on human capital (education, nutrition and health) of its members. PAL beneficiaries are selected in areas where the supply of services is not running in full capacity, meaning that Oportunidades co- responsibilities are not implementable. It is important to minimize the risk of overlap of the two programs, in order to avoid situations in which neighboring households in similar conditions are registered in different programs. The Bank supported the work done to develop minimum standards of health services provision in the context of Oportunidades that could then be used to progressively transition as many families as possible from PAL to Oportunidades as services became available. The Project itself did not finance PAL. 21 Box 2. Changes in targeting. In 2010 a law was introduced stating that CONEVAL guidelines to define, measure and identify poverty had to respond to multiple dimensions, and no longer could be limited to a measure of income/consumption. These guidelines were to be followed by social programs targeting the poor. At the same time, following the expansion of the Program to urban areas, targeting errors were increasing given that the targeting instrument had been initially developed and optimized for rural areas. A small technical group with representatives of Oportunidades, the Secretaria de Desarollo Social (SEDESOL), CONEVAL, the World Bank and the IADB was created to pursue the objective of reformulating the targeting instrument for Oportunidades. The group worked well, with full ownership of the Government and strong technical support provided by World Bank staff. The result was a stronger and more flexible targeting instrument that responded to the new vision of multidimensional welfare developed by CONEVAL, and potentially allows social programs with different objectives to use the same targeting instrument. The new instruments improved targeting outcomes and allowed to more easily take into account the need to introduce changes in eligibility criteria, for example due to price changes that impacted welfare status. Component 2: Technical Assistance to Oportunidades (US$2.8 million of loan financing). Component 2 financed activities aimed at providing technical support in the following areas, as set out in the original Project document: (a) Strategies to increase quality and accessibility of health and education services used by Oportunidades families, such as the pilots on early childhood interventions and on parent participation in secondary schooling, as well as the studies looking at the role of education quality and school performance of Oportunidades beneficiaries; (b) Strategies to increase self-sufficiency of Oportunidades families, such as the review of the program’s effects on health, nutrition, education, labor market outcomes in rural areas after ten years of intervention; (c) Actions to further improve the design and operation of Oportunidades, such as the studies on time limits in the Program for beneficiaries and new schemes of transfers/co- responsibilities, the development of an Oportunidades’ specific module for the National Health and Nutrition Survey (ENSANUT) to track beneficiaries’ outcomes, and the study on pertinence of services offered to indigenous population in urban areas; (d) Implementation of the Indigenous Peoples Plan (IPP). This work was identified by the client and was part of their external evaluation agenda, a long- standing feature of the Oportunidades program for which it is widely recognized. Table 3 summarizes key outputs (and results) under this component in each of these focus areas, and key results are summarized here. Supporting an already well-established evaluation agenda and mechanism meant that the Bank needed to get up to speed quickly on previous work and the priorities which had been identified for future work in the short-run. Similarly, the program faced the difficult task of coordinating an extensive area of engagement both externally with the Bank and the IADB, and internally with SEDESOL, Ministry of Education and Health, and CONEVAL. After some initial adjustments, coordination improved through better communication among all actors and an agreed set of procedures. The Government, with inputs from the Bank and the IADB, developed a flowchart 22 ratifying the changes in process for setting, implementing and approving work conducted in the context of the evaluation agenda. These changes were formalized in 2013. The TA was extensive and covered multiple technical areas. The Bank’s efforts to provide substantive technical inputs in a timely fashion were significant. This was challenging at times, considering that multiple vehicles for engagement (PKS, NLTA) were ongoing in parallel to the Project itself, led and implemented by largely the same team even though it successfully managed to tap into specific expertise from other Bank staff when needed (e.g., in areas such as targeting and education). 23 Table 3. Activities, results and comments for Technical Assistance financed under Component 2 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up (a) Strategies to increase quality and accessibility of health and education services used by Oportunidades families Study on Early Childhood • Findings suggest that adding PEI-CONAFE initiatives provides additional, measurable, Follow up survey was delayed, and Development (ECD) Program benefits to the nutritional status and development of Oportunidades beneficiaries’ some budget cuts affected PEI-CONAFE in synergy with children. implementation of PEI-CONAFE, but Oportunidades (Impact • No large differences appear in the way the program is implemented and effective in the follow-up survey was finally evaluation-IE) (2010) indigenous versus non-indigenous communities. conducted in 2012. PEI-CONAFE • The baseline and follow-up surveys show extremely high levels of malnutrition, proved to be an effective delivery Objective: assess the impact of highlighting nutrition and early childhood development as a key area of intervention for mechanism for behavior change in some adding Early Childhood Oportunidades in rural areas of the most vulnerable communities. Stimulation (through the PEI- The Program is considering Early CONAFE program) on nutrition, Childhood Development as a key pillar health, and development of rural of its redesign strategy. indigenous children benefitting from Oportunidades. Study on the role of community • No evidence of impact of training for vocales on relationship between community The pilot was under implementation committees in fostering parent committees/vocales and schools, perception of effectiveness of vocales, types of during a year in which Oportunidades participation in secondary participation of parents in school made substantial changes in its schooling through synergies with • Some important implementation problems: high rotation of vocales (90%) and lack of community committees. The higher than the Apoyo a la Gestion Escolar information on how training was received. average rotation of vocales created (AGE) program (IE) (2010) severe challenges for the success of the pilot. The pilot was a follow up Objective: improve quality of intervention, expanding a model that education and outcomes for had been successful at the primary level Oportunidades beneficiaries to secondary school. The lack of results studying in Telesecundaria schools is likely due to its implementation by fostering parent participation problems, suggesting need for better with the help of Oportunidades planning both of the intervention and its 24 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up community committees (and evaluation. vocales 22). Complementary study on the • Findings suggest that quality of education services is strongly correlated with education The results provide an important quality of education services that gaps – where quality is higher Oportunidades students gap versus non-Oportunidades analytical base to support the Program’s the Program offers its rural students is lower efforts to get more schools with beneficiary population (2009) • Schools attended by Oportunidades students are disadvantaged in their access to Oportunidades beneficiaries in the PEC important inputs and services and AGE programs, as well as efforts Objectives: Expand on the previous • Controlling for variables of context, there is no difference between the test scores of with PRONABES to enable more study which analyzed scores on Oportunidades students and other students Oportunidades students to receive standard tests by type of school for • Factors associated with good quality (“eficaz”) are leadership, good working tertiary level scholarships, contributing Oportunidades students by: (i) relationships among teachers, teacher stability, expectations of the teachers and to higher expectations by both students taking advantage of supplementary students, and infrastructure and teachers. data sources to identify factors that • The study highlights some recommendations to improve quality of education in can impact learning outcomes (and rural/poor areas: training for school directors; introduction of agreements between gaps), controlling for the authorities, schools and parents to maintain school infrastructure/equipment; prioritize socioeconomic context and school programs that incentivize permanence of teachers in rural schools; increase quality of model (“eficaz” and supervision of rural schools; strengthen teacher training for telesecundaria; prioritize “compensatoria”), (ii) understand investment in schools with highest concentration of Oportunidades beneficiaries and and analyze the availability of reverse decrease in budget allocations for compensatory schools; increase the inputs and the education processes number/type of available education transfers for lower and upper secondary by school model type. Study on the performance of • 21% reduction in the Spanish learning gap in primary school among Oportunidades Together with other education studies Oportunidades beneficiaries in beneficiaries on quality of services, it suggests that standardized education tests – • 55% reduction of the gap in secondary schools lower quality of schools strongly affects ENLACE (2012) • Access to inputs and schooling processes/modalities is inversely associated with the education outcomes for Oportunidades percentage of students who are Oportunidades beneficiaries: where the latter proportion beneficiaries. The study also suggests is higher, the quantity and quality of inputs/resources is lower. Between 2008 and 2011, that it is important to continuously the change in distribution of these inputs/resources was small. monitor school performance/outcomes 22 Vocales (world in Spanish) refers to Oportunidades beneficiaries’ mothers selected by the community to support the program implementation at local level. The vocales main role is to ensure that the needed communication related to the program is provided to all beneficiaries in a timely manner. 25 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up • School modalities that show the largest performance gaps - CONAFE schools and for Oportunidades children as this Objective: (i) assess and analyze indigenous schools – did not report positive changes. In these modalities, however, the represent an important instrument to changes in education gaps among difference between beneficiaries’ and non-beneficiaries’ outcomes is very small, while help target interventions to improve beneficiaries between 2008 and socioeconomic conditions among students are not as great as they are in other school education quality. 2011 through ENLACE results, as modalities well as the performance of schools • Greatest changes could be observed for schools using the Telesecundaria modality as a function of the percentage of (83.2% reduction in the gap) and technical secondary schools (44.4% reduction in gap) students in each; (ii) analyze access while in general secondary the reduction was 39.2%. to inputs and different school modalities of different schools. (b) Strategies to increase self-sufficiency of Oportunidaades families Summary document compiling The report summarizes findings of external evaluations on health, nutrition, education, and The summary document was very results of the external labor market outcomes. For the first time, the roles of quality of services and of the important to consolidate and evaluations of Oportunidades Program’s National Coordination were reviewed. The study includes an integrated, thematic communicate the results of a vast during 2007-2008 analysis of both the results and recommendations of the studies. Some key results were the number of evaluations/studies to non- following: experts and decision makers – and to Objective: The evaluation agenda • Reduced malnutrition in children < 2 years old facilitate follow-up on the findings and for 2007-2008 included a set of 17 • Improved cognitive skills in children age 7-11 years old that had participated in the recommendations. studies which focused on the Program since they were infants quality of services (health, • Increased use of health services (especially preventive) education, and National • Extended the education trajectory by as much as 1 year for youth enrolled when young, Coordination) and the and as much as 2 years when compared to previous generation identification of the effects of • Greatest changes in health and education could be observed in women and indigenous Oportunidades in rural areas after populations 10 years of intervention. The • In many instances, longer permanence in the Program was associated with better health objective was to compile and and education outcomes integrate the most significant • The Program still faces significant challenges in reaching its objective of breaking the findings and recommendations of intergenerational cycle of poverty. Nutrition, health, education and job insertion the 2007-2008 external evaluation indicators for the target population still lag significantly behind agenda and assess results achieved • Program implementation and services have improved greatly over time despite the and challenges remaining for the significant expansion in coverage. These operational improvements have been possible Program. in large part thanks to the implementation of the “esquema de microzonas” and the strengthening of the Social Network. These innovations have improved knowledge of the Program among beneficiaries, though significant communication barriers remain • Greater coordination between Oportunidades and other national, State and local 26 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up programs offering complementary services would help create conditions for the economic development that would support efforts to break the poverty cycle in what are often much marginalized areas in which the program operates. (c) Actions to further improve the design and operation of Oportunidades Study on education trajectories • 69% of beneficiaries surveyed had transitioned to secondary, with no difference by The methodology chosen for the study of minors of Oportunidades gender and the timeframe in which it was families (2010) • 56.6% of beneficiaries surveyed also completed lower secondary, and 10% transitioned conducted did not allow drawing to upper secondary conclusive results. This experience Objective: observe and analyze the • Over 80% of beneficiaries had a continuous record of school participation/receipt of highlighted the need for improvements educational trajectories of conditional transfers over the years in planning and supervision of studies. Oportunidades beneficiaries after • In general, being part of the program in earlier ages was correlated with greater However, a significant result of this the first 10 years of participation in progress/completion rates – especially for girls. effort is that the administrative data that the Program, in particular • Some persistent external factors negatively affect outcomes: discrimination and abuse was collected represents an historical examining why some are not on the part of teachers/school authorities, either with children themselves or the parents. baseline against which student follow receiving the education transfers. up could be carried out in the future. Study on Program time limits • Over time, household socioeconomic conditions improve, but at a declining rate The analysis fed into the Program’s and new schemes of transfers • On average, rural and urban households are no longer eligible after 24-32 and 18-27 strategy for recertification and and co-responsibilities (2010) years in the Program, respectively contributed to the determination of a • Evidence suggests that rising above the existing eligibility thresholds does not permanent verification phase” for those Objective: review trajectories of eliminate the risk of falling into poverty in the future households around the eligibility beneficiary families particularly • Different eligibility thresholds were estimated for rural and urban households, as well as threshold. The study also contributed to relating to their socio-economic conditions that would determine eligibility for a “permanent verification phase” for the discussions on the appropriate target characteristics; identify and suggest those households around the threshold. population for the Program and to later transparent and objective criteria • Possible criteria to re-certify beneficiaries were identified, with their respective pros efforts to analyze the gaps in secondary for periodic re-certification of and cons. school attendance. beneficiaries, in line with program • Program should primarily be directed at families with children. The Program needs to objectives. make efforts to improve secondary school attendance rates of beneficiaries and to improve targeting results. 27 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up Study on social mobility of young • Positive impact on social mobility: 40% of youth beneficiaries have a higher qualified The major impacts on social mobility Oportunidades beneficiaries from employment relative to the head of household (47% have similar employment). were through the effects on education, rural households (2010) • Inclusion of migrants in the dataset improved the overall impact on social mobility (and demonstrating the importance of paying on salaries). In the previous studies impact observed was limited attention to the gaps in secondary This study builds on a similar • Upward social mobility is higher for migrants and women school attendance. Results were worse analysis carried out in 2008 that • Length of permanence in the program is not correlated with higher upward social for those who remained in the same pointed at differences in the mobility, but education is (about 7.4% of mobility is explained by education locality, demonstrating the challenges of characteristics of youth that had incentivized through the Program). breaking the inter-generational moved from those who had • The greatest impact of the program on social mobility comes through education. In this transmission of poverty when the remained in their localities. The sense, while attendance is important, increasing the quality of education is critical resident localities are economically study was not able to incorporate • Controlling for education and migration effects, youth that have participated in the depressed – marginalized, rural areas in additional data on youth who had program receive up to 44% more earnings than those that have not and the impact is due Mexico. The results fed into the moved but when better data to the program. discussion on inequality of access to became available the Program tertiary education and the role of updated the previous analysis. scholarships, as well as more broadly on Objectives: (i) determine whether the importance of developing a length of participation in the productive agenda for youth in program has an impact on social challenging context of highly mobility (as determined by a higher marginalized localities. The qualified job relative to head of Government is increasing attention to household) and (ii) determine if this area. youth who have participated in the program have higher labor earnings relative to those that did not participate Oportunidades module developed • Positive impact of Oportunidades over the last 15 years in reducing prevalence of In order to target and reduce for the National Health and stunting and anemia. However, it remains important to strengthen interventions for beneficiaries’ health and nutrition gaps Nutrition Survey (ENSANUT) groups with the highest nutritional vulnerability that still show high stunting and anemia through improvements in coverage and 2011-2012 rates quality of health supply, it is important (2011) • Positive impact/efforts of Oportunidades in increasing health insurance affiliation: 7 out to keep monitoring outcomes. Given the of 10 beneficiaries report affiliation to Seguro Popular. Coverage is greater in poorest Program’s large national coverage, two quintiles, progressive impact being able to conduct a national survey, • Utilization rates of preventive services higher among beneficiaries than non- occasionally strengthened with Objective: Obtain strategic beneficiaries – and lower prevalence of drinking/smoking among adolescent Oportunidades specific sub-modules, is information relative to the impact beneficiaries than non-beneficiaries. Vaccination rates are around 90% for key to this effort. In this case, results 28 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up of Oportunidades on key health Oportunidades children under 5 once again provide useful evidence to and nutrition indicators. • Exclusive breastfeeding rates are higher among beneficiaries than non-beneficiaries , support efforts to improve quality of though overall prevalence remains low services for Oportunidades’ • Quality of services remains a challenge, with roughly 10% of beneficiaries who prefer beneficiaries. to use private health services even though comparable public service is free of charge. Study on profile of indigenous • Prejudice of institutions regarding indigenous people’s ability/capacity to benefit from The Program incorporated some of the population and pertinence of the Program remains a problem recommendations in the revised services offered by Oportunidades • Language barriers prevented beneficiaries from properly understanding information proposal for the urban health attention to indigenous population in necessary to comply with co-responsibilities, and there was evidence of complaints model (MAS - Modelo de Atención en urban areas (2010) from indigenous beneficiaries who were dropped from the Program for reasons that Salud). Findings also provided inputs were never explained to them for the development of the Indigenous Objective: Prepare a profile of the • Lack of familiarities with written language and certain processes creates challenges in People’s Communication Plan (PCI). indigenous population in urban fulfilling necessary paperwork (one key example is related to migration to rural-urban areas, analyze the cultural migration and associated paperwork) pertinence of Oportunidades for • Non-indigenous beneficiaries reported better perception of quality of health services urban indigenous population and than indigenous beneficiaries. Communication and cultural barriers in the provision of identify areas of opportunity to health services are persistent improve the attention and services • Discrimination within the education systems is frequent, and little or no recognition of for indigenous Program the need to take into consideration different culture/language participants in urban areas. • Need to coordinate and work closely with institutions responsible for indigenous population. (d) Implementation of the IPP Evaluation of the 2010 • The PCI provides an innovative instrument to communicate health prevention Roughly 30% of total beneficiaries are indigenous people’s messages in a culturally appropriate way currently making use of materials, communication plan (2011) • Some differences between federal proposal and state level implementation in some which have been successfully revised states following the evaluation. The Program Objectives: In response to evidence • Greater participation of frontline staff working closely with communities would and health authorities plan to further from previous external evaluations improve the design of materials and the selection of topics. expand coverage of the PCI to include on the linguistic and cultural • At the moment the main audience of the materials developed are women in reproductive additional languages, and are also barriers diminishing the age and children < 5, this should be expanded considering innovative ways to measure effectiveness of the Program’s • The materials should also be translated into more languages/variants so they could be the impact of the plan on behavior workshops on health practices, used in more states. Multilinguismo , with several languages spoken in one area, and change. during 2009-2010 the National presence of Spanish speaking staff, represents an issue. Commission for Social Protection in Health designed and 29 Studies and objectives Results and recommendations Comments, Lessons-learned, financed by the Project Follow-up implemented an Indigenous Peoples’ Communication Plan (PCI) The purpose of this study was to validate the materials, methodology, themes and design content of the PCI to ensure that were achieving their intended effect and to identify any obstacles to its implementation at the community level, including the adaption and use by Program staff. 30 The technical assistance provided in support of the evaluation agenda through the Project as well as the related knowledge services (Table 4) helped create the evidence base for a dialogue with the other social sectors on the importance of improving the quality of service provision for beneficiaries to increase the impact of the Program and improve outcomes. This critical area of engagement was supported through technical assistance and policy dialogue during supervision to strengthen coordination with other social programs, and is reflected in the indicators for the second PDO. Since activities in support of these objectives were conducted by the team itself as part of implementation support, they do not come through in the description of activities financed by the Project and are summarized here instead: • Education quality: the Bank TA and policy dialogue supported stronger linkages with the existing school-based management program to improve the quality of public schools, the Programa Escuelas de Calidad, or PEC. The objective was to improve the targeting of PEC (participation in which is voluntary and demand-driven) by supporting participation of Oportunidades schools in the Program. The number of Oportunidades schools participating in PEC increased during Project implementation, from 24,955 (23 percent) in 2006 (baseline), to 28,944 (25 percent) at the time of Project completion. • Access to tertiary education: the Bank TA and policy dialogue supported mechanisms to increase the coverage of the National Scholarship Program for Higher Education (PRONABES) and early interventions among Oportunidades students. During implementation, the number of Oportunidades beneficiaries with PRONABES scholarship increased from 36,941 (15 percent) to 68,181 (20 percent). In the case of tertiary education also the percentage of coverage of Oportunidades population increased in the program. • Health: the Bank TA supported dialogue with the health sector to ensure quality and supply of services provided to beneficiaries during program expansion. In particular, the Bank supported the increase in coverage of beneficiaries by the health insurance Seguro Popular in collaboration with the existing Bank health project. • Finally, the Bank supported the dialogue on issues such as the development of standards for sharing data, strengthening the model for state-level coordination, and the needs for changes in the operating rules of various programs to improve coordination and synergies with Oportunidades. This dialogue is not immediately evident by looking at the list of key outputs or indicators, but should be mentioned in this context. The establishment of sustainable connections with other programs proved challenging. Progress, though satisfactory, was less than had been initially hoped. The challenge was to work with programs such as PEC and PRONABES – in the education sector – and Seguro Popular – in the health sector – that impact the quality and coverage of services for Oportunidades beneficiaries. The Bank and the Government expected to leverage synergies within the World Bank financing itself (across the social protection, health and education teams which all had active projects in Mexico at the time) to better work with the central sectoral agencies on increasing their coverage of Oportunidades beneficiaries. While these represented an especially challenging area for 31 engagement, it was also one that became increasingly meaningful, as evidenced by the fact that it is now the focus of a new World Bank Loan. Table 4. Related Engagements, supporting Technical Assistance outputs Programmatic Knowledge NLTA: Improvements in CCT Programmatic Knowledge Services: Social Protection for Governance SFLAC (P125712) Services: Strengthening Social the Poor (P116169) Protection System (P129698) Revision of the targeting Entry-Exit of beneficiaries. Four Re-designing Oportunidades for instrument for Oportunidades- related studies looking at households increased impacts (promoting technical guidance for the analysis that have exited Oportunidades as a employability) – technical guidance and decision making process result of a Recertification process, document summarizing diagnostic and the possible determinants and options for redesign of the explaining the changes (or lack of) program in eligibility status Improvements in Service Quality Inter-sectoral coordination Broadening options for skills through culturally adapted • Review of the experience of development and employability of communication Oportunidades in coordinating Oportunidades youth – a study of • Video on early childhood with other national programs the alternative education institutions nutrition • Initiatives to strengthen that youth attend, identifying • Validation of training materials coordination between potential operational changes that on Oportunidades Program for Oportunidades and state-level the program could put in place to the health and education sectors social strategies and programs increase its impact and coverage • Communications strategy to orient and link secondary school graduates to complementary services, including the design of a new web page for youth (www.vas.gob.mx) Strategic Review for Model for the coordination of Oportunidades. Updating the long- activities between the local and term strategy for updating the national levels – Technical note analysis of the characteristics of its to explore options to improve the potential beneficiaries (target program’s modalities and population) mechanisms for coordination and articulation with other programs at the local level Support the development of state- Learning from past adjustments level service coordination models in the implementation of the • Review of state experiences on Oportunidades targeting service coordination mechanism – Technical Note • Impact evaluation of Contigo Vamos por Mas, the Guanajuato state program which aims to link Oportunidades beneficiaries with complementary services Qualitative analysis of the trajectories of families who have left the program following the re- certification of beneficiaries – Technical note The role of Oportunidades in the response to natural disasters – Case study 32 Annex 3. Economic and Financial Analysis Since the Mexican Government first introduced Oportunidades (originally, Progresa), social assistance spending in Mexico has increased significantly, and so has its efficiency. Oportunidades has expanded its coverage significantly, but gradually, as budgetary feasibility was always a key consideration. The Government progressively scaled down (and eliminated) food subsidies and replaced them with targeted transfers through Oportunidades and other social programs that have proven impact. The Program continues to be financially sustainable. At the time of appraisal, no economic or financial analyses were conducted. The following factors – which remain relevant today – were considered in determining the efficiency of the Project at the time of preparation and then Additional Financing: • The body of evidence on Oportunidades, based on the numerous external evaluations conducted since the program first started in 1997, shows significant positive impacts of the Program on household consumption, school enrollment and attendance, health prevention/access to health and nutrition – all of which are a part of the Project’s development objectives • The most recent cost-effectiveness analysis 23 was done monetizing the results of these impact evaluations, but was limited to returns to schooling. These simulations show mostly positive results, with positive cost/benefit ratios ranging from 1.4 – 7.1, except in some cases using a 10 percent discount rate. These results can be considered a lower bound on the expected benefits since they do not consider the impacts on health and nutrition (or consumption more generally). • The Project supported improvements in targeting that improved targeting efficiency to the poor. These lead to further improvements in Program effectiveness since studies have shown larger impact of CCTs among households that are poorer at the baseline. 24 Moreover, the Project supported increases in coordination of social policy which are expected to further increase efficiency and returns from investment: • Evaluations have shown that an important factor behind the persistent gaps in outcomes for Oportunidades beneficiaries is the lower quality of services received. Greater coordination with education and health sectors to improve the coverage and quality of services – as evidenced by the efforts to increase coverage of PEC and PRONABES – for Oportunidades population is thus expected to have led to greater impacts on final outcomes for the target population. • The unification of the targeting methodology across social programs, and the consolidation of the Beneficiary Registry of Oportunidades so that it can be used also by other programs administered by SEDESOL are important steps taken by the Government that are expected to progressively generate increased efficiency in overall social spending in Mexico. 23 Parker, S., Behrman, J. (2008) “Seguimiento de adultos jóvenes en hogares incorporados desde 1998 a Oportunidades: Impactos en educación, pruebas de desempeño.” 24 Conditional Cash Transfers: Reducing Present and future Poverty (World Bank, 2009), p. 136. 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Juan Carlos Alvarez Senior Counsel LEGES Patricia M. Bernedo Senior Program Assistant LCSHS-DPT Diana Isabel Cardenas Social Protection Economist LCSHS-DPT Maria E. Castro-Munoz Consultant LCSHH Barbara Hemetsberger Consultant LCSHS-DPT Theresa Jones Lead Operations Officer LCSHS-DPT Alessandra Marini Senior Economist LCSHS-DPT Felix Prieto Arbelaez Senior Procurement Specialist LCSPT Dena Ringold Lead Economist AFTSW Manuel Antonio Vargas Madrigal Lead Financial Management Specialist MNAFM Supervision/ICR Barbara Hemetsberger Consultant LCSHS-DPT Alessandra Marini Senior Economist LCSHS-DPT Gabriel Penaloza Procurement Specialist LCSPT Phillippe George Pereira Senior Social Protection Economist AFTSW Guimaraes Leite Evelyne Eugenia Rodriguez Ortega Consultant LCSEN Juan Carlos Serrano-Machorro Sr Financial Management Specialist LCSFM Maria Concepcion Steta Gandara Sr Social Protection Specialist LCSHS-DPT (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY09 30.18 222,082.40 FY11 (AF) 4.65 39,794.41 Total: 34.83 261,876.80 Supervision/ICR FY09 0.85 2,280.20 FY10 49.61 239,234.67 FY 11 27.29 149,649.69 FY 12 14.13 106,744.41 FY13 19.79 155,497.99 FY14 31.44 113,815.15 Total: 143.11 767,222.10 34 Annex 5. Beneficiary Survey Results (if any) N/A 35 Annex 6. Stakeholder Workshop Report and Results (if any) N/A 36 Annex 7. Borrower's ICR 25 and/or Comments on Draft ICR I. Appraisal of the objective, design, operational experience, and execution of operations a. Objective The Project had the following identical objectives under Loan Agreements 7708-MX and 7968-MX:  Increase capacities in health, nutrition, and education of poor families through human capital investment by promoting regular health check-ups, improving health status, and raising school enrollment and attendance.  Build sustainable connections between Oportunidades and other social programs of the Government of Mexico in order to improve health and education outcomes for Program participants. For the attainment of these objectives, both loans had the same two components: (a) direct support for Oportunidades beneficiary families; and (b) technical assistance through contracts for consultant services. The Project’s objectives were aligned with each of the components of the Human Development Opportunities Program (Programa de Desarrollo Humano Oportunidades – PDHO), referred to here as Oportunidades. Clearly related to the Program’s main overall component, its first objective was addressed through conditional cash transfers aimed at encouraging families to make the human capital investments needed in order to break the intergenerational cycle of poverty. Its second component involved a series of consultancies with the Oportunidades National Coordination that provided direct evidence contributing to: (a) improved quality and increased access to health and education services for the beneficiary population; (b) development of strategies for increasing the self-sufficiency of beneficiary families; (c) improvement of the design and operation of Oportunidades, and (d) implementation of the Indigenous Peoples Plan (Plan Indígena). It can be said that the Project’s first objective-component alignment has been satisfactory. The mechanism that defines Oportunidades and has enabled it to achieve the outcomes reported in various external evaluations has been the conditions attached to the cash transfers, whereby certain behaviors have been expected of the beneficiary families, including behaviors aimed at improving school attendance, food availability, and the health and nutritional status of the younger members of these families. The second objective-component alignment is considered to have been sufficient for the development of pilot-scale activities related to the design, planning, and evaluation of coordination and linking mechanisms proposed under the Project’s second objective. However, it is felt that this second alignment has been limited in terms of guaranteeing the 25 The original document, in Spanish, is in the Project files. 37 implementation and continuity of these coordination and linking activities, which have only been designed, planned, and implemented on a pilot scale. This observation raises concern especially when the activities in question are carried out with entities or agencies that do not participate directly in Oportunidades operations and are not party to the commitments assumed within the framework of the Loan Agreement. b. Design Oportunidades is a conditional cash transfer program. According to its Operational Rules, project execution involves three sectors: social development (Ministry of Social Development, through the Oportunidades National Coordination), education, and health. The institutional arrangements agreed upon for execution of the Oportunidades Project were appropriate for ensuring overall Project execution because they provided for participation by the sectors involved in operating Oportunidades pursuant to its Operating Rules. The Rules, in turn, are part of the Operating Manual for project execution, which also includes provisions governing financial administration and contracts, as well as the Indigenous Peoples Plan, prepared in accordance with World Bank policy. With regard to arrangements for ensuring that project execution complies with the Bank’s policy for preventing and combating fraud and corruption in projects financed by IBRD loans and IDA credits and grants (Guidelines dated October 15, 2006), the following specific provisions were established for ensuring the Bank that this policy will be observed, though under specific conditions that are explained in the following paragraph. During the course of Project execution under the Oportunidades Financial Support Category, the National Coordination had specific agreements with external agents (called settlement institutions) with which it subcontracted the service of delivering the cash transfers associated with financial expenditures under this heading. Since these contracts were not legal documents derived from the language of the lending agreement (i.e., legal covenants) but rather documents signed in the normal course of Oportunidades operations, the Bank did not have (nor could it have had) any legal relationship that would permit it to audit or directly review the Project’s records as required under the anti-corruption policy. Moreover, the financial resources granted by the Bank do not flow through the National Coordination but rather are dispensed directly by the Federal Treasury in the form of disbursements to cover eligible expenditures. Given these conditions, In order to comply with the anti-corruption policy, specific arrangements were established which included notifying the settlement institutions that, according to the policy, they are considered “channels” or “recipients” for the flow of funds identified with the financial resources that the Bank has granted for the Project, albeit indirectly. Key Project Indicators As indicated in the ICR (Section 3.2, Achievement of Project Development Objectives), the choice of some of the key project indicators was somewhat ambitious, based on the Bank’s interest in tracking progress in activities or interventions outside the direct control 38 of the National Coordination—for example, indicators related to the Quality Schools Program (Programa de Escuelas de Calidad—PEC), the National Program for Higher Education Scholarships (Programa Nacional de Becas para la Educación Superior— PRONABES), and implementation of a pilot Early Childhood Intervention Program (Programa de Educación Inicial—PEI) under the National Council on Educational Development (Consejo Nacional de Fomento Educativo—CONAFE). This last indicator was felt to be especially important because, even though there were funds allocated for an appraisal of this intervention under the Consultation Services Category, by the end of the Project it had not been possible to scale up or expand early education interventions in the Oportunidades beneficiary population, either because CONAFE was unable to serve this population or because there was no specific provision in the legal language of the Project documents that required CONAFE to take action in light of the findings from the appraisal that was funded for the Project. The lesson learned from this experience, or at least a suggestion for future technical assistance operations, is to involve all stakeholders that are in a position to ensure, and to commit to taking the necessary steps for, successful project execution starting from its design and early preparation, whether as co-executors or as participating entities that have a commitment or a mechanism for ensuing their participation in and responsibility for project activities. Operating Costs Category Loan Agreement 7968-MX (Additional Financing) included one more Investment Category in addition to those established under Loan Agreement 7708-MX—namely, Operating Costs to fund the administrative costs (salaries) of National Coordination personnel responsible for project administration. However, this type of expenditure was not accepted as fundable with financial resources granted by the Bank, inasmuch as it corresponded to the Ministry of Treasury and Public Credit. Therefore the funds allocated for this category had to be transferred to the Oportunidades Financial Support Category. This situation did not pose any risk to project execution: the transferred funds were disbursed, and the amount involved was very small relative to total funding (0.04%). However, this situation represents a lesson that may be taken into account in future assistance operations and financing by the Bank, since it is important to include, as part of a project, funds that may be identified in the budget as possibly being covered by resources from an external financing source. Indigenous Peoples Plan The Indigenous Peoples Plan prepared for the Project calls for several actions that fall under the responsibility of the health sector—for example, the Indigenous Communication Plan (Plan de Comunicación Indígena)—and the education sector (e.g., the inclusion and participation of Oportunidades beneficiaries in PEI-CONAFE. However, no mechanism was provided to guarantee fulfillment of these actions by the sectors in question. In the case of the Indigenous Communication Plan, the health sector took responsibility and carried out the activities specified in the plan as part of its overall commitment to 39 Oportunidades, but the PEI-CONAFE intervention was never implemented, much less expanded to include Oportunidades beneficiaries. A similar lesson, or suggestion for future technical assistance operations, can be learned with regard to the Project’s key indicators—namely, to involve all the stakeholders that might be in a position to ensure and commit to implementation of the actions needed in order to ensure successful project execution. c. Operational Experience and Execution of Operations According to the National Coordination (the Project’s implementing agency), the operational experience and execution of operations were satisfactory except for the change in the budgetary allocation of funds for operating the educational component (in which expenditures eligible for the Oportunidades Support Category were identified), which starting in fiscal year 2012 were assigned directly to the Ministry of Public Education and not to CONAFE. The decision to change the budgetary allocation of the resources for the Oportunidades educational component was taken by the education sector without considering its implications for project execution and without prior consultation with the other stakeholders that the borrower had involved in execution and supervision of the Project (the Ministry of Treasury and Public Credit, in particular its International Financial Affairs Unit (Unidad de Asuntos Internacionales de Hacienda), Nacional Financiera, S.N.C., and the administrative unit in the National Coordination responsible for administration of the Project). As a result, it was necessary to amend the Loan Agreement (7968-MX– Additional Financing), and the process was not completed until October 2012—in other words, 10 months after the start of the fiscal year. It is important to point out that the amendment to Loan Agreement 7968-MX (Additional Financing) was also necessary because of changes in the system for the payment of cash transfers under the Oportunidades Support Component. Starting in 2012, the National Coordination was only allowed to use the services of a settlement institution (Banco del Ahorro Nacional and Servicios Financieros, S.N.C.) to transfer cash support to Oportunidades beneficiaries. Once this change was reflected in the legal language of the loan agreement, it did not constitute a risk for the maintenance of an adequate financial administration system for the Project. II. Evaluation of the operational outcome in terms of the agreed objectives First Development Objective The funds granted by the Bank did not flow directly to the fiscal year budget authorized for operating the Oportunidades Program. Instead, they were received by the borrower (in the form of disbursements once the eligible expenditures had been made) through Nacional Financiera S.N.C. and then transferred to the Federal Treasury. 40 Still, indirectly, the funds granted by the Bank contributed to meeting the objective of building the capacity of Oportunidades beneficiaries in the areas of health and nutrition, since the funds were associated with expenditures already made under the Oportunidades Investment Support Category and disbursed accordingly. Table 1 in Annex 2 of the ICR shows the proportion represented by Bank funding of cash transfers to Oportunidades beneficiaries. Second Development Objective With regard to the second of the Project’s objective-component alignments mentioned in the previous section, a number of consultancies addressed the four broad topics considered in the Technical Assistance Component of the Project. The work performed under these consultancies is described below.  An external evaluation reviewed two pilot-scale projects aimed at improving the quality of health services and education and increasing access by the population served. Some of the projects implemented and evaluated were: (a) Community Promotion Committees (Comités de Promoción Comunitaria—CPCs), the School Management Support Program (Programa de Apoyo a la Gestión Escolar—AGE) under CONAFE, and distance secondary education (Escuelas Telesecundarias) aimed at promoting the participation of Oportunidades parents in secondary schools under the AGE program; and (b) early child development and nutrition for children under 4 years of age, designed to promote the participation Oportunidades families with children under 5 years of age in the CONAFE Informal Early Education Program (Programa de Education Inicial No Escolarizada—PEI- CONAFE]) to ascertain whether this connection actually raises levels of nutrition, health, and cognitive, motor, speech, social, and emotional development in rural children who are Oportunidades beneficiaries.  Two other consultancies also focused on rural areas and studied (a) temporary employment and new support schemes for beneficiary families based on shared responsibility; and (b) intergenerational social mobility of Oportunidades beneficiaries in rural areas. These consultancies yielded objective information for developing strategies to raise the level of self-sufficiency and improve the well- being of Oportunidades beneficiaries.  Consultancies on the following topics sought to obtain evidence to identify improvements in the design and operation of Oportunidades: (a) school performance of Oportunidades children and young people in rural areas; (b) compilation of data from the 2008 external appraisal of Oportunidades; (c) study of an Oportunidades program module in the National Health and Nutrition Survey 2011-2012 (Encuesta Nacional de Salud y Nutrición—ENSANUT); and (d) performance of Oportunidades scholarship winners in the ENLACE test, including changes between 2008 and 2011 in basic and lower secondary education. These consultancies gave the Oportunidades National Coordination the elements it needed to update its assessment of the population served in terms of the main indicators of health, food availability, nutrition and education, as well as objective elements for 41 (a) updating the diagnosis of the problem that the program addresses; and (b) developing a proposal to adjust the program design in order to address the changes identified in said diagnosis.  The following two consultancies were part of the activities under the Indigenous Peoples Plan: (a) a study of the beneficiary indigenous population in urban areas; and (b) an appraisal of the Indigenous Communication Program (Programa de Comunicación Indígena—PCI] implemented by the health sector. The results of the first consultancy gave the Oportunidades National Coordination a clear characterization and an overall diagnosis of the indigenous population living in cities and metropolitan centers, as well as an understanding of the challenges involved in providing care for this population group in urban areas. The second consultancy helped to improve the cultural appropriateness of the PCI materials, images, and messages on health and nutrition being used by the health sector in predominantly indigenous localities in the southern part of the country. The results of these consultancies helped enable the Oportunidades National Coordination to identify areas for improvement in the design and operation of the Program’s interventions in the areas of food availability, health, nutrition, and education. In keeping with national law, 26 the recommendations and areas for improvement identified in these appraisals and studies were analyzed, sorted, and classified for further action and reflection with a view to improving the Program’s design or operation. III. Evaluation of borrower performance and the project implementing agency during preparation and execution of operations, with special emphasis on lessons learned that could be useful in the future According to the National Coordination, borrower performance and its own performance were considered satisfactory during the preparation and execution phases of operations, with the exception of CONAFE participation, given that it dropped out of the project because of the change in the budgetary allocation of funds for the for the educational component of the Oportunidades Program beginning in fiscal year 2012. It is important to mention the participation of Nacional Financiera as the settlement agent designated by the borrower to administer and supervise project execution, especially its work with the National Coordination in the handling of contracts, disbursements, and overall project supervision. IV. Evaluation of World Bank performance during project preparation and execution of operations, the effectiveness of its relations, and lessons learned 26 The National Council on Social Development Policy Evaluation [Consejo Nacional de Evaluation de la Política de Desarrollo Social CONEVAL], working together with the Secretariat of the Treasury and Public Credit and the Secretariat of Public Administration, establishes the norms governing the process of improving federal social programs through its mechanism for monitoring areas subject to improvement identified in reports and evaluations of federal programs under Federal Public Administration. Information on the Oportunidades Program is available at: http://evaluacion.oportunidades.gob.mx:8010/es/asm/asm2011.php 42 According to the National Coordination, the Bank’s performance during the preparation and execution of operations was satisfactory, as was the participation of the project team, the specialists in financial administration and procurement, the legal team (during preparation of the legal language of the two loan agreements, 7708-MX and 7968-MX), and the experts in disbursements. During project execution, the Bank’s team was prompt in responding to questions, requests, and normal situations, supervising the operation on an ongoing basis and ensuring attainment of the development objectives. Other Lessons Learned Consultancies and appraisals on a broad variety of topics generated evidence for the effectiveness of some of the interventions tied to other federal programs and the performance of Oportunidades in the key indicators of its three components. The National Coordination identified the following lessons learned from its technical assistance activities:  The importance of implementing actions for early childhood stimulation in order to strengthen the Oportunidades results already observed in terms of the nutritional status of young children and making a positive impact on the various dimensions of child development, especially in the indigenous population;  The importance of strengthening the training activities of Community Promotion Committees to improve the effectiveness of the messages being communicated to the beneficiary population;  The progress made by Oportunidades in improving school attendance by children and young people, noting that this progress has not yet been reflected in substantial improvements in the indicators of social mobility, especially in rural areas;  The depth of the poverty in which most of the beneficiary families live and how it impacts on the time required to remain in the program in order to raise people out of this condition;  The progress already observed in the Oportunidades population in the indicators of food availability, health, and nutrition, as well as the improved grades of schoolchildren; and  Elements that require: (a) appropriate cultural attention in working with indigenous people living in urban areas; and (b) more focused attention in existing interventions, especially in the area of health. While the information and analyses emanating from these consultancies, appraisals, and studies have been of great value for designing, planning, and improving the coordination and ties that the Oportunidades National Coordination relies on, it is felt that it would also be valuable to have consultant activities that are more directly related to the implementation and execution of coordination and linking mechanisms. 43 Completion Report of the Borrower in Spanish 44 45 46 47 48 49 50 51 52 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A 53 Annex 9. List of Supporting Documents World Bank (2004), Country Partnership Strategy with the United States of Mexico FY05-08, Report No.28141-ME. World Bank (2008), Country Partnership Strategy with the United States of Mexico FY08-13, Report No.42846-MX. World Bank (2014), Country Partnership Strategy with the United States of Mexico FY14-19, Report No. 80800-MX. IEG, The World Bank Groups’ Response to the Global Economic Crisis, Phase II (2012). World Bank (2009), Conditional Cash Transfers: Reducing Present and future Poverty (2009) World Bank (2010), Poverty Reduction and Economic Management Sector Unit Latin America and Caribbean Region, Recent Trends and Forecasts of Poverty in Mexico, October 2010 World Bank (forthcoming), Understanding the Poverty Impact of the Global Financial Crisis in Latin America and the Caribbean, edited by Margaret Grosh, Maurizio Bussolo, Samuel Freije, Parker, S., Behrman, J. (2008) “Seguimiento de adultos jóvenes en hogares incorporados desde 1998 a Oportunidades: Impactos en educación, pruebas de desempeño.” World Bank (2005), Mexico: School Based Management Project (P115347), Report No. 33894- ME, Project Appraisal Document, November 14, 2005, Washington, D.C. World Bank (2005), Mexico: Tertiary Education Student Assistance APL Project (P085593), Report No: 3 1727-ME, Project Appraisal Document, November 15, 2005, Washington, D.C. World Bank (2010), Mexico: Support to the Social Protection System in Health Project (P116226), Report No.52142-MX, Project Appraisal Document, February 23, 2010, Washington, D.C. Study on Early Childhood Development (ECD) Program PEI-CONAFE in synergy with Oportunidades (Impact evaluation-IE) (2010) Study on the role of community committees in fostering parent participation in secondary schooling through synergies with the Apoyo a la Gestion Escolar (AGE) program (IE) (2010 Complementary study on the quality of education services that the Program offers its rural beneficiary population (2009) Study on the performance of Oportunidades beneficiaries in standardized education tests – ENLACE (2012) 54 Summary document compiling results of the external evaluations of Oportunidades during 2007- 2008 Study on education trajectories of minors of Oportunidades families (2010) Study on Program time limits and new schemes of transfers and co-responsibilities (2010) Study on social mobility of young Oportunidades beneficiaries from rural households (2010) Study on profile of indigenous population and pertinence of services offered by Oportunidades to indigenous population in urban areas (2010) Evaluation of the 2010 indigenous people’s communication plan (2011) 55 MAP 56