The World Bank Results in Nutrition for Juntos SWAp ( P117310 ) Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004175 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON THE IBRD-79610 LOAN IN THE AMOUNT OF US$ 25 MILLION TO THE REPUBLIC OF PERU FOR THE RESULTS IN NUTRITION FOR JUNTOS SWAP ( P117310 ) November 30, 2017 Social Protection & Jobs Global Practice Latin America And Caribbean Region The World Bank Results in Nutrition for Juntos SWAp (P117310) CURRENCY EQUIVALENTS Exchange Rate Effective November 30, 2017 Currency Unit = Peruvian Sol (PEN) 3.23 PEN = 1 US$ 1.42 US$ = 1 SDR FISCAL YEAR July 1 - June 30 Regional Vice President: Jorge Familiar Calderon Country Director: Alberto Rodriguez Senior Global Practice Director: Michal J. Rutkowski Practice Manager: Pablo Gottret Task Team Leader(s): Nelson Gutierrez ICR Main Contributor: Michele Davide Zini ii The World Bank Results in Nutrition for Juntos SWAp (P117310) ABBREVIATIONS AND ACRONYMS CCT Conditional Cash Transfer CLAS Comités Locales de Administración de Salud (Local Committees for Health Administration) CPS Country Partnership Strategy CPF Country Partnership Framework CRED Control de Crecimiento y Desarrollo (Health Check-ups) CUI Codigo Unico de Identificacion (Unique Identifier Code – in the birth certificate) DGPP Dirección Nacional Presupuesto Público (National Directorate for Public Budget) DIRESA Dirección Regional de Salud (Regional Health Directorate) DNI Documento National de Identidad (National Identity Number / Document) DO Development Objectives DPL Development Policy Loan ECD Early Childhood Development ENCRED Encuesta a Establecimientos de Salud en la Atención del Control de Crecimiento (Survey of Health Centers on Growth Monitoring) GoP Government of Peru INEI Instituto Nacional de Estadística e Informática (National Institute for Statistics and Informatics) IPSF Indigenous Peoples Strategic Framework MEF Ministerio de Economía y Finanzas (Ministry of Economy and Finance) MIDIS Ministerio de Desarrollo e Inclusión Social MINSA Ministerio de Salud (Ministry of Health) NLTA Non-Lending Technical Assistance PAN Programa Articulado Nutricional (Articulated Nutrition Program) PBB Performance-Based Budget RENIEC Registro Nacional de Identificación y Estado Civil (National Registry of Identification and Civil Status) SIS Seguro Integral de Salud (Integral Health Insurance Program) SWAp Sector-Wide Approach UCPS Unidad Ejecutora de Proyectos Sectoriales (Sectoral Operations Coordinating Unit) iii The World Bank Results in Nutrition for Juntos SWAp (P117310) TABLE OF CONTENTS DATA SHEET ........................................................................................................................... I I. OPERATION CONTEXT AND DEVELOPMENT OBJECTIVES .................................................. 1 II. OUTCOME ...................................................................................................................... 4 A. RELEVANCE OF PDOs ........................................................................................................................ 4 B. ACHIEVEMENT OF PDOs (EFFICACY) ................................................................................................. 5 C. EFFICIENCY ........................................................................................................................................ 9 D. JUSTIFICATION OF OVERALL OUTCOME RATING ........................................................................... 11 E. OTHER OUTCOMES AND IMPACTS ................................................................................................. 11 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 12 A. KEY FACTORS DURING PREPARATION ............................................................................................ 12 B. KEY FACTORS DURING IMPLEMENTATION ..................................................................................... 13 IV. BANK PERFORMANCE AND RISK TO DEVELOPMENT OUTCOME ....................................... 14 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................................................... 14 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ........................................................... 15 C. BANK PERFORMANCE ..................................................................................................................... 16 D. RISK TO DEVELOPMENT OUTCOME ............................................................................................... 17 VI. LESSONS AND RECOMMENDATIONS ............................................................................. 18 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 22 A. RESULTS INDICATORS .................................................................................................................... 22 B. KEY OUTPUTS BY COMPONENT ..................................................................................................... 26 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 29 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 30 iv The World Bank Results in Nutrition for Juntos SWAp (P117310) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P117310 RESULTS IN NUTRITION FOR JUNTOS SWAP ( P117310 ) Country Financing Instrument Peru Specific Investment Loan Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Ministry of Economy and Finance Republic of Peru Project Development Objective (PDO) The proposed operation has been designed to support the demand, supply, and governance of nutrition services provided by the Government of Peru (GoP), organized under the Articulated Nutrition Program (PAN). The objectives of the proposed operation are to: (i) increase demand for nutrition services by strengthening the operational effectiveness of the Juntos Conditional Cash Transfer (CCT) program; and (ii) improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos. Activities to support both objectives include a strong emphasis in promoting good governance to monitor the impact of the Government Programs and of the proposed intervention in nutrition outcomes. FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing IBRD-79610 25,000,000 25,000,000 25,000,000 Total 25,000,000 25,000,000 23,256,569 Non-World Bank Financing Borrower 29,000,0001 0 0 Total 29,000,000 0 0 Total Project Cost 54,000,000 25,000,000 25,000,000 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 08-Mar-2011 12-Jan-2012 07-Apr-2015 31-Mar-2016 30-Jun-2017 1 The US$29,000,000 borrower’s financing was defined at approval by estimating the budgetary contribution of the Peruvian Government to the Articulated Nutrition Program (PAN). However, monitoring the disbursement of these resources at the regional level proved to be too difficult and the borrower’s contribution was revised to 0. It should be however noted that the Peruvian Government ultimately devoted more than US$ 230 nationally between 2012 and 2017 alone to the implementation of the PAN, and more than US$ 29 million in the three regions covered by this operation (Amazonas, Cajamarca, and Huanuco). See https://www.mef.gob.pe/contenidos/presu_publ/documentac/PE0001_Programa_Articulado_Nutricional.pdf and “Informe de Proyecto – 2016” prepared by the Peruvian Ministry of Economy and Finance. i The World Bank Results in Nutrition for Juntos SWAp (P117310) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 10-Jul-2013 0 Change in Results Framework - Other Change(s) 26-Feb-2014 7.28 Change in Results Framework 26-Nov-2014 7.28 Other Change(s) 14-Jan-2016 18.02 Change in Loan Closing Date(s) 17-Sep-2016 23.26 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 08-Aug-2011 Satisfactory Satisfactory 0 02 13-Mar-2012 Satisfactory Satisfactory 0 03 13-Nov-2012 Moderately Satisfactory Moderately Satisfactory 0 04 17-Jun-2013 Moderately Unsatisfactory Moderately Unsatisfactory 0 05 24-Dec-2013 Moderately Satisfactory Moderately Satisfactory 7.28 06 22-Jul-2014 Moderately Satisfactory Moderately Satisfactory 7.28 07 04-Feb-2015 Moderately Satisfactory Moderately Satisfactory 7.28 08 10-Oct-2015 Satisfactory Moderately Satisfactory 11.72 09 05-May-2016 Satisfactory Satisfactory 18.02 10 03-Nov-2016 Satisfactory Moderately Satisfactory 23.26 11 22-Jun-2017 Moderately Satisfactory Moderately Satisfactory 23.26 ii The World Bank Results in Nutrition for Juntos SWAp (P117310) SECTORS AND THEMES Sectors Major Sector/Sector (%) Social Protection 100 Social Protection 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Social Development and Protection 60 Social Protection 60 Social Safety Nets 60 Human Development and Gender 40 Nutrition and Food Security 40 Nutrition 20 Food Security 20 ADM STAFF Role At Approval At ICR Regional Vice President: Pamela Cox Jorge Familiar Calderon Country Director: Carlos Felipe Jaramillo Alberto Rodriguez Senior Global Practice Director: Michal J. Rutkowski Practice Manager: Helena G. Ribe Pablo Gottret Task Team Leader(s): Alessandra Marini Nelson Gutierrez ICR Contributing Author: Michele Davide Zini iii The World Bank Results in Nutrition for Juntos SWAp (P117310) I. OPERATION CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context2 At the time of appraisal, Peru had witnessed a decade of high growth and declining poverty. Between 2005 and 2008, the country was one of the strongest performers in the Latin America and Caribbean Region, with growth accelerating from 6.4 percent in 2005 to 9.8 percent in 2008. Although GDP growth fell to 0.9 percent in 2009 in the wake of the global financial crises, Peru was one of the few economies in the region that avoided a recession and posted an 8 percent growth rate in 2010. Poverty, albeit still high by the late 2000s, fell from 48.7 percent in 2005 to 34.8 percent in 2009, and extreme poverty from 17.4 percent to 11.5 percent during the same period. Despite the progress, human development outcomes, and especially children’s chronic malnutrition,3continued to lag. After stagnating throughout the 1990s, the country’s stunting rate began to decline in the early 2000s, falling from 30 percent but only to a still worrisome rate of 23.8 percent in 2009. In parallel, between 2005 and 2009, an alarming increase in stunting in urban areas was recorded, from 13.5 to 16.2 percent. Great variations in chronic malnutrition rates continued to exist across regions, ranging from a high 53.6 percent in Huancavelica, in the south-west of Peru and the poorest region in the country, to a low 2.1 percent in Tacna, close to the Peruvian-Chilean border. Failing to tackle the challenge of chronic malnutrition would have affected the country’s productivity and future economic growth. Malnutrition can cause permanent damage to a child’s intellectual development and irreversible losses of human capital formation. With about half a million children starting life with such a large disadvantage in the early 2000s, Peru was severely limiting its ability to make further progress towards reducing poverty and inequality, and to sustaining growth in the long term. In the years leading to the operation’s appraisal, the country had developed a high and widely shared commitment towards reducing malnutrition. In the mid-2000s, the Peruvian Administration had placed nutrition at the forefront of its social policy with the 5-by-5-by-5 initiative, a pledge to reduce by 5-points- in-5-years (by 2011) chronic malnutrition in children under 5. Accordingly, to promote the integration of nutrition initiatives, in 2007 the Government of Peru (GoP) launched the CRECER strategy, an inter- institutional platform under the leadership of the Inter-Ministerial Commission for Social Affairs to coordinate malnutrition interventions. At the same time, the Ministry of Economy and Finance (MEF) formalized the commitment to achieving better results and greater accountability around nutrition outcomes by including the Articulated Nutrition Program (Programa Articulado Nutricional - PAN in Spanish) among the programs to be monitored within the Performance-based Budgeting (PBB) pilots. PAN offered the opportunity to break the institutional budgetary inertia and assign budgetary resources as a function of achieving better results in nutrition rather than on the basis of historical allocation.4 By 2009 2 This section is based on the Project Appraisal Document and on Marini, A., Rokx, C. and Gallagher, P. (2017), Pg. 31, “Standing Tall: Peru’s Success in Overcoming its Stunting Crisis”, World Bank. http://documents.worldbank.org/curated/en/815411500045862444/pdf/117053-OUO-FINAL-Peru-book-FA-SINGLE-PAGES- with-cover.pdf 3 Chronic malnutrition and stunting are often used as synonyms. Stunting is defined as when a child’s height is more than two standard deviations below the mean for its age and sex. See Marini et al. 2017. 4 The PAN was set up as the key strategic programs to be monitored within the Performance-based Budgeting (PBB) pilots implemented by the Ministry of Economy and Finance. The objective of PBB through the PAN was to concentrate the different 1 The World Bank Results in Nutrition for Juntos SWAp (P117310) budgetary programming for nutrition had already improved, and the allocation of resources for regions where malnutrition rates were highest increased significantly. Only the most impactful stunting reduction tools were allocated money, and hard data became critical in evaluating programs, achievements, and therefore funding. By 2010, and from then on, budgetary allocations and malnutrition rates were directly related. This occurred in part thanks to EUROPAN, a 60 million Euro grant from the European Community to support the goal of the in the three poorest regions of Peru (Huancavelica, Apurimac, and Ayacucho) through direct budget support to selected regional Governments based on progress in the provision of health checkups (Control de Crecimiento y Desarrollo – CRED in Spanish) and child immunizations. Theory of Change (Results Chain) International evidence called for a multi-sectoral approach mixing demand and supply-side interventions to achieve rapid results and to prevent new generations of children from being malnourished. Malnutrition is a complex problem related to high prevalence of infectious disease, inadequate feeding and caring practices, and low birth weight. However, the lack of awareness of the problem —and of the solution— by parents (especially mothers), the lack of accountability from providers, and the weak incentives to improve services, also contribute to rampant malnutrition. Experiences from other countries showed that reductions in malnutrition had been achieved through a multi-sectoral combination of demand-side interventions (e.g. Conditional Cash Transfer – CCT programs) and measures aimed at increasing the coverage and quality of health and nutrition services to complement the anticipated income effect arising from CCT programs. Promoting mothers’ behavioral changes with respect to child health, hygiene, care and feeding practices, also proved to be cost-effective and impactful. The operation was designed along a very clear results chain in which multi-sectoral interventions aimed to contribute to achieving the ultimate long-term outcome of lowering child malnutrition in focus areas. • On the demand side, the operation sought to improve the administrative capacities of the Juntos program, which provides cash to poor mothers in exchange for taking their children regularly to health, growth and nutrition check-ups, and ensuring that their children attended school (the beneficiaries’ co-responsibilities). By increasing the affiliation to Juntos of children younger than 12 months, and improving the verification of co-responsibilities of children younger than 36 months already affiliated to the program, the operation aimed to focus on the most crucial period of a child’s development, and stimulate the demand for health and nutrition services while changing poor households’ behavior with respect to early childhood nutrition. The change theory was that as cash reached poor households, demand for better health and social services in poor and rural areas would increase, as well as demand for food richer in nutrients and for caring practices around children’s health and nutrition. This was to lead to an improved diet, and ultimately to lower malnutrition rates. efforts (budgetary, logistical, and organizational) in those regions with the highest malnutrition rates. According to Marini et al., 2017, page 31 and 32, “The PAN program gave the regions the green light to increase their budgets for nutrition by 50 percent if they met targets. PAN focused on priority “products” in its drive to reduce stunting. First, it made vaccination of children against pneumococcus and rotavirus a priority. Rather than choosing 20 vaccines, Peru chose to focus on just two. This was based on international experience, which showed that these two vaccines would have the biggest impact on reducing chronic malnutrition. Second, it spent money on quality health services to ensure children grew properly. Third, it invested in reducing iron deficiency by providing supplements to families. Crucially, it did so by shifting public spending to target regions where malnutrition rates were the highest, such as Huancavelica. (…) The success of PAN rested on three pillars. The first pillar was the use of result-based budgeting. The second pillar was a unified approach. No one ministry or public body had “ownership” of nutrition. It was a shared priority and a shared responsibility. “They understood that the PAN was not owned by any sector but was a state program. (…) The third pillar was specialized training for public servants, including in the local offices of MEF, to ensure understanding and knowledge to tackle chronic malnutrition.” 2 The World Bank Results in Nutrition for Juntos SWAp (P117310) • On the supply side, the operation was designed to support the improvement of the delivery capacity of health providers, providing an incentive to local clinics to increase the quantity and the quality of prioritized preventive health and nutritional services for children under 12 months (CRED5, vaccinations, and demonstration sessions for parents on preparing nutritious and healthy food). More and better quality health services were to lead to greater prevention and lower incidence of infections and diseases among poor children, ultimately resulting in lower malnutrition in selected communities; • Finally, the operation aimed to strengthen the GoP's capacity to influence nutritional outcomes by improving budgetary planning and monitoring of results. By supporting the recently-established performance-based budgeting approach, and fostering the collection, maintenance, and use of up-to- date data on key nutritional and health outcomes, the operation sought to further increase the quality of health services, ultimately contributing to reducing malnutrition. PBB “led not only to a better allocation of financial resources but also to a change in organizational culture not only within MEF but also in the entities that needed to understand that the goal of the resources we were assigning had a final purpose” (Marini et al., 2017, pg. 29). Project Development Objectives (PDOs) In line with the results chain described above, the operation was designed to support the demand, supply, and governance of nutrition services provided by the GoP. The development objectives of the operation were to: (i) increase demand for nutrition services by strengthening the operational effectiveness of the Juntos CCT program; and (ii) improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos and targeted by this operation (Amazonas, Cajamarca, and Huánuco)6. Key Expected Outcomes and Outcome Indicators The operation supported the achievement of reducing chronic malnutrition in Peru. The key outcome indicators were: • (Increase in the) percentage of children under 12 months that received the complete CRED scheme according to their age in the areas of intervention of the Juntos Program targeted by the operation. This PDO indicator was, in a way, the most important as it focused on the first 12 months of a child’s life, the most critical period from a development perspective; • (Increase in the) percentage of children under 36 months that received complete CRED scheme according to their age in the areas of intervention of the Juntos Program targeted by the operation. This indicator, as reported in the operation’s Implementation Status Reports (ISRs) is considered a “control” indicator, and was intended to monitor the stability of the delivery of complete CRED schemes nationwide. 7 5 Controlo de Crecimiento y Desarollo, a program supporting parents in tracking a child's growth, health and nutrition and providing counseling to foster behavioral changes through community-based demonstration sessions. 6 These three regions were selected to complement the interventions of the EUROPAN project, which pre-dated the operation and covered the three poorest regions in the country (Apurimac, Ayacucho, Huancavelica). The regions of Amazonas, Cajamarca, and Huánuco were the next three poorest in at appraisal. 7 The “main” PDO indicator of the project is considered to be the one focusing on child younger than 12 months as these are the most critical for a child’s future development. The first 12 months count for 11 out of a total of 21 CRED visits in the first 3 years of age. On the other hand, the use of the second indicator, children under three years with completed CRED scheme, was considered to be a "control" indicator as a desirable threshold. The goal of this PDO was established with the objective of tracking CRED’s coverage after the first year of a child’s life. MINSA’s standard is set at 4 CRED visits during the second year, and 1 visit during the third year of age. 3 The World Bank Results in Nutrition for Juntos SWAp (P117310) Components The operation was structured around 3 components: • Component 1 - Support the strengthening and consolidation of the design and operational capacity of the Juntos program for families with children younger than 36 months. The activities executed under this Component sought to stimulate the demand for health and nutrition services by Juntos’ beneficiaries. The Component’s funds were disbursed based on outputs and “reimbursed” the borrower upon the achievement of pre-established results related to increasing early affiliation to Juntos, and the verification of beneficiaries’ compliance with the program’s co-responsibilities; • Component 2 - Improve the coverage and quality of the provision of basic preventive health and nutrition services in the Juntos areas. The funds allocated to this Component were disbursed through capitation payments8 which financed a set of pre-established results related to better health and nutrition. It sought to improve the delivery capacity of health providers by providing them with an incentive to increase and improve the supply of preventive health and nutritional services for children under 12 months in the Juntos areas covered by the operation; • Component 3 - Strengthen the GoP's capacity to influence nutritional outcomes by improving budgetary planning and monitoring of results for selected activities of the PAN. This Component supported an incentive mechanism to strengthen the GoP’s capacity towards: a) the implementation of a planning and monitoring system for health facilities; b) the establishment of a social monitoring of nutrition results at the municipal level; c) the technical verification of capacity of health facilities; d) the early affiliation of newborns to the Integral Health Insurance Program (Seguro Integral de Salud – SIS, in Spanish); and e) verification of CRED information. The results-based part of this component disbursed against the increase in the number of health micro-networks with accurate and timely information in SIGA9, against an increase in the number of municipalities with an installed social monitoring system, and against the increase in the affiliation of children younger than 12 months to SIS. The remaining funds allocated to this component disbursed based on a procurement plan agreed upon by the borrower and the Bank. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating By the mid-2000s, Peru had one of the highest rates of chronic malnutrition in Latin America. The rate stagnated throughout the 1990s and for the better part of the 2000s, despite rapid economic growth and significant investments in food and nutrition programs. By the time the operation was designed, tackling malnutrition – with its dramatic effects on citizens, communities, and the economy as a whole, had become a policy priority in the country. Furthermore, the operation was developed and designed following years of World Bank engagement and analytical work, which helped identify some of the key bottlenecks and tools to reduce malnutrition in the country. 8 In health care, capitation is defined as a payment method where “the provider is paid, in advance, a predetermined fixed rate to provide a defined set of services for each individual enrolled with the provider for a fixed period”. See Langenbrunner, J ohn C., Cheryl Cashin, and Sheila O’Dougherty (2010) “What, How, and Who: An Introduction to Provider Payment Systems.” In Designing and Implementing Health Care Provider Payment Systems : How-to Manuals”. 9 Integrated Administrative Management System (Sistema Integrado de Gestión Administrativa). SIGA now covers over 7000 health centers across the country and allows the health sector’s to monitor budget and assess whether inputs at the micro health center level (CRED packages, vaccines, etc.) are aligned with demand (for instance, the number of children in a jurisdictions). 4 The World Bank Results in Nutrition for Juntos SWAp (P117310) Because of the above, the development objectives of the operations were highly relevant. Moreover, the operation’s key expected outcome of reducing chronic malnutrition in Peru was very much aligned with the World Bank Group’s Country Partnership Strategy (CPS) 2007-2011. The CPS agreed in fact to support activities aimed at promoting and developing a new social contract in education, health, and nutrition, with an emphasis on investing in people to reduce extreme poverty and social exclusion. Upon closure, the operation’s development outcomes remain highly relevant. Despite the remarkable progress in tackling childhood malnutrition, the country continues to face several serious nutritional challenges that will need to be addressed in the coming years. About 13 percent of the country’s children are still stunted, with huge disparities across the country; chronic malnutrition still settled at over 26 percent in 2016 in rural areas (against less than 8 percent in urban areas). Malnutrition remains especially high among children speaking indigenous languages – with rates three to four times higher relative to Spanish speaking children. Furthermore, urban malnutrition has stagnated during the 2011-2016 period, pointing to the need for tailored interventions in these areas to achieve further reduction. Anemia, often related to diet deficiencies, remains a major concern across Peru, with around 59 percent of children between 6 and 11 months suffering from it, and 44 percent of children between 6 and 36 months being anemic (Marini et al., 2017). Recognizing such ongoing challenges, one of the three pillars of the World Bank Group’s 2017-2021 Country Partnership Framework (CPF) for the Republic of Peru focuses on services for citizens, and includes among its objectives the improvement of health and nutrition services for the bottom 40 percent of the population. Due to the above, the relevance of PDOs is assessed as high. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome The first objective of the operation – increasing the demand for nutrition services by strengthening the operational effectiveness of the Juntos CCT program, was achieved. The percentage of Juntos beneficiaries below 24 months old registered in the program’s information system with birth certificate, National Identity Number (Documento Nacional de Identidad - DNI) and SIS affiliation experienced an outstanding increase between 2011 (18 percent) to 2016 (74 percent), well above the target of 67 percent, according to SIS data. According to Juntos’ administrative data, the percentage of Juntos children registered in the Program information system before 30 days of birth reached a total of 39.9 percent essentially meeting the 40 percent target. The second objective of the operation – improving the coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos, was partly achieved. Over the duration of the operation, the capacity of health facilities to offer basic health and nutrition services was increased. This is evident by the large increase in the “share of children under 12 months receiving the complete CRED package in the three areas of intervention of the Juntos CCT Program targeted by this operation” (the first of the two PDO indicators), which reached 86.4 percent in 2016 based on INEI10 data, up from 63.9 percent in 2011, and exceeding the target of 80 percent set at appraisal. The share of children under 36 months that received complete CRED scheme according to their age in the areas of intervention (the second PDO indicator) peaked at 72.7% in 2015, but ultimately did not meet its end target of 73 percent and settled at 70 percent by project closure, up 2.8 percentage points relative to the 2012 baseline value and on a clear upwards overall trend since 2011. In terms of 10 National Institute for Statistics and Informatics (Instituto Nacional de Estadistica e Informática – INEI in Spanish). 5 The World Bank Results in Nutrition for Juntos SWAp (P117310) quality of services, the operation contributed to much progress. The large increase in the share of health clinics endowed with the necessary equipment to deliver the complete CRED package and vaccines in Juntos’ districts (from 0 in 2011 to 91 percent in 2016, and 51 percentage points above the target value of 40 percent) is a clear testament of how the operation supported better quality of health and nutrition services. The overall objective of the operation – contributing to reducing malnutrition in the regions covered by the operation’s activities, was achieved. Peru’s success story reducing malnutrition during the last decade is remarkable and by now well-known. Based on INEI data, stunting rates in Peru for children under 5 decreased by a staggering 15.4 percentage points between 2007 and 2016, from 28.5 percent to 13.1 percent. The reduction in malnutrition rates observed in Amazonas (10 percentage points), Cajamarca (8.2 percentage points) and Huánuco (11.7 percentage points) – the three regions covered by the operation, was higher than the national average of 6.3 percentage points. This outstanding policy outcome was achieved due to an evidence-based, multi-sectorial and integrated set of interventions which the operation under review supported. The operation contributed to the achievement of such goals. It did so by, among other things: • Indirectly providing an incentive to local Governments to provide better health services and to focus on nutrition outcomes. The operation supported the extension of convenios de apoyo presupuestario (budget support agreements) to the three regions covered by the operation. The convenios are agreements between MEF and regional governments according to which incremental fixed and variable resources are transferred from the central to regional and local administrations on the basis of the achievement of certain results11. Once the convenios were set up, by reimbursing MEF based on the achievement of nutrition and health related results, the operation implicitly helped MEF provide incentives to regional Governments12, and by extension to local Governments and local health service providers, as well as specific agencies such as the National Registry of Identification and Civil Status (Registro Nacional de Identificación y Estado Civil - RENIEC in Spanish) and SIS13, to push forward the nutrition agenda; • Financing, through Component 3, technical assistance aimed at improving the collection and use of data around nutrition inputs, outputs, and outcomes. First and foremost, this was achieved by financing the development of the Padron Nominal, a live platform administered by RENIEC and accessible by local health centers and the Ministry of Health (Ministerio de Salud – MINSA in Spanish), MEF, the Ministry of Development and Social Inclusion (Ministerio de Desarrollo e Inclusión Social – 11 “The Convenios were supported by the EUROPAN project. The fixed tranche disbursements were linked to fulfillment of administrative/logistical processes needed to implement the PAN (including strengthening operational capacities of budget units). Variable tranches were disbursed against achievements of pre-established targets in coverage (percentage of children with immunizations; percentage of children with CRED; percentage of children affiliated to SIS, etc.).” See Marini, A., Rokx, C. and Gallagher, P. (2017), Pg. 31, “Standing Tall: Peru’s Success in Overcoming its Stunting Crisis”, World Bank. http://documents.worldbank.org/curated/en/815411500045862444/pdf/117053-OUO-FINAL-Peru-book-FA-SINGLE-PAGES- with-cover.pdf 12 Health is de facto a regional function in Peru. Doctors, for instance, report functionally to MINSA, but operationally to the local governments through the executing agencies of local Governments. For instance, MINSA has no power to dismiss doctors or to give them incentives. This one of the main reasons as to why the convenios were so important. Previously, the central Government (MEF or MINSA) had no tool at their disposal to provide doctors or local health clinics incentives to deliver more and better services. However, through the convenios, the central Government could make available additional variable budget to local governments (and, in turn, to health clinics and doctors) on the basis of results. 13 Under the EUROPAN project for instance, the central Government signed convenios with regional Governments. In turn, regional Governments signed their own convenios with RENIEC and SIS 6 The World Bank Results in Nutrition for Juntos SWAp (P117310) MIDIS in Spanish), the Ministry of Education, and local Governments. It contains individual level information on children under 6 years of age at the Centro Poblado level for the country’s 1,869 districts. The Padron Nominal, which represented a turning-point in Peru’s ability to identify, track, and monitor children and children’s nutrition and health services and outcomes, has been instrumental towards improving early registration to the country’s civil registry and affiliation to Juntos and SIS. The operation also supported the elaboration and dissemination of disaggregated data and the promotion of culturally-sensitive service delivery to highlight inequality in access to services to support better nutritional outcomes among indigenous households. Operation resources financed the design and implementation of four ENCRED surveys (Encuestas de Productos Priorizados), a nation-wide technical verification run by INEI among primary health care centers to assess their input needs and ability to provide nutrition-related services14 as specified by the standards of MINSA. These surveys, in turn, became instrumental to improving the quality of services provided by health clinics and to the increase in the share of clinics capable of delivering CRED packages and vaccines to Juntos’ beneficiaries. Finally, the operation improved the management of information at the district level by financing training activities for local government officials to develop and manage the Padron Nominal. This enabled local Governments to interpret and monitor nutrition results and service coverage indicators, providing continuous feedback and supporting the goal of more and better health services; • Supporting the preparation and implementation of the early affiliation strategy for newborns to SIS and Juntos. This included promoting the involvement of mayors of Centros Poblados (Small Towns) in the cycle of documentation and affiliation of children younger than 12 months to SIS. The focus on children younger than 12 months was a fundamental change supported by the operation, as these are considered the crucial months for development. Through Component 1 the operation also financed the expansion and the improvement of Juntos’ information system (Sistema de Información the Trasferencias Condicionadas – SITC in Spanish). As a result, since 2014, the SITC has been a web- based platform, with interoperability with the databases of MoE and SIS, connected to the Padron Nominal, and allowing for the close monitoring of a much higher number of beneficiary households, in line with the doubling of the program’s intake observed since 2007.15 • Supporting, through Component 3, the preparation and implementation of the social monitoring strategy (Monitoreo Social), aimed at strengthening the monitoring of nutrition outcomes, and coverage and quality of primary-level health services.16 Among other things, the operation financed the hiring of 15 consultants which gave direct technical support to regional and local actors in the implementation of the social monitoring strategy. The operation was instrumental towards: (i) the provision of timely, reliable and easy-to-understand information about the local status of key nutrition indicators, access to health and nutrition services, and health inputs availability; (ii) improvements in local coordination to enhance service delivery and close gaps in the supply of priority services (CRED, vaccines and micronutrients); and (iii) the establishment of social accountability platforms on nutrition outcomes (Foros de Rendición de Cuentas) in 148 districts. In turn, these achievements helped establish child nutrition as a priority in the districts targeted by the operation. A 2017 assessment of social monitoring found a strong positive correlation between the implementation of the social monitoring strategy and nutrition results. 14 Information sessions for mothers (sesiones demonstrativas), availability of CRED packages, availability of vaccines and additional tools, compliance with children’s affiliation to the Padron Nominal, etc. 15 According to Juntos’ administration, the number of beneficiaries increased from about 373,000 in 2007 to 735,000 households in June 2017. 16 The Monitoreo Social strategy envisions the close monitoring, at the municipal level, on behalf of the mayor’s office, the health sector, Juntos, and SIS, of a set of indicators such as: proportion of children younger than 12 months with a DNI, registered in SIS, having received the appropriate CRED packages and vaccination, etc. 7 The World Bank Results in Nutrition for Juntos SWAp (P117310) • Financing technical assistance aimed at increasing the adequacy and visibility of services delivered to indigenous populations, which tend to be the most affected by malnutrition. The GoP – with the support of the operation, produced a nationwide social assessment and delivered recommendations to overcome barriers that limit access of indigenous populations to Juntos and PAN services. The operation also supported the preparation of communication materials targeted to indigenous communities to increase their affiliation to Juntos and SIS. Justification of Overall Efficacy Rating Peru’s success story is a result of many initiatives, one of which is the operation under review. These reforms were accompanied by firm political commitment, a strong focus on evidence, incentives and results, interventions which simultaneously addressed the demand and the supply of health and nutrition services, and efforts aimed at changing behaviors (Marini et al., 2017). The operation undoubtedly contributed to the reduction of malnutrition in the country. The operation has been critical in ensuring greater availability of key nutrition services like immunizations and growth monitoring (CRED) checkups in target areas. It has been instrumental in increasing early-childhood affiliation to the Integral Health Insurance (SIS), and in making health and nutrition services more accountable to regional and local governments, and civil society. The operation improved cross-sectoral coordination in the operation’s focus areas by strengthening the link between Juntos’ administration and the delivery side of health and nutritional services. It bolstered the demand for services by supporting the dissemination of information on good nutrition and its importance among poor and vulnerable communities, and by supporting the expansion of Juntos and its monetary incentives to poor families. Finally, and more generally, as reported in the Borrower’s Project Closing Report, the operation provided an additional incentive towards more cooperation among agencies and actors, leading to joint efforts towards the fight against malnutrition. The selected financing instrument for this operation - the Sector-Wide Approach17 tool, proved to be the right one. A DPL was considered not appropriate as this financing vehicle typically provides non- earmarked budgetary support to client countries linked to the achievement of high-level policy and/or macroeconomic goals. Among the instruments which allowed for a more sectorial and operational focus, an Investment Project Financing (IPF) was considered. This type of vehicle exclusively disburses the proceeds of the loan against specific eligible expenditures and is typically more input-based and better suited for capital intensive projects. On the other hand, the SWAp instrument allowed a clear focus on results by disbursing against the achievement of predefined and tangible objectives. It was consistent, and in fact reinforced, the ongoing move towards performance-based budgeting in Peru led by MEF. While this particular SWAp operation did not entail a pooling of resources from different donors (a procedure which was more common to operations in the Africa Region), the SWAp supported an ongoing sector- wide Government initiative to complement the resources that both the Peruvian Government (through the PAN) and international donors (through the EUROPAN) had devoted towards the pursuit of a set of outputs (e.g., the provision of complete CRED services, to name one), and ultimately to the achievement of a clear overall outcome (the reduction of malnutrition in the country). The SWAp approach, at least on paper, did not impose an additional set of financial management rules on the country, simply reimbursing 17 Sector wide approaches (SWAps) operations were designed to strengthen the partnership between Governments and development partners. “By using a systems approach, SWAps aimed to strengthen national ownership and coherence around policies, spending and results. At the same time, they attempted to reduce transaction costs. SWAps could be supported by one or more sector programs which, in turn, could be financed by (one or more) donors using different modalities (e.g. general/sector budget support, pooled funds, project aid, debt relief).” See “Moving towards a sector-wide approach (SWAp) for health in fragile states: Lessons learned on the state of readiness in Timor Leste, Sierra Leone and Democratic Republic of Congo”, Royal Tropi cal Institute, 2011, pg.iii. 8 The World Bank Results in Nutrition for Juntos SWAp (P117310) the GoP for eligible expenses on the condition of having met certain results. Finally, it should also be mentioned that other Bank lending instruments (such as the Performance-for-Results – P4R financing tool) were not available at the time of this project’s preparation. While the operation has performed well overall, PDO indicators achievement have fluctuated somewhat, ranging from fully achieved to displaying slippages. At closing, one of the two PDO indicators was met while the other fell just short of its target despite being on a clear positive trend, and having essentially met the target value in 2015. Seven out of nine intermediate indicators were fully achieved (or exceeded their targets); however, two intermediate indicators were not met. Due to the above, the overall efficacy rating of the operation is deemed substantial. C. EFFICIENCY Assessment of Efficiency and Rating From an economic point of view, even though it is very difficult to estimate the direct benefits of operations such as this one, the activities supported by the SWAp are very likely to have resulted in net gains. By the time of approval, the primary causes of malnutrition in Peru were relatively well-understood and included the high incidence of diseases due to low access to services and poor health and hygiene practices, and inadequate food intake18. As discussed in the previous section, the operation directly focused on tackling these issues by; (i) contributing to the strengthening of Juntos program, which had been shown to forester nutritional improvements among beneficiaries, and to lead to change in the utilization of health services, and; (ii) contributing to the improvement in the coverage and quality of basic preventive health services. Specifically, the operation directly supported the expansion of the registration of children to Juntos, the improvement in the functioning of the program19, the complete affiliation of Juntos’ beneficiaries to SIS (which in turn increased the participation and use of health services), and the enforcement of the verification of the program’s co-responsibilities. In parallel, it supported “a centralized and coordinated approach to spending money in health facilities to ensure that the increased demand of services promoted by Juntos was met by adequate coverage and minimum quality of health and nutrition services.” (Marini et al., 2017, pg. 33). These activities likely contributed to reducing malnutrition in Peru which, in turn, likely led to significant overall economic gains as malnutrition “is not only responsible for nearly half of all deaths of children under five, it is estimated to reduce the world’s GDP by $1.4 -$2.1 trillion.” (Marini et al., pg. 17). From an implementation perspective, the operation witnessed some initial delays, which resulted in effectiveness being declared 10 months after approval; in turn, this led to delays in disbursement. As the Juntos Results for Nutrition operation was the first SWAp operation in the social sectors in Peru, there period after approval was marked by a steep learning curve. As the Peruvian Government is legally allowed to borrow from International Organizations (IOs) such as the World Bank only for incremental and investment expenses, concerns arose with respect to the operation financing activities related to Juntos, a recurrent budgetary item. Furthermore, the fact that a sizeable share of the operation’s finances was to be disbursed against the achievement of results seemed to be inconsistent, from the GoP’s side, with the need to have an ex-ante procurement plan for much of the operation’s resources. The original 18 Rogers, Beatrice L., Serena Rajabiun, James Levinson, and Katherine Tucker (2002), “Reducing Chronic Malnutrition in Peru: A Proposed National Strategy”, Tufts University Discussion Paper. 19 Annex 4 provides further rationale and details on the economic rationale for the strong focus of the operation on the Juntos program and details on how the operation supported the improvement of its implementation. 9 The World Bank Results in Nutrition for Juntos SWAp (P117310) phrasing of some of the results indicators was deemed at odds with the results-based nature of part of the operation, and the need to align indicators with the legalities of the disbursement procedures became apparent shortly after the operation’s approval. Finally, the creation, in 2011, of MIDIS, and the placing of Juntos under MIDIS’ responsibility (instead of the President’s Office) further delayed implementation. The operation was restructured five times. Three low-level restructurings were processed between 2012 and 2014 to amend some of the operation’s legal covenants and results indicators to improve implementation and speed disbursements. The latest two restructurings were processed in early 2016 and in late 2016 to extend the closing date of the operation to ensure the completion of key activities. These extensions were sought in part due to the lag between approval and effectiveness and aimed to facilitate the sustainability of the Padron Nominal and Monitoreo Social, two of the main achievements of the operation, and to improve the overall performance of the operation by transferring the knowledge accumulated by MEF to MINSA and MIDIS. Part of the reason behind the early-2016 extension was also related to the extreme weather conditions linked to the El Nino pattern, which temporarily put a strain on the ability to field the Encuesta de Productos Priorizados (ENCRED) survey, which was and remains vital to monitoring results. None of these restructurings or delays altered the initial design and scope of the operation in a fundamental manner; in fact, they should be mostly seen as the “price to innovate”. The Peruvian Government had requested an operation with a specific focus on results, and the idea of “fungibility of resources” was a relatively new one in the country at the time. Success therefore required a major shift in the way in which MEF thought about donors’ interventions and funding. On the other hand, the Bank, to respond to a pressing need in the country, had to rapidly adapt and learn how to get off the ground an instrument previously untested in the social sector in Peru at a time when the “Program -for-Result” instrument was not available. Both the World Bank team, and the Government counterparts, proactively responded to problems and unforeseen circumstances as they arose, as evidenced by the five restructurings. While it is important to take into account and to record hiccups, these were to be expected given the innovative nature of the operation, and need to be weighted accordingly when rating the overall operation’s efficiency. By late 2013 the operation had stabilized and implementation was smooth until closing, with the loan fully disbursing. After being downgraded to moderately unsatisfactory in mid-2013 mostly due to the above-mentioned delays, the rating of the operation’s implementation progress was upgraded shortly thereafter and maintained a rating of moderately satisfactory (or above) until closing. Disbursement reached 100 percent. Financial management and procurement were rated moderately satisfactory or better throughout the project life. The overall efficiency of the operation is assessed as substantial. Early challenges were mostly due to the innovative nature of the operation in the Peruvian context, and mostly related to delays in finalizing procurement plans and reaching an agreement between the World Bank and the GoP on measurable results’ indicators and legal covenants which allowed a speedy disbursement of the operation’s resources. All stakeholders were able to find mutually acceptable solutions for the benefit of the operation, thereby finding a sustainable and smooth implementation path until closing. Furthermore, the delays experienced at the beginning of the operation, as challenging as they may have been, also allowed the Bank to proactively draw some important lessons, which ultimately paved the way for a smoother design and implementation of other SWAp operations in social sectors20. 20 See, for instance, the Basic Education Project in Peru (P123151). 10 The World Bank Results in Nutrition for Juntos SWAp (P117310) D. JUSTIFICATION OF OVERALL OUTCOME RATING The overall outcome of the operation is assessed as satisfactory. This emerges by weighing the three sub-ratings relative to the relevance of the PDOs (assessed as high), the efficacy of the operation (assessed as substantial), and the efficiency of the operation (assessed as substantial). It should be stressed once more that the type of delays and initial challenges listed above were to be expected given the innovative nature of the operation. As the first SWAp operation in social sectors in Peru, the steep learning curve for both the Bank and the GoP was anticipated. The results-oriented and multi-sectoral approach of the operation called for evidence-based decisions and was to rely on data which was mostly not available at appraisal. The delays and hiccups described above are weighed in the ratings and are arguably the main reason why the operation is ranked as satisfactory as opposed to highly satisfactory. But one should be clear that this is mostly the price to pay when implementing new investment tools in a country / sector, and testing new grounds. E. OTHER OUTCOMES AND IMPACTS Institutional Strengthening The operation contributed substantially to institutional coordination and strengthening. In particular, the operation: • Strengthened the stewardship of local governments in the coordination of different institutions and sectors to increase the quality and quantity of health and nutrition services and programs for children younger than 12 months. One of the main ways in which the operation contributed to this goal was by supporting the expansion and institutionalization of the convenios de apoyo presupuestario described above. The convenios provided and still provide a significant monetary incentive to local government to pursue agendas and objectives; they also foster coordination within different levels of government and among agencies, as achieving results in multi-sectorial areas such as nutrition required an integrated vision and shared implementation plans. As mentioned, the operation also supported the development and implementation of the Monitoreo Social, an inter- agency and multi-stakeholder monitoring mechanism led by local governments with the participation of indigenous organizations, local civil society, health-service providers, Juntos staff, and SIS representatives; • Supported the further strengthening of the implementation of results-based budgeting. This, aside from the convenios, was achieved thanks to the choice of instrument: by reimbursing operation activities based on targets met, the SWAp operation aligned itself with the changes which Peru was undergoing in the area of management of public expenditures, and de facto supported and reinforced Peru’s performance-based budgeting approach, from the top-down; • Lobbied for the strengthening of the Municipal Civil Registry Offices (Oficinas de Registros de Estado Civil – ORECs, in Spanish), and the establishment of well-equipped Auxiliary Registry Offices in hospitals and other birth facilities to support the online issuance of the Certificate of Live Birth. This subsequently allowed timely processing of Birth Certificates and National ID numbers, and facilitated inter-agency exchange of information to improve early access to health and nutrition services. Other Outcomes and Impacts The operation’s focus on children’s early affiliation to Juntos and SIS had important spillovers on the health sector as a whole, and on the identification of children throughout the country. Traditionally, in 11 The World Bank Results in Nutrition for Juntos SWAp (P117310) Peru, having a National Identity Card (Documento Nacional de Identidad – DNI, in Spanish), which can only be issued by RENIEC officials, was a precondition to access health services and register with SIS. Such requirement de facto excluded many children, who at the time of the operation’s approval did not have a DNI, from benefitting from programs such as Juntos, which provided a strong incentive to seek and receive nutrition services, vaccines and immunizations in a timely manner. This was particularly true in remote rural areas, where RENIEC offices are not present. Analysis showed, however, that most children had a Unique Identification Code (Codigo Unico de Identidad – CUI, in Spanish), a citizen’s unique identifier printed on birth certificates, and therefore CUIs, are issued not only by RENIEC but also by ORECs, the latter of which are much more prevalent across the country’s territory relative to RENIEC. To facilitate early affiliation to SIS and Juntos, and therefore access to health services for children in remote and poor areas, the operation actively partnered with and provided technical assistance to RENIEC to improve the National Civil Registry database. This assistance resulted in RENIEC’s platform being able to receive in real time newborn birth certificate and CUI information from ORECs; as such, households could register children with SIS and affiliate to Juntos from the moment of birth (using the CUI number) and while they were waiting for a DNI. In parallel, the task team also actively supported the establishment of Auxiliary RENIEC’s offices in hospitals capable of issuing the birth certificate and automatically initiating the Child DNI issuance process at the time of birth. Currently, there are such Auxiliary offices in more than 150 hospitals all over the country. According to RENIEC, 98.5 percent of Peruvian children currently have a DNI, a feat that has dramatically improved the Government’s ability to better monitor the country’s demographics, children’s health, health services utilization, etc. It is important to note that this achievement appears permanent; even in the absence of the program, Peruvian newborns will continue receiving a CUI and a DNI upon birth. The operation was also instrumental towards improving the governance system around the collection and use of data in Peru. The renewed focus on evidence, the push towards having more and better data (from ENCRED to ENDES, to SIS’ platform, to the Juntos’ SITC, to the Padron Nominal), and the use of a strong evidence-base combined with constant monitoring of key data to take informed decisions towards nutrition results, now seem to be a common practice (or at least a goal) across the GoP. Furthermore, the focus on building integrated IT systems, connecting, updating, and maintaining platforms, and making them accessible to local Governments and other stakeholders, contributed to improving institutional coordination at different levels, aligning strategies and interventions. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION The Bank was well positioned to support the country’s efforts to improve nutritional outcomes, and the design of the operation benefitted from lessons learned from other World Bank interventions in this area. The operation was designed as part of an ongoing and long-standing engagement between the Bank and the GoP to tackle chronic malnutrition. Problems with the incentive system and financing of the country’s health system had been examined by the Bank through the RECURSO non-lending technical assistance activity, and some reforms had been supported through the Results and Accountability Development Policy Loan (REACT DPL) series. The operation built on the Bank’s global experience in designing and supporting the implementation of CCT programs around the world - from Brazil’s Bolsa Familia, to Mexico’s Oportunidades (now Prospera), to Colombia’s Familias en Acción, but also leveraged the experience gained through several years of non-lending technical assistance provided to the Juntos program. Finally, the operation benefitted from the lessons learned, tools, and capacity built through the 12 The World Bank Results in Nutrition for Juntos SWAp (P117310) Scaling Up Handwashing Project, a World Bank coordinated public-private partnership of 50 organizations from 23 regions of Peru aiming to fight malnutrition and promote behavioral change in hygiene. As mentioned in Section A, the operation was designed along an evidence-based causal chain which reflected the latest thinking and best practice interventions in the nutrition area. More specifically: • The demand side of the operation focused on strengthening Juntos’ implementation built on both international and country-specific evidence. Research showed that CCT programs across the world not only had the potential to be effective poverty alleviating tools, but could also help generate adequate incentives for poor families to invest in the education, health, and the nutrition of their children. CCTs could also act as triggers to improve supply of basic social services by boosting demand.21 An evaluation of Juntos showed that the program, while improving beneficiaries’ dietary intake and use of health services, was not showing much impact on final outcomes such as reduction of chronic malnutrition. This was partly due to lack of coverage and quality of the associated health and nutrition services, and partly to shortcomings in the program’s design and implementation, in particular with respect to the verification of beneficiaries’ compliance with health and nutrition co- responsibilities.22 Marini et al. (2017) report that, at appraisal, Juntos was struggling with verifying that mothers were bringing all their children for height and weight checkups as part of their co- responsibilities, and that there was a lack of clarity on which health services pregnant women and children could expect to receive in return for their cash payments.23 The focus on co-responsibilities and on improving the management of the program followed directly from this evidence. • The supply-side component of the operation also reflected the latest lessons in the region on health financing and health services. Chiefly, the operation envisioned incentives to strengthen the governance and accountability framework built into the performance-based budgeting system adopted by Peru since 2007, which aimed at breaking the institutional budgetary inertia by assigning (incremental) resources as a function of achieving better and traceable results in nutrition rather than on the basis of historical allocations. Furthermore, the health interventions supported by the operation drew on several interventions that had been demonstrated effective in reducing malnutrition according to international experience (breastfeeding promotion, complementary feeding promotion strategies, micronutrient interventions, and support strategies to improve community nutrition and control disease). B. KEY FACTORS DURING IMPLEMENTATION As highlighted in the efficiency section, effectiveness of the operation was delayed, ultimately leading to the closing date being extended. Project effectiveness was delayed by about 10 months after approval. This in turn affected the schedule of implementation and disbursements. The reasons for these delays are listed in Section C. After the GoP and the World Bank agreed on the simplification of disbursement requests, on the specifications of technical audits, and modified some results’ indicators, the operation progressed smoothly. 21 See Ariel Fiszbein and Norbert Schady (with Francisco H.G. Ferreira, Margaret Grosh, Niall Keleher, Pedro Olinto, and Emmanuel Skoufias), 2009, “Conditional Cash Transfers: Reducing Present and Future Poverty”, The World Bank. 22 Perova and Vakis, 2009, “Welfare impacts of the Juntos Program in Peru: Evidence from a non-experimental evaluation”, The World Bank mimeo. 23 See Marini, A., Rokx, C. and Gallagher, P. (2017), “Standing Tall: Peru’s Success in Overcoming its Stunting Crisis”, World B ank. http://documents.worldbank.org/curated/en/815411500045862444/pdf/117053-OUO-FINAL-Peru-book-FA-SINGLE-PAGES- with-cover.pdf 13 The World Bank Results in Nutrition for Juntos SWAp (P117310) IV. BANK PERFORMANCE AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design The PDO and key outcome indicators were clear and monitorable; however, the intermediate outcome indicators had to be simplified and reduced to align them more closely to the operation’s objectives, and to allow for better monitoring. At approval stage, the operation had 18 intermediate outcomes (five for each of the first two components, and eight for the third component). Following two restructurings, some indicators were dropped, some reworded, some introduced ex novo. By closing, the operation had a more streamlined total of nine intermediate indicators. The reasons behind these changes included: the lack of data/ baseline information for some indicators, the need to exploit existing data when monitoring the operation, facilitate disbursement, and simplify the drafting of the operation’s technical audits. The operation’s monitoring and evaluation procedures were set up to support the GoP’s efforts to strengthen existing information systems and their utilization. The monitoring and evaluation activities supported by the operation were designed to further develop and harmonize the monitoring systems of MEF, Juntos, SIS, and MINSA. Monitoring activities were envisioned to be a critical element of the capitation payment system utilized to fund activities under Component 2 and 3. M&E Implementation M&E implementation faced some initial challenges. As mentioned above, the inability to compile the baseline figures of some intermediate indicators, and the lack of data to monitor others, called for the operation’s restructuring. Baseline estimates for some indicators became available by 2012. Furthermore, in 2015, as mentioned above, El Nino prevented the collection of ENCRED data, the only source of information for the monitoring of some intermediate results; the absence of such data put disbursement on hold and was one of the main reasons behind one of the extensions of the project’s closing date. Despite the above, many of the most important achievements of the operation are linked to the collection and improved use of data in view of better nutrition results. However, the delays described above led to a rating of Moderately Satisfactory in M&E implementation by closing. The operation was critical in introducing an innovative social monitoring mechanism to oversee nutrition results and quality of health and nutrition services in 160 districts in the three targeted regions. The mechanisms contributed to elevating the issue of malnutrition and introducing a much greater level of social accountability within poor communities. It did so by making available to local authorities a small, carefully crafted list of district- level indicators (children with DNI, children affiliated to SIS, children affiliated to Juntos, gaps in CRED coverage and in immunization/vaccines). Such indicators, in turn, allowed mayors and local Governments to identify and monitor individualized outcomes of children, and to focus efforts and prioritize interventions. The social monitoring mechanism rests on the Nominal Census ("Padron Nominal"), as previously noted a major achievement supported by the operation, which consolidates into a single platform information from local and central databases and health establishments to provide real-time information to MINSA and MEF, and to local stakeholders participating in the local social accountability fora. This information is also utilized by the Performance Incentives Fund (In Spanish, Fondo de Estimulo al Desempeño – FED24) and other results-based budgetary instruments implemented by MEF and MIDIS 24FED is a three-year budget support agreement between regional Governments, MIDIS, and MEF which promises additional resources to regional governments based on the achievement of several results (which include nutrition related outcomes). 14 The World Bank Results in Nutrition for Juntos SWAp (P117310) (such as the Municipal Incentives Plan) to monitor the fulfillment of targets agreed at the local level. A third major accomplishment of the operation, as mentioned above, was supporting the creation of ENCRED, a now-vital survey of primary health care centers focused on assessing their input needs to provide services; ENCRED, among other things, enabled regional and local Governments to truly assess their capacity to deliver nutrition-related health services, and their constraints, and allowed the Juntos’ information system to better verify the program’s co-responsibilities. M&E Utilization The operation made extensive use of existing and newly acquired data to monitor results. Results indicators, and the overall implementation of the operation were monitored using several data sources such as ENCRED, information captured by the Padron Nominal, Juntos and SIS’ platforms, and the Survey on Demographics and Household Health (Encuesta Demográfica y de Salud Familiar – ENDES in Spanish), run by INEI. After years of analytical work and technical assistance on behalf of the World Bank, and thanks in part to the operation’s support and its focus on evidence-based decisions a vast amount of nutrition- related data is now available and used at a much lower level of disaggregation (district level, individual level, etc.) allowing for a more granular understanding of local constraints. Justification of Overall Rating of Quality of M&E Because of the significant achievements listed above, the extensive use of data made during the operation’s duration, and the way in which the operation overcame the initial challenges related to data availability and results monitoring, the overall quality of M&E is rated as substantial. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental Safeguards. This operation did not trigger any environmental safeguard policies. Procurement. Procurement processes were rated as Satisfactory for most of the operation’s lifecycle. Procurement ratings were downgraded to Moderately Satisfactory in mid-2013 due to delays in the procurement processes managed by MEF’s Sectoral Loans Coordinating Unit (UCPS), and to delays on behalf of the World Bank Group in approving the procurement of two large contracts. The ratings were upgraded to Satisfactory about one year later, and remained so until closing, once the World Bank procurement team was able to verify that all Bank procedures were being followed and steady collaboration between the World Bank’s team and MEF had been established. Financial management. The FM ratings started out as Satisfactory as ex ante planning, budgeting, accounting, internal controls, flow of funds, financial reporting, and auditing arrangements were adequately set up. The ratings were downgraded to Moderately Satisfactory in mid-2013 due to delays in the preparation of the terms of reference of the operation’s technical audit. Internal processes and procedures within between MEF’s General Directorate of Public Budget (Dirección General de Presupuesto Público – DGPP) and UCPS improved, and administration of funds followed suit; however, delays in the submission of technical audits remained throughout the project, as well as some coordination issues between the DGPP and the UCPS. As such, the rating remained constant at Moderately Satisfactory until closing. Overall, the teams managed well the challenges of the first result based operation in the social sectors in Peru and the fact that the legal framework had to be adapted to ensure smooth disbursement. Indigenous People. The operation triggered the Bank's Indigenous Peoples Policy OP 4.10 and was expected to have positive impacts on poor marginalized groups. It indeed did so by helping bring national 15 The World Bank Results in Nutrition for Juntos SWAp (P117310) attention to the huge chronic nutrition gaps between indigenous households and non-indigenous households. At approval, according to the operation’s Project Appraisal Document (PAD), indigenous households represented approximately 41 percent of poor households in Peru and their participation in Juntos made up 45 percent of the total number of households participating in the program. Chronic malnutrition showed the highest rates in departments with high numbers of indigenous households, including Huancavelica, Apurímac, Cajamarca, Huánuco and Cusco. However, the exact gap between indigenous and non-indigenous districts, and the conditions behind the uneven performance of nutrition and health programs in different territories and cultural contexts, were mostly unknown before the operation. To fill this information gap, before approval, the GoP conducted a nationwide social assessment and prepared an Indigenous Peoples Strategic Framework (IPSF). Throughout the operation, the GoP consulted with indigenous communities and organizations to ensure better and culturally appropriate service delivery and information sessions to change behaviors, and supported a systematic effort to disaggregate data to track nutrition results in different cultural and geographic settings. The rating of the ISPF’s implementation started off as Satisfactory before being downgraded to Moderately Satisfactory in 2012 due to overall operational delays. By mid-2015, and until closing, the progress towards the implementation of the IPSF was deemed Satisfactory. The operation contributed to much progress when it comes to indigenous people. Juntos’ staff carried out a diagnostic of main access barriers faced by indigenous households nationally, developed a strategy and devoted a team to improving nutritional outcomes within indigenous communities. The diagnosis indicated that indigenous communities faced serious challenges in terms of timely registration of children to the Juntos Program, and in terms of households' compliance with nutrition co-responsibilities. Accordingly, with the support of the operation, the GoP arranged consultations with regional Indigenous People (IP) organizations throughout the country, aligned the indigenous people strategies of the Ministry of Health and of the Civil Registry, and began hiring Native field agents to better connect with indigenous populations. MINSA adopted a differentiated strategy for indigenous people which included the definition of a special package of inputs to serve indigenous communities according to their territorial, institutional and cultural specificities, and began offering nutrition services adapted to territorial and demographic conditions of Amazon communities. The operation also allowed for higher per capita reimbursements to regional health services to reflect the higher incremental costs of operating in the Amazons. Overall, the operation contributed to a large reduction in the rate of stunting among indigenous children - almost 10 percentage point relative to the 47 percent registered in 2005 according to Peru’s Demographic and Family Health Survey (ENDES). However, at 38 percent, chronic malnutrition among indigenous children remains in need of a strong policy focus. C. BANK PERFORMANCE Quality at Entry The World Bank Group designed a timely operation that was highly relevant to the Peruvian context. The PDO’s focus on nutrition capitalized on changes in the country’s political economy which, in the years leading to the operation’s approval, led to malnutrition moving to the top of the GoP’s policy priorities. The focus on demand, supply, and capacity building activities was consistent with the holistic and multi- sectorial approach needed to tackle malnutrition. Furthermore, the selected SWAp tool, by disbursing against results, was not only an innovation in the Peruvian context but was also fully aligned and reinforced the GoP’s move towards performance based budgeting, which started in the mid-2000s and was gaining momentum. 16 The World Bank Results in Nutrition for Juntos SWAp (P117310) The World Bank accurately assessed risks. At approval, the team deemed as substantial the risk of implementation constraints due to the lack of prior experience within the GoP with the implementation of SWAp operations in social sectors. The Bank also assessed as substantial the risk of the GoP not being able initially to generate information on outcomes to justify the disbursements linked to results. Indeed, these two risks materialized at the early stages of the operation, but the Bank team proactively restructured the project to address the situation, leading to a smoother implementation thereafter. Quality of Supervision The World Bank mobilized resources and conducted an active and effective supervision that was instrumental to the operation’s achievements. As the risks mentioned above materialized, thanks to the constant supervision of the task team, the World Bank was able to effectively respond to the challenges by restructuring the project to facilitate implementation and disbursement. The regular supervision and implementation support missions were key to identifying issues and potential delays in implementation as early as possible, and led to additional restructurings which proved vital to largely achieving the operation’s goals. Furthermore, in the years leading to the operation, as mentioned above, the Bank mobilized additional resources to establish a coordinated and multi-sectoral approach to combatting child chronic malnutrition, which led to the development of the operation. During implementation, the task team secured a US$2 million Japanese Social Development Fund Trust Fund to complement the operation’s resources and support the development of protocols/materials for indigenous populations. The GoP considers the World Bank supervision and support satisfactory and useful. As discussed above, some initial delays in agreeing on the legalities of the SWAp design and disbursement occurred, and there were implementation delays. Finding IT experts to be hired thought the operation’s resources, and agreeing on their terms of references between MEF, MIDIS, and the World Bank task team, was a very lengthy process which often caused delays. However, the respectful, collegial, and friendly working relationship which was built between the World Bank and the GoP’s team is undeniable. Furthermore, the technical assistance which the Bank provided is considered extremely valuable, and was arguably one of the main reasons behind the request for the operation. The positive relationship between the Bank’s task team and the executing teams of the GoP was not only specifically mentioned in the Borrower’s Project Closing Report, but was apparent at the operation’s closing event, during which many stakeholders stressed how having a unified team working towards a common objective majorly contributed to overcoming the initial difficulties and eventually to the operation’s overall success. Justification of Overall Rating of Bank Performance Based on the above, the Bank’s performance is assessed as satisfactory. D. RISK TO DEVELOPMENT OUTCOME The sustainability of some of the achievements of the operation may face some challenges. • The future of multi sectoral interventions in Peru remains uncertain. As the operation (and the EUROPAN) wound down, the previously mentioned Fondo de Estimulo al Desempeño (FED) took over part of the incentive provider role to regional Governments within the performance-based budgeting approach to fighting malnutrition. Malnutrition (and more recently anemia) continue to be on top of the Government’s agenda25, and most of the infrastructure built through the operation (especially IT- 25 In 2017 the Peruvian Government announced its intention to reduce chronic malnutrition rate in children by a further 7 17 The World Bank Results in Nutrition for Juntos SWAp (P117310) related) will prove useful going forward. However, the political support behind multi-sectoral MEF- led interventions, a major factor behind Peru’s success in reducing malnutrition, seems to be fading as line Ministries are seen as better placed to lead interventions in their sectors, putting at risk some of the achievements of the past years. MIDIS’ role in overcoming this risk will be key going forward; • As highlighted in the Borrower’s Project Closing Report, it remains unclear how the Padron Nominal will be continuously updated and maintained, who and how will support the future implementation of the Monitoreo Social in the three areas covered by the operation, and who will finance future ENCRED surveys, arguably the biggest accomplishments supported by the operation. RENIEC, the administrator or the Padron Nominal, benefitted from the operation and the Government’s financial support building and maintaining the Padron Nominal, and will struggle to continue doing so past 2018 in the absence of the operation’s resources and unless general budget resources are allocated for this purpose beyond 2018. Local authorities (mayors in particular) will continue to be key towards the collection of the data on which the Padron Nominal and, in turn, the Monitoreo Social rest. However, in the absence of the operation, and once the FED closes, as variable budgets at the disposal of local Governments decline, officials will face more challenges collecting, maintaining, and updating the vital information needed to continuously assess nutrition results and the quality of primary health and nutrition services at the district level. The overall sustainability of the Padron Nominal (at the national level) and of the Monitoreo Social (in the three focus areas of the operation) will be highly dependent on the level of ownership and capacity at the regional and district level, potentially resulting in great variation in the use and quality of these tools across the country. This could affect especially lower density indigenous areas, where active participation in social monitoring tends to be lower and the cost of maintaining these tools higher.26 Furthermore, after being run four times thanks to the operation’s resources, the ENCRED survey is unfunded and there are currently no plans to implement it. In the absence of a wider strategy and allocation of resources towards the maintenance of these tools, some of the achievements of the operation may be in jeopardy; • The push towards further increasing children’ early affiliation to Juntos could lose momentum. As described in this report, the program ties cash support to selected households to the fulfillment of co- responsibilities linked to children’s nutrition and health; however, the benefit amount received by Juntos households is independent from the number of children in the household. Therefore, after registering their first-borns, household have and will continue to have little incentive to register their other children to SIS and affiliating them to Juntos upon birth. This could, in turn, lower the demand for health and nutrition services at the local level (vaccines and immunizations, CRED packages, etc.), negatively affecting the gains in reducing malnutrition achieved over the last decade. VI. LESSONS AND RECOMMENDATIONS Lessons Learnt A successful operation needs ex ante clear answers to the questions “why, when, and how. As to the why and the when, in the face of a stunting rate among children under five of 28 percent in 2005, the World Bank and the GoP clearly understood that the tide in the fight against malnutrition had begun to turn and that the country’s political class and civil society were demanding action. Accordingly, in the mid- percentage points (to 6 percent) by 2021, and anemia rates to 19 percent. 26 The Regional Governments of Huánuco and Cajamarca established and incentive scheme for their districts according to which “bonuses” are allocated on the basis of the achievement of nutrition and health related goals. According to officials of the Huánuco region, the Logro Social scheme (as the scheme is known) has been very influential towards the nutrition agenda and the success of the monitoreo social. Such experiences should be explored forward and potentially expanded to other areas. 18 The World Bank Results in Nutrition for Juntos SWAp (P117310) and late-2000s, the GoP introduced Juntos, the Integral Nutrition Program, moved to performance-based budgeting, and implemented the EUROPAN project, which was the first convenio de apoyo presupuestal focused on the fight against malnutrition. At the same time, the Bank began supporting Peru’s efforts in promoting standards for nutrition through budget-support financing for reforms and through analytical and work on communication; the Bank also started providing TA to the Juntos program. The SWAp operation emerged against this background as a natural next step in the fight against malnutrition and in the relationship between the Bank and the GoP. Despite the initial legal and administrative difficulties in setting up the operation, which were mostly due to the novelty of the SWAp tool in the context of the Peruvian social sector, the loan contributed to institutionalize and strengthen positive dynamics which were undergoing in the country (e.g. the convenios, performance based budgeting, evidence-based decision making, etc.). As far as the how is concerned, as described above, the multi-sectoral approach to a multi-faceted problem, and the use of evidence-based interventions ensured a technically rigorous design. Lesson 1: A strong understanding of client’s political economy dynamics, presence in a country, availability of resources, combined with cutting-edge solutions need to come together and be mutually reinforcing factors for change to happen. Implementation arrangements which combine strong technical competencies, high administrative capacity, and the necessary level of political clout are vital features of a successful operation. The GoP’s decision to allocate the implementation responsibility to MEF proved to be the right one for the task at hand. The ability of MEF - and of its Department of Public Expenditure, to bring together different agencies, Ministries, and levels of Government, and to provide them with tangible incentives should not be underestimated. Most importantly (and less visibly), the decision to hire within MEF a handful of health practitioners/doctors with a deep understanding of malnutrition and extensive prior experiences with health projects in the context of performance based financing, proved to be key. The creation of this technical team of health experts not only ensured that the operation (and the malnutrition agenda more generally) had “key allies” within MEF, but that it could also benefit from much needed expertise on issues somewhat external to a Finance Ministry. It gradually allowed malnutrition to become a forefront issue within MEF - a key driver of policy making and an agenda setter in Peru, and to remain so for the foreseeable future, as many experts of the MEF’s Mini-MINSA were civil servants. Lesson 2: Where possible, for implementation arrangements to be effective, the main responsibility should be assigned to an administratively capable and politically influential agency, while ensuring the availably of the necessary technical competencies within the agency. Understanding where incentives are needed is crucial. As discussed above, a major achievement of the operation was to build upon the reform process which had been going on in the area of public expenditure management in Peru, away from path-dependent and inertial budget allocations, and towards performance based financing. Piggy-backing on the positive experiences of the EUROPAN project, and capitalizing on the establishment of the crucial convenios de apoyo presupuestario in EUROPAN regions, the operations successfully lobbied and extended the use of the convenios to its three focus areas27. Such agreements allowed the transfer of additional resources to local Governments based on results, thereby aligning the incentives of several different agents (from regional Governors to local mayors, from executing agencies, to local heal clinics and doctors). Importantly, the convenios allocated the fixed part of transfers based on better planning. This meant that the poorest regions in Peru, which were previously having difficulties with administrative and logistical processes, started having a clear incentive to learn how to manage result based budgeting, better plan for their use of resources, track their expenses, and 27As reported in the Borrower’s Project Closing Report, many of the regions which signed convenios de apoyo presupuestal with the central Government showed greater improvements in services coverage and in the reduction of malnutrition. 19 The World Bank Results in Nutrition for Juntos SWAp (P117310) account for children in their areas (i.e., the demand side of local services). The incentives which performance based budgeting and the convenios presupuestario gave to the different levels of Government, and which the operation reinforced, enabled Peru to focus on the availability of health centers and services for the poorest and most remote areas, where making progress was and is the hardest. At the same time, this incentive system gave regional Governments and remote areas more voice, more resources, and more attention. It brought together different stakeholders to discuss and work towards a common objective.28 Lesson 3: To walk that last mile, setting the right incentives is key. Constantly monitoring results is key to foreseeing issues and to rapidly troubleshooting for success. As this report shows, success required the “perfect storm” of smooth implementation and coordination, by and of all actors involved and at the same time, as well availability of human and financial resources. For malnutrition to decline so significantly, many variables had to interact simultaneously, and even small shocks to this often precarious and actively pursued equilibrium can entail important setbacks. As this experience shows, even exogenous and uncontrollable factors such as extreme weather conditions or staff turnover can significantly impact or delay success. Lesson 4: Frequent and detailed monitoring at the lower level of disaggregation possible becomes all the more important towards results as it allows stakeholders to focus their attention and efforts where most needed and when most needed. Recommendations For all its successes, an objective reading of the challenges faced by the operation’s implementation may help draw some useful recommendations useful for both the GoP and the World Bank going forward. World Bank legal and procurement rules and procedures, while vital to fulfill fiduciary responsibilities of borrowers and lender, need to allow for some flexibility and to take into better account national legislation, especially when testing new grounds. As mentioned above, some of the initial delays were due to lack of clarity among the GoP and the World Bank about the legalities of the operation’s disbursement procedures. For instance, the operation was legally restricted to financing only incremental costs sustained by the Government, a restriction which seemed at odds with the focus on results of the SWAp approach. Solving this impasse took several months. Also, delays occurred as the World Bank’s procedures for investment lending required a procurement plan to be drafted and approved for the whole operation, despite its results-based nature. This was particularly true for Components 1, which was not truly procurable given that it disbursed based on the achievement of results. On the other hand, Component 3, which was essentially a technical assistance component, had to be artificially transformed into a “procurable results-based financing” operation. Finally, the requirement to have yearly technical independent audits of the project was at odds with the Peruvian legislation. These requirements on behalf of the World Bank were often cited by the borrower as a source of frustration and were mentioned in the Borrower’s Project Closing Report. Overall, despite the great appreciation that MEF’s team have for the support received by the Bank, notwithstanding the innovative features of the operation, and even though opting for a SWAp vehicle was the right decision at the time of approval given the available options, the instrument was deemed accounting-wise and administratively too cumbersome by the borrower. Recommendation 1: Even though a set of leaner and explicitly results-based investment vehicles has been devised by the World Bank (e.g. the Performance-for-Results instrument), greater flexibility on the 28Many stakeholders interviewed for the preparation of this ICR stressed the importance of the regional Mesas de Concertación para la Lucha contra la Pobreza (Discussion Forum for the Fight Against Poverty) chaired by the regional Gerente de Desarrollo Social (the Head of the Social Development Department) and bringing together Government and civil society, towards reaching a common understanding around nutrition issue and having a common vision and strategy. 20 The World Bank Results in Nutrition for Juntos SWAp (P117310) side of the World Bank will be continue to be needed when piloting innovative interventions. The health sector was not sufficiently involved in the operation’s design and implementation, both from the side of the World Bank and the GoP; this shortcoming likely reduced support for the operation from the sectoral side. Internally, within the World Bank, the Health Global Practice was consulted during preparation but had a limited role during the operation’s implementation, reducing the likelihood of further World Bank future engagements in the area of nutrition. From the borrower’s side, despite the innovative idea of staffing MEF with former health practitioners, institutionally, MINSA supported but was never directly involved in the operation (for instance as a co-executor)29. Having MEF as the main responsible agency for the operation’s implementation may have been the right choice from a political economy, administrative capacity, and accountability perspective, and ownership around the operation’s activities and sustainability was overall high within MEF, MIDIS, and local Governments. However, not having MINSA actively involved likely reduced the level of support of a key sector and line Ministry for World Bank operations going forward. In other words, the consequences of coordination issues within the Bank, and the failure to involve as many stakeholders as possible on the borrower’s side can be far reaching. Recommendation 2: World Bank teams, as well as borrowers, should make a conscious effort to be as coordinated as possible – and align incentives accordingly, from the design phase onwards. Despite its many achievements, the operation represents somewhat of a missed opportunity for the World Bank’s engagement in Peru. As the operation closes, the World Bank’s portfolio in the social protection and nutrition areas is significantly winding down. Additional financing, while discussed at length, was ultimately not requested by the GoP, nor a continuation of the operation. The Social Protection and Jobs Global Practice does not currently have an engagement with MEF or MIDIS going forward, despite the overall positive experience of the SWAp. Finding the resources to disseminate internationally Peru’s experiences fighting malnutrition will be a challenge. This is likely the result of many factors, from the just-mentioned limited involvement of the health sector in the operation, to the legal and procurement complexities of these financing tools when applied to the Peruvian context, to changes in the political economy of the country which reduce the appeal of multi-sectoral interventions. Certainly, this can simply be due to the fact that the Government of Peru, having access to international markets and to different funding sources, may not need World Bank loans as much as in the past, especially in some sectors. And while the nutrition agenda per se seems not be at risk, the business development potential of this engagement from the World Bank’s side was not exploited to the fullest. This is especially relevant as the issue of anemia gains more and more attention in Peru, as the country may have found useful the Bank’s support in this area. Recommendation 3: Greater attention to the changes mentioned above may have increased the chance of continued engagement. 29 As described in the earlier sections of this report, MINSA was involved in the operation indirectly. This mostly happened at the regional and local level and through DIRESA’s offices (Dirección Regional de Salud - Regional Health Directorate), which closely cooperate with local Government’s executing agencies. However, even though MINSA supported the operation, its involvement in the implementation of the operation was limited, as reported in the Borrower’s Project Closing Report. 21 The World Bank Results in Nutrition for Juntos SWAp (P117310) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increase demand for nutrition services by strengthening the operational effectiveness of the Juntos Conditional Cash Transfer (CCT) program Objective/Outcome: Improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of Juntos Children under 12 Percentage 63.90 80.00 80.00 86.40 months that have received the 31-Dec-2011 31-Dec-2011 30-Jun-2017 30-Dec-2016 complete CRED scheme according to their age in the areas of intervention of the Juntos CCT Program targeted by this operation. Comments (achievements against targets): Target Achieved and Exceeded (108% of Target). This is considered the main PDO indicator of the project, since it focuses on the critical and most intensive period of interventions to overcome the risk for a child of being stunted. The first 12 months count for 11 out of a total of 21 CRED visits in the first 3 years of age. Source: INEI and ENDES Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of Children under 36 months Percentage 67.20 73.00 73.00 70.00 that have received complete 30-Mar-2012 31-Dec-2011 30-Sep-2016 30-Dec-2016 CRED scheme according to their age in the areas of intervention Comments (achievements against targets): Partly Achieved (96% of Target). The use of this indicator, children under three years with completed CRED scheme, was a "control" indicator as a desirable threshold. The goal of the second PDO was established with the objective of tracking maintenance interventions, which account for 4 CRED visits during the second year, and 1 visit during the third year of age. Its actual value peaked at 72.7% in 2015 and ultimately exceeded the baseline by 2.80 percentage points, but did not reach the desirable threshold value of 73%, the indicator’s end target. Source: INEI and ENDES 22 The World Bank Results in Nutrition for Juntos SWAp (P117310) A.2 Intermediate Results Indicators Component: Component 1: Support the strengthening and consolidation of the design and operational capacity of the Juntos program for families with children younger than 36 months Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Juntos beneficiary Percentage 18.00 0.00 67.00 74.10 children under 24 months registered in the Juntos information system (SITC) with birth certificate, identity 31-Dec-2011 30-Jun-2017 30-Jun-2017 document or CUI, and SIS affiliation. Comments (achievements against targets): Target Achieved and Exceeded. Source: SIS and Juntos' Administrative Data. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of children born Percentage 9.55 0.00 40.00 39.90 each year in Juntos households and registered in the SITC 31-Dec-2011 30-Jun-2017 30-Jun-2017 before 30 days. Comments (achievements against targets): Target Achieved. Source: SIS and Juntos' Administrative Data Component 2: Improve the coverage and quality of the provision of basic preventive health and nutrition services in the Juntos areas Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Children younger Percentage 43.00 0.00 65.00 88.50 than 36 months affiliated to SIS and with complete and timely CRED scheme in the areas of intervention of Juntos in 31-Dec-2011 30-Jun-2017 30-Jun-2017 Amazonas, Cajamarca and Huánuco. Comments (achievements against targets): Target Achieved and Exceeded. Source: SIS 23 The World Bank Results in Nutrition for Juntos SWAp (P117310) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of mothers in Juntos Percentage 0.00 0.00 50.00 28.80 families, with children under 24 31-Dec-2011 30-Jun-2017 30-Jun-2017 months, who have attended at least to one demonstrative session in Amazonas, Cajamarca and Huánuco. Comments (achievements against targets): Target Not Achieved. Despite huge effort to expand the demonstrative session as part of parents counseling process, the end target was not achieved. The main reason is there is not a protocol and registration process in MINSA and MIDIS. In other words, families may have attended the sessions more than this indicator suggests, but the registration system does not allow the full quantification of families participating to these sessions. Source: ENDES Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of health facilities Percentage 0.00 0.00 40.00 91.00 with basic equipment to give 31-Dec-2011 30-Jun-2017 30-Jun-2017 CRED and vaccines in Juntos districts Comments (achievements against targets): Target Achieved and Exceeded. Source: SIGA Component: 3: Strengthen the GoP's capacity to influence nutritional outcomes by improving budgetary planning and monitoring of results Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of health facilities Percentage 98.40 0.00 100.00 100.00 with updated and complete 31-Dec-2011 30-Jun-2017 30-Jun-2017 asset information in SIGA Comments (achievements against targets): Target Achieved. Source: SIGA. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of districts in the first Number 0.00 0.00 75.00 156.00 quintile where there have been 31-Dec-2011 30-Jun-2017 30-Jun-2017 discussion forums about CRED coverage and vaccines (social monitoring) and are registered on MEF’s web page 24 The World Bank Results in Nutrition for Juntos SWAp (P117310) Comments (achievements against targets): Target Achieved. Source: MEF Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of health micro- Percentage 0.00 0.00 60.00 44.10 networks annually verified because they meet the certification criteria in 31-Dec-2011 30-Jun-2017 30-Jun-2017 Amazonas, Cajamarca and Huánuco Comments (achievements against targets): Target Not Achieved. The information collected by ENCRED corresponds to 2016 and was presented by INEI in April of 2017. The ENCRED survey was not launched in 2017 because it is the census planning year. End target was partially reached because of continuing difficulties faced by the Regional Health Directorates and the networks to reach distant health centers to deliver the required supplies in a timely manner. However, the value for 2015 presented by MEF and INEI was 33.2 percent, therefore pointing to a positive trend in the last two years. Source: INEI and ENCRED Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of newborns registered Number 11759.00 0.00 15000.00 19878.00 in SIS before 30 days in Juntos districts in the areas of 31-Dec-2011 30-Jun-2017 30-Jun-2017 intervention of the operation. Comments (achievements against targets): Target Achieved and Exceeded. Source: SIS 25 The World Bank Results in Nutrition for Juntos SWAp (P117310) B. KEY OUTPUTS BY COMPONENT Component 1: Strengthening and consolidating of the Juntos CCT program for families with children under 36 months (Cost $5.50M) 1. Increase demand for nutrition services by strengthening the operational effectiveness of the Juntos Conditional Cash Transfer (CCT) program. 2. Improve coverage and quality of the supply of basic preventive health and nutrition Objective / Outcome services in the communities covered under the PAN including Juntos. Improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos. 1. Percentage of Juntos Children under 12 months that have received the complete CRED scheme according to their age in the areas of intervention of the Juntos CCT Program Objective / Outcome Indicators targeted by this operation. 2. Percentage of Children under 36 months that have received complete CRED scheme according to their age in the areas of intervention. 1. Percentage of Juntos beneficiary children under 24 months registered in the Juntos information system with birth certificate, identity document or CUI, and SIS affiliation. 2. Percentage of children born each year in Juntos households and registered in the SITC Intermediate Results Indicators before 30 days. 3. Number of newborns registered in SIS before 30 days in Juntos districts in the areas of intervention of the operation. 1. Supporting the preparation of the early affiliation strategy for newborns to the Integral Health Insurance (Seguro Integral de Salud – SIS) and Juntos, and the involvement of mayors of Centros Poblados in the cycle of documentation and affiliation of children younger than 12 months to SIS. Key Outputs by Component 2. Supporting the implementation of specific procedures and budgetary provisions to (linked to the achievement of the Outcomes 1 and 2) improve timely access of Indigenous households to nutrition counseling and health services, and to SIS and Juntos benefits. 3. Supporting the identification of legal frameworks hindering the timely registration of indigenous children to Juntos and SIS, and their access to nutrition services, and recommendations to overcome those legal barriers. 26 The World Bank Results in Nutrition for Juntos SWAp (P117310) Component 2: Improving coverage and quality of the provision of basic preventive health and nutrition services in Juntos areas (Cost $5.50M) 1. Increase demand for nutrition services by strengthening the operational effectiveness of the Juntos Conditional Cash Transfer (CCT) program. 2. Improve coverage and quality of the supply of basic preventive health and nutrition Objective / Outcome services in the communities covered under the PAN including Juntos. Improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos. 1. Percentage of Juntos Children under 12 months that have received the complete CRED scheme according to their age in the areas of intervention of the Juntos CCT Program Objective / Outcome Indicators targeted by this operation. 2. Percentage of Children under 36 months that have received complete CRED scheme according to their age in the areas of intervention. 1. Percentage of Children younger than 36 months affiliated to SIS and with complete and timely CRED scheme in the areas of intervention of Juntos in Amazonas, Cajamarca and Huánuco. Intermediate Results Indicators 2. Percentage of mothers in Juntos families, with children under 24 months, who have attended at least to one demonstrative session in Amazonas, Cajamarca and Huánuco. 3. Percentage of health facilities with basic equipment to give CRED and vaccines in Juntos districts. 1. Supporting the dissemination of disaggregated data and promotion of culturally- sensitive service delivery to highlight inequality in access to services to support better nutritional outcomes among indigenous households. 2. Supporting the strengthening of the Municipal Civil Registry Offices (OREC), and the establishment of well-equipped Auxiliary Registry Offices in hospitals and other birth facilities articulated with MINSA, RENIEC (National Registration Service) and the Regional Key Outputs by Component Governments, to support the online issuance of the Certificate of Live Birth. (linked to the achievement of the Outcomes 1 and 2) 3. Supporting the preparation of information on coverage of CRED checkups by mother- tongue and ethnicity; this data allowed estimating and making visible the results and access gaps experienced by indigenous households to nutrition services. 4. Supporting the design and deliver of trainings on nutritious and healthy food preparation for parents of children under 12 months as well as intensified interpersonal counselling, and technical demonstration sessions for verification of Historias Clinicas and FUAs (Formatos Unicos de Atención in Spanish). 27 The World Bank Results in Nutrition for Juntos SWAp (P117310) Component 3: Strengthen the GoP's capacity to influence nutritional outcomes by improving budgetary planning and monitoring of results for selected activities of the PAN (Cost $14.00M) 1. Increase demand for nutrition services by strengthening the operational effectiveness of the Juntos Conditional Cash Transfer (CCT) program. 2. Improve coverage and quality of the supply of basic preventive health and nutrition Objective / Outcome services in the communities covered under the PAN including Juntos. Improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the PAN including Juntos. 1. Percentage of Juntos Children under 12 months that have received the complete CRED scheme according to their age in the areas of intervention of the Juntos CCT Program Objective / Outcome Indicators targeted by this operation. 2. Percentage of Children under 36 months that have received complete CRED scheme according to their age in the areas of intervention. 1. Percentage of health facilities with basic equipment to give CRED and vaccines in Juntos districts. 2. Number of districts in the first quintile where there have been discussion forums about Intermediate Results Indicators CRED coverage and vaccines (social monitoring) and are registered on ME F’s web page. 3. Percentage of health micro-networks annually verified because they meet the certification criteria in Amazonas, Cajamarca and Huánuco. 1. Supporting the strengthening the current Results-based financing approach prioritized by the Government of Peru as part of its overall Early Childhood Development (ECD) policy. 2. Supporting the development and scaling-up the use of management data on key outcomes, aligned with the monitoring systems of the different institutions (Juntos, SIS, Key Outputs by Component MINSA, etc.). (linked to the achievement of the Outcomes 1 and 2) 3. Identifying and monitoring individualized outcomes of children through the Nominal Census ("Padron Nominal"), which provides real-time information to MINSA and the MEF and permits planning activities and allocating resources based on real demand. 4. Increasing the awareness and social appropriation of nutritional outcomes and services for children through the Social Monitoring strategy. 28 The World Bank Results in Nutrition for Juntos SWAp (P117310) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Nelson Gutierrez Task Team Leader(s) Selene del Rocio La Vera, Juan Carlos Martell Rivera Procurement Specialist(s) Patricia De la Fuente Hoyes Financial Management Specialist Claudia Rokx Team Member Patricia O. Orna Team Member William Reuben Social Safeguards Specialist Alessandra Marini Team Member Rory Narvaez Team Member Sara Burga Team Member Hugo Martin Brousset Chaman Team Member Javier Soriano Tabares de Nava Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle US$ (including travel and consultant No. of staff weeks costs) Preparation FY10 0 369,519.55 FY11 0 157,878.74 FY12 0 - 210.08 FY13 0 0.00 Total 0.00 527,188.21 Supervision/ICR FY11 0 43,310.76 FY12 0 183,350.57 FY13 0 178,174.48 FY14 45.860 146,799.22 FY15 46.766 191,468.11 FY16 31.700 146,894.31 FY17 25.485 242,313.19 FY18 1.375 6,268.51 Total 151.19 1,138,579.15 29 The World Bank Results in Nutrition for Juntos SWAp (P117310) ANNEX 3. PROJECT COST BY COMPONENT Actual at Percentage Amount at Approval Project Closing of Approval Components (US$M) (US$M) (US$M) Component 1: Strengthening and consolidating of the Juntos CCT program for families with children 0 5.50 0 under 36 months. Component 2: Improving coverage and quality of the provision of basic preventive health and 0 5.50 0 nutrition services in the Juntos areas. Component 3: Strengthen the GoP's capacity to influence nutritional outcomes by improving 0 14.00 0 budgetary planning and monitoring of results for selected activities of the PAN. Total 0.00 25.00 0.00 30 The World Bank Results in Nutrition for Juntos SWAp (P117310) ANNEX 4: THE ECONOMIC RATIONALE FOR THE ROLE OF JUNTOS IN REDUCING MALNUTRITION The Juntos CCT program “was created at the same time as the acceleration in stunting reduction began. The program was designed as a safety net program to tackle chronic poverty and contribute to the accumulation of human capital. Over time, it included the reduction of chronic malnutrition as an explicit objective” (Marini et al. 2017, pg. 49). 30 As mentioned in the main text, the operation was designed with a strong focus on strengthening the functioning of Juntos. This design feature had a strong rationale: • As mentioned in the main text, a vast body of international evidence pointed to the important role that conditional cash transfer (CCT) programs could play towards households investing more in human development. By the operation’s approval, extensive literature, including experimental studies and non-experimental evaluations, testified to the potential of CCT programs such as Juntos to “achieve significant results in reducing chronic malnutrition when accompanied by adequate coverage and quality of health and nutrition services that promote behavioral changes in mothers with respect to child health, hygiene, care and feeding practices.”31 • More specific to the Peruvian context, by 2011, at least three rigorous evaluations of Juntos had shown that the program improved beneficiaries’ dietary intake and increased their use of health services. A 2009 evaluation conducted by Perova and Vakis32, covering the 2006 and 2007 period, combined several data sources including the National Household Survey (ENAHO), the National Register of Municipalities, and Juntos’ administrative data. This work showed that the program, while having moderate impacts on poverty reduction, led to increases in “the utilization of health services for both children and women, improved the nutritional intake of participating households (…) and (led to) significant increases in school registration and attendance”. A second evaluation by Perova and Vakis, published in 2012 but available before the operation’s approval, applied instrumental variable techniques to nationally representative household data from 2006 to 2009 to estimate the overall impact of Juntos five years after its rollout33. This research broadly confirmed the previous findings, showing that Juntos increased utilization of health services among children under the age of 6 and women of childbearing age. Furthermore, the study found that the likelihood of Juntos’ beneficiary children attending school, conditional on being enrolled in school, increased by 25 percentage points. Finally, by 2011, Jaramillo and Sanchez had measured the impacts of the Juntos program on nutritional outcomes. Relying on ENDES data from 2008 to 2010, and using a combination of difference-in- difference and matching techniques, the authors found that the program, while not having a significant impact on overall chronic malnutrition, did in fact contribute to the reduction of early 30 See Marini, A., Rokx, C. and Gallagher, P., 2017, “Standing Tall: Peru’s Success in Overcoming its Stunting Crisis”, World Ba nk. http://documents.worldbank.org/curated/en/815411500045862444/pdf/117053-OUO-FINAL-Peru-book-FA-SINGLE-PAGES- with-cover.pdf 31 See World Bank, 2011, pg. 39, “Project Appraisal Document on A Proposed Loan In The Amount Of Us$25.0 Million To The Republic Of Peru for the Juntos Results For Nutrition Project”, and Ariel Fiszbein and Norbert Schady (with Francisco H.G. Ferreira, Margaret Grosh, Niall Keleher, Pedro Olinto, and Emmanuel Skoufias), 2009, “Conditional Cash Transfers: Reducing Present and Future Poverty”, The World Bank. 32 Perova and Vakis, 2009, “Welfare impacts of the Juntos Program in Peru: Evidence from a non -experimental evaluation”, The World Bank mimeo, as quoted in Perova and Vakis, 2012, pg. 54, “5 Years in Juntos: New Evidence on the Program’s Short and Long-Term Impacts”, Economía Vol. XXXV, N° 69, pp. 53-82. 33 Perova, E. and Vakis, R., 2012, “5 Years in Juntos: New Evidence on the Program’s Short and Long-Term Impacts”, Economía, Vol. XXXV, N° 69, pp. 53-82. 31 The World Bank Results in Nutrition for Juntos SWAp (P117310) chronic malnutrition (i.e. in children below 5 years of age). 34 • Juntos’ limited impact on final outcomes such as the reduction of chronic malnutrition was due to a number of fixable shortcomings. According to a 2009 evaluation of the program’s implementation led by the World Bank35, despite being the best targeted social program in Peru, Juntos suffered from design and implementation issues, in particular with respect to the verification of beneficiaries’ compliance with their health and nutrition co-responsibilities. Marini et al. (2017) report that, at the time of the operation’s approval, Juntos was struggling with verifying that mothers were bringing all their children for height and weight checkups as part of their co-responsibilities. Key bottlenecks in the program’s implementation had been identified. There was a lack of clarity on which health services pregnant women and children could expect to receive in return for their cash payments as standard package of health services for Juntos’ beneficiaries had not been defined. The boundaries of roles and responsibilities between the verification process of the co-responsibilities and cash payments to beneficiaries were blurred, and the beneficiaries’ registry was seldomly updated. Spot checks on beneficiaries were not carried out, and quality control mechanisms were weak.36 • In line with the above, improving the functioning of Juntos became a priority. As reported in the operation’s appraisal document (pg. 39), by the time of approval, “with help from the World Bank Juntos for Nutrition technical assistance, a multi-agency think-tank had examined ways to increase Juntos’ impact on nutrition outcomes by better articulating the demand and supply side of the program and adjusting operational procedures”. At the same time, strengthening the implementation and verification of beneficiaries’ co-responsibilities, so that the program could achieve its full potential and contribute more effectively to the fight against malnutrition, became a cornerstone of the PAN. As the operation was envisioned to be a key supporter of the implementation of the PAN, the World Bank team designed its demand side component with the clear objective of improving the functioning of Juntos, in line with the findings mentioned above. At the same time, the above research also stressed that the limited coverage and quality of the associated health and nutrition services was preventing Juntos from achieving important results in terms of poverty and malnutrition reduction; accordingly, the supply-side of the operation was designed with the clear objective of providing incentives for increasing the quantity and the quality of services at the local level. Looking back, the importance that the operation placed on Juntos seems to have been quite appropriate. As Marini et al. (2017) report, even though it is obviously impossible to attribute the reduction in malnutrition witnessed in Peru solely to Juntos, the program “was certainly an important catalytic force for improvements in the supply of health/nutrition services across the country” (pg. 49). 34 Jaramillo, M. and Sánchez, A., 2011, “Impacto del programa Juntos sobre nutrición temprana”, Lima: GRADE. 35 World Bank Programa Juntos: Antecedentes, Avances y Resultados de Evaluación, April 2009. 36 World Bank, 2013, Improving Nutrition Through Multisectoral Approaches. 32