URUGUAY Noncommunicable Diseases Prevention Project Report No. 131480 DECEMBER 6, 2018 © 2019 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2019. Uruguay— accuracy of the data included in this work. The Any queries on rights and licenses, including Noncommunicable Diseases Prevention Project. boundaries, colors, denominations, and other subsidiary rights, should be addressed to Independent Evaluation Group, Project information shown on any map in this work do World Bank Publications, The World Bank Performance Assessment Report 131480 not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC Washington, DC: World Bank. World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 131480 PROJECT PERFORMANCE ASSESSMENT REPORT URUGUAY NONCOMMUNICABLE DISEASES PREVENTION PROJECT (IBRD-74860) December 6, 2018 Human Development and Economic Management Independent Evaluation Group Currency Equivalents (annual averages) Currency Unit = Uruguayan peso (UR$) 2008 $1.00 UR$20.94 2013 $1.00 UR$20.48 2009 $1.00 UR$20.58 2014 $1.00 UR$23.25 2010 $1.00 UR$20.06 2015 $1.00 UR$27.33 2011 $1.00 UR$19.31 2016 $1.00 UR$30.16 2012 $1.00 UR$20.31 2017 $1.00 UR$28.66 Abbreviations ASSE State Health Services MSP Ministry of Public Health Administration (Administración (Ministerio de Salud Pública) de los Servicios de Salud del M&E monitoring and evaluation Estado) NRFS National Risk Factor Survey AGESIC Agency for the Development of NCD noncommunicable disease Electronic Government and PAHO Pan American Health Information and Knowledge Organization Society (Agencia para el PDO project development objective Desarrollo del Gobierno de PPAR Project Performance Assessment Gestión Electrónica y la Report Sociedad de la Información y PPENT Noncommunicable Diseases del Conocimiento) Prevention Project (Programa CVD cardiovascular disease de Prevención de Enfermedades No Transmisibles) FONASA Fondo Nacional de Salud PSCU Project Support and IAMC Collective Medical Assistance Coordination Unit Institutions (Instituciones de RAP primary care networks (Redes Asistencia Médica Colectiva) de Atención Primaria) ICR Implementation Completion SNIS National Integrated Health and Results Report System (Sistema Nacional IEG Independent Evaluation Group Integrado de Salud) WHO World Health Organization All dollar amounts are U.S. dollars unless otherwise indicated. Fiscal Year Government: January 1 – December 31 Director-General, Independent Evaluation Ms. Sophie Sirtaine Director, Human Development and Economic Management Mr. Auguste Tano Kouame Manager, Corporate and Human Development Ms. Emanuela Di Gropello Task Manager Ms. Gisela M. Garcia ii Contents Preface ................................................................................................................................................................vii Summary .......................................................................................................................................................... viii 1. Background and Context ..........................................................................................................................1 Project Context .............................................................................................................................................4 2. Relevance of the Objectives and Design.............................................................................................6 Objectives .......................................................................................................................................................6 Relevance of the Objectives ..................................................................................................................... 7 Design ..............................................................................................................................................................8 Components ................................................................................................................................................................ 8 Implementation Arrangements ............................................................................................................................. 9 Relevance of Design....................................................................................................................................9 3. Implementation ......................................................................................................................................... 11 Implementation Experience .................................................................................................................... 12 Safeguards Compliance ......................................................................................................................................... 12 4. Achievement of the Objectives ............................................................................................................ 13 Objective 1: Strengthen Health Delivery Services ......................................................................................... 13 Outputs ........................................................................................................................................................................ 14 Outcomes ................................................................................................................................................................... 14 Objective 2: Strengthen MSP’s Health Policy Framework for NCDs ...................................................... 21 Outputs ........................................................................................................................................................................ 21 Outcomes .................................................................................................................................................................. 24 5. Efficiency ...................................................................................................................................................... 28 6. Ratings .......................................................................................................................................................... 29 Outcome ...................................................................................................................................................... 29 Risk to Development Outcome ............................................................................................................ 29 Bank Performance .................................................................................................................................... 30 Quality at Entry ........................................................................................................................................................ 30 Quality of Supervision ............................................................................................................................................ 31 Borrower Performance ............................................................................................................................ 32 Government Performance.................................................................................................................................... 32 iii Implementing Agency Performance................................................................................................................. 33 Monitoring and Evaluation .................................................................................................................... 33 Design ......................................................................................................................................................................... 33 Implementation ....................................................................................................................................................... 34 Use................................................................................................................................................................................ 35 7. Lessons.......................................................................................................................................................... 35 Bibliography..................................................................................................................................................... 40 Boxes Box 4.1. Montevideo: Continuous Commitment to Noncommunicable Disease Prevention 23 Box 4.2. Uruguay National Response to Noncommunicable Diseases: Relevant World Health Organization Indicators ........................................................................................................... 25 Figures Figure 1.1. Deaths Attributed to Noncommunicable Diseases, Uruguay and Comparable Countries .........................................................................................................................................2 Figure 1.2. Population Aged 60 Years and Older in Uruguay and Selected Countries ...............3 Figure 1.3. Cause of Death by Age Group in Uruguay, 2016 ...............................................................3 Figure 4.1. Prevalence of NCD Risk Factors in Population Aged 25–64 Years ............................ 19 Figure 4.2. Death Rates for Cardiovascular Diseases and Diseases of the Circulatory System in Uruguay ................................................................................................................................... 20 Tables Table 1.1. Main Noncommunicable Diseases as Cause of Death ....................................................... 1 Table 4.1. Original and Revised Targets of Selected Outcome Indicators .................................... 15 Appendixes Appendix A. Basic Data Sheet ................................................................................................................... 45 Appendix B. Methods and Evidence ....................................................................................................... 49 Appendix C. Other Programs Contributing to Project Development Objectives .................. 58 Appendix D. List of Persons Met.............................................................................................................. 62 This report was prepared by Gisela M. Garcia, task team leader, and Natalia Araujo, consultant, who assessed the project in April 2018. The report was peer reviewed by Daniel Maceira and panel reviewed by Judyth Twigg. Yezena Yimer provided administrative support. iv Principal Ratings Indicator ICR ICR Review PPAR Outcome Moderately satisfactory Moderately satisfactory Moderately satisfactory Risk to development Modest Modest Substantial outcome Bank performance Moderately satisfactory Moderately satisfactory Moderately satisfactory Borrower performance Moderately satisfactory Moderately satisfactory Moderately satisfactory Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of the ICR. PPAR = Project Performance Assessment Report. Key Staff Responsible Division Chief or Sector Project Task Manager or Leader Director Country Director Appraisal Luis Orlando Pérez Keith E. Hansen Pedro Alba Completion Luis Orlando Pérez Daniel Dulitzky Jesko S. Hentschel v IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About This Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, country assistance strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, highly unsatisfactory. Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes). The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vi Preface This is a Project Performance Assessment Report (PPAR) by the Independent Evaluation Group (IEG) of the World Bank Group on the Noncommunicable Diseases (NCD) Prevention Project (P050716; known as PPENT in keeping with its Spanish acronym). The project was selected for a PPAR to capture lessons from one of the first exclusively NCD-focused projects supported by the World Bank. The project was approved on August 28, 2007, and the closing date was extended twice from the original December 31, 2012, to December 31, 2015, through two level II project restructurings. The project was financed with an International Bank for Reconstruction and Development (IBRD) loan of $25.3 million; total expenditure was below estimate ($21.7 million), including borrower’s contribution of $2.6 million. The project development objective of the NCD Prevention Project was “to support the borrower’s efforts to further strengthen its health delivery services and the current health policy framework for NCDs through the expansion of access and the quality of primary health care services related to NCD early detection, and the provision of specialized medical care to avoid or reduce exposure to NCD risk factors and their health effects.” (World Bank 2007, 5). This PPAR presents IEG’s findings and conclusions based on a review of the World Bank’s project documentation, combined with a field mission to Uruguay carried out between April 16 and April 26, 2018, and consequent data collection. IEG conducted interviews with a range of project stakeholders, including staff responsible for the project coordination and implementation, government counterparts and partners, World Bank staff, and representatives from the academia and civil society. Appendix B provides information on evaluation methods and sources. Appendix D shows the list of people met. Following standard IEG procedure, the draft PPAR was shared with relevant government officials in Uruguay for their review and comment; no comments were received from the borrower. vii Summary This is a Project Performance Assessment Report (PPAR) on the Uruguay Noncommunicable Diseases (NCD) Prevention Project (P050716 [Programa de Prevención de Enfermedades No Transmisibles; PPENT]). The project development objectives (PDOs) were “to support Government's efforts to further strengthen its health delivery services and health policy framework for NCD.” (World Bank 2007, 5). These objectives did not change during implementation. The relevance of the PDOs is rated substantial. The project was highly relevant at appraisal and remained relevant during the whole implementation period and after completion, encouraging the prevention and early detection of the NCDs with the highest prevalence in the country. PPENT objectives were timely, accompanying important structural changes to the health care system. Shortcomings had to do with the narrow definition of the objectives to strengthen the health policy framework, despite recognizing the need to go beyond the health sector to effectively address NCDs. The relevance of project design is rated substantial. The design had several robust elements; for example, strengthening the public provider primary care services was a crucial element to improve access to quality health care services, and one highly relevant pilot supported evidence-based learning for changes in incentives in providers’ payment. However, the project was very ambitious in its design, underestimating the time and resources needed to equip, motivate, and train providers at the primary care level on the importance of preventing NCDs, and then define, measure, and monitor providers’ performance. In addition, the project could have included more discussion about its complementarities with activities led by other ministries, municipalities, and key actors outside the government. Project implementation was affected by several changes of government and the implementation model chosen by the government of Uruguay. The project was approved on August 28, 2007, became effective on January 9, 2008, and closed on December 31, 2015 (three years after its original closing date). The project used existing administrative arrangements and staff, which led to a high level of institutionalization and capacity building but also caused meaningful delays. Four restructurings amended implementing arrangements, reallocated funds because of changes in activities, revised the results framework, and extended the closing date twice. The total cost ($21.7 million) was 76 percent of initial estimates. The objective of strengthening health services was rated modest given insufficient information regarding providers’ performance to show increased access and quality of services, especially for the public provider. The objective of strengthening the health viii policy framework was rated substantial due to its contributions to Uruguay’s progress on developing a national response to NCDs, especially in the areas related to integrated strategy, guidelines and regulations, data generation, and a set of national targets. The project increased the capacity within the Ministry of Public Health (Ministerio de Salud Pública; MSP) and the main public health service provider (the State Health Services Administration [Administración de los Servicios de Salud del Estado]; ASSE) to respond to the epidemiological shift in the country. This entailed investments in technology, equipment, capacity building, information systems, and the development of new management and decision-making tools. The project helped Uruguay align its health goals with NCD epidemiological priorities by supporting attention to evidence at the national level. The project made important contributions to Uruguay’s progress on developing a national response to NCDs, especially in the areas related to guidelines and regulations, data generation, definition of national targets, and change in the incentive structure for providers. Data generated by the project was key to set targets for NCDs at the national and provider levels. Because of the evidence generated by the project, Uruguay’s health goals gave prominence to the prevention and timely diagnosis of NCDs. The incentive structure introduced by the project promoted a change in the payment model to all providers, emphasizing the attention to performance measures that went beyond NCDs. Sustained attention to providers’ performance measurement and improvements is high on the agenda of the government. The implementation model led to significant institutional strengthening and capacity building within the ASSE and MSP. Given the consensus-based approach in the Uruguayan government, working with staff in MSP and ASSE was central to promoting sustained changes in approaches and management tools after project closure. Strengthening ASSE’s primary care was key given the low performance of the public provider and the new responsibilities introduced by the reform. Available data confirm the importance of project investments in institutional capacity, technology and equipment, and information systems for the public provider. Yet, given its broader scope, limitations in the design of methodology for monitoring and evaluation, and delays in having the equipment and technology fully operational, the project fell short in showing results of these efforts on the accessibility and quality of health services within its implementation period. The limited information available on providers’ performance and ASSE’s relatively inferior performance compared with private providers when the information was available, hindered achievement of the objective of strengthening health services. ix Additionally, insufficient budget allocation and fragmented attention to health promotion activities limited efficacy in achievement of the objective of strengthening the health policy framework and represent a risk. Attention to integrated NCD prevention and health promotion activities was low throughout project implementation, with limited evidence of effective coordination with main actors beyond MSP to effectively address the multidimensionality of NCD risk factors. Project efficiency is rated modest. Preventing NCDs can be highly cost-effective, and available data for Uruguay indicate that this may indeed be the case. Two shortcomings are noted, however: (i) significant delays in implementation and limited coordination reduced cost efficiency; and (ii) attribution of reductions in mortality and morbidity to project investments is complex. Taking into account the substantial rating for relevance of objectives and design, the ratings for the two objectives—modest and substantial—and the modest rating for efficiency, the overall outcome is rated moderately satisfactory. Based on the degree of institutionalization of some of the activities supported by the project, and given the shortcomings associated with fragmented attention to health promotion and insufficient budget allocation to providers’ performance monitoring, the risk to development outcome is rated substantial. Bank performance is rated moderately satisfactory due to a moderately unsatisfactory rating for quality at entry and a moderately satisfactory rating for World Bank supervision. Project components were relevant to achievement of the PDOs, but the design was overly ambitious and underestimated the time and capacity needed to fully implement the wide range of activities included at entry, given the implementation modality and the ongoing reform process. Yet, the continuous support received by the World Bank team during implementation was unanimously praised, government officials appreciated the commitment of the World Bank supervision team to understand the local context to overcome obstacles and propose solutions. Borrower performance is rated moderately satisfactory due to a moderately unsatisfactory rating for government performance and a moderately satisfactory rating for implementing agencies’ performance. Changes in government, as well as procurement issues, negatively affected project performance. Capacity issues were a constant in explaining performance, yet the government of Uruguay did not ensure the sustainability of the efforts promoted by the project. However, MSP and ASSE’s commitment to the project was high, and staff dedicated extra hours, beyond their normal responsibilities, to ensure ownership and relevance of activities supported by the project. x Monitoring and evaluation is rated modest. The project supported important efforts in data collection that were instrumental in decision making at the policy level but of limited use for assessing providers’ performance. Shortcomings were noted on monitoring and evaluation design, implementation, and use. These include lack of adequate indicators to measure PDOs, process indicators to assess providers’ performance not included or dropped at restructuring, and administrative data not collected from providers except on a limited basis for the Previniendo pilot. Four lessons emerge: • Preventing NCDs requires a multidimensional approach that goes beyond strengthening the role of MSP and health services. This understanding was embedded in the project appraisal document; however, it was not fully articulated in project objectives or design. The project focused on secondary prevention under the control of MSP, paying limited attention to behavior change activities beyond that of providers. Although strategies to influence changes in the behavior of the population with respect to NCD risk factors were implemented in schools and at the community level, their emphasis and scale were limited, and continuity and sustainability relied mostly on interest on the part of individuals or municipalities. Better strategic use of other actors’ roles and capacities could have improved the focus, efficacy, and efficiency of the resources devoted to public awareness and prevention activities. The extent to which MSP can effectively work with and influence other ministries and key civil society actors should not be taken for granted but should instead be an explicit goal of these types of projects. Moreover, attention to promoting behavior change of the population should be as important as investments in equipment, technology, and incentives for health providers. • Projects implemented during important reform processes must take into consideration the timing of the reform and adjust project expectations and ambitions accordingly. The project introduced incentives in providers’ payments in three departments (an administrative subdivision of the country) to motivate greater focus on prevention of NCDs. But before the project was able to show results, these incentives were scaled up to include all 19 departments of the country. As project scale increased, expectations of results within the lifetime of the project became unrealistic: providers had not previously reported on performance; providers’ structures, population served, and incentives to comply with performance measures differed; defining performance indicators for NCD management proved difficult; no information system was in place and there was limited capacity to support effective reporting; and the MSP did not have the xi capacity to monitor providers’ compliance. Yet, the World Bank did not reassess feasibility or adjust project expectations accordingly. • Projects with a strong focus on capacity building need to be more realistic about what can be achieved within the project lifetime. Project design underestimated the time and resources needed to equip, motivate, and train providers at the primary care level on the importance of preventing NCDs, and to then define, measure, and monitor providers’ performance (and the learning process involved in continuously redefining and adjusting those measures). Much of the institutional capacity, technology, and equipment funded by the project was not fully operational until later during implementation, which limited its use for NCD management at the provider level. • Innovative projects like the PPENT should devote more attention to capture learning from implementation. In its original design, the project included an impact evaluation of the pilot but did not include provisions to capture or systematize the learning from an innovative and appealing implementation model that relied extensively on the existing structure and staff of the MSP. All stakeholders contacted by the Independent Evaluation Group confirmed the significant institutional strengthening and capacity building that ASSE and MSP received and the learning-by-doing facilitated by the project. This is especially evident in sustained changes implemented in management tools, evidence-based decision making, and information systems. Yet, this learning was not well captured by the World Bank, missing an important opportunity to share this experience in other contexts. Auguste Tano Kouame Director Human Development and Economic Management Independent Evaluation Group xii 1. Background and Context 1.1 Uruguay is a high-income country with some of the highest human development indicators in the Latin America and the Caribbean Region. Uruguay’s socioeconomic indicators are comparable to those of Organisation for Economic Co-operation and Development countries, including the adult literacy rate (98.5 percent) and life expectancy (77 years). The country has one of the highest rates of health service coverage and financial protection in Latin America and the Caribbean (World Bank 2017). 1.2 Noncommunicable diseases (NCDs) are the main cause of mortality and morbidity in Uruguay, accounting for 86 percent of total deaths (26,000 people a year), 76 percent of premature deaths, and 89 percent of years lived with disability.1 Cardiovascular diseases (CVDs) are the main cause of death, accounting for 31 percent of total deaths and 24 percent of premature years of life lost (WHO 2014). The share of total deaths attributable to CVDs has experienced a steady decrease over the years, going from 40.5 percent in 1990 to the current 31 percent. Cancer has been the second leading cause of death for decades and the first cause of premature death, responsible for more than 9,000 deaths a year or about 28 percent of total deaths and 30 percent of premature deaths.2 Diabetes and chronic respiratory diseases are responsible for 7 percent and 6 percent of total deaths, respectively. The relative importance of these diseases has experienced slight change over time (see table 1.1). Table 1.1. Main Noncommunicable Diseases as Cause of Death 1990 2004 2016 Percentage Percentage Percentage of total Ranking of total Ranking of total Ranking Cause of Death deaths (no.) deaths (no.) deaths (no.) Cardiovascular diseases 40.5 1 34.6 1 31.0 1 Neoplasms 25.9 2 27.2 2 27.7 2 Diabetes 5.2 3 5.6 5 6.8 4 Chronic respiratory 4.7 4 6.2 3 6.3 5 diseases Neurological disorders 4.3 5 5.7 4 7.2 3 Source: Independent Evaluation Group based on data from Global Burden of Disease. 1.3 Compared with the regional average, these figures are relatively high, and resemble more closely those of richer countries. Although the burden of NCDs in Uruguay has remained relatively stable at almost 90 percent of total deaths since 1990— as is the case in other high-income countries—the average for Latin America and the Caribbean, although increasing, remains relatively lower, with deaths from NCDs increasing from fewer than 60 percent in 1990 to 76 percent in 2016 (figure 1.1). 1 Figure 1.1. Deaths Attributed to Noncommunicable Diseases, Uruguay and Comparable Countries 100% 89% Percent of total deaths 87% 86% 76% 58% 0% 1996 2013 1990 1991 1992 1993 1994 1995 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2014 2015 2016 High-income Uruguay Latin America and Caribbean Source: Independent Evaluation Group based on data from Global Burden of Disease. 1.4 Varying demographic profiles help explain the differences in the prevalence of NCDs across countries. Uruguay was one of the first countries in Latin America and the Caribbean to undergo the so-called first demographic transition, characterized by continuous decreases in fertility and mortality rates and consequent increases in life expectancy. As a result, Uruguay has one of the oldest populations in the region, with 19.1 percent aged 60 years or older; and, among those older than 60, 22 percent are aged 80 years or older. These percentages are high not only compared with those of other countries in Latin America and the Caribbean but also internationally (figure 1.2). 1.5 The prevalence of NCDs increases with age, especially after age 50 years. The demographic transition in Uruguay was accompanied by what is known as the epidemiological transition, a process through which the importance of NCDs as the main cause of mortality and morbidity overtakes that of infectious diseases. Figure 1.3 shows the distribution of deaths by cause for different age groups in Uruguay. 2 Figure 1.2. Population Aged 60 Years and Older in Uruguay and Selected Countries Source: Independent Evaluation Group based on data from the United Nations Population Division Note: LAC = Latin America and the Caribbean. Figure 1.3. Cause of Death by Age Group in Uruguay, 2016 7.6% 3.4% 10.5% 45.4% 42.4% 39.7% Injuries 89.5% 87.8% Non-communicable 49.3% diseases 46.2% 49.8% Communicable, maternal, neonatal, and 8.4% 8.3% 7.0% 4.6% nutritional diseases Under 5 5-14 15-49 50-69 70+ Source: Independent Evaluation Group based on data from Global Burden of Disease. 1.6 Unhealthy lifestyles also lead to the development of NCDs. Poor diet, lack of physical activity, and tobacco and alcohol consumption negatively affect obesity, high blood pressure, and high sugar and cholesterol levels in blood. Data from the Global Burden of Disease study (GBD) show behavioral and metabolic risks are responsible for 27 percent and 23 percent, respectively, of the burden of mortality and morbidity 3 attributed to NCDs in Uruguay.3 These figures are very similar to those available at the time of project appraisal in 2006 (30 percent and 24 percent, respectively). 1.7 Behavioral and metabolic risks responsible for NCDs are common in countries in Latin American and the Caribbean. Twenty-three percent of the region’s population is obese, and 58 percent is overweight, and these figures are expected to increase (Institute for Quality of Life 2016; Weber et al. 2012). Rising levels of hypertension are seen in these countries, with prevalence rates ranging from 20 percent to 40 percent of the adult population (Rasolt 2014; Salicrup, Ordunez, and Engelgau 2018). 1.8 Uruguay is not different. Hypertension is one of the main health problems affecting the Uruguayan population, and its prevalence there is among the highest in the region. Hypertension can cause CVD (that is, stroke, heart attack, or aneurysm), kidney disease, and even blindness. If not controlled or treated, ultimately it can lead to death. Evidence suggests that high levels of under-diagnosed and undertreatment exist among those with the condition. 1.9 Obesity and overweight rates were estimated at about 60 percent of the adult population (those 25 to 64 years old) in 2006 (Ministry of Public Health 2006). Furthermore, data from the Second National Survey of Risk Factors for NCDs carried out in 2013 shows that about 37 percent of the Uruguayan population aged 15 to 64 can be considered at high risk of developing NCDs, showing three or more risk factors simultaneously. 1.10 The poor endure more suffering than the rich, not only because of the large financial cost associated with NCD treatment but also the higher level of exposure to risk factors. Higher exposure to risk factors such as tobacco, lack of exercise, and poor diet are in many cases associated with lack of information and resources. In Uruguay, the prevalence of both high blood pressure and diabetes has been shown to be higher among the poor and those with lower levels of education (Sandoya 2016). 1.11 Uruguay shows important inequalities in access to health care because of the cost of health care. The 2014 National Health Survey shows that 35 percent of the population considers the cost of health care (including for example, out-of-pocket payments or transportation costs) to be one of the main barriers to access. Socioeconomic inequalities, favoring the relatively better-off, have also been found with respect to waiting times, number of health care visits, and preventive tests and treatments (Balsa et al. 2009). Project Context 1.12 The Noncommunicable Diseases Prevention Project (Programa de Prevención de Enfermedades No Transmisibles; PPENT) was designed to respond to the demographic 4 and epidemiological profile described in the previous section when Uruguay was undergoing a major health care system reform. Through the launch of the National Integrated Health System (Sistema Nacional Integrado de Salud; SNIS) in 2007, the reform brought the previously fragmented private and public subsystems under one umbrella to provide more comprehensive and equitable health coverage to all. It separated the Ministry of Public Health (Ministerio de Salud Pública; MSP) from the direct provision of health services, assigning the public provision to the State Health Service Administration (Administración de los Servicios de Salud del Estado; ASSE). Data from MSP indicate that over 1.5 million people were incorporated to the SNIS and that coverage for the poorest increased (Ministry of Public Health 2015a). 1.13 The reform aimed at better aligning the country’s health care delivery system to its epidemiological profile and shifting from an assistance-based or treatment-oriented model toward greater emphasis on primary health care and health promotion and prevention activities.4 Consequently, the country put into motion several initiatives aimed at the prevention, treatment, and control of the main NCDs affecting the population and their risk factors. 1.14 To motivate providers to devote more attention to key prevention activities, in October 2008 the health care reform changed the way health care providers were paid for their services. The reform combined a risk-adjusted capitation payment scheme (based on sex and age of the beneficiary population) with incentives tied to the attainment of health goals (metas sanitarias) based on the country’s health priorities.5 1.15 Approximately 9 percent of the total resources health providers receive from the Fondo Nacional de Salud (FONASA)6 were tied to compliance with these goals (Buglioni et al. 2011). Goals were set for providers based on the epidemiological profile of their users. Providers were expected to close the gap between the expected and actual performance based on survey data and the established goals. Table B.3 in appendix B provides a summary of the different goals set and their timeline. 1.16 The level of providers’ compliance with these goals varied, with private providers (Collective Medical Assistance Institutions [Instituciones de Asistencia Médica Colectiva]; IAMC) showing the best performance. ASSE, the public provider, has so far shown poorer performance than IAMC (FEMI n.d.). Incentives for compliance within the ASSE are lower than for private providers because ASSE receives FONASA funds for fewer than one-third of its beneficiaries and as a public provider cannot make use of the additional resources as private providers do. In the case of the Collective Assistance Institutions, these payments account for 5–6 percent of their total net income.7 5 1.17 Health care reform was accompanied by an increase in public expenditures on health. Health care expenditures in Uruguay currently account for about 9–10 percent of gross domestic product (GDP), higher than the regional average of 7.4 percent but lower than the average expenditure of high-income countries (12 percent).8 During the past decade, health care expenditure in the country has kept pace with GDP, and increased at an average annual rate of 6.4 percent, measured in real terms, from 2007 to 2014.9 1.18 Primary care networks (Redes de Atención Primaria; RAP) were created, and several actions were introduced to strengthen primary care. The Inter-American Development Bank is currently supporting ASSE RAPs in the south region of the country in the form of a pilot. This pilot includes several of the activities originally included in the PPENT, such as integrated information systems, guidance and protocols, training of health personnel, referral systems, management tools, and performance measures. The Pan American Health Organization is currently supporting MSP’s systematization of good practices to further strengthen the primary care model.10 1.19 Other actors not directly supported by the project were very active in NCD prevention. This includes the Honorary Commissions for Cancer and Cardiovascular Diseases,11 the National Alliance for Preventing NCDs (Alianza Nacional para Prevención de ENT), the Secretary of Sports and Recreation, the National Institute of Food (Instituto Nacional de Alimentación), and the Network of Physical Activity of Uruguay (Red de Actividad Física del Uruguay). 2. Relevance of the Objectives and Design Objectives 2.1 The project development objective (PDO) of the PPENT as stated in the loan agreement was “to support the Borrower’s efforts to further strengthen [i] its health delivery services and [ii] the current health policy framework for NCDs through the expansion of access and the quality of primary health care services related to NCD early detection, and the provision of specialized medical care to avoid or reduce exposure to NCD risk factors and their health effects” (World Bank 2007, 5). 2.2 The PDO stated in the project appraisal document was slightly different: “to support the Government’s efforts to further strengthen its health delivery services and the current health policy framework for NCDs” (World Bank 2008, 10). The document continues by saying “the specific development objectives of the proposed operation would be (i) to expand accessibility and quality of primary health care services related to selected NCDs early detection and medical care; and (ii) to avoid and reduce exposure to selected NCDs risk factors as well as their health effects.” 6 2.3 This PPAR, following World Bank and Independent Evaluation Group (IEG) guidelines, concurs with the Implementation Completion and Results Report (ICR) and ICR Review in understanding the project's objectives to have been the stated in the loan agreement. Relevance of the Objectives 2.4 The relevance of focusing on avoiding and reducing exposure to risk factors for selected NCDs and their health effects is unquestionable given the data presented in the background and context sections. The PPENT was highly relevant at appraisal and remained relevant during the entire implementation period. It tackled the country’s main health issues, encouraging the prevention and early detection in primary care facilities of NCDs with high prevalence in the country and focusing on the risk factors responsible for an increasing share of the NCD burden and not being addressed by other governmental programs. 2.5 PPENT objectives were timely in that they accompanied important structural changes to the health care system. Stakeholders contacted by IEG consistently referred to the importance of the World Bank’s support to help the country clarify the health goals pursued by the reform and adopt an incentive mechanism to align providers’ performance with these goals. 2.6 The PDOs are still highly relevant to Uruguay’s national health strategy as outlined in the country’s 2020 National Health Objectives (Objetivos Sanitarios Nacionales 2020; MSP 2015). The 2020 National Health Objectives prioritize NCD prevention, screening, and treatment. This is consequently reflected in the country’s health goals and the associated performance indicators by which providers are being paid.12 2.7 The inclusion of indicators related to NCD prevention among the MSP’s revised health goals reinforces the continuous relevance of project objectives and targeted the “incentives to performance” approach promoted by the project. This renewed attention to NCDs in the health goals is also a strong signal of the country’s commitment to addressing NCDs targeted by the project, not only throughout the project implementation period but even after closure (see table B.3 in appendix B). 2.8 The PPENT maintained its relevance to the World Bank Group country strategies developed throughout its implementation period. The project was fully aligned with the Bank Group 2005–10 Country Assistance Strategy and the FY10–15 Country Partnership Framework. The project contributed to the former’s pillar on improving living standards through its strengthening of access to and quality of health services for NCDs and to the 7 latter’s pillar on Social Inclusion and Equity through its support to the government's health sector reform agenda. 2.9 There were shortcomings, however. Although the literature and the project narrative in the project appraisal document recognized the need to go beyond the health sector to address NCDs—reflected in some of the actions supported by the project at the municipal and community levels and in the proposal developed by the project for a strategic plan—objectives were narrowly defined to strengthen the health policy framework. Changing Uruguayans’ behavior affecting targeted risk factors could have been part of the project objectives. 2.10 The project objectives also could have made the indirect poverty targeting more explicit. Although the project did not target the poor per se, it acknowledged that NCDs disproportionally affect those who are relatively worse off (World Bank 2008, 7–9). Consequently, the project aimed at improving services to the poor by giving greater focus to strengthening ASSE’s health services noting that ASSE’s beneficiaries are mainly low-income families, many of them informal workers or inactive, without IAMC health coverage. However, it did not include provisions to monitor the results for this subgroup. 2.11 The relevance of objectives is rated substantial. Design Components 2.12 The project included four components, three technical, described in the following paragraphs, and one devoted to project management by coordinating administrative processes through the Project Support Coordination Unit (PSCU). 2.13 Component 1: Strengthening the capacity of MSP to address the country’s changing epidemiological profile. This included the development of (i) an integrated health information system, an epidemiological surveillance on NCDs, and a performance monitoring system for public and private providers; (ii) a national health promotion strategy to educate the public on NCD risk factors and promote healthy lifestyles; and (iii) regulatory frameworks to enhance the effectiveness of NCD programs. 2.14 Component 2: Improving access to quality health care services for prevalent NCDs in public primary care facilities aimed at strengthening the capacity of the public health system to screen for NCDs (hypertension, CVDs, obesity, diabetes, and selected preventable cancers). Activities included the provision of technology and medical 8 equipment to primary care facilities for detection and treatment of NCDs; management tools, including adoption of quality standards and improved referral systems; and training to health workers on NCD prevention, screening, and management. 2.15 Component 3: Implementation of an NCD prevention pilot program (Previniendo) in three departments (an administrative subdivision of the country) to enhance NCD control and risk factor prevention through financial incentives. Health providers offered a package of preventive interventions and activities to high-risk populations and received in exchange financial incentives based on screening outcomes. This pilot was discontinued when the government of Uruguay decided to increase financial incentives across all 19 departments of the country. Implementation Arrangements 2.16 MSP and ASSE were the main project implementers. The PSCU was housed in MSP’s General Directorate of Secretariat (Dirección General de Secretaría). MSP’s General Directorate of Health (Dirección General de Salud) was responsible for the implementation of components 1 and 3. ASSE was responsible for the implementation of the activities under component 2. 2.17 Annual performance agreements were signed between MSP and each implementing agency. Each agency identified the technical assistance, training, and goods to be provided under the project. Administrative and line staff in MSP and ASSE were responsible for implementing project activities but did not have full-time dedication to the project. The PSCU was responsible for overall coordination, monitoring and evaluation (M&E), technical support, administration, and fiduciary issues (World Bank 2008 16, 61). Relevance of Design 2.18 The statement of objectives articulated in the loan agreement and in the project appraisal document presents measurement challenges, complicating the reconstruction of the theory of change. The PDOs in the project appraisal document included avoiding and reducing “exposure to selected NCD risk factors as well as their health effects” (World Bank 2008, 10). Although the project may have contributed to the reduction of risk factors, attributing results to the project is problematic. In addition, framing the higher-level outcomes (“health effects”) in the objective as an output creates confusion in understanding the project’s intended theory of change. 2.19 The PDOs in the loan agreement lack specificity. The narrative in the project appraisal document suggests that the accessibility and quality of primary health care services and the overall capacity of MSP were expected to be affected, but this is not an 9 obvious understanding of the objective to “strengthen” (World Bank 2008, 10). Table B.2 in appendix B presents the project theory of change constructed following the project’s results framework and information available on the project document at appraisal. 2.20 The PPENT presented several positive design features. NCDs and the associated risk factors targeted by the PPENT were those with high prevalence in Uruguay. These included CVDs (hypertension), diabetes, and obesity.13 This is confirmed by the data presented in section 1 and remained relevant during the project implementation period. 2.21 The PPENT rightly focused on strengthening primary health care. Strengthening ASSE’s primary care was a crucial element to improve access to quality health care services. This included technology and medical equipment in ASSE primary care facilities for detection and treatment of NCDs; updates and investments in information systems; training to health workers on NCD prevention, screening, and information and management tools; and incentives to focus more on prevention programs. International evidence confirms that strengthening primary care in public health care facilities can indeed have a positive impact on health promotion and on the prevention of NCDs and is the strategy recommended by the World Health Organization (WHO; Demaio et al. 2014). 2.22 The inclusion of the Previniendo NCD prevention pilot program was an innovative way to test incentives for providers to focus on NCD prevention and promotion of healthy lifestyles. Previniendo combined the investments in equipment and training to providers with a change in the way providers were compensated. International evidence shows that the payment scheme used to remunerate providers for their services can have a significant impact on the quantity and the quality of the services delivered. Although a capitation payment method has proved to be an effective cost-control mechanism, its direct effects on quality are dubious (Cashin et al. 2014). Evidence on the effects of performance-based financing mechanisms on access and quality of health services is limited; its results are mixed but promising (World Bank 2018). 2.23 Nonetheless, the project was too ambitious in its design. Project design underestimated the time and resources needed to equip, motivate, and train providers at the primary care level on the importance of preventing NCD, and to then define, measure, and monitor providers’ performance (and the learning process involved in continuously redefining and adjusting those measures). 2.24 The project included little discussion about its complementarities with activities led by other ministries, municipalities, and key actors outside the government. The Ministry of Education and the Ministry of Social Development, the Honorary 10 Commissions for Cancer and Cardiovascular Diseases, and civil society efforts such as the NCD Alliance and the Diabetes Association had a significant role to play in NCD prevention. Other donors were also involved in strengthening the primary level of care in public institutions and promoting relevant community- and municipal-level activities, including the Inter-American Development Bank and the Pan American Health Organization. However, the project did not discuss complementarities with these efforts. 2.25 The relevance of design is rated substantial. 3. Implementation 3.1 The project was approved on August 28, 2007 and became effective on January 9, 2008. The project midterm review was conducted as planned in December 2010. The project closing date was extended twice from the original date of December 31, 2012, to December 31, 2015, to complete activities (for a total extension of three years). 3.2 There were no changes to the project objectives during implementation, although there were changes in planned activities and indicators with consequent reallocation of funds. • January 2008: Loan agreement amended to eliminate the commitment charges and interest waivers and to include default interest and the front-end fee amount. • November 2011: Loan agreement amended to redefine the “conversion rate.” • November 2012: Project design changed in response to government decision to scale up the Previniendo pilot; targets for several key outcome indicators were reduced to reflect actual prevalence data from the National Risk Factor Survey (NRFS; not available at project design); one outcome indicator was added to assess project impact on mortality; and implementation arrangements were slightly modified. At that time, 45 percent of the loan had been disbursed. The project closing date was extended from December 2012 to August 2014. • August 2014: Project closing date was extended from August 2014 to December 2015. 3.3 Actual project cost was $21.7 million, 76 percent of the appraised cost of $28.8 million. Actual disbursements for each component were lower than originally planned, except for project management. Lower disbursements were mainly due to changes in the Previniendo and counterpart contributions that may not be totally accounted for in project documents because of the project implementation 11 arrangements. Counterpart financing was lower than the expected $3.5 million, totaling $2.6 million. Planned versus actual disbursements by component are presented in appendix A. Implementation Experience 3.4 Project implementation was affected by several changes of government and the implementation model chosen by the government of Uganda. The administration changed three times during the project implementation period, which lead to rotation in MSP and ASSE staff, including five health ministers. The project used existing administrative arrangements and staff, which led to higher levels of institutionalization and capacity building within MSP and ASSE. However, this implementation model also generated important delays (discussed in detail in section 5, Efficiency). 3.5 Implementation progress was rated satisfactory or moderately satisfactory throughout project implementation except for a brief period in 2011. The disbursement rate until the last semester of 2009 was very low. At the time of the midterm review, about 50 percent of resources were committed, but it took some time after the project midterm review for actual disbursement to occur. Lower execution was associated with the World Bank’s and government of Uruguay’s procurement rules and budget cycles. 3.6 The project benefited from remarkable continuity in the project supervision team; the PPENT had the same task team leader from inception until closure. All government officials interviewed were very appreciative of the supervision team, highlighting the support and commitment received by the project task team leader (see the discussion of Bank performance in section 6 on Ratings). Safeguards Compliance 3.7 At appraisal, the project was assessed as an environmental category B project due to medical waste and provision of medical equipment. The safeguard policy on Environmental Assessment (OP/BP 4.01) was triggered. 3.8 Monitoring data and reporting included in the World Bank’s operational portal and in project files showed that the project complied with relevant environmental standards. At appraisal, it was noted that the government already had adequate legislation and practices in place to manage medical waste. Mitigation measures were prepared in response to the safeguard policy, meeting borrower requirements. According to the ICR, no significant negative environmental impacts were identified during project implementation (World Bank 2016, 12). 12 3.9 New environmental legislation passed during project implementation required health providers to develop mandatory health care waste management plans. The project provided support in developing guidelines for these waste management plans and in assessing the current state of waste management plans. However, because of delays in contracting policies these plans were not implemented, leading to a rating of moderately satisfactory. Financial Management and Procurement 3.10 Financial management was satisfactory, with regular financial reporting submitted on time. World Bank documentation in project files shows that no serious issues were found in fiduciary missions. 3.11 Procurement followed World Bank policies and procedures. The World Bank’s team provided thorough training and support in setting up systems consistent with World Bank procedures. Minor procurement issues arose related to record keeping, but these were addressed through regular supervision missions and did not cause any undue impact on the rollout of activities. 3.12 The project benefited from using existing government control and fiduciary systems. All payments were subject to strict ex ante controls by the Accountant General Office and the Court of Auditors with no observations. This process caused implementation delays, which the World Bank team could have better anticipated, but it also gave the procurement process greater transparency and accountability. 3.13 The initial project implementation period was marked by extensive procurement delays due to strict protocols required by the World Bank and the borrower for fiduciary oversight and control. Project supervision identified minor issues and delays associated with multiple controls from the government side and cumbersome administrative procedures. Limited experience of the implementing agencies with World Bank procedures caused slowness and delays. These issues were solved through changes in World Bank staff responsible for procurement, revisions of implementation arrangements, and closer supervision and procurement training provided by the World Bank. 4. Achievement of the Objectives Objective 1: Strengthen Health Delivery Services 4.1 Health delivery services for NCD prevention were strengthened through several activities, including investments in technology and medical equipment for primary care facilities for detection and treatment of NCDs; introduction of new management tools at national and departmental levels; development and adoption of quality standards and referral systems; investments in management information systems and epidemiological 13 surveillance; training to health workers on NCD prevention, screening, and management; and incentives to health providers for the adoption of prevention programs. 4.2 Information on increased access and quality of health services as a result of project investments is not available given the absence of process or intermediate indicators that connect the outputs with project outcomes.14 Project indicators relied mostly on information from two national risk factors surveys; administrative data from providers that could have complemented this information is either not available or incomplete. Private providers seem to have responded more positively to incentives and changes in the health care delivery model than did the public provider (ASSE). Outputs 4.3 Key outputs supporting the achievement of PDO 1 include (i) 122 public and private primary health care facilities certified and accredited on medical care for NCDs; (ii) technology and medical equipment in 117 primary care facilities for detection and treatment of NCDs, including a digital mammography network for ASSE that equipped 85 centers with digital imaging capability; (iii) management tools, including adoption of quality standards and improved referral systems; (iv) 38 vehicles and medical equipment that facilitated regular rural health care visits; (v) monetary incentives to health providers for greater attention to NCD prevention programs; (vi) an NCD epidemiological surveillance system in place; and (vii) 2,399 primary care teams receiving training on NCD prevention and treatment. Outcomes 4.4 By the end of the project implementation period, all revised targets in indicators related to screening for NCD risk factors and follow-up care for patients with hypertension, diabetes, and obesity were surpassed (table 4.1). 14 Table 4.1. Original and Revised Targets of Selected Outcome Indicators (percent) Indicator Original Target Revised Target Project End Population aged 45–64 years covered by the — 30 36.7 SNIS Hypertension cases diagnosed and under 60 — 62.6 follow-up care (baseline 55) Diabetes cases diagnosed and under follow-up 83 73 77.6 care Obesity cases under treatment and control 50 20 34.3 Source: Independent Evaluation Group based on project documentation. Note: See table B.4 in appendix B for a complete list of indicators. — = not available. 4.5 IEG methodology warrants a split rating given that original targets were reduced for most of the indicators and original targets were only achieved for the hypertension cases diagnosed. However, given that the project is being assessed several years after its completion and additional information is available, formal targets are not as relevant to understanding achievement, and therefore a split rating is not deemed necessary. 4.6 One key limitation of the indicators is that they were not based on actual administrative data from health services providers supported by the project but were instead from the NRFS. Administrative data are limited but when available bring attribution into question. Facilities participating in the Previniendo failed to reach all the associated targets. In the case of the SNIS population screened for NCD factors, for instance, project records suggest that compliance with this indicator comes from private providers; data for ASSE indicate a mere 1.3 percent toward a target of 22 percent. In addition, the most recent data on hypertension cases identified among ASSE users 25–64 years old is 13.87 percent.15 4.7 Moreover, although clinical guidance and protocols for treatment of patients with hypertension, diabetes, and obesity were produced, and extensive training was conducted, information regarding their adoption at the primary level of care is limited. IEG found implementation plans for ASSE but no information on adoption and results (except for scattered references as part of the Previniendo reporting). 4.8 Project funds supported the development and pilot test of quality standards for managing NCDs in primary care facilities in the Previniendo departments. Since 2012, a participatory self-assessment tool on quality standards has been developed and applied in public and private facilities in five departments (Maldonado, San Jose, Canelones, Florida, and Montevideo). Only one department has applied the voluntary self- assessment in all its public and private primary care facilities (Maldonado). Data on 15 good practices were collected at project closure, but MSP does not enforce or incentivize providers to periodically conduct this self-assessment and report the results. 4.9 Project support for equipment and technology in ASSE contributed to greater availability and quality of oncological services. Radiology equipment and computed radiography for digital imaging funded by the project directly contributed to improve the quality of the diagnostics. The PPENT contributed to the development of guidelines and protocols for the management of breast and cervical cancer. Equipment and technology also contributed to better management of oncological records, thanks to complementary activities funded by AGESIC.16 However, despite investments in infrastructure, equipment, and training supported by the project, and the existence of mandatory screenings established by law,17 breast cancer screening did not improve in ASSE. The associated PDO indicator to measure the PDO was not met for either original or revised targets. MSP in its 2020 National Health Objectives recognizes incomplete coverage and registration and the absence of a system for follow-up and treatment.18 The ICR also points to these reasons and adds equipment installation delays and technical capacity constraints to manage the equipment (World Bank 2016, 84). 4.10 ASSE’s performance is concerning given that screening for breast cancer is common in Uruguay. According to the NRFS, in 2014, 73.2 percent of women aged 40–64 years had received a mammogram during the past two years, but the percentage of female ASSE beneficiaries aged 50–69 having had a mammogram was never greater than 15 percent throughout implementation. Although this figure may underestimate actual coverage due to issues with reporting, additional efforts may be needed to reach the poorest women. IEG could not find updated data for ASSE as the public provider does not regularly report on this indicator. 4.11 WHO data reviewed to complement the available information shows mixed progress for Uruguay on health system strengthening toward a national response to NCD. Out of the six indicators tracked by WHO, only two were achieved by 2017: the availability of essential medicines to treat NCDs and the availability of basic technologies to treat major NCDs. The project may have contributed the most to the second of these indicators. WHO data show fewer than 50 percent of health facilities used CVD guidelines. No information is available for the three remaining indicators: proportion of population at high risk for CVD or with existing CVD; proportion of high- risk persons receiving any drug therapy and counseling to prevent heart attacks and strokes; and proportion of primary health care centers reported as offering CVD risk stratification.19 4.12 Probably one of the most important contributions of the project was to influence the definition of Uruguay’s health goals and its associated targets for public and private 16 providers receiving FONASA funds. Goal 2 was inspired by the Previniendo model and aimed at strengthening the primary care level and enabling continuous monitoring and follow-up of patients, making earlier diagnosis of NCDs and management of their risk factors easier, avoiding possible complications, and preventing the onset of diseases.20, 21 IEG reviewed compliance data with goal 2 to complement the evidence available related to project outcomes. 4.13 Since its first introduction, goal 2 was redefined several times to better match the country 2020 National Health Objectives. It first focused on defining the ideal mix of staff in primary care facilities. Then, indicators were added on the number of population assigned to a referring physician, training for medical providers, and mandatory clinical tests on a regular basis for at-risk patients. In its latest form (redefined in 2017), goal 2 has 4 performance indicators providers need to comply with. Two of them related to the surveillance and treatment of population with hypertension and cardiovascular diseases that apply to all health care providers. The other two indicators can be selected by each provider in agreement with the MSP, based on the epidemiological profile of their population. 4.14 Compliance with the referring physician goal was higher among private providers than in ASSE. JUNASA data for the years 2015 and 2016 shows more than half of the private providers surpassed the established targets. Information for ASSE was incomplete and, when available, showed a much lower level of compliance. Consequently, although private providers collected 86 percent of the maximum amount that they could potentially receive from goal 2, ASSE collected only 2.8 percent in 2015 and 15 percent in 2016. 4.15 Moreover, IEG found that the assignment of a referring physician was seen by providers just as an administrative requirement for a capitation payment, with no real impact on the quality of care or its accessibility. The indicator was found ineffective and was later redefined to include training of medical personnel in selected NCDs risk factors. Another indicator was added to include “extended-hours positions” to reduce the number of doctors with multiple employments (very common in ASSE) and encourage greater allocation of time to primary care and training/education activities. 4.16 Information on compliance with the newly defined goal 2 on risk factors controls is not available, suggesting continuous struggles with information reporting and monitoring. Public officials in the MSP confirmed that there were no payments associated with the compliance of these goals yet. The lack of payments associated with goal 2 has to do with: (i) delays among providers in capturing and reporting the required information and, (ii) the limited availability of human resources needed to double check the accuracy of the information reported by providers to ensure 17 compliance. Given the resources the project dedicated to information systems, this lack of data is discouraging. 4.17 Changes in NCDs risk factors and their health effects are discussed in the next paragraph but need to be taken with some caveats. These indicators are of limited relevance to the achievement of the PDO given that they are higher-level objectives, associated to both PDOs that may take longer than the project implementation period to materialize, and their attribution is complex. Other government programs contributing to the PDO are discussed in appendix C. 4.18 Risk factors for NCDs in Uruguay do not show a consistent reduction trend during the project implementation period.22 In 2013 compared with 2006, some risk factors declined, some increased, and some did not show statistically significant change. (figure 4.1). These figures likely underestimate the prevalence of some of these risk factors given the large magnitudes of underdiagnoses estimated in Uruguay. The second NRFS estimated that more than 60 percent of people with hypertension and 50 percent of people with diabetes are not aware of their condition (NRFS 2016). 4.19 Moreover, other available (but not comparable) data suggest that high prevalence is worrisome in certain age groups. For instance, more than 40 percent of the population older than 60 years is diagnosed with hypertension23and 15 percent of children attending public schools in Montevideo suffer from the condition; the prevalence is even higher among those children that are overweight.24 In 2011 almost 8 percent of children under 5 years old25 and 30 percent of children in high school26ere obese or overweight. Per 2013 data, overweight affects almost 10 percent of children under 2 years old, and more than 11 percent of children aged 2 to 3.27 And in 2015, 12.6 percent of children aged 2 to 6 were overweight, and 40 percent of children aged between 10 and 13 attending Montevideo public schools were overweight or obese. 28 18 Figure 4.1. Prevalence of NCD Risk Factors in Population Aged 25–64 Years (percent) a. Increasing prevalence in NCD risk factors b. Decreasing prevalence in NCD risk factors 7.6 Diabetes 9.5 5.3 Binge drinking 12.1 25.3 Sedentarism NCD Risk Factor 18.2 NCD Risk Factor High cholesterol 21.5 Elevated salt 29.9 33.9 consumption 28.6 36.6 22.8 High-blood pressure Physical inactivity 30.4 29.9 64.9 Overweight/Obesity 56.6 Tobacco 28.8 Deficient intake of 32.7 90.9 fruits and vegetables 84.9 Percent Percent 2013 2006 2013 2006 Source: Independent Evaluation Group based on data from Second National Risk Factors Survey for noncommunicable diseases. Note: Definitions of risk factors are as follows. Binge drinking: at least one episode of binge drinking in the past week; physical inactivity: fewer than 150 minutes a week of moderate physical activity, or fewer than 75 minutes a week of vigorous physical activity (or any combination); sedentary lifestyle: more than seven uninterrupted hours sitting or resting; tobacco: daily tobacco consumption; elevated salt consumption: addition of extra salt to prepared meals; deficient intake of fruits and vegetables: fewer than five portions a day of fruits or vegetables; other categories are self-explanatory. NCD = noncommunicable disease. 4.20 Although the observed increase in prevalence of NCD risks factors could be explained by better diagnosis, to which project efforts may have contributed, attribution is not possible. The PPENT likely contributed to better diagnosis and follow-up care through technology and medical equipment, information systems, guidance and protocols, and strengthened capacity. However, neither project nor administrative records provide sufficient information to show results in outreach, diagnosis, and follow-up care. 4.21 IEG examined trends and changes in mortality due to CVD and diseases of the circulatory system, both associated with the risk factors targeted by the project. In 1990 28 percent of deaths from NCDs could be attributed directly to hypertension; in 2016 this figure had dropped to 22 percent.29 Furthermore, the importance of behavioral and metabolic risk factors associated with CVD and diseases of the circulatory systems deaths decreased by 40 percent and 30 percent, respectively, in the same period. 4.22 An indicator on the mortality from diseases of the circulatory system in the population under 70 years was included in the project M&E at the 2012 project restructuring. The target on this indicator was surpassed, reaching 60.60 percent at project closing (target was 67.50 percent), from a baseline of 75.18 percent. This indicator 19 was included because it was considered a good proxy of overall progress in preventing and controlling NCDs and has the advantage that data are collected annually. 4.23 Attributing changes in this indicator (or similar indicators) to project-supported activities is not possible, however; thus, it should be used with caution as evidence of achievement of project PDOs. Trends in both the mortality due to CVD and that to diseases of the circulatory system began descending well before the project implementation period (figure 4.2). Thus, these achievements cannot be solely attributed to better diagnosis and follow-up of hypertension promoted by the project. 4.24 Moreover, this declining trend can be observed in many other countries in Latin America and the Caribbean (Ordunez et al. 2015). Some of the reasons behind this trend are reductions in metabolic and behavioral risks; better diagnosis and treatment; and, both primary and secondary prevention (Ezzati et al. 2016). The literature also points to the importance of increases in per capita income and improvements in cardiovascular health. Improved cardiovascular health in South America was enough to compensate the effects of population aging on CVD (Roth et al. 2015). Figure 4.2. Death Rates for Cardiovascular Diseases and Diseases of the Circulatory System in Uruguay a. Deaths from cardiovascular disease b. Deaths from diseases of the circulatory system 140 124.1 140 Mortality rate (per 100,000 Mortality rate (per 100,000 120 120 100 98.2 100 pop) pop) 80 68.6 78.0 80 78.4 60.6 60 68.6 60 59.0 40 40 1990 1995 2000 2005 2010 2015 1998 2003 2008 2014 Source: Global Burden of Disease data for mortality due to cardiovascular disease; Ministry of Public Health Vital Statistics for mortality due to diseases of the circulatory system. 4.25 It is noteworthy, however, that in Uruguay the reduction of premature mortality due to CVD was more impressive than that of the region: mortality among those aged 15–49 was reduced by 50 percent. Consequently, the number of years lost because of premature mortality decreased almost 40 percent (from 6,383 in 1990 to 3,936 in 2016).30 The regional level was only 13 percent on average. 4.26 Information collected by IEG supports a modest rating on achievement of PDO 1. This modest rating is sustained by limited attribution of changes observed in some of the 20 PDO-related indicators to project efforts, insufficient information on health providers’ performance, and limited progress in the areas where that information is available, especially for ASSE (despite sustained project efforts in information systems). Objective 2: Strengthen MSP’s Health Policy Framework for NCDs 4.27 Despite narrowly defining its objective to focus on the MSP health policy framework, the project supported the implementation of relevant measures at the national, departmental, municipal, and community levels and in primary health centers to strengthen the government’s capacity and response to NCDs. Achievements are more noticeable on the regulatory and stewardship capacity of the MSP and, to a lesser extent, in its monitoring and surveillance functions and in the overall direction of its health promotion and prevention strategy. Outputs 4.28 The project contributed to the achievement of PDO 2 through its direct support to (i) NCD data generation and analysis, (ii) epidemiological surveillance at the departmental level, (iii) the foundations of an integrated health information system, and (iv) community-level interventions to prevent NCDs. 4.29 Country NCD data and capacity to collect and analyze such data increased because of the PPENT support to the second NRFS, the GBD study, and the first National Health Survey. All stakeholders consulted for this evaluation agree on the importance of the first two. A great deal of effort went into disseminating these data among providers and ensuring its understanding and the implication for prioritization and organization of services. Capacity building to conduct such studies and analyze the data remained in MSP. 4.30 The surveillance capacity of MSP improved as a result of the creation of a national network of epidemiology units in all departments, improvements in the quality of hospital discharges registries, and development of guides on reportable diseases and health events. The epidemiology point person in each department, funded by the PPENT during project implementation, is now included in the MSP budget. By project completion, 75 percent of ASSE hospital discharges were reported and coding errors were reduced to less than 3 percent. A national surveillance system on NCDs was developed. 4.31 The project strengthened the quality of medical records at the provider level but fell short of its original objective of having an integrated information system. The project improved the quality of multiple and fragmented databases on life events, reportable diseases, hospital infections and discharges, and national risks factors. MSP’s 21 assessments confirmed important progress in the quality of registration yet acknowledged continuous challenges to effectively capture and monitor providers’ performance (see discussion regarding lack of data on goal 2 under objective 1). Challenges are greater for the public provider given its size and presence across the country: only 30 percent of ASSE beneficiaries were registered in electronic medical records at project completion. 4.32 As part of its prevention and promotion efforts, the project supported several activities to promote healthy lifestyles. These included the healthy communities and municipalities strategy (Monge et al. 2012), small health promotion projects at the community level, and the healthy schools strategy.31 4.33 MSP encouraged healthy lifestyles with the active participation of local communities and citizens through the healthy communities and municipalities strategy. Local populations were educated on the benefits of healthy habits such as physical activity or healthy eating. Over 63 agreements were signed between MSP and local actors. The project helped define the criteria for its accreditation. 4.34 Several municipalities added to these efforts by promoting exercise and healthy habits through different channels. Supported activities included training and communication on healthy habits and exercise, the expansion of walking and bike paths, and the installation of “healthy stations” and “outdoor gyms” in strategic points for people to use free of charge (100 were installed in 89 municipalities in all departments across the country32; approximatively one-third of these were paid for with PPENT funds). In 2013, 22 municipalities were considered “healthy municipalities.”33 IEG was not able to obtain the most updated information on this, except for the departments of San Jose and Canelones, and the city of Montevideo (see box 4.1). 22 Box 4.1. Montevideo: Continuous Commitment to Noncommunicable Disease Prevention The municipality of Montevideo is probably the best example of continuous commitment to noncommunicable disease prevention. A 2014 decree prohibited bars and restaurants from exhibiting salt-shakers and other high-sodium products to the public and were required to display a sign reminding customers of the harmful effects of salt consumption. a Local bakeries were invited to sign a voluntary agreement with the municipality committing to reduce the content of sodium in their products.b The city also established a partnership in October 2017 with the Bloomberg Foundation on its Healthy Cities Initiative, planned to last 18 months.c,d Montevideo is the only city focusing on reducing salt intake. It aims to do so by (i) enforcing current salt-reduction policies, and (ii) introducing new policies such as a media campaign to warn people about the dangers of salt and providing training to restaurant and school lunchroom staff to prepare and make available low-sodium food and condiments. Also, through this partnership, Montevideo joins five other cities in Latin America and the Caribbean (Bogotá, Cali, Lima, Medellín, and Quito) in developing policies to limit the availability and advertising of sugary drinks and processed or high-sodium foods in public environments, while making healthy options easier and cheaper. Source: World Health Organization Noncommunicable Disease Progress Monitor. Note: a. http://www.montevideo.gub.uy/aplicacion/resolucion?parametro=1694-14&alto=8100 b. http://www.msp.gub.uy/comunicado/monitoreo-del-sodio-en-las-panader%C3%ADas-en-el-marco-de-la-campaña- “-sal-salud.” c. https://partnershipforhealthycities.bloomberg.org/. d. http://www.montevideo.gub.uy/institucional/noticias/montevideo-ciudad-saludable. 4.35 In addition, the PPENT funded 36 health promotion projects in 17 of the 19 departments, but these initiatives, although individually successful, were rather isolated and not part of a strategic direction toward health promotion. MSP attention and overall support to these initiatives was irregular, funding was small (between $10,000 and $12,000), sustainability of activities relied on individuals’ or municipalities’ interest, and the local capacity and interest generated was not fully leveraged to promote substantive change. 4.36 Through the healthy schools’ strategy, the project promoted healthy lifestyles among children attending public and private schools. Approaches include the promotion of healthy eating habits, exercise and physical activity. At project completion, the strategy had reached 16 percent of the primary schools in the country (379 schools distributed across all departments). This initiative was further supported by a law requiring the availability of healthy food choices in cafeterias and canteens and promotion of the benefits of healthy nutrition habits to prevent and control obesity and hypertension.34 As a result, a list of recommended foods and beverages to be offered at schools was developed with the consequent prohibition of displaying or advertising any 23 product not on that list. Also, school menus had to be designed and approved by a professional nutritionist. 4.37 The information collected during the IEG mission showed a low level of compliance with this law, due in part to lack of control and in part to lack of knowledge about what constitutes healthy or unhealthy food. Although the law prohibits the display of unhealthy food to school-age children, it does not limit the selling of it. Uruguay recently approved legislation on food labeling, which is expected to help in this regard.35 Outcomes 4.38 In the absence of an outcome indicator to assess progress on the strengthening of MSP’s health policy framework, this PPAR uses WHO indicators as a reference. WHO regularly tracks progress toward developing national responses to NCDs across its member countries using several indicators.36 4.39 Of the 10 indicators tracked by WHO in 2015, 7 were fully covered by the project and 1 was partially covered. Of the 7 fully covered by the project, 4 had to do with setting national targets and associated systems to generate routine reliable data and developing multisectoral policies toward the achievement of those targets. Another indicator was associated with the existence of protocols for NCD management at the primary care level (see box 4.2). Of these 5 indicators, the first 2 were fully achieved, the other 2 partially achieved, and the last 1 not achieved. The remaining 2 indicators were associated with public education and awareness campaigns promoting physical activity, to which the project only made limited contributions. Regarding the first indicator, stakeholders consulted for this evaluation unanimously agreed on the importance of the PPENT in raising awareness of NCD prevention and health promotion and its consequent inclusion in the 2020 Health Objectives and associated targets. Similarly, stakeholders agreed on the importance of the PPENT in changing the incentive structure of providers, tying their performance to the country’s health goals. Despite the lack of information on compliance with related NCD goals, the impact the project had in the definition of NCD goals should not be minimized. The country’s health goals gave prominence to health promotion and disease prevention activities and timely diagnosis, many focusing on NCDs and their main risk factors. Half of the 14 indicators that providers can choose from are directly related to NCDs and prevalence estimates from the NRFS are used as baseline values (MSP 2017a). Eighty- three percent of the providers chose for their performance to be measured against NCD- related targets.37 24 Box 4.2. Uruguay National Response to Noncommunicable Diseases: Relevant World Health Organization Indicators Indicator 1. Uruguay has set national NCD targets (time-bound, address NCD mortality and key risk factors, based on the nine voluntary global targets and the WHO Global Monitoring Framework). Indicator 2. Uruguay has a functioning system for generating reliable cause-specific mortality data on a routine basis. Indicator 3. Uruguay has a STEPS survey or another risk factor survey every 5 years.a Indicator 4. Uruguay has an operational multisectoral national strategy or action plan that integrates major NCDs (diabetes; cancer; chronic, cardiovascular, and respiratory diseases) and their shared risk factors (unhealthy diet, physical inactivity, harmful use of tobacco and alcohol). Indicator 9. Uruguay has evidence-based national guidelines or protocols or standards for the management of major NCDs through a primary care approach, recognized or approved by government or competent authorities. Source: World Health Organization, NCD Progress Monitor. Note: NCD = noncommunicable disease. a. STEPS is the WHO STEPwise approach to surveillance. 4.40 Regarding the second indicator, the PPENT contributed in setting the foundations for an integrated health information system. Through its focus on equipment, technology, and capacity building, the project initiated the integration of key databases on life events, reportable diseases and health events, hospital infections and discharges, and national risk factors. Although the overly ambitious goal of standardization, quality and automatization of the information needed for the integration of the different systems was not achieved, progress is noticeable. 4.41 For instance, the electronic medical record was hardly adopted by ASSE providers; yet, it was considered a needed step toward the creation of an electronic clinical history, currently being developed. Stakeholders consulted for this evaluation confirmed that several lessons derived from the limitations of the electronical medical record were incorporated in the design of the electronic clinical history: capability to work off-line, accessibility to all health personnel (not only to doctors), automatic alerts to users, and so on. The electronic clinical history is being implemented by AGESIC with support from the IADB.38 4.42 Per stakeholders interviewed by IEG, the project promoted a cultural change in ASSE information systems, that went beyond technology adoption. Yet, the project overestimated the time needed to overcome the challenges, rigidities and resistance to change within ASSE. At the time of project design ASSE had multiple and fragmented information systems and did not even have a registry of beneficiaries. ASSE was also under no obligation to report on performance to MSP, this changed in 2010. 25 4.43 Regarding the third indicator, the PPENT ensured the continuity of having an NRFS every five years, as recommended by WHO. The second NRFS (as well as the GBD studies) funded by the PPENT were broadly used to set relevant NCD targets at the policy and provider levels. The third NRFS was planned for 2018, but the survey is not included in the MSP budget. Uruguay is searching for alternative funding. 4.44 Regarding the fourth indicator, the project supported the development of a Proposal for a Strategic Plan for Health Promotion, Prevention, and Control of NCDs but was unable to fully implement it. The proposal outlines key areas to effectively address NCDs, stressing the importance of a horizontal and multisectoral approach to NCD and its social determinants.39 This proposal was developed in 2010, but implementation of several of the key initiatives started only in 2013/14. Delays were associated with lack of MSP leadership in the area to engage key partners outside the health ministry. The strategic plan was developed in 2015 but was never implemented. 4.45 Project contributions to promoting behavioral change in physical activity and tobacco and alcohol use were limited. They included community-supported interventions with funding from the Uruguay Solidarity and Inclusive Fund, and small- scale projects and education campaigns funded as part of the healthy municipalities and schools strategies. Their reach was limited. 4.46 Overlapping roles between the multiple actors supporting NCD prevention activities and promoting behavioral change, and a lack of unified operating strategy, resulted in a lack of overall guidance and coordination, undermining results in this area. The project focused mostly on secondary health promotion activities within MSP’s control, with little coordination with activities performed by other key actors aiming at changing the behavior of the population regarding risk factors. For an integrated multisectoral strategy to function, the work done by the National Alliance for Preventing NCDs, the Honorary Commissions, the Secretary of Sports and Recreation, INDA and RAFU should have been better coordinated and leveraged.40 4.47 Limited coordination efforts were also found with the OPS/OMS local office and with the work supported by the IADB in the sector. The OPS/OMS have been promoting the healthy municipalities strategy in Latin America and the Caribbean since 1985, WHO is a known leader on NCDs and through its local office it has supported Uruguay in the definition of its 2020 Health Objectives. Similarly, the IADB has been supporting the e- government effort in AGESIC41 and had several small projects aiming at strengthening the Primary Health Center Network. Yet, there were limited signs of collaboration and coordination. 26 4.48 Stakeholders outside of the MSP interviewed for this evaluation confirmed the lack of coordination with project activities. Although this lack of coordination may have to do with limited visibility of PPENT given the implementation model (stakeholders could have worked with MSP officials unaware that they were implementing PPENT activities), still it raises questions regarding the focus and strategic direction of the NCD promotion and prevention activities pursued by the project. 4.49 Moreover, the effort on NCD prevention was not accompanied by additional budgetary provision for health promotion activities or greater attention at the national level. Although during the project implementation period the social determinants of NCDs and the importance of an integrated strategy to address them was emphasized, currently MSP does not have a directorate for health promotion and disease prevention. Rather, these activities are organized within each vertical program. 4.50 Only a very small fraction of the total health care budget is devoted to health promotion activities, and this allocation did not change over time. Data for 2012 shows that only 4 percent of total expenditures were devoted to health promotion and prevention activities. 42This figure is identical to what was estimated at appraisal (PAD p.7) and showed slight variations over time: 3 percent in 2015 and 2 percent in 2016. Of the total ASSE budget, only 12 percent is allocated to the primary care networks, whereas 30 percent is allocated to the central level of care (IDB 2015). GoU’s resources to key entities supporting these activities, including the Honorary Commissions for Cancer and Cardiovascular Diseases do not show a consistent pattern in recent years. 43 For the Commissions, public funds represent more than 90 percent of their total income. Other key actors, such as the Uruguayan Diabetic Association, do not receive financial support.44 4.51 Finally, the project developed and pilot-tested several guidelines/protocols/ standards for NCD management at the primary care level but their adoption among providers is limited. The project helped develop a voluntary self-assessment tool for NCD management in primary care facilities but so far it has been of limited use (see discussion under PDO #1). 4.52 In summary, the project made important contributions to Uruguay’s progress on developing a national response to NCDs, especially in the areas related to integrated strategy, guidelines and regulations, data generation, set of national targets, and, to a lesser extent, health promotion. The project supported relevant policies, strategies, and action plans on NCD prevention health promotion and set the foundations for developing an integrated information system. Thus, the project’s contribution to this objective is rated substantial. 27 5. Efficiency 5.1 Empirical evidence shows that prevention and early detection of NCDs can save lives and money. A study by Abegunde et al. (2007), based on 23 developing countries, shows that targeted NCD prevention and control programs could prevent 24 million deaths and save about $8 billion in these countries. For the same 23 countries, another study by Asaria et al. (2007) shows that two preventive interventions targeted at reducing salt intake and tobacco consumption among the population could save 13.8 million lives in 10 years, at a cost of only $0.5–1.00 per person in upper-middle- income countries. 5.2 The project economic analysis estimated high value for money of the investment. The results suggest that the intervention was indeed cost-effective, as the cost per disability-adjusted years of life lost averted as a result of the project ranges between $334 and $2,371 (depending on the percentage of contribution given to the project and the discount rate considered for the calculation of the present values). 5.3 Economic analysis at appraisal highlighted the cost-effectiveness of preventing NCDs, comparing its costs with NCD treatment. At closing, a cost-effectiveness analysis based on the reduction of the premature mortality and morbidity due to CVD that could be attributed to the project was included in the ICR. The ICR followed WHO guidance to calculate cost-effectiveness and included a sensitivity analysis using different discount rates. The ICR’s most conservative assumption assumed a 30 percent contribution of the project to the overall mortality. There was no rationale behind the use of 30 percent. 5.4 Including a discussion on the economic costs NCDs impose, at both the individual and country levels, with available data from Uruguay could have enriched the analysis. NCDs are characterized by their long duration and expensive treatments, thus imposing important direct costs in medication and treatments and indirect costs due to reduction in labor supply and productivity losses because of illness (Bloom et al. 2017). 5.5 As an example of the magnitude of these costs, the economic burden of tobacco in the country has been estimated at 2.6 percent of GDP,45 and the cost of alcohol at 0.5 percent, considering both direct and indirect costs.46 In the case of obesity, although there are no estimates available for Uruguay, research for other countries in the region suggests that the economic cost of this condition is high, running from 0.54 percent of GDP in Chile to 2.5 percent and 2.4 percent of GDP in Mexico and Brazil, respectively (Institute for Quality of Life 2016). 28 5.6 Efficiency in implementation was low, resulting in a total extension of three years. Delays were associated with frequent turnover in project leadership, insufficient counterpart budget allocations, and procurement. Important delays had to do with the implementation model. However, the latter also facilitated greater institutional capacity within the government that should be considered among project benefits. Inefficiencies were also found in the limited coordination with other efforts in the sector. 5.7 Project efficiency is rated modest. Preventing NCDs can be highly cost-effective, and available data for Uruguay indicate that this may indeed be the case. Two shortcomings are noted, however: (i) significant delays in implementation and limited coordination reduced cost efficiency; and (ii) attribution of reductions in premature CVD mortality and morbidity to project investments is complex. 6. Ratings Outcome 6.1 Overall outcome rating is moderately satisfactory. The objective of strengthening health services was rated modest given insufficient information regarding providers’ performance to show increased access and quality of services, especially for the public provider. The objective of strengthening health policy framework was rated substantial due to its contributions to Uruguay’s progress on developing a national response to NCDs, especially in the areas related to integrated strategy, guidelines and regulations, data generation, and set of national targets. The project had substantial ratings for relevance of its objectives and design and a modest efficacy rating. Risk to Development Outcome 6.2 The goal of NCDs promotion and prevention remains highly relevant to GoU’s health goals. NCDs are undoubtedly a priority for the GoU, as reflected in the 2020 National Objectives and its associated health goals 6.3 Project’s implementation arrangements limited the risk to development outcomes. Using MSP’s internal administrative structure, not only ensured that the strengthened institutional capacity remained within the MSP and ASSE but also led to the institutionalization of many activities, including M&E and health information systems. Data reported in the ICR indicated project implementation did not have a major fiscal impact on MSP and ASSE budgets, taking an average of 0.3 percent of their budgets throughout the project implementation period (World Bank 2016, 53). 6.4 The results-based financing design promoted by Previniendo was adopted by the MSP in its health goals and went beyond the screening of chronic diseases to change the 29 overall incentives to providers. Government officials and other stakeholders interviewed for this evaluation agreed that the current definition of goal 2 comes from the incentive scheme first established by the Previniendo pilot. The importance of the Previniendo pilot in redefining goal 2 is also acknowledged in MSP publications.47 This scale up of monetary incentives for performance monitoring is an indication of the government’s commitment to continue with the innovations introduced by the project. 6.5 Evidence collected by IEG suggests that the widespread use of monetary incentives was not accompanied by additional resources needed to monitor its compliance. Both the accurate reporting on performance indicators by providers and the verification of compliance, as well as the evidence to set up the goals and indicators, require sustained investments in human resources. Essential elements to define performance targets for providers, such as the third NRFS, are not currently budgeted for (the survey should have been conducted in 2018). Additional capacity is needed for providers to accurately report quality information. Despite important advances in information systems, the interconnectivity and integrality are still missing. Serious capacity issues in MSP limit the verification of compliance and associated payments. Although the Inter-American Development Bank is currently providing support to strengthen MSP auditing and monitoring functions and information systems, insufficient budget allocation poses a serious sustainability risk in the future. 6.6 Insufficient budget allocation and fragmented attention to health promotion activities is also worrisome. Financial support to activities promoted by the Honorable Commissions of Cardiovascular Health and for the Fight Against Cancer have experienced limited change over time. IEG found few examples of effective interministerial coordination on NCD prevention and limited use of large-scale communication or behavior change campaigns. The absence of a centralized unit devoted to this within MSP and the return to activities organized under each vertical program pose a risk in effectively addressing the multidimensionality of NCD risk factors. 6.7 Given these shortcomings associated to insufficient budget allocation to providers’ performance monitoring and fragmented attention to health promotion, the risk to development outcome is rated substantial. Bank Performance Quality at Entry 6.8 The project design drew on prior analytic work on the country’s health sector and international evidence and was highly consistent with the government’s sector 30 priorities and challenges. Stakeholders interviewed agreed the World Bank brought valuable experiences and knowledge from other countries and gave Uruguay the time needed to shape the reform and helped to keep the focus on the country’s ultimate goals. 6.9 Nevertheless, the project was overly ambitious and underestimated the time and capacity needed to fully implement the wide range of activities included at entry. Given the implementation modality and the ongoing reform process, the sequencing of activities needed to produce expected changes was not properly considered at entry. Equip and train providers, change provider’s approach to NCD management, set providers’ goals and monitor their progress toward those goals through a new surveillance and monitoring information system took much more time than envisioned. Variety in organizational culture, norms, and capacity within different providers, ASSE and the MSP itself were not properly accounted for. 6.10 The results framework presented important limitations (as described in the discussions of relevance of design and M&E). The absence of indicators to capture results of the extensive institutional strengthening pursued by the project and indicators on different providers’ performance limited the ability to show project achievements. Committing to results on access and quality of health services was highly dependent on the success of project efforts to strengthen information systems. 6.11 Procurement and financial management requirements by the government as well as the implications of the five-year budget cycle could have been better anticipated by the World Bank. Mitigation measures at appraisal did not include plans for the additional time that compliance with government audit measures would require nor was the budget cycle and its implications for project implementation mentioned. 6.12 Given these shortcomings, quality at entry is rated moderately unsatisfactory. Quality of Supervision 6.13 All government officials in MSP and ASSE expressed high appreciation for the work of the World Bank team and of the project task team leader in particular. The continuous support received by the World Bank team during implementation was mentioned by all interviewees, and government officials appreciated the commitment of the World Bank supervision team to understand the local context to overcome obstacles and propose solutions. The ICR found at least two supervision missions and more than three annual technical visits were conducted during the whole period of implementation (World Bank 2016, 22). Slowness and delays associated with the World Bank’s procurement rules in the first years of implementation were solved as a result of closer supervision, greater responsibility for procurement given to the PSCU, and technical assistance and training. 31 6.14 The World Bank team candidly reported on implementation in the midterm review and in the ICR, and was comprehensive in its aide-mémoire, although concerns were not always reflected in ratings given in the Implementation Status and Results Reports. Delays in restructuring after the midterm review were not discussed in the World Bank’s supervision reports and concerns raised regarding insufficient budget allocation were mentioned only in passing. Ratings in the Implementation Status and Results Reports were consistently moderately satisfactory and satisfactory during project implementation, except for a brief period. 6.15 The World Bank team could have adjusted project ambitions when complexity grew during implementation. This occurred when institutional capacity and rigidities within existing systems in the implementing agencies were evident, but also when the government decided to scale up the incentives structure of the Previniendo to assess providers’ performance. The capacity needed at the level of MSP and ASSE to fully implement activities was not reassessed in the restructuring. 6.16 On balance, the quality of World Bank supervision is rated moderately satisfactory. Quality at entry was rated moderately unsatisfactory. Taken together, and given the outcome rating of moderately satisfactory, these lead to an overall rating of Bank performance of moderately satisfactory. Borrower Performance Government Performance 6.17 The five-year budget cycle led to significant implementation shortcomings and delays. Procurement was negatively impacted by recurrent delays in ensuring formal budget allocations to the project during the first years of implementation. The MSP budget for the cycle 2005–09 was approved before the project, but its implications were not adequately planned for. Lack of project disbursement, in turn, created a vicious cycle of insufficient budget allocation. 6.18 Capacity issues were a constant in explaining performance, yet the GoU did not ensure the sustainability of the efforts promoted by the project. The project found a creative way to work with government officials and remunerate their efforts (paying for extra hours to their regular schedule) but the implications of this implementation modality for the execution time and its sustainability were not considered. 6.19 The limited coordination with other key actors working on NCD prevention contributed to constrained project efficacy and efficiency. Better leveraging the work of the Honorary Commissions, the NCD Alliance, OPS, IADB and other ministries could have sharpened the focus, increased outreach and sustainability of prevention activities. 32 6.20 Changes in government, including turnover of high-level authorities negatively impacted project performance. Per the information available in supervision reports, there were important delays in taking strategic decisions. This included but is not limited to a late restructuring, processed three months before the original closing date. 6.21 Considering all this, government performance is rated moderately unsatisfactory. Implementing Agency Performance 6.22 Despite high turnover at the beginning of project implementation, MSP and ASSE’s commitment to the project was high. This commitment is well documented in World Bank preparation and supervision documents and is reflected in a dedicated team that implemented the project with little changes since 2010, as well as in the close collaboration with the World Bank and participation both throughout project preparation and implementation and after project completion. MSP and ASSE staff dedicated extra hours, beyond their normal responsibilities, to ensure ownership and relevance of activities supported by the project. 6.23 Nonetheless, inexperience working with World Bank processes resulted in procurement delays that negatively affected implementation. Inadequate support to implement project activities and limited staff and capacity, especially regarding procurement activities, were addressed during project implementation through extensive support and training and ultimately by assigning this function to the PSCU. 6.24 Implementing agency performance is rated moderately satisfactory. Government performance was rated moderately unsatisfactory. Taken together, and given the overall outcome rating of moderately satisfactory, these lead to an overall borrower performance rating of moderately satisfactory. Monitoring and Evaluation Design 6.25 The original results framework used the statement of objectives included in the project appraisal document (not in the loan agreement) and included few appropriate indicators to measure the PDOs. Strengthening health services was to be measured by indicators on the percentage of population screened for NCD factors and on the control and follow-up of patients with hypertension, diabetes, obesity/overweight, as well as the percentage of women who had a mammogram. However, only one of these indicators captures actual coverage of services in public primary health care facilities supported by the project. 33 6.26 In addition, the M&E at design presented other important limitations to capture the project’s efficacy, including (i) unbalanced number of indicators to measure the different objectives, giving more prominence to PDO 1 (seven indicators compared with one for PDO 2); (ii) lack of indicators to measure the institutional strengthening supported by component 1, despite being mentioned in the PDO wording; (iii) lack of indicators to capture the multidimensional nature of the changes needed in the policy framework for NCDs, rather focusing only on those related to the health sector; (iv) lack of indicators to measure the quality of the public primary health care services, despite being the focus of component 2; and (v) an indicator on the percentage of newborns with disabilities as an imperfect measure of the project PDOs. 6.27 Indicators on the actual coverage and effectiveness of the services provided were included for the Previniendo pilot. The pilot included an impact evaluation at the time of project design that aimed at assessing results of the key changes promoted by the project in three departments. In addition, the pilot also planned for independent audits of providers’ data, including surveys of beneficiaries regarding their participation and satisfaction. Implementation 6.28 Much of the institutional capacity building and provision of equipment funded by the project was not fully realized until later during project implementation. This delay limited the possibility of showing results of changes in providers’ NCD management. For instance, quality standards for primary care attention were operational in 2012, self-evaluation started in 2013, and feedback and guidance were given to primary care facilities only during the last year of project implementation. The epidemiology units established at the departmental level only started to provide regular information for decision making in 2017. Furthermore, despite important advances in data generation and analysis, such as ASSE’s registry on hospital discharge, the goal of an integrated health information system was not achieved. 6.29 Not all the measures to assess progress in results included in the M&E framework were implemented as planned. At design, a set of key performance indicators were established for an annual monitoring of progress and an impact evaluation was planned to assess the impact of the Previniendo pilot. Per information provided by the PSCU, the annual progress review was conducted as planned. The impact evaluation of the pilot was never implemented. Project supervision refers to some sort of evaluation of the Previniendo carried out by the GoU before the scale up of the incentives but IEG could not locate such report. 34 Use 6.30 The project supported important efforts in data collection that were instrumental in decision making at the policy level but of limited use for assessing providers’ performance. Information from the NRFS and GBD study was used to establish the country health goals and associated targets. National data were used to estimate (and adjust) baseline and targets of PDO indicators with limited discussion of project attribution. Relevant process indicators that could have helped assess providers’ performance were not included or dropped at restructuring. Administrative data were not collected from providers except on a limited basis for the Previniendo pilot. 6.31 Given the limitations in existing M&E systems in ASSE and MSP, the project was too ambitious in its objective of building an integrated electronic information system to inform decision making. Lack of resources and capacity within both MSP and ASSE, along with limited user-friendless and capabilities of some of the instruments promoted by the project, placed serious constraints on the generation of accurate administrative data at department and provider levels and, even more, on the verification and use of that information for decision making. 6.32 Overall, project M&E is rated modest. 7. Lessons 7.1 Preventing NCDs and promoting health requires a multidimensional approach that goes beyond strengthening the role of MSP and the provision of health services. Although this understanding was embedded in the project appraisal document, it was not fully articulated in project objectives or design. The project focused on secondary prevention under the control of MSP, paying limited attention to behavior change activities beyond that of providers. Strategies to influence behavior change of the population toward NCD risk factors were implemented at school and community levels, but their emphasis and scale were limited, and continuity and sustainability relied mostly on individuals or municipality interest. Better strategic use of other actors’ roles and capacities on NCD prevention could have improved the focus, efficacy, and efficiency of the resources devoted to this area. The extent to which MSP can effectively work with and influence other ministries and key civil society actors should not be taken for granted but rather should be an explicit goal of these types of projects. Moreover, attention to promoting behavior change of the population should be as important as investments in equipment, technology, and incentives for health providers. 7.2 Projects implemented during important reform processes have to take into consideration the timing of the reform and adjust project expectations and ambitions 35 accordingly. The project introduced incentives in providers’ payments in three departments to motivate greater focus on NCD prevention and promotion of healthy activities. However, before the project was able to show results, these incentives were scaled up to all 19 departments with the reform. As project scale increased, expectations of results within the lifetime of the project became unrealistic: providers were not used to reporting on performance; providers’ structures, population served, and incentives to comply with performance measures differed; defining performance indicators for NCD management proved difficult; there was no information system in place and limited capacity to support effective reporting; and MSP did not have the capacity to monitor provider’s compliance. Despite these factors the World Bank did not reassess feasibility, nor did it adjust project expectations accordingly. 7.3 Projects with a strong focus on capacity building need to be more realistic about what can be achieved within the project lifetime. Project design underestimated the time and resources needed to equip, motivate, and train providers at the primary care level on the importance of preventing NCDs, and to then define, measure, and monitor providers’ performance (and the learning process involved in continuously redefining and adjusting those measures). Much of the institutional capacity, technology, and equipment funded by the project were not fully operational until later during implementation, which limited their use for NCD management at the provider level. 7.4 Innovative projects such as the PPENT should devote more attention to capture learning from implementation. The project included in its original design an impact evaluation of the pilot but did not include provisions to capture or systematize the learning from an innovative and appealing implementation model that relied extensively on the existing structure and staff of MSP. All stakeholders contacted by IEG confirmed the significant institutional strengthening and capacity building that ASSE and MSP received and the learning-by-doing that the project facilitated. This is especially evident in sustained changes implemented in management tools, evidence- based decision making, and information systems. However, this learning was not well captured by the World Bank, missing an important opportunity to share this experience in other contexts. 1 Global Burden of Disease Compare Tool Website: https://vizhub.healthdata.org/gbd-compare/. 2 Global Burden of Disease Compare Tool Website: https://vizhub.healthdata.org/gbd-compare/. 3Behavioral risks include child and maternal malnutrition, tobacco, alcohol and drug use, dietary risks, sexual abuse and violence, and low physical activity; metabolic risks include: high fasting plasma glucose (blood sugar), high total cholesterol, high body-mass index, low bone mineral 36 density, and impaired kidney function. Global Burden of Disease Compare Tool Website: https://vizhub.healthdata.org/gbd-compare/. 4See chapter 5 of the National Integrated Health System (Sistema Nacional Integrado de Salud (SNIS) creation Law 18211, articles 34 and 36. 5The new payment method seeks to align providers’ incentives to the country’s health care objectives, by linking part of the payment they receive to compliance with the goals. These goals, as well as the manner and frequency with which providers are evaluated and paid, are set by MSP. MSP reviews the metas goals periodically based on the country’s health priorities and the epidemiological profile of the populations served by different providers. 6Fondo Nacional de Salud is the financial entity in charge of collecting, managing, and redistributing public health funds. 7http://www.msp.gub.uy/publicaciones/direcci%C3%B3n-general-del-sistema-nacional- integrado-de-salud/econom%C3%ADa-de-la-salud/sistema 8 https://data.worldbank.org/indicator/SH.XPD.CHEX.GD.ZS. 9 Observatorio del Sistema de Salud de Uruguay (2017). Gasto y Financiamiento del Sector Salud. Press Release, March 13th, 2018. http://www.msp.gub.uy/noticia/se-extiende-el-plazo-para-la- 10 presentaci%C3%B3n-de-experiencias-de-buenas-pr%C3%A1cticas-en. 11Honorary Commissions are autonomous public organizations whose mission is to promote, coordinate, and develop plans and programs for cancer and cardiovascular diseases prevention, early detection, rehabilitation, and research. 12MSP health goals available at http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/INSTRUCTIVO%20META%202%2 0%28Parte%20II%29%20%20-%20Abril2017%20a%20Marzo2019%20versión%2007junio2017.pdf. 13 Tobacco was not included as there was a separate program devoted to it. 14Examples of such indicators could include primary health care facilities reporting through the diagnosis and monitoring system; equipment operational and functioning in primary health care facilities; primary health care facilities implementing changes in NCD management after training received; number of population screened for diabetes, hypertension, and obesity in rural visits; and so on. Some of these indicators were included in the original M&E especially with regard to the Previniendo pilot but were dropped at restructuring. 15 https://atuservicio.msp.gub.uy/ 16Specifically, the Integrated Diagnostic Imaging Network (Red Integral de Diagnostico por Imagenes, RIDI), and the Oncologic Electronic Health Record (Historia Clinica Oncologica, HCEO). At the time of the IEG mission, RIDI was fully operational in ASSE. 17Breast and cervical cancer prevention programs are mandatory by law. MSP resolution No. 402/006 and Presidential Decree No. 571/006 require mammograms and pap smears for working women who want to receive a health card and maintain or obtain a new job. In 2006, women aged 40–59 had to have a mandatory—and free—mammogram every two years; this requirement 37 changed in 2015 to apply to women aged 50–69. Women are given a day off each year to have a gynecological examination and screening and are unable to work without a card certifying that they have been screened. 18 2020 National Health Objectives p.147. 19 http://www.who.int/nmh/countries/2018/ury_en.pdf?ua=1 20MSP. Jornadas de Intercambio. A diez años de iniciada la reforma sanitaria: evaluación y desafíos del Sistema Nacional Integrado de Salud. 21FEMI (n.d). Sistema de Pago por Cumplimiento de Metas Asistenciales: Antecedentes, Situación Actual, y Proyección. https://www.femi.com.uy/archivos_v/Antecedentes%20y%20proyeccion%20del%20pago%20por %20metas.pdf 22Although not strictly covering the whole project implementation period, two NRFSs (2006 and 2013) provide relevant data. 23 Encuesta Longitudinal de Protección Social, 2013 24Diario El País, 28 March 2017. Un Problema de Infancia: el 40% de los escolares de Montevideo sufre sobrepeso. https://www.elpais.com.uy/informacion/escolares-montevideo-sufre- sobrepeso.html 25FAO (2016). Panorama de la seguridad alimentaria y nutricional en América Latina y el Caribe. http://www.fao.org/3/a-i6747s.pdf 26MSP (2012). Segunda Encuesta Mundial de Salud Adolescente. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/II_Encuesta_Mundial_de_Salud_A dolescente_.pdf 27Instituto Nacional de Estadística (2015): First National Survey of Health, Nutrition, and Infant Development 2013 (ENDIS): First Results. Available at http://www.ine.gub.uy/web/guest/encuesta-de-nutricion-desarrollo-infantil-y-salud-endis-2018-/- /asset_publisher/8nW0ZKdgKuqR/content/endis-2013-primeros- resultados/maximized?_101_INSTANCE_8nW0ZKdgKuqR_redirect=%2Fencuesta-de-nutricion- desarrollo-infantil-y-salud-endis-2018- 28MIDES (2018): Second National Survey of Health, Nutrition, and Infant Development 2015 (ENDIS). Available at http://www.mides.gub.uy/innovaportal/file/99392/1/informe-final.pdf 29 GBD Results Tool 2016. 30 GBD Results Tool 2016 website https://vizhub.healthdata.org/gbd-compare/ 31 https://escpromotorasdesalud.weebly.com/. 32 http://www.deporte.gub.uy/plazas-de-deportes.html. 33MSP Press Release, September 23th, 2013: http://www.msp.gub.uy/noticia/“municipios- saludables”-en-canelones. 38 34Law 19140 of October 2013 https://legislativo.parlamento.gub.uy/temporales/leytemp3073449.htm. 35Presidencia de la República Oriental del Uruguay Press Release, August 30, 2018: https://www.presidencia.gub.uy/comunicacion/comunicacionnoticias/basso-rotulado-alimentos- envasados. 36WHO has data for 2011, 2015, and 2018 on NCD progress, but its indicators changed over time and hence they are not fully comparable. 37Most of the providers chose to focus on the reduction of premature deaths from cervical and colon cancer and on reduction in complications from diabetes. 38The Salud.uy program initiated in 2012 established the technical and enabling regulatory environment to enable and secure national electronic medical records (historia clinica electrónica nacional, HCEN). The HCEN will allow health care teams to access the information they need from each patient in real time, from anywhere in the country, ensuring a more accurate and better quality care, and reducing waiting times. 39Priority areas included attention to healthy eating, physical activity, tobacco and alcohol control, and strengthening the level of primary care. 40With RAFU’s support a network of champions of physical activity in each department was created. Physical activity guidelines were prepared by the MSP and the Secretary of Sports and Recreation with support from OPS. 41http://www.agesic.gub.uy/innovaportal/v/4434/19/agesic/novedades-programa- saluduy.html?idPadre=4431 42First year in which disaggregated information is available. See https://www.cgn.gub.uy/innovaportal/file/3801/1/tomo_ii_parte_2_inciso_12.pdf 43Information on public funds to the Honorary Commission for the Fight Against Cancer, and the Honorary Commission for Cardiovascular Health are available at their websites http://www.comisioncancer.org.uy/uc_371_1.html and http://www.cardiosalud.org/organizacion/presupuestos Asociación de Diabéticos del Uruguay web page: https://www.adu.org.uy/index.php/adu/que- 44 hacemos-en-adu. 45 Interview with staff at NCD Alliance. 46 Experts’ estimates at Junta Nacional de Drogas. 47See for instance: Reforma de la Salud: Transformar el futuro; metas cumplidas y desafíos renovados para el Sistema Nacional Integrado de Salud (p.69). 39 Bibliography Abascal, W., E. Estevez, B. Goja, F. Gonzalez, F., and A. Sica. 2013. “Impact of the tobacco control policies. 2006-2009 national Program for Tobacco Control.” Arch Med Interna Octubre 35 (Supl 4): S01-S16. http://www.scielo.edu.uy/pdf/ami/v35s4/v35s4a01.pdf. Abegunde, D. O, C. D. Mathers, T. Adam, M. Ortegon, and K. Strong. 2007. “The Burden and Costs of Chronic Diseases in Low-Income and Middle-Income Countries.” Lancet 370: 1929–1938. http://www.who.int/choice/publications/p_2007_Chronic_disease_burden_Lancet.pdf. Administración de los servicios de salud del Estado. 2014. Total de usuarios del país ASSE: por tipo de cobertura según sexo y grupos de edad. Montevideo, Uruguay: Departamento de Gestión de la Información de ASSE. Asaria, P., D. Chisholm, C. Mathers, M. Ezzati, and R. Beaglehole. 2007. “Chronic Disease Prevention: Health Effects and Financial Costs of Strategies to Reduce Salt Intake and Control Tobacco Use.” Lancet 370: 2044–2053. http://www.who.int/choice/publications/p_2007_Salt_Tobacco_Lancet.pdf. Balsa, A., D. Ferres, M. Rossi, and P. Triunfo. 2009. “ Inequidades socioeconómicas en el uso de servicios sanitarios del adulto mayor montevideano.” Estudios Económicos 24 (1): 35–88. http://www.jstor.org/stable/40311551?seq=1#page_scan_tab_contents Bloom, D., C. Simiao, M. Kuhn, M. McGovern, L. Oxley, and K. Prettner. 2017. “The Economic Burden of Chronic Diseases: Estimates and Projections for China, Japan, and South Korea.” NBER Working Paper 23601, National Bureau of Economic Research, Cambridge, Massachusetts. http://www.nber.org/papers/w23601.pdf Bonilla-Chacín, María Eugenia, ed. 2014. Promoting Healthy Living in Latin America and the Caribbean: Governance of Multisectoral Activities to Prevent Risk Factors for Noncommunicable Diseases. Directions in Development. Washington, DC: World Bank. Buglioni, M., A. Dean, F. Esponda, and E. Raffaele. 2011. Reforma de la salud: Transformar el futuro; metas cumplidas y desafíos renovados para el Sistema Nacional Integrado de Salud. Montevideo, Uruguay. Ministerio de Salud Pública. Cashin, C., Y. -L. Chi, P. Smith, M. Borowitz, and S. Thomson (eds). 2014. Paying for Performance in Healthcare: Implications for Health System Performance and Accountability . European Observatory on Health Systems and Policies. Berkshire, United Kingdom: Open University Press. http://www.euro.who.int/__data/assets/pdf_file/0020/271073/Paying- for-Performance-in-Health-Care.pdf Contaduría General de la Nación. 2012. Rendición de cuentas y balance de ejecución presupuestal. ejercicio 2012. Montevideo, Uruguay: Contaduría General de la Nación. https://www.cgn.gub.uy/innovaportal/file/3801/1/tomo_ii_parte_2_inciso_12.pdf. 40 Demaio, A. R., K. K. Nielsen, B. P. Tersbøl, P. Kallestrup, and D. W. Meyrowitsch. 2014. “Primary Health Care: A Strategic Framework for the Prevention and Control of Chronic Non- Communicable Disease.” Global Health Action 7 (1). DOI: 10.3402/gha.v7.24504. http://doi.org/10.3402/gha.v7.24504. Economist Intelligence Unit. 2017. Cancer Control in Uruguay an Example of Regional Best Practice. London: Economist Intelligence Unit. https://www.eiuperspectives.economist.com/sites/default/files/images/CancerControlinU ruguay.pdf. Ezzati, M.; Obermeyer, Z.; Tzoulaki, I.; Mayosi, B.; Elliott, P. and Leon, D. 2015. “The Contributions of Risk Factor Trends and Medical Care to Cardiovascular Mortality Trends.” Nature Reviews Cardiology 12 (9): 508–530. https://www.nature.com/articles/nrcardio.2015.82#auth-52015 FEMI (Federación Médica del Interior). n.d. Sistema de pago por cumplimiento de metas asistenciales: antecedentes, situación actual, y proyección. Montevideo, Uruguay: FEMI. https://www.femi.com.uy/archivos_v/Antecedentes%20y%20proyeccion%20del%20pago %20por%20metas.pdf. González Raga y Asociados. 2017. “La atención de la salud pública según la opinión pública uruguaya.” CIFRA. http://radiouruguay.uy/wp-content/uploads/2017/11/Informe- Interno-ASSE-oct2017.pdf. Global Adult Tobacco Survey. 2018. Comparison Fact Sheet: Uruguay 2009 and 2017. Atlanta, GA: Centers for Disease Control and Prevention. Inter-American Development Bank (IADB). 2015. “Estrategia de País 2016-2020, Uruguay.” Nota Técnica de Salud. Washington DC. IADB. Instituto Nacional de Estadística. 2015. Encuesta nacional de salud, nutrición y desarrollo infantil 2013 (ENDIS): primeros resultados. Montevideo, Uruguay: Instituto Nacional de Estadística.http://www.ine.gub.uy/web/guest/encuesta-de-nutricion-desarrollo-infantil- y-salud-endis-2018-/-/asset_publisher/8nW0ZKdgKuqR/content/endis-2013-primeros- resultados/maximized?_101_INSTANCE_8nW0ZKdgKuqR_redirect=%2Fencuesta-de- nutricion-desarrollo-infantil-y-salud-endis-2018-. Institute for Quality of Life. 2016. The Social and Economic Costs of Obesity in Latin America: A Call to Action. Gaithersburg, MD: Sodexo Quality of Life Services. http://www.qualityoflifeobserver.com/sites/default/files/latam-obesity-call-to-action- institute-qol-report.pdf. Levcovitz, E., G. Antoniol, D. Sánchez, and M. Fernández Galeano (eds.). 2011. Atención primaria de salud en Uruguay: Experiencias y lecciones aprendidas. Montevideo: OPS. 41 Ministerio de Desarrollo Social. 2018. Encuesta de nutrición, desarrollo infantil y salud: Informe de la segunda ronda 2015 (ENDIS). Montevideo, Uruguay: Ministerio de Desarrollo Social. http://www.mides.gub.uy/innovaportal/file/99392/1/informe-final.pdf. Ministerio de Desarrollo Social / Junta Nacional de Drogas. 2012. Adolescencia: Un mundo de preguntas. II Encuesta mundial de salud adolescente. Montevideo, Uruguay: Ministerio de Desarrollo Social. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/II_Encuesta_Mundial_de_S alud_Adolescente_.pdf. Ministry of Public Health (Ministerio de Salud Pública). 2006. Primer encuesta nacional de factores de riesgo de enfermedades no transmisibles. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/1er_enfrecnt_2006_1.pdf ———. 2011. Derechos y beneficios de usuarios del Sistema Nacional Integrado de Salud, y su correlato legal. Dirección General del SNIS—MSP. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/Derechos_y_Beneficios_y_s u_correlato_legal.pdf ———. 2013. Segunda encuesta nacional de factores de riesgo de enfermedades no transmisibles. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/2DA_ENCUESTA_NACIO NAL_final2_digital.pdf ———. 2015a. Desempeño de los prestadores integrales del sistema nacional integrado de salud: 2010 –14. Área Economía de la Salud, Dirección General del Sistema Nacional Integrado de Salud. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/Desempeño%20Sectorial%2 02010-%202014.pdf ———. 2015b. JUNASA: Rendición de cuentas, Ejercicio 2015. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/RC.pdf ———. 2016a. JUNASA: Rendición de cuentas, Ejercicio 2016. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/comunicado/rendición-de-cuentas-junasa- ejercicio-2016 ———. 2016b. Objetivos sanitarios nacionales 2020. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/Objetivos%20Sanitarios%20 Nacionales%20FINAL%2022%20JUNIO%202016.pdf ———. 2017a. Metas asistenciales: Instructivo meta 2—Abril 2017-Marzo 2019. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/INSTRUCTIVO%20META 42 %202%20%28Parte%20II%29%20%20- %20Abril2017%20a%20Marzo2019%20versión%2007junio2017.pdf. ———. 2017b. A diez años de iniciada la reforma sanitaria: Evaluación y desafíos del sistema nacional integrado de salud. Jornadas de intercambio. Montevideo, Uruguay: Ministerio de Salud Pública. ———. 2017c. Encuesta de satisfacción con los servicios del primer nivel de atención: Informe de resultados 2017. Montevideo, Uruguay: Ministerio de Salud Pública. Monge, A., K. Sass, G. Cataldi, and A. Mazzei. 2012. Municipios y comunidades saludables. Estrategia nacional de promoción de salud, Unidad de Descentralización Territorial. DIGESA, MSP. Montevideo, Uruguay: Ministerio de Salud Pública. http://www.msp.gub.uy/sites/default/files/Municipios_y_Comunidades_Saludables_- _diciembre_2012%20%282%29.pdf. Ordunez, P., E. Prieto-Lara, V. Pinheiro Gawryszewski, A. J. M. Hennis, and R. S. Cooper. 2015. “Premature Mortality from Cardiovascular Disease in the Americas: Will the Goal of a Decline of ‘25% by 2025’ Be Met?” PLoS ONE 10 (10): e0141685. https://doi.org/10.1371/journal.pone.0141685. Oreggioni, I. 2014. Incentivos a la eficiencia en el sistema nacional integrado de salud. Montevideo, Uruguay: Ministerio de Salud Publica. Rasolt, D. H. 2014. “Rising Hypertension, Cardiovascular Disease, in Latin America.” Defeat Diabetes Foundation. https://defeatdiabetes.org/rising-hypertension-cardiovascular- disease-latin-america/. Roth, G., Mohammad H. Forouzanfar, Andrew E. Moran, Ryan Barber, Grant Nguyen, Valery L. Feigin, Mohsen Naghavi, George A. Mensah, and Christopher J. L. Murray. 2015. “Demographic and Epidemiologic Drivers of Global Cardiovascular Mortality.” New England Journal of Medicine 372 (14): 1333–1341. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4482354/ Salicrup, L. A., P. Ordunez, and M. M. Engelgau. 2018. “Hypertension Control Activities in Latin America and the Caribbean: Opportunities for Late-Stage (T4) Translation Research.” Revista Panamericana de Salud Pública 42: e22. DOI: 10.26633/RPSP.2018.22. http://iris.paho.org/xmlui/bitstream/handle/123456789/34869/v42e222018.pdf?sequence=1 &isAllowed=y. Sandoya, E. 2016. “Diabetes y enfermedad cardiovascular en Uruguay.” Revista Uruguaya de Cardiología 31: 505–514. http://www.scielo.edu.uy/pdf/ruc/v31n3/v31n3a12.pdf. Sollazo, A., and R. Berterreche. 2011. “El abordaje de la atención primaria de salud, modelos organizativos y prácticas: caso de un centro de salud público urbano de Montevideo, Uruguay 2011.” Saúde em Debate 36 (94): 461–472. http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-11042012000300017. 43 Organización de las Naciones Unidas para la Alimentación y la Agricultura. 2016. América Latina y El Caribe: Panorama de la seguridad alimentaria y nutricional: Sistemas alimentarios sostenibles para poner fin al hambre y la malnutrición. Santiago, Chile: Food Agriculture Organization. http://www.fao.org/3/a-i6747s.pdf. Webber L., F. Kilpi, T. Marsh, K. Rtveladze, M. Brown, and K. McPherson. 2012. “High Rates of Obesity and Non-Communicable Diseases Predicted across Latin America.” PLoS ONE 7(8): e39589. https://doi.org/10.1371/journal.pone.0039589 World Bank. 2005. Uruguay—Country Assistance Strategy for 2005–10. World Bank: Washington, DC. ———.2007. Uruguay—Non-Communicable. Disease Prevention Project Loan Agreement. Washington, DC: World Bank Group. ———.2008. Uruguay—Non-Communicable. Disease Prevention Project Appraisal Document. Washington, DC: World Bank Group. ———.2015. Uruguay—Country Partnership Strategy for 2010–15. World Bank: Washington, DC. ———.2016. Uruguay—Non-Communicable Diseases Prevention Project Implementation Completion and Results Report. Washington, DC: World Bank Group. ———.2017. Global Monitoring Report: Tracking Universal Health Coverage. World Bank, Washington, DC. ———. 2018. World Bank Group Support to Health Services: Achievements and Challenges. Independent Evaluation Group. Washington, DC: World Bank. World Health Organization. 2015. Noncommunicable Diseases Progress Monitor 2015. Geneva: World Health Organization. ———. 2017. Noncommunicable Diseases Progress Monitor 2017. Geneva: World Health Organization. ———. 2018. Noncommunicable Diseases Uruguay Country Profile. Geneva: World Health Organization. http://www.who.int/nmh/countries/ury_en.pdf. 44 Appendix A. Basic Data Sheet Noncommunicable Diseases Prevention Project (BIRD-74860) Table A.1. Key Project Data Actual or Current Appraisal Estimate Estimate Actual as Percent of Financing ($, millions) ($, millions) Appraisal Estimate Total project costs 32.80 21.77 76 Loan amount 25.30 19.16 76 Cofinancing Cancellation Table A.2. Cumulative Estimated and Actual Disbursements Disbursements FY08 FY09 FY10 FY11 FY12 FY13 FY14 FY15 FY16 FY17 Appraisal 4.00 11.50 18.10 23.20 25.30 25.30 25.30 25.30 25.30 25.30 estimate ($, millions) Actual 0.00 0.35 0.85 2.44 10.59 14.39 16.45 19.76 19.76 19.76 ($, millions) Actual as 0.00 3.04 4.70 10.51 41.86 56.87 65.01 78.10 78.10 78.10 percent of appraisal Date of final disbursement: September 2016 Table A.3. Project Dates Event Original Actual Concept review 01/15/2000 05/15/2006 Negotiations 07/10/2005 07/10/2005 Board approval 11/30/2000 08/28/2007 Signing 12/13/2007 12/13/2007 Effectiveness 01/09/2008 01/09/2008 Closing date 12/31/2012 12/31/2015 45 Table A.4. Staff Time and Cost World Bank Budget Only Staff time Costa Stage of Project Cycle (no. weeks) ($, thousands) Lending FY99 0.00 1,314.00 FY00 0.20 516.87 FY01 0.00 0.00 FY02 0.00 0.00 FY03 0.00 0.00 FY04 3.57 20,370.06 FY05 4.31 28,054.90 FY06 30.24 123,290.70 FY07 32.53 121,082.71 FY08 10.21 26,637.15 Total 81.06 321,266.39 Supervision/ICR FY08 11.24 47,943.23 FY09 29.78 102,076.90 FY10 22.91 96,672.45 FY11 18.97 92,377.52 FY12 8.76 97,556.27 FY13 11.28 97,049.90 FY14 12.99 115,040.59 FY15 17.92 69,477.62 FY16 14.84 74,085.18 Total 148.69 792,279.66 Note: ICR = Implementation Completion and Results Report. a. Including travel and consultant costs. 46 Table A.5. Task Team Members Names Title Unit Responsibility or Specialty Lending Luis Orlando Pérez Sr Public Health Specialist GHN04 Task Team Leader Cristian Baeza Director HNP Public Health Gaston Mariano Blanco Sr Social Protection Specialist GSP04 Operations Isabella Anna Danel Sr Public Health Specialist SASHD Public Health Jose Pablo Gomez-Meza Sr Economist LCSHH Health Ricardo Eduardo Lugea Sr Procurement Specialist GGO04 Procurement Alejandro Roger Solanot Sr Financial Management Specialist GGO22 Financial Management Mariana Montiel Sr Counsel LEGLA Legal Santiago Scialabba Program Assistant LCC7F Administrative Support Febe Susana Libert Program Assistant LCC7F Administrative Support Marcelo Daniel Barg Consultant LCC6C Institutional Development Juan Sanguinetti Consultant HLNP Economic and Financial Analysis Jorge Gosis Consultant GHN04 Public Health Analia Stasi Consultant GHN04 Public Health Supervision/ICR Luis Orlando Pérez Sr Public Health Specialist GHN04 Task Team Leadera Ricardo Eduardo Lugea Sr Procurement Specialist GGODR Procurement Armando Sanjines Sr Procurement Specialist GGO04 Procurement Alejandro Roger Solanot Sr Financial Management Specialist GGO22 Financial Management Mariana Montiel Sr Counsel LEGLA Legal Diego Ambasz Sr Operations Officer GED04 Operations Daniela Romero Operations Officer GHN04 Operations Marcelo Morandi Consultant GENDR Environmental Specialist Febe Susana Libert Consultant GEEDR Administrative Support Mariela Alvarez Program Assistant LCC7 Administrative Support Silvestre Rios Centeno Program Assistant LCC7 Administrative Support Marcelo Daniel Barg Consultant LCC6C Institutional Development Analia Stasi Consultant GHN04 Public Health Jorge Gosis Consultant GHN04 Public Health Alfredo Perazzo Consultant GHN04 Health Economist Mario Virgolini Consultant GHN04 Health Promotion Osvaldo Rico Consultant GHN04 Epidemiologist Natasha Zamecnik Consultant GHN04 Economist Note: ICR = Implementation Completion and Results Report. 47 Table A.6. Other Project Data: Planned versus Actual Expenditure by Component Appraisal Percentage of Component Estimate Actual Costs Appraisal 1. Strengthening MSP capacity to address the 6.1 3.16 52 country's changing epidemiological profile 2. Improving access to quality health care 15.8 14.12 89 services for prevalent NCDs in public primary care 3. Implementation of an NCD prevention pilot 3.8 1.18 31 program, Previniendo 4. Project management 2.1 3.25 155 5. Unallocated 0.9 contingencies Front-end fee IBRD 0.06 Total project cost 28.7 21.71 76 48 Appendix B. Methods and Evidence This evaluation is based largely on (i) interviews with more than 45 project stakeholders in Uruguay carried out during a field mission in April–May 2018; (ii) interviews with World Bank staff and staff from other international agencies working in Uruguay during the project implementation period; and (iii) review of project documents and additional data supplied by the project coordination unit, interviewees, or other sources. In addition, the evaluation benefited from the analysis of a large set of available databases, including Ministry of Public Health administrative data, the 2008 and 2013 National Risk Factors Surveys for NCDs; the 2015 Burden of Disease Study; the 2016 National Health Survey; 2017 data collected by WHO on national responses to NCDs; Departmental/ Municipal-level administrative data; and a large range of technical/analytic studies and research undertaken by a range of agencies/actors in the country. Table B.1 presents the data sources by evaluation question. Table B.1. Data Sources Data collection methods Review of WBG /Govt Review of academic and policy literature Review of project Secondary Data Interviews with Stakeholders documents strategies Site visits sources (NDP) Relevance of Objectives ✓ ✓ ✓ ✓ ✓ Relevance Relevance of Design ✓ ✓ ✓ ✓ Objective 1 to support Government's efforts to further strengthen its ✓ ✓ ✓ ✓ ✓ ✓ health delivery services for NCD Efficacy Objective 2 to support Government's efforts to further strengthen its ✓ ✓ ✓ ✓ ✓ ✓ health policy framework for NCD Efficiency Implementation and technical ✓ ✓ Sustaina to what extent project outcomes and ✓ ✓ ✓ ✓ ✓ bility key outputs are sustained? Long changes on NCD prevalence and NCD- term ✓ ✓ related mortality and morbidity? impacts Valued WB's additionality (beyond ✓ ✓ ✓ added financing) activities designed and implemented to benefit the most disadvantaged? Reaching how did disadvantaged population ✓ ✓ ✓ ✓ ✓ ✓ the poor benefited? 49 The mission focused on Montevideo and included meetings with central government, department directorates of central government, staff in the project coordination unit, municipalities, private sector, academia, and civil society (see appendix D for a list of all people met). Emails were sent to all departments. Visits to sites that had received funds from the Uruguay Solidarity and Inclusive Fund were not conducted as they were well documented by existing evidence, and as secondary sources through interviews confirmed this evidence. The project coordination unit and the World Bank supervision team had collected a solid based of evidence that was made available to the IEG team. IEG evaluation methods sought to complement this existing evidence. Stakeholder interviews focused on generating lessons from the program, and on specific questions around: (i) project design relevance and effectiveness, (ii) changes in NCD-related policies and their connection (or not) with the project, (iii) the additionality of the World Bank’s role; (iv) prospects and barriers for replication in other countries, and (v) lessons from project experience. Stakeholders also volunteered information on the performance of the PSCU and the World Banks’ supervision team and consultants, the institutional strengthening in the Ministry of Public Health and ASSE that the project contributed to, and the transformational nature of the project. Findings have a high confidence level, as there was a very high degree of consistency (and sometimes unanimity) among stakeholders on most findings. 50 Table B.2. Project Results Chain PDO/Outcomes Long-Term Components Outputs (and indicators) Outcomes (i) Strengthen health delivery Integrated health services information system; NCD data and Percentage of diagnosed knowledge products; hypertension, diabetes, obesity NCD epidemiological cases under follow-up care from surveillance; primary care teams. Performance Avoid and reduce Percentage of newborns with monitoring system for disabilities being monitored by exposure to selected NCD public and private risk factors and their Early Detection and Treatment providers; health effects Units. 1. Strengthening the Regulatory Percentage of women aged 50–69 capacity of MSP to frameworks; years and covered by the state address the country's Technology and health insurance that had a changing medical equipment to mammogram. epidemiological primary care facilities NCD quality standards for The crude mortality rate profile. for detection and accreditation of primary health from "diseases of the treatment of NCDs; care facilities. circulatory system" for Management tools, Population screened for NCD risk those in the population including adoption of factors and received follow-up under 70 years old. 2. Improving access quality standards and under Previniendo guidelines to quality health care improved referral services for prevalent systems; NCDs in public primary care Training to health (ii) Strengthen MSP’s health workers on NCD policy framework for NCD prevention, screening, and management; 3. Implementation of Incentives to health Development of a Strategic Plan a NCD prevention centers for prevention for Health Promotion, Prevention, pilot program programs; and Control of NCDs. National health Healthy municipality strategies. promotion strategy; Healthy school strategies. 4. Project Municipal- and management community-level Population aged 45–64 years activities screened for NCD risk factors. Source: Independent Evaluation Group based on project documents. 51 Table B.3. Uruguay Health Goals Implementation period Children’s health Control and referral Oct.2008 Apr. 2017 Mar. 2019 GOAL 1 Maternal health Prenatal care and timely checkups Oct. 2008 Mar.2019 Changes in primary care facilities and training Human resources structure Oct. 2008 Jul. 2009 Referring physician for teenagers and adults Jul 2009 Mar. 2017 Human resources training Jul. 2009 Mar. 2017 GOAL 2 Goals linked to 2020 Health Objectives Control of hypertension Apr. 2017 Mar. 2019 Cardiovascular health Apr. 2017 Mar. 2019 Indicators chosen by providers based on epidemiological profile Apr. 2017 Mar. 2019 Elderly care GOAL 3 Regular checkups (health care card) Jul. 2009 Apr. 2014 Referring physician Jul. 2009 Apr. 2014 (later reintroduced in goal 2) Incentives to primary care physicians GOAL 4 May Salary increases/High dedication positions Jun 2019 2012 52 Source: Independent Evaluation Group based on MSP information and other sources. Note: Goal 1 as originally defined included six indicators on children’s health: (i) newborns referral (follow-up control scheduled before baby is 10 days old), (ii) neonatal care (control that the scheduled follow-up of the newborn takes place), (iii) medical controls during first year of life, (iv) up-to-date vaccinations, (v) breastfeeding, (vi) hip ultrasound at four months; six indicators regarding maternal health: (i) at least six prenatal controls, (ii) timely care for pregnant women, before the third month of gestation, (iii) human immunodeficiency virus and sexually transmitted disease tests, (iv) perinatal clinical record up-to-date, (v) dental check-up, (vi) referral to dentistry of pregnant women, and indicators regarding the response of the health system to gender-based violence. In April 2017, indicators for Goal 1 changed, although the focus remained child and maternal health. Three objectives were defined around children’s health, related to the control and follow-up of newborns, the control and follow-up of newborns “at risk” (with the definition of being at-risk depending not only on health factors, but also on socioeconomic context), and the medical control of children up to five years old. Each of these objectives is composed of multiple indicators. A fourth objective is related to the adequate and timely control of pregnancies; and finally, as before, one component is related to gender-based violence. Goal 2 was initially established as a target on the number of general practitioners, pediatricians, and gynecologists that each institution ought to have to adequately cover their affiliated population. Later, the goal was divided into two components: (i) the referring physician goal set targets for the percentage of users who have a referring physician of their choice and assign to the referring physician the responsibility of monitoring the health of their patients following national protocols and rules, and (ii) the HR training component, that aimed at the training of the medical personnel in areas such as hypertension, diabetes, tobacco consumption, and gender violence. In April 2017, Goal 2 was changed again to better align it to the 2020 Health Objectives through the promotion of adequate control and diagnosis of hypertension, cardiovascular diseases, and two conditions that are tailored to the epidemiological profile of the population covered by each provider. Indicators providers can choose include: (i) reduction of premature deaths due to cancer (focusing on cervical and colon cancer) – selected by 24 providers; (ii) reduction in complications due to diabetes – selected by 15 providers; (iii) reduction of overweight and obesity, focusing on health education – selected by six providers; (iv) reduction of current smokers and exposure to second-hand smoking – selected by five providers; (v) reduction of mortality due to COPD – selected by five providers; (vi) promotion of healthy eating habits and physical activity – selected by two providers; (7) reduction of premature deaths by modifying preventable risk factors – selected by 1 provider. Goal 3 focuses on the elderly and was established in July 2009 with a twofold aim: (i) to promote the prevention and early diagnosis through annual mandatory checkups (free of charge) for those older than 65 (these controls were linked to the delivery of the Health Card of the Elderly); and (ii) to encourage the role of the referring physician in the elderly population. Goal 3 was discontinued and replaced by Goal 4 in 2014. Goal 4 aims at changing the way health care professionals are hired and paid, by creating the so-called “extended-hours positions” to minimize the multiple employments that characterize the sector. Doctors hired under this new scheme ought to meet certain criteria regarding the allocation of time among primary care, urgent care, and training/education activities. Health promotion is encouraged in the primary care and is to be achieved by working not only with the patient, but also with their family and social context (Oreggioni 2014). 53 Table B.4. Key Project Development Objective Indicators, Targets, and Progress Value at Original Original Revised Revised Project Adequacy of Indicator to Measure PDO and Indicator Baseline Target Baseline Target Closure Related Information after Project Closure Percentage of hypertension cases diagnosed 49% 60% 54.7% 62.6% NRFS data were used for these indicators, not and under follow-up by primary care teams. administrative data by providers. Indicators do not measure adequacy of care nor performance for the Percentage of diabetes cases diagnosed and 73% 83% 63.9% 73% 77.6% different providers. Elevated levels of under diagnosis may under follow-up by primary care teams. affect reliability of the data. Targets were reduced for diabetes and obesity after NRFS prevalence data were Percentage of obesity/overweight cases NA 50% 13% 20% 34.3% available. Related indicators for Health Goal 2 should diagnosed and under follow-up by primary better capture this but as of September 2018 there is no care teams compliance data available. Percentage of women 50–69 covered by the 25% 45% 9.9% 20% 11.63% Target population changed at restructuring to cover only public provider who had mammography last ASSE. Available information for ASSE suggests a limited year impact of the project on the public provider performance (that goes beyond this indicator). Percentage of newborns with disabilities NA 60% 75.6% Inadequate indicator to measure PDOs. being monitored by Early Detection and Dropped at restructuring. Treatment Units Crude mortality rate from diseases of the 75.18% 67.5% 60.3% This indicator was added at the 2012 restructuring. It circulatory system (I00-I99) in the population better captures the long-term outcomes, but attribution is under 70 years complex (decreasing trend before project implementation). Population aged 45–64 covered by the NA 65% 30% 36.7% Target population extended at restructuring (originally National Integrated Health System and covered only ASSE). Results come from private providers screened for noncommunicable disease risk as data for ASSE indicates a mere 1.3% toward a target of factors 22%. Indicator does not capture follow-up care. Related indicators for Health Goal 2 should better capture this but as of September 2018 there are no compliance data available. Source: Independent Evaluation Group based on project documentation NRFS = National Risk Factor Survey; PDO = project development objective. 54 Table B.5. Project Contributions to WHO Indicators on NCD Progress (2015) WHO Indicators NCD Progress Monitor Data PPENT Contributionsa 1. Uruguay has set national NCD Fully achieved The PPENT was key in raising targets (time-bound, address NCD https://extranet.who.int/ncdccs/Data/U awareness on the importance of mortality and key risk factors, and RY_Uruguay_NCD_targets.pdf NCDs. The Previniendo pilot was a based on the nine voluntary global precursor of the health targets set by targets and the WHO Global Data collected every two years the country. The PPENT funded the Monitoring Framework) second NRFS and the Global Burden http://www.who.int/ncds/surveillance/n cd-capacity/en/ of Disease study, crucial to identify the main NCD risk factors so that targets could be set accordingly. 2. Uruguay has a functioning system Fully achieved The PPENT contributed to setting the for generating reliable cause-specific foundations for an integrated health mortality data on a routine basis Data collected yearly information system and initiated the integration of key databases on: life http://www.who.int/healthinfo/tool_co d_2010.pdf events, reportable diseases and health events, hospital infections and discharges, and national risk factors. 3. Uruguay has a STEPS survey or Partially achieved The PPENT funded the second NRFS. another risk factor survey every 5 The third NRFS should have been yearsb Data collected every two years done in 2018, but the survey is not in the MSP budget. The country is http://www.who.int/ncds/surveillance/n cd-capacity/en/ searching for alternative funding. 4. Uruguay has an operational Partially achieved The PPENT supported the multisectoral national strategy/ action https://extranet.who.int/ncdccs/Data/U implementation of a National Health plan that integrates major NCDs RY_Uruguay_NCD_targets.pdf Promotion Strategy, and the creation (diabetes; cancer; chronic, of a Health Promotion Network cardiovascular, and respiratory Data collected every two years composed of intersectoral diseases) and their shared risk factors http://www.who.int/ncds/surveillance/n government officials at the national (unhealthy diet, physical inactivity, cd-capacity/en/ and departmental levels. harmful use of tobacco and alcohol) 5. (a) Uruguay has implemented (a) Partially achieved The PPENT had nothing to do with measures to reduce affordability by the observed progress on tobacco increasing excise taxes and prices on policies. This was a result of a tobacco products successful government program, led (b) Fully achieved (b) Uruguay has implemented by the president directly. measures to eliminate exposure to second-hand tobacco smoke in all indoor workplaces, public places, and public transport (c) Uruguay has implemented plain and standardized packaging or large (c) Fully achieved graphic health warnings on all tobacco packages (d) Uruguay has enacted and enforced comprehensive bans on 55 tobacco advertising, promotion, and (d) Fully achieved sponsorship (e) Uruguay has implemented effective mass media campaigns that (e) Not achieved educate the public about the harms of smoking/tobacco use and second- hand smoke Data collected every two years http://www.who.int/tobacco/global_r eport/ http://www.who.int/tobacco/global_r eport/2017/technical_note_I.pdf?ua=1 6. (a) Uruguay has enacted and (a) Partially achieved enforced restrictions on the physical The PPENT did not have any specific availability of retailed alcohol (via alcohol-related actions. reduced hours of sale) (b) Not achieved (b) Uruguay has enacted and enforced bans or comprehensive restrictions on exposure to alcohol advertising (across multiple types of media) (c) Partially achieved (c) Uruguay has increased excise taxes on alcoholic beverages Data collected every three to four years http://www.who.int/entity/substance_ abuse/activities/survey_alcohol_health _2016.pdf?ua=1 7. (a) Uruguay has adopted national (a) Partially achieved Several regulations led directly by the policies to reduce salt/sodium Montevideo Municipality aimed at consumption decreasing salt intake among the (b) Not achieved (b) Uruguay adopted national policies population. The PPENT did not have that limit saturated fatty acids and anything to do with this. virtually eliminate industrially produced trans fatty acids in the food However, the PPENT contributed supply (c) Fully achieved through the implementation of (c) Uruguay has implemented the Healthy Schools Campaigns, which WHO set of recommendations on encouraged healthy habits in schools marketing of foods and nonalcoholic (d) Partially achieved prioritizing better nutrition and beverages to children Data collected every two years physical activity. The PPENT also (d) Uruguay has legislation/ supported regulations regarding regulations fully implementing the http://www.who.int/ncds/surveillance/ healthy snacks for schools and a International Code of Marketing of ncd-capacity/en/ decrease in the amount of salt used Breast-milk Substitutes http://www.who.int/nutrition/publicat in school menus. ions/infantfeeding/code_report2016/ en/ 56 8. Uruguay has implemented at least Not achieved The PPENT helped in raising one recent (within past five years) awareness of the importance of national public awareness program physical activity, particularly among Data collected every two years and motivational communication for http://www.who.int/ncds/surveillance/children, through small-scale Healthy physical activity, including mass Schools Campaigns. Also, the PPENT ncd-capacity/en/ media campaigns for physical activity financed the investment in equipment behavioral change to support the promotion of physical exercise, including the installation of 38 active places with physical activity stations and the purchase of 60 bicycles for the Community Bike project implemented in the city of Canelones. 9. Uruguay has evidence-based Not achieved PPENT supported the development national and pilot test of quality standards for guidelines/protocols/standards for managing NCDs in primary care Data collected every two years the management of major NCDs http://www.who.int/ncds/surveillance/facilities. As a result of this pilot, a through a primary care approach, voluntary self-assessment tool on ncd-capacity/en/ recognized/approved by government quality standards has been developed or competent authorities and applied in public and private facilities in five departments (Maldonado, San Jose, Canelones, Florida, and Montevideo). 10. Uruguay has provision of drug Don’t Know The PPENT contributed to an increase therapy, including for control for in the percentage of people diabetes mellitus and hypertension, diagnosed and under follow-up Data collected every two years and counseling for eligible persons at http://www.who.int/ncds/surveillance/treatment for diabetes, obesity, and high risk of heart attacks and strokes, hypertension. However, the PPENT ncd-capacity/en/ with emphasis on the primary care did not directly fund the provision of level drugs for diabetes/hypertension, or counseling for at-risk individuals. Source: WHO NCD Progress Monitor (2015 Data) complemented by IEG evidence to assess PPENT contributions. Note: NRFS = National Risk Factor Survey. a. PPENT = Noncommunicable Diseases Prevention Project. b. STEP= WHO’s STEPwise approach to surveillance. 57 Appendix C. Other Programs Contributing to Project Development Objectives In addition to the activities directly supported by the project, other measures implemented during the project implementation period may have contributed to the progress observed. Among the measures impacting NCDs targeted by the project, the most notable include national programs for tobacco and cancer control, programs for secondary prevention, and discounts on medicines for diabetes and hypertension. National Program for Tobacco Control:48 in September 2004, Uruguay adhered to the WHO Framework for Tobacco Control. Key measures implemented through this program included the prohibition of smoking in closed public places and in work places; a systematic rise in tobacco prices through tax increases; regulation, and later prohibition, of tobacco related marketing and publicity; prohibition of exhibiting tobacco products at points of sale; and free diagnosis and treatment of tobacco-dependency problems, as part of the services provided at the primary level of care (Abascal et al. 2013).49 Uruguay’s approach to and success in tobacco control are known as best practice in the region.50 National Program for Cancer Control:51 created in 2005, with the objective of reducing the incidence of cancer and its associated mortality rates through (i) promoting education and prevention among the population, and (ii) providing assistance, rehabilitation, and palliative care to those with the disease. Several publications have highlighted the success of this program.52 Programs for Secondary Prevention:53 since 2004, the National Resource Fund (FNR) runs four programs focused on smoking cessation, cardiovascular diseases, kidney disease, and obesity. Programs are directed to FNR beneficiaries, except for the smoking cessation program, which is available to all. Evaluations of these programs, performed periodically by the FNR, show positive results in terms of risk factors, health complications, and mortality.54 Discounts on diabetes and hypertension medications: to guarantee access to adequate treatment and medication for those diagnosed with hypertension and diabetes, medication for these conditions were subjected to discounts and exemptions.55 48 Law 17793 of July 2004 and Law 18256 of March 2008. 49 Global Adult Tobacco Survey, Comparison Fact Sheet: Uruguay 2009 & 2017. 50 See for instance Bonilla (2014) and The Economist Intelligence Unit (2017). 51 Decreto N° 202/005. https://www.impo.com.uy/bases/decretos/202-2005 58 52 See for instance The Economist Intelligence Unit (2017). 53 Secondary prevention refers to interventions that aim to control and avoid complications once a disease has been already developed. 54 Fondo Nacional de Recursos, Programas de Prevención: http://www.fnr.gub.uy/programas_prevencion 55 Ministerio de Salud Pública. 2011. Derechos y Beneficios de usuarios del Sistema Nacional Integrado de Salud, y su correlato legal. Dirección General del SNIS – MSP. Additional Data on Promotion and Prevention Figure C.1. Public Expenditures on Noncommunicable Disease Prevention a. Honorary Commission for Cardiovascular Health b. Honorary Commission for Cancer 100.0 100 90.0 90 80.0 80 70 70.0 U$ (millions) 60 60.0 $U (millions) 50 50.0 40 40.0 30 30.0 20 20.0 10 10.0 0 2011 2012 2013 2014 2015 2016 - Nominal $ Real $ (2010) Nominal $ Real $ (2010) Sources: Income statements from the Honorary Commission for the Fight Against Cancer, and the Honorary Commission for Cardiovascular Health. Available at http://www.comisioncancer.org.uy/uc_371_1.html and http://www.cardiosalud.org/organizacion/presupuestos Note: both commissions receive additional support from donations, members’ fees, and other sources. MSP indicators on providers’ performance suggest a change in the model of attention in the direction sought by the health care reform and supported by the PPENT. Greater focus on prevention can be inferred by increases in the ratio of coordinated or nonurgent consultations and admissions over urgent consultations. Available data for the period 2012–171 show marked differences between the public provider ASSE, the Private Health Insurers, and IAMC (figure C.2). Private insurers show higher ratios of coordinated admissions and consultations. These figures need to be taken with caution given the differences in the socioeconomic profile of those affiliated with the different providers and issues with underreporting in ASSE. Also, ASSE performance does not show a consistent picture (the first indicator shows an improvement during the period, but the second one 59 suggests a decline). Moreover, important differences can be observed within regions—ASSE providers in the south region consistently show better performance.2 Figure C.2. Ratio of Coordinated to Urgent Admissions and Consultations a. Coordinated vs Urgent Admissions b. Coordinated vs Urgent Consultations 5.72 6.13 6.18 0.98 0.91 0.94 5.24 5.29 0.89 5.10 Ratio of consultations Ratio of admissions 5.33 5.33 5.21 5.31 5.36 5.10 0.65 0.59 0.65 0.62 0.58 0.37 0.40 0.41 2.50 2.38 2.53 2.38 2.32 2.24 0.31 0.39 2013 2014 2015 2016 2017 2012 2013 2014 2015 2016 2017 ASSE SPI IAMC ASSE SPI IAMC Source: SINADI (Sistema Nacional de Información), Ministerio de Salud Pública. Note: ASSE = State Health Services Administration; IAMC = Collective Medical Assistance Institutions; SPI = Private Health Insurers. Additional information regarding ASSE’s performance at the level of primary care is limited. A study conducted in an ASSE primary care network facility in an urban area of Montevideo in 2011 using the Primary Care Assessment Tool methodology reported good performance on “first point of contact,” “comprehensiveness” of services offered, and human resources capacity.3 Performance was found intermediate on coordinated care and integration of services and low on access and the family and community centeredness of the network. Barriers to access were linked to organizational and functional aspects (Berterretche and Sollazzo 2012). Another study conducted in 2017 for ASSE RAP south region using the same methodology suggests that low performance on accessibility is still an issue (IDB 2017). Information collected during field interviews suggests that ASSE is still working on improving its primary care. The Inter-American Development Bank is currently supporting ASSE RAPs in the south region of the country with a pilot that includes several of the activities originally included in the PPENT, such as integrated information systems, guidance and protocols, training to health personnel, referral systems, management tools, and performance measures. Given that the south region is the best performer among ASSE, the existence of such a pilot suggests that the accessibility and quality of care in ASSE is not yet achieved (even if not specific to NCD management). MSP data indicate user satisfaction with primary care services is relatively high (MSP 2017). Users of ASSE primary care in the past year ranked their overall satisfaction at 8, on a scale of 1 to 10, and 93.3 percent of users would prefer to stay in ASSE, given the choice of changing providers. The level of satisfaction among ASSE beneficiaries is similar to that of IAMC’s 60 beneficiaries but lower than levels observed among beneficiaries of private insurers. Within ASSE, providers in the south region present higher levels of satisfaction. However, 2017 data suggest a slight decline in user satisfaction among ASSE beneficiaries, compared with levels observed in 2014.4 References Abascal, W., Estevez, E., Goja, B., Gonzalez, F., Sica, A. (2013). Impact of the tobacco control policies. 2006-2009 national Program for Tobacco Control. Arch Med Interna Octubre 2013; 35 (Supl 4):S01- S16. http://www.scielo.edu.uy/pdf/ami/v35s4/v35s4a01.pdf Berterretche, R., and A. Sollazzo. 2012. “ The Approach of Primary Health Care, Organizative Models and Practices: Case of an Urban Public Health Center in Montevideo, Uruguay 2011.” Saúde Debate [online] 36(94): 461–72. Bonilla-Chacín, M.E. (ed). 2014. Promoting Healthy Living in Latin America and the Caribbean: Governance of Multisectoral Activities to Prevent Risk Factors for Noncommunicable Diseases. Directions in Development. Washington, DC: World Bank. Economist Intelligence Unit. 2017. “Cancer Control in Uruguay, An Example of Regional Best Practice.” https://www.eiuperspectives.economist.com/sites/default/files/images/CancerControlinUruguay. pdf. IDB (Inter-American Development Bank). 2017. “Fortalecimiento de la resolutividad del primer nivel de atención de ASSE.” Washington, DC: IDB. Ministry of Public Health (MSP). 2017. “Encuesta de Satisfacción con los Servicios del Primer Nivel de Atención: Informe de Resultados 2017.” Área de Economía de la Salud – JUNASA MSP. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/Informe%20Encuesta%20de%20Sati sfacción_0.pdf. 1 Reporting on these indicators was not mandatory until 2012/13. 2 MSP (2017). Desempeño de los Principales Prestadores del SNIS, 2014 –16. A 10 Años de la Reforma Sanitaria. http://www.msp.gub.uy/sites/default/files/archivos_adjuntos/Desempeño%20principales%20prestadores %20del%20SNIS%202014-%202016%20DIC17.pdf 3 The methodology was developed by John Hopkins University and adapted to the Uruguayan context and consists of surveys of facilities, consumers, and providers. 4 This information contradicts findings in another phone survey conducted in October 2017 that shows that 6 out of 10 Uruguayans considered that the level of care in ASSE has improved. http://radiouruguay.uy/wp-content/uploads/2017/11/Informe-Interno-ASSE-oct2017.pdf 61 Appendix D. List of Persons Met Names Title Organization Luis Orlando Pérez Task Team Leader World Bank Daniela Romero ICR Author World Bank Mario Sanchez IADB Local Office Rita Sorio IADB Local Office Martín Sacchi Project Coordinator MSP Cecilia Reolón PMU–M&E MSP Dr Jorge Basso Ministro Salud Pública Dra. María José Rodríguez Responsable Area Programatica NCD MSP Dr Jorge Quian Subsecretario de Salud Pública MSP Andrea Mazzei Coordinadora Programa Promoción de la MSP Salud y Determinantes Sociales Rosario Berterreche Director Sistemas de Información ASSE Dr. Mario Zelarayán Director Ejecutivo Comisión Honoraria para la Salud Cardiovascular Daniel Olesker Asesor del PIT-CNT Instituto Cuesta Duarte Ida Oreggioni Directora Economia y Salud MSP Giselle Jorcin MSP Eduardo Bianco Presidente Centro de Investigacion para la Epidemia del Tabaquismo, Dr Alfredo Toledo Alianza Uruguay para las ENTs Dr Miguel Asqueta Alianza Uruguay para las ENTs Dardo Curti Banco Central Uruguay Gustavo Sóñora Legal Advisor Tobacco Control Anabella Marchese Directora de Unidades Asistenciales ASSE Especializados Elizabeth Ortega Coordinadora del programa de capacitacion ASSE Ing José Clastornik Director AGESIC Wilson Benia Asesor MSP Alicia Ferreira Directora Fondo Nacional de Recursos Gabriela Lamique Directora Red Atención Primaria ASSE Alicia Sosa Directora Red Atención Primaria ASSE Miguel Fernández Galeano ASSE Jorge Venegas Asesor Federación Uruguaya de la Salud Soledad Bonapelch Departamento de Sociología, FCS Adriana Brescia DIGECOOR MSP Oscar Garateguy DIGECOOR MSP Gabriel Rossi MSP 62 Raquel Sánchez Asociación Uruguaya de Nutricionistas y Dietistas María Rosa Curutchet Directora del Observatorio de Seguridad MIDES Alimentaria Oscar Díaz Arnesto Red de Actividad Física del Uruguay Gilberto Ríos Director del Área de Relaciones MSP Internacionales y Cooperación Rosana Gambogi Co-Encargada de la Dirección Técnico Fondo Nacional de Recursos Médica Héctor Suárez Coordinador Junta Nacional de Drogas Ximena Moratorio Responsable del Programa Nacional de MSP Nutrición Ricardo Rodríguez Buño PAHO/WHO Giovanni Escalante Representative PAHO/WHO Uruguay Raquel Rosa Sub Directora General de Salud MSP Leticia Zumar Asesora Económica MEF Martín Vallcorba Asesor Asesoría Macroeconómica y MEF Financiera Analice Beron Directora de Salud Intendencia de Montevideo Marcelo Amado Director de la Dirección de Regulación Intendencia de Montevideo Alimentaria Alicia Musso Programa "Escuelas Saludables" Álvaro Luongo Presidente Comisión Honoraria de Lucha contra el Cáncer Javier Gonzalez Director Asociación de Diabéticos del Uruguay Pablo Caggiani Consejero Electo Consejo de Educación Inicial y Primaria 63