VERIFICATION OF PERFORMANCE IN RESULTS-BASED FINANCING (RBF): THE CASE OF PANAMA'S HEALTH PROTECTION FOR VULNERABLE POPULATIONS (PSPV) PROGRAM DISCUSSION PAPER AUGUST 2015 Alfredo Perazzo Carmen Carpio Renzo Sotomayor VERIFICATION OF PERFORMANCE IN RESULTS-BASED FINANCING (RBF) THE CASE OF PANAMA’s HEALTH PROTECTION FOR VULNERABLE POPULATIONS (PSPV) PROGRAM Alfredo Perazzo, Carmen Carpio and Renzo Sotomayor August 2015 Health, Nutrition and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population (HNP) Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. For information regarding the HNP Discussion Paper Series, please contact the Editor, Martin Lutalo at mlutalo@worldbank.org or Erika Yanick at eyanick@worldbank.org. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522- 2422; e-mail: pubrights@worldbank.org. © 2015 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii Health, Nutrition and Population (HNP) Discussion Paper Verification of Performance in Results-Based Financing Programs: The Case of Panama’s Health Protection for Vulnerable Populations (PSPV) Program Alfredo Perazzo a, Carmen Carpio b and Renzo Sotomayor c a Health, Nutrition & Population, The World Bank Group, Washington, DC, USA b Health, Nutrition & Population, The World Bank Group, Washington, DC, USA c Health, Nutrition & Population, The World Bank Group, Washington, DC, USA This study was produced with support from the Health Results Innovation Trust Fund (HRITF) administered by the World Bank and generously supported by the Governments of Norway and the United Kingdom. Abstract: Verification differentiates results-based financing (RBF) from other health-financing mechanisms, and it is considered an important process of RBF program design and implementation. Despite the vital role it plays in RBF, not much has been written about verification as a process, and information about different elements of the process—frequency, cost, and direct and indirect effects among others—is scarce. Panama’s Health Protection for Vulnerable Populations Program (PSPV) uses an RBF mechanism to deliver health services to the country’s rural poor. As in many RBF schemes, a major component of the PSPV is the verification of results. This study focuses on PSPV’s verification process, highlights its results and their application, and identifies lessons learned. Such information is useful to policy makers and technical experts interested in or designing RBF mechanisms. Keywords: verification, PSPV, Panama, results-based financing (RBF), lessons learned Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Correspondence Details: Petra Vergeer, 202-473-7181, Pvergeer@worldbank.org, The World Bank Group, 1818 H St NW, Washington, DC, USA iii Table of Contents LIST OF TABLES AND FIGURES ................................................................................................. VI ACKNOWLEDGMENTS ................................................................................................................ VI ABBREVIATIONS……………………...………………………………………………………………..VIII PREFACE ...................................................................................................................................... VII INTRODUCTION ............................................................................................................................. 9 METHODOLOGY .......................................................................................................................... 10 CONTEXT: RBF IN PANAMA ...................................................................................................... 11 INTERNAL VERIFICATION IN THE PSPV .................................................................................. 15 General Facts about the Internal Verification Process .................................................................. 15 How is the Beneficiary Population Register Internally Verified? ................................................... 15 How are the Coverage and Service-Provision Internally Verified? ............................................... 16 MAJOR CHARACTERISTICS OF THE THIRD-PARTY VERIFICATION PROCESS ................. 17 General Facts about the Third-Party Verification Process ............................................................ 17 How Is the Beneficiary Population Register Verified by the ATE? ................................................ 18 Stage One .................................................................................................................................. 18 Stage Two .................................................................................................................................. 18 How Are the Coverage and Service-Provision Indicators Verified by the ATE? ........................... 18 Stage One .................................................................................................................................. 19 Stage Two .................................................................................................................................. 19 How is the Social Audit Performed? .............................................................................................. 19 Implementation Challenges ........................................................................................................... 20 FINDINGS OF THE VERIFICATION METHODS.......................................................................... 21 What Are the Results of the Verification? ...................................................................................... 21 What Are the Findings Used for? .................................................................................................. 23 Adjustments to Payments .......................................................................................................... 23 Determination of Penalties ........................................................................................................ 24 Establishing Targets for Indicators ............................................................................................ 25 VERIFICATION COSTS ................................................................................................................ 26 LESSONS LEARNED ................................................................................................................... 27 What Has Been Learned? ............................................................................................................. 27 The Use of IT Systems in the Verification Process ................................................................... 27 The Synchronization of the Verification of Different UBAs ........................................................ 27 The ATE’s Relationship with the PVSP Stakeholders at the MOH ........................................... 27 The Sample Size of Beneficiary Records and Indicators .......................................................... 27 The Verification of Protocol Compliance ................................................................................... 27 The Need for the Simplification of Payment and Verification Processes .................................. 28 CONCLUSION............................................................................................................................... 29 REFERENCES .............................................................................................................................. 30 ANNEXES……………………………………………………….……………………………………...…31 ANNEX 1. Package of prioritized health interventions…………...………………..................… 31 ANNEX 2. Social audit questionnaire…………..…………...………………...………………….... . 32 iv LIST OF TABLES AND FIGURES Table 1 Basic Facts about Panama 9 Table 2: Population Coverage, Number, and Capitation Payment by Region 12 Table 3 Five Coverage Indicators 13 Table 4 Ten Service-Provision Indicators 14 Table 5. Final Fiver Percent Payment 20 Table 6. UBAs RBF Coverage Indicators (2013-2014 Period) 21 Table 7. UBAs RBF Service-provision Indicators (2013-2014 Period) 21 Table 8. Average percent of indicators accepted by DPSS and rejected by ATE 22 Table 9. Adjustments for the Coverage Indicators 24 Table 10. Penalties and Adjustments 25 Table 11. Costs Associated with the ATE 26 Figure 1 Map of Panama 10 Figure 2 Organizational Structure of the PSPV 11 Figure 3 Actors’ Roles in the Internal Verification Process and the Flow of Information 15 Figure 4 Third-Party Verification Process and the Role of the ATE 17 v ACKNOWLEDGMENTS A number of individuals were key to the production of this case study. Dr. Alex Gonzalez and Dr. Tania Chung of the Ministry of Health’s Financial and Administrative Management Unit in Panama provided invaluable assistance in clarifying the complexities of the program and in answering the authors’ queries. The paper also benefited from Petra Vergeer’s expertise on verification and her guidance on developing the structure of this case study. The authors also wish to thank Evelyn Rodriguez, who provided technical and translation assistance, and Sarah McCune, who provided significant editorial and writing support. Finally, the authors thank the Health Results Innovation Trust Fund (HRITF) at the World Bank for funding this study. The content and conclusions expressed within are those of the authors, and do not represent the views of the World Bank, Panama’s Ministry of Health, or the funders. Any errors and omissions are the authors’ alone. The authors are grateful to the World Bank Group for publishing this report as an HNP Discussion Paper. vi ABBREVIATIONS ATE External Technical Audit (Auditoria Técnica Externa) CODIPRO Technical Coordination Board (Consejo Directivo de Proyectos) CPP Portfolio of Incentivized Services (Cartera de Prestaciones Priorizadas). DPSS Directorate of Health Service Provision (Dirección de Prestación de Servicios de Salud) DRS Directorate for Regional Health Services (Dirección Regional de Salud) EEC Health Coverage Expansion Strategy (Estrategia de Extensión de Cobertura) GI Institutional Groups (Grupos Institucionales) GOP Government of Panama HEPI Health Equity and Performance Improvement Project MOH Ministry of Health OE External Organizations (Organizaciones Externas) PAISS Basic Health Services Package (Paquete de Atención Integral de Servicios de Salud) PSPV Health Protection for Vulnerable Populations (Protección en Salud para Poblaciones Vulnerables) RBF Results-Based Financing RDO Opportunities Network (Red de Oportunidades) UBA Basic Health Unit (Unidad Básica de Salud) UGSAF Financial and Administrative Health Management Unit (Unidad de Gestión de Salud, Administrativa y Financiera) vii PREFACE This case study is part of a World Bank-published series that examines verification in results-based financing (RBF) programs in six countries. The series is intended to provide RBF practitioners, technical experts, and policy makers with practical information they can use to inform the design and implementation of the verification process in their respective RBF programs. This particular case study on Panama’s Health Protection for Vulnerable Populations Program (PSPV) follows a framework developed to permit cross-country comparisons. More specifically, this study presents an overview of the PSPV’S RBF mechanism and the program’s verification process. The study also outlines the findings and results of verification, as well as the costs associated with verification. Finally, it provides a brief summary of lessons learned. The complexity of verification in Panama’s PSPV makes it an especially interesting and informative case. viii INTRODUCTION Panama is an upper-middle income country located at the southern tip of Central America. The country of 3.86 million boasts high economic and development indicators. In 2013, its Gross National Income (GNI) per capita was USD 10,700, and its Human Development Index (HDI) was .765, ranking it 65th in the world out of 187 countries and fourth out of the 20 countries in the region (UNDP, 2013). Panama’s health spending is generally in line with its income status and level of development. Total health expenditure in 2012 accounted for 7.6 percent of the country’s GDP, and per capita spending was USD 723. On average, the country performs well on key health indicators. Panama’s infant mortality rate in 2012 was 16 per 1,000 live births, and under-five mortality was 19 per 1,000 live births. The country’s maternal mortality rate in 2013 was 85 per 100,000 live births (World Bank, 2013). Table 1. Basic Facts about Panama Basic Facts about Panama Population (million) 3.86 GNI per capita (current USD) 10,700 HDI .765 Total health expenditure (% GDP) 7.6 Total health expenditure per capita 723 (current USD) Maternal Mortality Rate (per 100,000) 85 Infant mortality rate (per 1,000) 16 Under-five mortality rate (per 1,000) 19 Source: World Bank Open Data, 2013 These averages, however, mask the disparities in health outcomes between urban and rural and indigenous areas where access to quality health services is less equitable. For example, in 2010 the infant mortality rate in the rural province of Bocas del Toro was 26.6, and in Guna Yala, an indigenous area, it was 22.3 (PAHO, 2012). Under-five mortality was, respectively, 2.6 and 0.6 times higher than the national average (PAHO/MINSA, 2012). Over the past decade, the Government of Panama (GOP) has used different results-based financing (RBF) approaches to improve health outcomes and to address the inequity of health services. In 2008, it introduced the World Bank-financed Health Equity and Performance Improvement (HEPI) project to implement the Health Protection for Vulnerable Populations (PSPV) program. The PSPV is one of two components of the Ministry of Health’s (MOH) Coverage Extension Strategy (EEC), and provides health services to the rural poor. Since its implementation, the PSPV has provided more than 200,000 beneficiaries with direct and continuous access to a package of health services. As in many RBF schemes, a major component of the PSPV is the verification of results. This study, therefore, will focus on PSPV’s verification process, highlight its results and their application, and identify lessons learned. 9 Figure 1. Map of Panama METHODOLOGY: This case study is part of a larger, six-country series on verification, and it follows a framework developed to permit cross-country comparisons. This comparative framework includes the following sections: i.) an overview of the PSPV; ii.) a description the program’s verification process; iii.) the findings and results of verification; iv.) the costs associated with verification; and v.) a brief summary of lessons learned. The case study, which covers the years 2011 and 2012, is primarily based on two sources of information: quarterly reports and interviews with government counterparts. The quarterly reports, which formed the basis of the results section, were produced during the verification process and are known as the Informe de Certificación del Padrón de Población Beneficiaria (Report on the Certification of the Population Registry). The interviews were conducted with officials involved in the implementation of the PSPV program, and the information yielded in these interviews concerned the program, the verification process, and lessons learned. 10 CONTEXT: RBF IN PANAMA Panama has used different RBF approaches to address the country’s gaps in access to health services and to improve health outcomes. In 2006, the GOP introduced the Opportunities Network (RDO), a conditional-cash transfer program targeting poor households. Under this program, cash transfers are made once households have met certain conditions. Two conditions of the RDO are health related: i.) completing antenatal care for pregnant women and; ii.) ensuring that vaccinations of children under five are up-to-date. Another example is the Health Coverage Expansion Strategy (EEC, for its Spanish acronym), which uses capitation payments that create financial incentives for providers to achieve better results. The EEC facilitates the delivery of two related packages of health services: i.) the Integrated Package of Health Care Services (PAISS+N), targeted to indigenous communities; and ii.) the Health Protection for Vulnerable Populations program (PSPV, for its Spanish acronym) targeted to the rural poor, non-indigenous populations. The PSPV is the focus of this case study. The PSPV program’s organizational arrangements are the following: the MOH’s Technical Coordination Board, (CODIPRO) directs the PSPV program at the central level. The Directorate of Health Service Provision (DPSS) and the Health Management, Administrative, and Financial Unit (UGSAF) are responsible for its technical oversight and financial management. The Directorate for Regional Health Services (DRS) supervises the Basic Health Units (UBAs). The organizational structure of the PSVP is represented visually in Figure 2 below. Figure 2 Organizational Structure of the PSPV Source: Operative Regalement (PAISS + N / PSPV) -2012 11 The PSPV’s package of health services is delivered through basic health units (UBA), which also identify and enroll beneficiaries. There are 79 UBAs in 12 regions, and they operate as mobile health teams. Mobile health teams could be: i) not-for-profit private health provider contracted by MOH central level to deliver the health package, referred to as external organizations (organizaciones externas - OE), or ii) mobile health teams consisting of government health professionals designated by the MOH’s Directorate for Regional Health Services (DRS), referred to as institutional groups (grupos institucionales - GI). Table 2 below highlights the type (i.e. GI or OE), population coverage, and capitation payment of the UBAs in the regions in which they operate. Table 2. Population Coverage, Number, and Capitation Payment by Region, 2012 Population Covered by Capitation Payment Type of UBA Number of UBAs a Region (USD) GI OE Total GI OE Total GI OE Total Bocas del 14,333 14,333 5 5 44.58 44.58 Toro Chiriquí 37,457 37,457 7 7 40.88 40.88 Coclé 11,749 24,023 35,772 2 4 6 40.88 40.88 40.88 Colón 20,768 20,768 4 4 50.22 50.22 Darién 19,393 19,393 7 7 53.21 53.21 Herrera 9,219 9,219 2 2 40.88 40.88 Kuna Yala 14,719 14,719 3 3 53.21 53.21 Los Santos 8,236 8,236 2 2 40.88 40.88 Ngobe Bugle 18,561 166,995 185,556 3 22 25 44.58 44.58 44.58 Panamá 9,999 9,999 3 3 53.21 53.21 Este Panamá 9,983 28,291 38,274 2 4 6 40.88 40.88 40.88 Oeste Veraguas 10,845 31,890 42,735 2 7 9 40.88 40.88 40.88 Total b 127,037 309,424 436,461 31 48 79 47.01 43.35 44.79 a All of the UBAs deliver the EEC, but only some participate in the PSPV. b In the case of Capitation Payment (USD) columns, the average not the total is presented. Source: Perazzo, A, Consultancy report to World Bank, July 2012. Under the PSPV, capitation is used to pay the UBAs, all of which have different capitation rates, which can range between USD 41 and USD 53 annually. The calculation of these capitation payments is based on the costs associated with the delivery of services (including human resources and supplies/equipment) and the cost of performance-based payments for the indicators described below. The capitation is divided into three separate payments. One payment is made every two months, following the completion of an UBA’s health visits to rural areas called “rounds”, based on the percentage of targets achieved for five coverage indicators. This payment can be a maximum of 65 percent of the capitation rate (Tables 2 and 3). 12 Table 3. Five Coverage Indicators Indicator Definition Communities visited UBAs should visit at least 85 percent of the communities in a population group during each health round. Population groups UBAs should “protect” a minimum of 80 percent of the protected resident beneficiary population in the communities belonging to the population group that they are responsible for during each round. A population group is considered to be protected when at least one of its inhabitants has received the package of PSPV preventive and curative services via the UBA in its community or corresponding community center. Population groups UBAs should treat a minimum of 50 percent of the receiving appropriate protected population through the different promotion, preventive and curative preventive and curative services established in the services portfolio of incentivized services (CPP). Children (under 24 UBAs should treat a minimum of 80 percent of the months) receiving registered population younger than 24 months old, through appropriate preventive growth and development check-ups outlined in the CPP. services Days of services UBAs should provide the CPP with 38 days of direct medical services to the population, per health round. Source: Operative Regalement (PAISS + N / PSPV) -2012 Another payment, which is made every four months, is based on the achievement of ten service- provision indicators (Table 4). Each indicator is weighted equally, and over the course of a year, health facilities can be paid a maximum of 30 percent of the capitation rate. Taken together, the five coverage indicators and the ten service-performance indicators form a portfolio of 15 incentivized indicators (CPP). Notice that there are no indicators related to quality of care, however, compliance with protocols is part of the verification process and sanctions are made accordingly. Based on scientific evidence, the MOH of Panama, similar to its Latin-American pairs, has developed and approved protocols for medical care. The protocols aim to put evidence into practice and standardize it, establishing what, how, when and where preventive and curative health services must be provided. In the case of Panama, there has been an innovation: compliance with protocols is part of the agreement with UBAs and therefore is monitored and evaluated as part of the verification process. For instance, there is a MOH protocol for the management of patients with diabetic and hypertension that UBAs must follow. If protocols are not properly followed, in areas such as clinical records, the delivery of services, or even in the reporting of the attentions in the information system, penalties will be imposed to the UBAs. 13 Table 4. Ten Service-Provision Indicators No. Indicator 1 Percentage of pregnant women with at least three antenatal check-ups (one per trimester) by the end of the third trimester 2 Percentage of pregnant women registered out of estimated total 3 Percentage of pregnant women with second dose or booster of TT or TD 4 Percentage of births attended by trained staff 5 Percentage of women 20 years or older who have had pap smears 6 Percentage of children under one year who have had four or more growth-and- development check-ups 7 Percentage of children under one year with a complete vaccination record 8 Percentage of children aged four with at least two growth-and-development check– ups 9 Percentage of children aged one to four years old with a complete vaccination record 10 Percentage of symptomatic respiratory diseases recorded out of estimated total Source: Operative Regalement (PAISS + N / PSPV) -2012 The final third payment is made annually based on a social audit, which evaluates patient- satisfaction through a survey given to beneficiaries. This payment is five percent of the capitation rate. It should be noted that the UBAs receive a 20-percent advance in order to sufficiently cover operational costs. This amount is deducted from subsequent payments based on the verification of achieved indicators. 14 INTERNAL VERIFICATION IN THE PSPV General Facts about the Internal Verification Process An internal verification process is conducted by the MOH prior to the verification process performed by an external third party. This internal process is linked to payment. Two components make up the internal verification process: 1. Verifying the authenticity of PSPV’s beneficiary population information using the National Database of Civil Registration and Identity Cards. 2. Verifying the five coverage and ten service-provision indicators. There are four actors involved in this verification process, namely the UBAs, the DRS, the DPSS, and the UGSAF. Figure 3 (below) visually represents the role of these actors in the internal verification process, and it highlights the flow of information. Figure 3. Actors’ Roles in the Internal Verification Process and the Flow of Information Source: Operative Regalement (PAISS + N / PSPV) -2012 How is the PSPV’ Beneficiary Population Register Internally Verified? The objective of verifying PSPV’s beneficiary population register is to ensure that all eligible beneficiaries are listed and that the information required for enrollment (i.e. demographic data) is accurate and complete. The accuracy and completeness of this information affects the UGSAF’s payment to the UBAs. 15 The Process The UBAs are responsible for initially enrolling individuals in the PSPV. An enrollment form consisting of demographic data is completed for each beneficiary. A copy of this form is kept in the beneficiaries’ records, and another is sent to the DRS. The UBAs send monthly reports to the DRS in which they detail any additions (i.e. births or individuals previously not enrolled) or subtractions (i.e. death or change of residence) to the list of program beneficiaries. The UBAs also send the DRS an updated report after each health round; quarterly reports are also sent. The UBAs enter and update all beneficiary information in the EEC database. The DPSS consolidates the beneficiary lists and information from the UBAs’ monthly, post-health round, and quarterly reports, and then sends a report to the DPSS. The DPSS is responsible for beneficiary records at the national level and maintains the PSPV beneficiary population register. Once it has received the DRS’ report, the DPSS crosschecks beneficiary names and associated information in the EEC’s database against the Identification Verification System, which draws from the GOP’s National Database of Civil Registration and Identity Cards. Any missing beneficiary information or discrepancies are noted, and changes are made to the beneficiary population register accordingly. The DPSS notifies the UGSAF of these changes. Based on these changes, the UGSAF issues payment. How are the Coverage and Service-Provision Internally Verified? The objective of verifying the coverage and service-provision indicators is to ensure that program beneficiaries have received at least one incentivized service and that providers have met quarterly performance targets. The Process The UBAs register all the services they provide in the EEC’s database and beneficiary health- related information in the MOH’s health information statistics system. They also detail their fulfillment of coverage and service-provision indicators in their monthly, post-health round, and quarterly reports to the DRS. The OE UBAs have their own data entry personnel; the GI UBAs use the MOH teams already in the field. The DRS consolidates this information and sends it to the DPSS. The DPSS compares the UBAs’ coverage and service-provision targets with the information reported in the EEC’s database, and subsequently determines which indicators have been achieved. The UGSAF then receives a report on the indicators and issues payment to the UBAs. 16 MAJOR CHARACTERISTICS OF THE THIRD-PARTY VERIFICATION PROCESS General Facts about the Third-Party Verification Process Under the PSPV program, a private firm with experience in project evaluations is contracted as third-party (ATE) to carry out a verification process, which includes the following three components: 1. Verification of the beneficiary population registered in the National Database of Civil Registration and Identity Cards. 2. Verification of the coverage and service-provision indicators. 3. Carrying out a social audit. Two teams from the ATE jointly conduct the verification process. One team is technically oriented, and is typically composed of nine physicians. This technical team ensures that the reported services have been delivered following the appropriate protocols. The other team is administrative, and its members include three accountants and eleven data-entry clerks. This administrative team reviews the data associated with the beneficiary list. In total, the verification process requires an average of 326 UBA staff hours per region. This number is high due to the fact, as previously noted, that each region has multiple UBAs. Figure 4 (below) outlines how the ATE carries out its verification process. Figure 4 Third-Party Verification Process and the Role of the ATE Source: Operative Regalement (PAISS + N / PSPV) -2012 17 How Is the Beneficiary Population Register Verified by the Third Party Verifier (ATE)? The ATE’s verification of the beneficiary population register is carried out on a quarterly basis. The verification is completed in two stages, and each stage has multiple stages. Stage One In stage one, an ATE team reviews the beneficiary population register in its entirety (one-hundred percent of listed beneficiaries) to ensure that the information entered for each beneficiary is complete and accurate. This information consists of a range of demographic data. Any missing or inaccurate information is noted. The information for each beneficiary listed in the beneficiary population register is then checked against the Identification Verification System. If discrepancies are found between the beneficiary population register and the Identification Verification System, the information in the latter is used and entered in the beneficiary population register. Crosschecking the two databases allows the ATE team to identify beneficiaries not included in the beneficiary population register, deceased beneficiaries still included in the Identification Verification System, or cases where it is impossible to crosscheck the data. Stage Two Stage two occurs in the field and is carried out using records of a sample of beneficiaries. The sample, which draws from the beneficiaries listed in the verified beneficiary population register, represents ten percent of a population group linked to a UBA and is composed of a sub-sample of records from the following demographic categories:  Pregnant women  Children less than one year  Children between one and two years  Children between two and three years  Children between three and four years  Children between four and five years  Individuals with high blood pressure  Individuals with diabetes mellitus  Women over 20 years  General Population Once the sample has been determined, the ATE requests that the facility make the corresponding beneficiary records available. A team from the ATE then goes to the facility. The team first confirms that the requested beneficiary records are available, and notes the availability of these records in a monitoring matrix. The team then assesses if the records include beneficiary enrollment forms and if the beneficiary information is complete. If any gaps in information have been noted, the ATE team checks the beneficiary information on the enrollment forms against the beneficiary information listed in the beneficiary population register. The team then accepts or rejects the records based on a set of previously established administrative and reporting criteria. This process results in a verified list of beneficiaries and a report of rejected records. How Are the Coverage and Service-Provision Indicators Verified by the third party verifier (ATE)? There are also two stages associated with the verification of the coverage and service-provider indicators, and there are multiple steps in each stage. These stages are: 1. Verification of services provided and received during health rounds. 2. Verification of quarterly performance objectives and administrative processes. 18 Stage One In stage one, an ATE team first verifies beneficiary information listed in the EEC’s database by crosschecking it against the information listed in the Identification Verification System. This process ensures that the services recorded in beneficiary records were actually delivered and that the beneficiary has received at least one incentivized service. Each service is associated with a checklist of required data. Any missing or inconsistent data is noted, and results in rejection of the record. During stage one, a team from the ATE also reviews the UBAs’ reported number of community visits and the number of days of services they provided. This information is checked against the EEC’s database. Currently, this process is done manually using Excel, but the ATE is in the process of developing an automated system. Stage Two Stage two is carried out in the field and draws from the same sample of beneficiary records selected in stage two of the beneficiary population register verification process. These beneficiary records are crosschecked against the clinical records kept at the facility level. Beneficiary identities are matched to these clinical records, and the ATE team confirms whether or not the beneficiaries actually received the services reported in the clinical records. Compliance with treatment protocols is also verified. Records that were accepted during stage one may be rejected during stage two due to inconsistencies in information or an absence of documentation. Over the course of 120 rounds of verification, 1,168 inconsistencies or lapses in information were found. During stage two, the ATE team also reviews the facilities’ administrative processes. These processes include accuracy of documentation and compliance with the PSPV’s financial regulations, such as use of funds and incentives. The ATE reports the results of the verification to the UGSAF. The DPSS and the UBAs are also informed of the results. If there is disagreement with the results, the UBAs can file complaints, which are then reviewed by the UGSAF. How is the Social Audit Performed? The ATE carries out a social audit, which is designed to assess patient satisfaction. The social audit aims to inform the population about the services covered by the GOP; promote transparency and accountability in the management of public resources; improve how healthcare professionals handle the doctor-patient relationship; and get information about challenges for the proper delivery of health services. Social audits are carried out in each of the 12 regions and in all of the UBAs (either OE or GI). Social audits are performed three times a year: • Twice a year. Immediately after receiving medical attention in the field by the UBA, five people from the community are interviewed. • Once a year. When an UBA has provided health services to a community for a year, five people from the community comprising the protected group are interviewed about UBA’s performance. Social audits evaluate the complete package delivered comprising promotion, preventive and curative services. The questionnaire contains 10 simple questions covering waiting times, proper explanation of procedures and prescriptions, and bedside manners of health personnel (Annex 2). Finally, based on the interviews described above, a report is prepared giving the UBA a score from 1 to 5. As previously noted, the results of the social audit are directly linked to the final five percent 19 of the payment to the UBA. According to the score obtained by the UBA, the final 5 percent of the payment is made according to the table below: Table 5. Final Five Percent Payment Patient Satisfaction Which % of the final 5 percent payment is Score awarded? Percentage Meaning 4 to 5 points Good 100% The final 5% is fully paid. 3 to less than 4 Regular 60% 60% of the final 5 points percent is paid. 0 to less than 3 Bad 0% The final 5 percent is points not paid Source: MOH-UGSAF Progress Report for the Period July 1, 2014 to December 31, 2014 Implementation Challenges Throughout the life of Project implementation, the RBF processes followed were generally the same. All the procedures mentioned above heavily relied, at least at the beginning of the program, on paper reports and manual procedures. Later, a software solution was implemented that allows the use of CDs which helps to reduce the number of paper reports produced. However, the verification process still demands a significant amount of time from the human resources of all the organizations involved. Despite having an RBF approach, the verification process involves several actors performing repetitive tasks. For instance, reports prepared by UBAs are reviewed by the DRS, DPSS, UGSAF and finally the ATE and in many cases the information reviewed by these offices was similar such as: the name and existence of a beneficiary to prevent false reporting, the correct recording of each beneficiaries’ information and the health service delivered, and the results indicators. In addition, the verification cycle requires a high number of reports per UBA: one every two, six and twelve months. In total, approximately 720 reports had to be reviewed every year by the offices mentioned above. Key personnel from the MOH and UGSAF dedicated several hours to oversee this procedure. To complete the verification process for each report took at least three months if there was no discrepancy. However, in cases with discrepancies, the process could take up to a year or more. In these cases, significant delays in payments occurred which subsequently affected the provision of health services in the rural areas by the mobile teams. For instance in 2010, from June to August, services were not delivered. Finally, in order to strengthen the stewardship role of the MOH and support the decentralization process, the program decided to involve the sub-national levels of health, the Directorate for Regional Health Services (DRS), and established that funds were transferred from the central level MOH to the DRS and finally to the UBAs. Although it was a good initiative, implementation shows that it adds an additional layer of compliance with laws and control mechanisms (both the national and sub national levels have to comply with these mechanisms) increasing the number of administrative steps from 16 to 32, contributing to the delays in the verification process and payments. 20 FINDINGS OF THE VERIFICATION METHODS What Are the Results of the Verification? The RBF indicators monitored are, on the one hand, coverage levels and, on the other hand, service-provision of priority health services. Table 5 shows the results of UBAs for the coverage indicators. These are the most recent results available, they belong to the 2013-2014 period. Table 6. UBAs RBF Coverage Indicators (2013-2014 Period) Indicator Definition Result Communities visited UBAs should visit at least 85 percent of the communities 99% in a population group during each health round. Population groups UBAs should protect a minimum of 80 percent of the 99% protected resident beneficiary population in the communities belonging to the population group that they are responsible for during each round. Population groups UBAs should treat a minimum of 50 percent of the 62% receiving appropriate protected population through the different services preventive and curative (promotion, preventive or curative services) established services in the portfolio of incentivized services. Children (under 24 UBAs should treat a minimum of 80 percent of the 97% months) receiving registered population younger than 24 months old, appropriate preventive through growth and development check-ups outlined in services the PSPV. Days of services UBAs should provide the PSPV with 38 days of direct 100% medical services to the population, per health round. Source: MOH-UGSAF Progress Report for the Period July 1, 2014 to December 31, 2014 The first payment was based on coverage indicators and was made every two months (at the end of each health round) and could be a maximum of 65 percent of the per capita amount. Payment was based on achieving target coverage levels for the 5 indicators in Table 5 above. If these targets were not met, the percentage of the per capita amount that the UBA received was adjusted, it will be explained in the next section. Table 6 shows the results of UBAs for the service-provision indicators. These are the most recent results available, they belong to the 2013-2014 period. In this case, it has been possible to separate them by the results achieved by OEs and GIs. As the table shows, the OEs had a better performance in achieving the service-provision indicators compared to the GIs Table 7. UBAs RBF Service-provision Indicators (2013-2014 period). Type of UBA No. Indicator OEs GIs 1 Percentage of pregnant women with at least three 78% 17% antenatal check-ups (one per trimester) by the end of the third trimester 2 Percentage of pregnant women registered out of 80% 22% estimated total 3 Percentage of pregnant women with second dose or 89% 21% booster of TT or TD 4 Percentage of births attended by trained staff 88% 23% 21 Type of UBA No. Indicator OEs GIs 5 Percentage of women 20 years or older who have had 67% 11% pap smears 6 Percentage of children under one year who have had four 92% 16% or more growth-and-development check-ups 7 Percentage of children under one year with a complete 94% 20% vaccination record 8 Percentage of children aged four with at least two growth- 87% 13% and-development check–ups 9 Percentage of children aged one to four years old with a 89% 15% complete vaccination record 10 Percentage of symptomatic respiratory diseases 13% 3% recorded out of estimated total Source: MOH-UGSAF Progress Report for the Period July 1, 2014 to December 31, 2014 The second tranche payment, paid every 4 months, was based on service-provision levels achieved and could be for a maximum of 30 percent of the per capita amount. The results from the previous period were supposed to be used as the basis for setting the targets for the next period; and the MOH, UGSAF and the DRS would meet every four months to establish the threshold levels by indicator and by population group. However, the complexity of the process generated delays in the flow of information that made this planning exercise difficult. Table 7 shows the average percentage of service-provision indicators by region accepted by DPSS and rejected by ATE for the entire territory covered by the EEC. In this case, the information presented is for the 2011-2012 period. Table 8. Average percent of indicators accepted by DPSS and rejected by ATE. 2011 (%) 2012 (%) Region Q1 Q2 Q3 Q1 Q2 Q3 Bocas del 0.80 -1.00 0.40 0.20 -0.20 - Toro Chiriquí 1.86 5.00 5.29 0.29 0.43 1.00 Coclé 2.33 0,00 5.50 1.00 1.33 1.33 Colón 4.00 4.25 6.75 -0.25 0.75 0.75 Darien 1.14 2.71 2.14 -0.29 0.14 - Herrera - 2.00 4.00 - - 1.50 Kuna Yala 1.00 2.00 2.00 0.67 - 1.67 Los Santos - 1.00 2.00 - 1.00 1.00 Ngobe Buglé 3.88 6.44 6.84 0.76 0.92 1.44 Panamá Este 0.67 2.33 2.33 -0.33 0.67 - Panamá 1.50 6.00 6.00 0.50 1.50 1.17 Oeste Veraguas 6.22 2.33 2.33 0.22 0.56 1.00 Total general 2.81 4.22 4.65 0.39 0.70 1.01 Source: Developed by authors based on UGSAF´s data (electronic files “Convenios”) In collectively assessing these results, they indicate that there are differences in the indicators achieved by region and type of UBA delivering the package of health services (OE or GI). The main differences are seen in Bocas del Toro, Darien, and Panama Este. These regions may struggle to 22 achieve the indicators due to their remote locations or difficult climates. These challenges often prevent the OE UBAs from making health rounds, or individuals from accessing the GI UBAs. In general, OE UBAs tend to perform better because they are structured as firm contracts, and have access to additional funding they can use to make health rounds happen. GI UBAs, conversely, lack access to this additional funding. What Are the Findings Used for? The results of the verification process are used for three purposes: 1. Adjustments to payments. 2. Determination of penalties. 3. Establishing targets for indicators. Adjustments to Payments The results from the verification process are used to determine payment adjustments. The ATE makes its recommendations to the UGSAF, and the UGSAF then adjusts payments accordingly. In 2011, for example, the UGSAF’s adjustments amounted to USD 1.1 million compared to the ATE recommendation of USD 560,709, approximately 2.8 percent of the total amount of capitation. The UGSAF’s adjustments can be greater than what the ATE recommends because the total of the UGSAF’s adjustments include penalties, which are described in the sub-section below. Table 8 shows how these adjustments are calculated for the coverage indicators. As the table illustrates, payment is related to the degree of achievement of the indicators. The rule used to determine the payment was choosing the lowest percentage achieved for the five indicators. For the service-provision indicators, no payment is made if an indicator is not achieved. 23 Table 9. Adjustments for the Coverage Indicators. Coverage Indicators Payment Population (%) groups Children (under Protected receiving 24 months) Communities Population appropriate receiving Days of Services Provided (%) Groups preventive and appropriate (%) curative preventive services services (%) (%) 85-100 80-100 50-100 80-100 Equal or greater than 38 days 100 76-84 72-79 45-49 72-79 Less than 38 days, equal to 90 34 days 67-75 64-71 40-44 64-71 Less than 34 days, equal to 80 30 days 58-66 56-63 35-39 56-63 Less than 30 days, equal to 70 26 days 49-57 48-55 30-34 48-55 Less than 26 days, equal to 60 22 days 40-48 40-47 25-29 40-47 Less than 22 days, equal to 50 18 days 31-39 32-39 20-24 32-39 Less than 18 days, equal to 40 14 days 22-30 24-31 15-19 24-31 Less than 14 days, equal to 30 10 days 13-21 16-23 10-14 16-23 Less than 10 days, equal to 6 20 days 4-12 8-15 5-9 8-15 Less than 6 days, equal to 2 10 days Source: Operative Regalement (PAISS + N / PSPV) -2012 Determination of Penalties ATE’s verification results are used to penalize UBAs for non-compliance with protocols, delays in service provision, and other issues. Penalties are expressed in terms of days that are given a monetary value based on the capitation payment. Generally, one penalty is associated with one day, which results in a deduction of payment between USD 10 and USD 25, depending on the size of the UBA’s catchment population. In 2011, there were 18,667 days of penalties at the rate of an average value of USD 8.64 per day. These penalties and the reasons behind them are shown in Table 9 below. 24 Table 10. Penalties and Adjustments for 2011. Penalty Category Days Amount % (USD) Services supplied to certified beneficiaries 618 4,902.8 3.0 Problems associated with service 88 904.3 0.6 delivery Formal problems in the clinical 530 3,998.5 2.5 records Package of services 607 5,083.2 3.2 Problems of delivery of services 545 4,390.1 2.7 Formal problems in the clinical 62 693.1 0.4 records Protocols 7344 62,916.1 39.0 Problems in the quality of services 153 1,751.9 1.1 Problems of delivery of services 6987 58,888.1 36.5 Formal problems in the clinical 204 2,276.0 1.4 records Primary Records vs Data Base 4822 39,461.5 24.5 Problems in the quality of services 48 448.0 0.3 Problems of delivery of services 4239 34,829.4 21.6 Formal problems in the clinical 532 4,166.2 2.6 records None specified 3 17.9 0.0 Forms 5227 48,582.7 30.1 Problems in the quality of services 23 135.1 0.1 Problems of delivery of services 5 39.3 0.0 Formal problems in the clinical 5198 48,398.7 30 records Non specified 1 9.6 0.0 Accountability 49 341.0 0.2 Problems of Reporting 49 341.0 0.2 Total 18667 161,287.1 100.0 Source: Authors calculations based on data from USGAP. Establishing Targets for Indicators The results from the verification process are also used to set the coverage and service-provision targets. The UGSAF, DPSS and the DRS meet every four months to establish the targets for each UBA. However, as described above, the verifications process involves several actors and is manually intense, which generates delays in the flow of information that made the planning exercise very difficult. 25 VERIFICATION COSTS The firm Gesaworld SA was hired to conduct the ATE. The firm’s initial contract was for two years (from 2010 to 2012), and it has been extended for another two. Excluding taxes, the two-year contract amounted to USD 1,595,000; including taxes the contract was for USD 1,834,250. Approximated, the cost of verifying one UBA per month (based on eight DRSs and five UBAs, i.e. 13 entities and 24 months) equates to USD 5,879. Gesaworld is responsible for verifying the results of the PSPV and the conditional-cash transfer program RDO, which they do in tandem. Therefore, the estimated cost of contracting the firm for five years to carry out verification for both programs would be approximately USD 4.59 million (based on 2.5 times USD 1.83 million). This figure translates to almost eight percent of the total cost of the PSPV program and 12.8 percent of IBRD loans. Table 11. Costs Associated with the ATE. Program Cost ATE PSPV Total Cost IBRD Loan (in millions of USD) (in millions of USD) (in millions of USD) PSPV 36.8 25.8 RDO 20.7 10.0 Total 4.59 57.5 35.8 ATE Cost as percentage 7.98 % 12.81% Source: Developed by authors based on Gesawodld Reports. 26 LESSONS LEARNED What Has Been Learned? This case study has presented an overview of the verification process used in Panama’s PSPV program, and from this overview, several thematic lessons emerge. They are discussed below. The Use of IT Systems in the Verification Process The verification process uses multiple IT systems, including the EEC database, the MOH’s health information statistics system, and the Identification Verification System. These systems, however, are not well integrated, and a manual review is subsequently required in some stages of the verification process. Indeed, during stage one of the verification of the coverage and service- provision indicators, the ATE must manually check the information in the EEC database against the Identification Verification System. Such manual reviews are inefficient, carry high transaction costs, and can result in higher error rates of reporting. Integrating the IT systems, specifically the Identification Verification System and the EEC database, would thus improve the efficiency of the verification process. Synchronizing the Verification Process among Different UBAs The different UBAs do not coordinate the dates of their health rounds and the subsequent submission of reports. This lack of coordination results in a disjointed, internal monitoring process, and ultimately, in an unsynchronized verification process. The unsynchronized process leads to greater complexity and increased inefficiency, like delays in payment to the UBAs. Such payment delays can undermine the RBF incentive system. The ATE’s Relationship with the PVSP Stakeholders at the MOH The ATE’s role in the PSVP and its relationship with different stakeholders at the MOH (i.e. DPSS and UGSAF) have evolved over the years. The UGSAF’s development of guides and semi- computerized tools—a process that presented a learning curve for all program stakeholders— altered the ATE’s role and its set of responsibilities. Its role and responsibilities are now highly specific and sensitive to changes in norms, procedures, and staffing of the program. Due to this, the ATE could be better integrated into the overall program to strengthen its relationship with the internal verifier, the DPSS. The role of the ATE needs to be reconsider; a better integration among ATE and MOH stakeholder’s roles, namely the DPSS, could result in gains in efficiency and program administration. The Sample Size of Beneficiary Records and Indicators The sample of beneficiary records constructed during the verification process represents the total population within the UBA catchment area in ten population sub-categories (i.e. children between three and four years). The sample does not represent each indicator, and therefore, may be too large to accurately capture indicator achievement. For example, in six of the ten indicators, a sample of 100 percent of the population sub-category (i.e. pregnant women) is required because it is small relative to the total population. This requirement, however, may distort the actual achievement of the indicators. The constructed sample and its size have cost implications: overpayment could occur for indicators that were not actually achieved. A sample that is representative of the indicators could be more cost-effective. The Verification of Protocol Compliance The ATE’s detailed verification of protocol compliance was not initially included in the process, but was later incorporated after associated data collection and analytical tools were developed. This 27 aspect of the verification process is particularly interesting because its results have value beyond affecting payment. Indeed, verifying protocol compliance, and more specifically, the degree of protocol compliance, has quality-related implications for the implementation of the EEC, which gives the verification an added health value. The Need for Simplification of Payment and Verification Processes The simplification of payment and verification processes is needed to ensure a focus on the actual results achieved and not the processes. Currently, the complex payment and verification processes require multiple steps by multiple actors, and requires the preparation of approximately 72 reports per quarter. Such complexity causes the program to lose sight of the end goal (i.e. the achievement of results) and instead focuses on process compliance. Additionally, the complexity of these processes has caused significant payment delays. These delays can ultimately negate the objective of the incentive payment—the reason that the payment is being received is forgotten due to the delay. Simplifying these processes will be particularly important as the MOH develops a new global strategy that builds on previous experiences (that is, World Bank-financed PMES Inter- American Development Bank-financed Fortalecimiento) to use simplified and coordinated payment and verification systems. These lessons demonstrate there are several opportunities for improving the verification process, which would allow for program focus to rest on tasks that are more cost-effective and have greater added value. In sum, there are six key lessons: 1. The information technology (IT) systems used during the verification processes should be better coordinated. 2. The verification process of the different UBAs should be better synchronized. 3. The relationship between the 3rd party verifier (ATE) and MOH stakeholders should be better integrated. 4. The sample size used in the third-party verification process should better reflect the indicators. 5. The verification of protocol compliance has value beyond simply affecting payment. 6. The simplification of payment and verification processes is needed to ensure a focus on the results achieved and not the processes. 28 CONCLUSION The complexity of the verification process in Panama’s PSPV makes it an especially interesting and informative case, and one that provides a number of learning opportunities. As the MOH develops a countrywide RBF strategy, it can draw on the lessons learned from the PSPV verification process. RBF practitioners, technical experts, and policy makers in other countries can learn from the case study and apply this learning while designing or implementing their respective RBF programs. 29 REFERENCES COSO (Committee of Sponsoring Organizations of the Treadway Commission). 2013. (www.coso.org). Gesaworld, PRODUCTO 2: Propuesta Metodológica Versión Ajustada, Panamá, 30 de Enero. 2013 Gesaworld. 2012. Guía de Auditoría Técnica, Panama. Gesaworld, Technical Audit Guidelines, 2012. www.gesaworld.com Gesaworld. 2012. Guía de Auditoría Administrativa Financiera, Panama. Gesaworld, Administrative and Finance Audit Guidelines. 2012. www.gesaworld.com Gesaworld. 2014. Informes Bimensuales (varias ediciones). Gesaworld, Bi-monthly Reports (several editions). www.gesaworld.com _____. 2014. Gesaworld, Quarterly Reports (several editions). www.gesaworld.com Gesaworld. 2012. Primer Informe de Auditoría Técnica Externa para la Certificación Del Padrón de Población Beneficiaria de la Estrategia de Extensión de Cobertura Periodo 2011 – 2012 – Versión Ajustada, Panamá, Noviembre 2012. Health in the Americas. 2012: Panama Country Chapter. Pan American Health Organization. Human Development Report. UNDP. 2013. http://hdr.undp.org/ Indicadores Basicos de Salud, Panama. 2009-2010. Ministerio de Salud de Panama y la Organizacion Panamericana de la salud. OLACEFS. 2008. Auditoría de los Programas Sociales, El Salvador, Julio de 2008 Perazzo, Alfredo. 2012. Proyectos: “Mejora de la Equidad y Desempeño en Salud” y “Protección Social – Apoyo a las Redes de Oportunidades” -Revisión y Análisis del Sistema de Metas, Banco Mundial, Mimeo, junio. UGSAF. 2012 Caja de Herramientas para el Agente de Verificación, Panamá. UGSAF. 2014 Informes de Evaluación. Financial and Administrative Health Management Unit, Ministry of Health, Evaluation Report. UGSAF. 2010. Provisión del PAISS+N y PSPV a través de giras institucionales - Reglamento operativo. Panamá, Octubre. UGSAF. 2010. Provisión del PAISS+N y PSPV a través de Organiaciones Extrainstitucionales - Reglamento operativo, Panamá, Octubre. World Bank. 2013. Open Data. http://data.worldbank.org/country/panama World Bank. 2008. Project Appraisal Document. Health Equity and Performance Improvement Project (2008) and Social Protection: Opportunities Network (2007). 30 ANNEXES Annex 1. Package of prioritized basic health services. Preventive Health Services Prenatal control Post-partum control Provision of Folic Acid and Iron for women in fertile age Development and growth monitoring for children under 5 years old Family Planning Pap smear and breast cancer preventive examination Health Care Services Acute Respiratory Infections (ARI) treatment Acute Infectious Diarrhea Screening, diagnosis and treatment of malnutrition for children under five Screening, diagnosis and treatment of malnutrition for women during pregnancy and post- partum Provision of supplementary food for women with low weight during pregnancy and post-partum Screening and treatment of micronutrient deficiency in pregnant women and children under 5 years old Triage and referral to appropriate health care unit for institutional delivery Screening and treatment of tuberculosis Screening, diagnosis and treatment of malaria Screening, diagnosis and treatment of leishmaniasis Screening, diagnosis and treatment of hypertension Ambulatory treatment and follow up of other adult chronic conditions and child morbidity (primary health care) Education and Health Promotion Preventive nutritional education Education and training on evaluation of water sources, treatment and quality control Education and training on sanitation activities in rural areas Education and training for solid waste management and disposal in rural areas Health promotion 31 Annex 2. Social Audit Questionnaire. 32 33 34 Verification differentiates results-based financing (RBF) from other health-financing mechanisms, and it is considered an important process of RBF program design and implementation. Despite the vital role it plays in RBF, not much has been written about verification as a process, and information about different elements of the process—frequency, cost, and direct and indirect effects among others—is scarce. Panama’s Health Protection for Vulnerable Populations Program (PSPV) uses an RBF mechanism to deliver health services to the country’s rural poor. As in many RBF schemes, a major component of the PSPV is the verification of results. This study focuses on PSPV’s verification process, highlights its results and their application, and identifies lessons learned. Such information is useful to policy makers and technical experts interested in or designing RBF mechanisms. ABOUT THIS SERIES: This series is produced by the Health, Nutrition, and Population Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author/s whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Martin Lutalo (mlutalo@ worldbank.org) or HNP Advisory Service (healthpop@worldbank.org, tel 202 473-2256). For more information, see also www.worldbank.org/hnppublications. 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org