Document of The World Bank Report No:ICR0000365 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-70620 JPN-26319) ON A LOAN IN THE AMOUNT OF US$30.0 MILLION TO THE BOLIVARIAN REPUBLIC OF VENEZUELA FOR A CARACAS METROPOLITAN HEALTH SERVICES PROJECT October 1, 2007 Human Development Sector Management Unit Bolivia, Ecuador, Perú, República Bolivariana de Venezuela Country Department Latin America and the Caribbean Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not be otherwise disclosed without World Bank authorization. 1 CURRENCY EQUIVALENTS ( Exchange Rate Effective ) Currency Unit = Bolivar 1.00 = US$ 0.000465 US$ 1.00 = 2,150 Bs. Fiscal Year January 1 ­ December 31 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immunodeficiency Syndrome AM Mayor of the Metropolitan District of Caracas (Alcalde Mayor) BA The Barrio Adentro Program CAS Country Assistance Strategy DO Development Objectives FMT Financial Management Team GDP Gross Domestic Product GOV Government of Venezuela HIV Human Immunodeficiency Virus ICB International Competitive Bidding IDB/IADB Inter-American Development Bank IDA International Development Association IEC Information, Education, and Communication IVSS Venezuelan Institute of Social Security (Instituto Venezolano de Seguridad Social) MDC Metropolitan District of Caracas MOF Ministry of Finance MSDS Ministry of Health and Social Development (Ministerio de Salud y Desarrollo Social) NGO Non-Governmental Organization PAHO Pan-American Health Organization PCU Project Coordination Unit PDVSA Petróleos de Venezuela S.A. (the state-owned oil company) QSA Quality of Supervision Assessment STI Sexually Transmitted Infections Vice President: Pamela Cox Country Director: Marcelo Giugale Sector Manager: Keith E. Hansen Project Team Leader: Carlos Marcelo Bortman 2 Venezuela VE- CARACAS METROPOLITAN HEALTH CONTENTS 1. Basic Information 2. Key Dates 3. Ratings Summary 4. Sector and Theme Codes 5. Bank Staff 6. Project Context, Development Objectives and Design 7. Key Factors Affecting Implementation and Outcomes 8. Assessment of Outcomes 9. Assessment of Risk to Development Outcome 10. Assessment of Bank and Borrower Performance 11. Lessons Learned 12. Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Results Framework Analysis Annex 2. Restructuring Annex 3. Project Costs and Financing Annex 4. Outputs by Component Annex 5. Economic and Financial Analysis Annex 6. Bank Lending and Implementation Support/Supervision Processes Annex 7. Detailed Ratings of Bank and Borrower Performance Annex 8. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 9. List of Supporting Documents MAP 3 1. Basic Information VE- CARACAS Country: Venezuela Project Name: METROPOLITAN HEALTH Project ID: P050495 L/C/TF Number(s): IBRD-70620,JPN- 26319 ICR Date: 01/03/2007 ICR Type: Core ICR Lending Instrument: SIL Borrower: Original Total Commitment: USD 30.3M Disbursed Amount: USD 4.4M Environmental Category: B Implementing Agencies Unidad Coordinadora Proyecto Salud Cofinanciers and Other External Partners 2. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 11/21/2000 Effectiveness: 03/28/2002 03/28/2002 Appraisal: 05/08/2001 Restructuring(s): Approval: 06/26/2001 Mid-term Review: 09/12/2005 10/17/2005 Closing: 12/31/2006 12/31/2006 3. Ratings Summary 3.1 Performance Rating by ICR Outcomes: MU Risk to Development Outcome: S Bank Performance: MS Borrower Performance: MU 3.2 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating: Potential Problem Project at any time (Yes/No): No Quality at Entry (QEA): None Problem Project at any time (Yes/No): Yes Quality of Supervision (QSA): MU DO rating before Closing/Inactive status: Unsatisfactory 4 4. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 4 4 Law and justice 1 1 Health 95 95 Original Priority Actual Priority Theme Code (Primary/Secondary) Decentralization Primary Secondary Law reform Secondary Secondary Health system performance Primary Primary Access to urban services and housing Primary Primary HIV/AIDS Secondary Secondary 5. Bank Staff Positions At ICR At Approval Vice President: Pamela Cox David de Ferranti Country Director: Marcelo Giugale Olivier Lafourcade Sector Manager: Keith E. Hansen Charles Griffin Project Team Leader: Carlos Marcelo Bortman Patricio Márquez ICR Team Leader: Carlos Marcelo Bortman ICR Primary Author: Jorge Barrientos 5 6. Project Context, Development Objectives and Design 6.1 Context at Appraisal Economic Context Over the last decades, the performance of the Venezuelan economy has been rather poor, largely due to unproductive public sector investments, poor macroeconomic management and the inability of economic policymakers to cope with oil cycles. In 2001, when the Project was being appraised, despite a then relatively high oil price (above US$20 per barrel on average), the consolidated non-financial public sector registered a deficit of about 6.4 percent of GDP, slightly above the one in 1998 when the average oil price was about US$10 per barrel. Shortly after taking office in 1999, the Chávez administration embarked on an ambitious agenda of reforms in Venezuela. These new policies, which marked a departure from previous governments, incorporated increased state intervention in many aspects of the economy, and also opened the door to greater citizen participation in governance. Aspects of these new policies-- which were supported by a sizeable portion of the population--included an array of price and exchange rate controls, direct government participation in the production and distribution of goods and services, more governmental authority over public enterprises (in particular the state- owned oil company, PDVSA), and a number of new initiatives to improve social outcomes for Venezuela's poor. The social programs implemented by the Chávez administration have had a clear focus on the poor. Although outcome indicators are not available for many programs, there can be little question that in areas such as education, health care, and urban living environment, Venezuela's underprivileged have seen a dramatic increase in government support. However, the overall economic results are less clear; Venezuela experienced a sharp economic downturn in 2003, due to political strife and a strike in PDVSA. National income rebounded strongly starting in 2004, driven in large measure by very high international oil prices and the recovery of oil production. Growth in 2005 and 2006 remained strong, with a major share of the recovery relying on the non- oil sector supported by the consumption boom resulting from higher government expenditure. Despite this significant recovery, overall unemployment and informality remain at high levels, and the relatively high budget deficits, despite historically high oil prices, fixed exchange rate and high banking sector exposure to government debt represent a latent danger to macroeconomic stability should oil prices fall. Health Sector Context Indicators of the health status of Venezuelans suggest that it continues to lag behind that of the populations of countries with similar per capita income. This situation is due in part to Venezuela's relatively low levels and unusually high fluctuations in public expenditures on health, and to numerous inefficiencies and inequities in the health system. By the time the Project was appraised, the health system was suffering from significant gaps in the area of public health interventions, had an unusual level of fragmentation and duplication of medical care services, was undergoing an incomplete and imperfect decentralization process, often poorly managed, and it had been unsuccessful for reaching the poor. Most of these issues continued to affect the health sector during Project implementation. It was only during the last three years, that the Government of Venezuela (GOV) made an attempt to increase coverage, reaching the poor, by launching the "Barrio Adentro" program (BA). However, the BA program started with little coordination with other primary health services and without a well defined referral system. 6 A significant proportion of Venezuela's health resources are concentrated in the capital city of Caracas, yet the Metropolitan District of Caracas (MDC) suffers, perhaps more than other regions, many problems of inefficiency and inequity. Health care in the MDC is generally provider oriented, curative rather than preventive in nature, impersonal, fragmented, and of low quality. The 17 hospitals in the MDC work without coordination with approximately 150 health centers scattered in the marginal urban areas. The ambulatory centers are small facilities, most of which are in poor physical condition and have insufficient and outdated medical and support equipment. The majority of the ambulatory centers provide general medical consultations and pediatric and basic trauma care; some offer gynecology and obstetrics services, and some dental care services. Very few have a small laboratory for basic diagnostic tests, with limited personnel and medical supplies. Physicians and nurses have little knowledge of community health problems, including risk factors associated with common illnesses and with emerging public health problems. Late in 2003, with cooperation from the Cuban government, the Chávez administration launched the BA program aimed at providing basic health care in low income areas in the main cities. Facilities were established with Cuban professionals and supplies. Little information is available on the outcome of this program, but there is some degree of consensus that it has expanded the supply of basic services, particularly to the poorest strata of the population in urban areas, including Caracas. Lately, efforts are being made to improve coordination with other service providers. Political context After a failed attempt by the Pérez administration (1989-93) to redefine the private and public sector roles and to reorient the economy toward market-led growth stalled in 1992, an interim administration followed without well defined objectives and lacked coherent economic programs. In 1994, President Caldera took office immediately after a deep systemic financial crisis. Partly in response to this crisis, unorthodox economic policies were adopted, including a wide range of price and exchange controls. These measures, together with the financial crisis, brought the economy to the brink of collapse by the end of 1995. By that time, the experiment was reversed with more market-oriented policies, including abandoning price and exchange controls and opening the oil sector to private investment. This approach brought rampant inflation and currency decline. The program became unsustainable when the bloated public sector resisted further reforms and the government reversed previous wage policies. It was against this background, that Venezuelans expressed their dissatisfaction with the failure of past governments by electing Hugo Chávez in December 1998 with the largest margin in 40 years. The sweeping electoral victory gave President Chávez a mandate to change radically from past policies and to focus his administration on political, social, institutional and economic reforms that had not been previously considered. This fundamental break with the policies of past governments led to a situation of great uncertainty and political polarization. Mr. Chávez used the initial stages of his administration to make deep institutional changes. These have included adopting a new Constitution and, within this new framework, changing more than one hundred laws. By mid-2000, a new Constitution had been put in place and the long process to define the new legal framework had started, including the re-election of all officials and Mr. Chávez for a new, six-year, term. At that point, in response to social pressures and high levels of unemployment, the President increased the focus of his administration on socioeconomic reforms. The social and economic objectives of the Chávez government largely focused on the reduction of poverty and inequality, as well as the provision of education and health services for the poor. 7 During the first years of the administration, the authorities promoted a state-led growth model with strong public sector intervention in the provision of goods and services, including the development of government-sponsored communities that called for the relocation of the poor from urban slums. Opposition movements started to emerge in 2001, including many of the government's former supporters, business leaders, labor unions, and some members of the Catholic Church. Opposition to different aspects of this model, aggravated by strikes in Petróleos de Venezuela S.A. (PDVSA), generated a conflict that escalated into an attempt to take over the government on April 2002. Civil unrest continued during the next thirty months leading to a referendum to revoke President Chávez term, that took place in August 2004. The favorable outcome strengthened the power base for the government, which consolidated its approach for a larger role for the public sector on a wide range of social and economic aspects. In late 2006, Mr. Chávez was eventually reelected, with over 60% of the vote, for another six year period. Rationale for Bank assistance The Project was consistent with the prevailing interim Country Assistance Strategy (CAS), which emphasized improving access to and quality of health care services for the poor. It was also focused on enhancing sustainable growth with stability; promoting social development; and modernizing the public sector. The Caracas Metropolitan Health Services Project (the Project) was designed to support all three of these goals, to varying degrees. It was seeking to promote economic growth and social development in the long term by improving health status of the poor via the provision of more accessible, equitable, and good quality health services. Additionally, it was designed to contribute to the modernization of the public sector by supporting the decentralization process of the MDC health system. Project design was also consistent with a poverty report on Venezuela, prepared by the Bank in March 2001, Investing in Human Capital for Growth, Prosperity, and Poverty Reduction. This report called for several measures to address two key problems: (i) great inequity in access to health services and the distribution of resources, and (ii) inefficiency and a lack of coordination among institutions providing health services. The poverty report acknowledged that Bank involvement could become a conduit for exchange of international experience and exposure to best practices, and would continue to provide the country counterpart team with access to relevant international experience in health sector development. Project design fit well into the objectives highlighted by the poverty report and it sought to make use of the Bank experience with decentralizing health systems. 6.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The Project's development objectives were: · to expand access to and improve the quality and efficiency of ambulatory care services, including the outpatient services of hospitals, for poor people living in selected parishes and municipalities of the MDC; · to improve the management of the health care system and support the decentralization process in the health system of the MDC; and, · to develop HIV/AIDS prevention and control strategies and activities in the MDC. The Project was also expected to contribute in the medium-term to the improvement of the health status of the most vulnerable population in the MDC. The Project followed up on some activities of a previously closed Project, the Health Services Reform Project (3823-VE) and the Endemic 8 Diseases Control Project (3538-VE). Key indicators for the Project included (i) the coverage of better quality ambulatory services for the poor population of the MDC, quality and efficiency of health services provided, management and civil society participation; (ii) strategies and activities for HIV/AIDS prevention implemented; and (iii) legal and organizational instruments and models implemented. 6.3 Revised PDO and Key Indicators (as approved by original approving authority), and reasons/justification PDO and key indicators were not revised 6.4 Main Beneficiaries, original and revised The universe of potential beneficiaries was estimated at 2.4 million persons, representing 70% of the total population of the MDC and 11% of the population of Venezuela. Most beneficiaries were expected to come from low income groups living in Caracas. Beneficiaries would receive health care services capable of resolving most clinical and surgical problems either at ambulatory facilities, or by referral, at hospital facilities. It was designed to develop an integrated system of community health, preventive, family medical and surgical care, and including endemic and environmental control through community participation and inter-institutional coordination. 6.5 Original Components (as approved) Component 1: Strengthening Health Care Networks (Total Cost US$ 44.07 million). Under this component, the Project supported: (i) the delivery of a comprehensive health care program targeted at the vulnerable population of the MDC by supporting the development of a new health care model, emphasizing the organization in networks of existing ambulatory centers, pre-hospital emergency medical care services (ambulance system, communication system, and paramedic teams), and outpatient departments of hospitals, in each of the selected districts and municipalities of MDC; and (ii) the improvement of the diagnostic, treatment, and medical waste management capacity of network facilities to provide appropriate and timely quality health care on an ambulatory and inpatient basis. Under this component subprojects would be financed for the rehabilitation and/or construction of physical infrastructure, and providing equipment, communication systems and ambulances for patient referral and emergency care. It would also support the training of health personnel, education and communication activities, and provide technical assistance for the improvement of supply systems, and the development of maintenance systems for physical infrastructure and equipment. Component 2: Institutional Development (Total Cost US$ 6.84 million). To ensure the virtual integration of health facilities into networks and efficient and effective delivery of a comprehensive health care program, the Project supported under this component the institutional modernization of health networks. It was seeking to: (i) implement new resource allocation mechanisms, developed under a previous Project, to promote allocative and technical efficiency; (ii) develop accounting, budgeting, contracting, billing and collection systems, including cost recovery from financial intermediaries; (iii) develop managerial protocols for strategic planning, human resources development and management, supplies and installations; 9 (iv) develop clinical protocols to support performance improvement, quality assurance; the revision and updating of the existing norms and guidelines for the handling and disposal of medical waste in health care facilities; and measurement of outcomes and user satisfaction, and management and continued education programs; and (v) reengineer processes, including the development of minimum accreditation standards, procedures and services in ambulatory and type II hospital networks. The Project financed institutional development subprojects that provided technical assistance, training and systems development to modernize health care organization and management structures and practices, including management information, health care financing and quality assurance systems, and human resources development. Component 3: Development of Strategies and Activities for HIV/AIDS Prevention and Control in the MDC (Total Cost US$ 2.74 million). This component supported the formulation and implementation of prevention and control strategies and activities for HIV/AIDS and Sexually Transmitted Infections (STI). It included: (i) information, education, and communication campaigns to raise awareness and change behavior to reduce HIV/AIDS and STI transmission, particularly among high-risk groups; (ii) development of a second generation HIV surveillance system in the MSDS; (iii) development of an information system for the control and follow up of HIV/AIDS patients; (iv) training of health personnel; and (v) supply of equipment for two HIV/AIDS diagnostic centers. Component 4: Development of Legal and Organizational Instruments and Models in the MDC (Total Cost US$0.80). The Project would finance technical assistance and activities to support health sector reform efforts at the MDC. The basic aim was to cover issues that might affect the health services network modernization objectives in the MDC, and those that could facilitate the definition of new institutional roles. More specifically, it attempted to: (i) develop a legal framework to support greater autonomy of district and municipal health directorates and service providers; (ii) develop new governance models such as foundations, allowing local authorities and communities to share greater responsibility for the management of the ambulatory care and hospital networks; (iii) adjust existing norms and regulations governing health organization and managerial arrangements; (iv) develop and process the approval of legal instruments for establishing decentralized and unitary health care organizational arrangements; (v) design new policies and mechanisms for managing human resources; (vi) carry out assessments and develop proposals for reforming the supply of drugs and other medical supplies; and (vii) design instruments to help make operational new health reform legislation, including the social security reform law. Component 5: Project Administration and Evaluation (Total Cost US$ 5.54 million) This component financed project management, monitoring, supervision and impact evaluation at the different levels of the system for the Project. 6.6 Revised Components Project components were not revised during implementation. 6.7 Other significant changes Due to extremely slow progress in implementation during the first three years, the government requested a cancellation of US$11.86 million (39% of loan amount) in early 2005. Another US$12.3 million were cancelled in early 2006 in response to another government request. Both cancellations, de facto reduced the Project's scope by about eighty percent affecting 10 achievements in all Project components, with the exception of the HIV/AIDS component. 7. Key Factors Affecting Implementation and Outcomes 7.1 Project Preparation, Design and Quality at Entry The Project can be considered adequately designed with clear and well defined overall objectives. It was based on the positive experience gained during execution of a previous Project ­ the Health Services Reform Project ­ which was focused on the implementation of integrated health care models in four states. The positive experience in the Aragua state was particularly relevant, since it had been led by the newly appointed Minister of Health (Dr. Rodriguez Ochoa), who enthusiastically sponsored the Project. However, there was little specific design work done during preparation which led to the Project having a flexible structure, where many aspects were left to be decided over the course of implementation. The uncertain political climate was indeed a key consideration when opting for this approach. However, this design feature eventually backfired during early stages of implementation, when activities had to focus on preparing these designs and making an assessment of investments needed for improving facilities. Since many political actors were involved, with some of them from the opposition to the Chávez administration, this design process was trapped in the political climate prevailing at the time. Although technical teams had agreed on key design aspects, allocation of resources and control of project activities became an insurmountable obstacle. Additionally, the flexible design approach turned out to be a liability in the Venezuelan political context as it precluded the Bank from having an instrument to exert leverage. Monitoring indicators were defined at appraisal and a base line established by external consultants. However, it appears that there was no borrower ownership of this process leading to a successful design and implementation of systems needed for an effective monitoring of Project outcomes. Project risks were well identified but their assessment and mitigating measures were not successful in the political context that followed during implementation. In particular, coordinating the number of key actors involved, including the Ministry of Health and Social Development (MSDS), the Ministry of Planning, one state government (Miranda), the Caracas umbrella municipality ­ "Alcalde Mayor" (AM) and the five municipalities involved (Libertador, Sucre, Baruta, Chacao and El Hatillo), became an impossible coordination task. As mentioned earlier, the tensions that took place among these actors within the political context, from early 2002 to mid 2004, stalled Project implementation and led the MSDS to abandon practically the decentralization approach. 7.2 Implementation There was practically no progress in Project implementation during the first three years (2002- 2004). By the end of 2004 only about three percent of the loan amount had been disbursed. This was largely due to the difficulties mentioned earlier for reaching consensus among the different actors involved, in defining roles, responsibilities and for agreeing on priorities and a modus operandi for implementing the Project. The political climate prevailing during that period was not conducive to a favorable environment for carrying out a meaningful dialogue. The main reason for this was that some key actors - the AM and three of the five municipalities ­ were in the political opposition to the Government. Additionally, GOV opted for using its own 11 resources to finance several items originally included under Project components, while other Project activities were postponed. It should be noted however, that the GOV is well aware that the Project's objectives remain valid, and that it badly needs to improve coverage and quality of health services in the MDC. By late 2004, the Government had gained considerable strength after the outcome of the referendum to revoke the presidential term and the MSDS took absolute control over the Project. A new implementation plan was agreed and the GOV requested the first partial cancellation of loan funds. By that time the BA program was well under implementation. Since the program's basic aim was to expand coverage, creating a network of ambulatory services in the low income neighborhoods of Caracas and other major cities, its objective was highly coincident with one of the Project's DO. From the start, the BA program was highly resisted by the health sector community and the political environment made it impossible to agree on a referral system to support the basic care provided under the program. When the political climate cooled off after the August 2004 referendum, the Bank saw the BA program as an opportunity for improving implementation. Additionally, the Project's objectives and description were sufficiently broad, allowing the Bank to support those activities, which the Government selected for financing, without the need to restructure the Project. After a careful assessment, the Bank agreed to finance, under the Project, the design of protocols, training programs and some infrastructure and equipment for ambulatory centers under the BA program. A Quality Supervision Assessment (QSA) was conducted during 2006. The QSA acknowledged that, notwithstanding the supervision team efforts to resolve implementation problems, the political environment had created highly unfavorable conditions for an adequate implementation. The QSA report raised the issue of whether the Project should have been restructured, since it was highly unlikely that, under the prevailing climate, the Project's DO were going to be achieved. On the other hand, and given the difficulties in country dialogue that occurred during the latter part of Project implementation, the Bank had opted for using the Project as one of the few channels available for trying to maintain a dialogue with the Government. Instead of entering into a complex negotiation process under a difficult climate for restructuring the Project, the Bank opted for a supervision strategy focused on supporting those activities within the Project for which the Government had requested support, and on suggesting that the Government cancel remaining loan funds that were not going to be required. Moreover, the GOV had given clear signals that it did not wish to seek an extension of the closing date beyond the original date of December 31, 2006. Eventually, cancellation took place in two stages. The first one took place in April 2005 and the second one in late June 2006, and the loan was closed on the original closing date. Implementation improved considerably during the last two years and relative progress was achieved for expanding coverage in low income areas through the construction of 26 new health centers for the BA program and the rehabilitation of another 8 existing centers. It also provided the equipment needed for operating these centers. Additionally, it made modest progress for improving the quality of health services provided by constructing an ambulatory surgery system, rehabilitating a reference clinic to support primary care centers and by supplying communication and information equipment for emergency services. It also achieved consensus among local practitioners on the importance to strengthen referral systems in an integrated manner and to start developing programs to address these issues. It should be noted that, notwithstanding the difficulties described above, the HIV/AIDS component was implemented relatively smoothly and practically all activities originally foreseen 12 were completed satisfactorily. 7.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The Project Coordination Unit (PCU) was responsible for Project monitoring and evaluation, with support from the MDC health agencies. Progress reports were set up to monitor Project implementation results and expenditures. A set of monitoring indicators was designed to assess periodically progress towards meeting outcomes regarding coverage and quality of services provided. A baseline survey was conducted during Project preparation, to collect information on MDC's epidemiological and health services delivery aspects. However, the PCU failed to implement these designs and activities were not monitored regularly nor were progress reports issued regularly. Some monitoring indicators were only reported in the Borrower's completion report. 7.4 Safeguard and Fiduciary Compliance Although the Project experienced problems and failed to achieve planned disbursement levels, the main causes were not related to poor procurement management. There was a high turn over of procurement staff in the PCU precisely when progress was picking up during 2005. This resulted in delays due to lack of familiarity with Bank guidelines. Several training missions had to be fielded to correct this shortcoming. However, there is ample evidence that the procurement activities were satisfactory and the agreed guidelines and procedures were complied with. The level of financial management supervision was adequate for the low level of Project activity. The Financial Management Team (FMT) participated in one mission in February 2005 and produced a detailed report with a number of recommendations. The audit reports raised issues of internal control which were in turn raised with the Project team. Although, there was little follow up, none of them were likely to have had a material impact on the Project's financial management. All construction and rehabilitation of facilities financed by the Project complied with environmental guidelines, particularly regarding disposal of medical waste. 7.5 Post-completion Operation/Next Phase The GOV has emphasized that expanding access to health care services continues to be a priority. Consequently, it intends to carry on with its strategy consolidating the first phase of the BA program and it has launched a second phase establishing diagnostic, rehabilitation and high technology centers to provide support to ambulatory services. Subsequently, it plans to refurbish the hospital networks with the aim of setting up a fully integrated system. This strategy also includes a comprehensive training program to eventually replace the Cuban staff in the BA program over the next few years. The plan also includes strengthening of horizontal coordination with other ambulatory centers and establishing a referral system with other health care providers. Budgetary allocations for the health sector are expected to increase substantially, reaching about 6 percent of GDP by 2008. 8. Assessment of Outcomes 8.1 Relevance of Objectives, Design and Implementation 13 The objectives established for the Project were well in line with efforts to address the most critical health sector issues identified during appraisal. These were well founded on the Bank sector experience gained through years of involvement in analytical work and projects financed. The GOV has acknowledged that the stated objectives remain relevant at present and the strategy outlined in 7.5 above has been designed to continue to address prevailing issues. There is, however, one aspect of the Project's design that lost its relevance under the present circumstances. The GOV changed it emphasis related to the decentralization process and, today, its preference is for using a more centralized approach towards the provision of health services through the MSDS, the Venezuelan Institute of Social Services (IVSS) and the BA program. 8.2 Achievement of Project Development Objectives The Government had a much wider program in the MDC and it is difficult to isolate the effect of the Project. Well founded information on actual coverage levels is not available, but rough estimates indicate that there was an expansion of over one million persons in the low income areas of Caracas. The BA program managed its own information system and statistics were not readily available to the MSDS. Also, the BA used different definitions for measuring services provided and it was not possible to consolidate the statistics reported annually with the official MSDS statistical data base. In any case, it is widely accepted by the health sector community that the BA program has improved coverage in low income areas. The Project's contribution to this increase in coverage was marginal since it only covered about ten percent of the new ambulatory centers opened during the Project's life. Rough estimates indicate that the overall coverage expansion has reached about 2 million poor people, short of the target of 2.4 million established at appraisal. There was modest progress on improving efficiency and quality of services. A protocol, designed under the Project, was established for the BA program and some training took place for staff in the MDC on new approaches for providing health services. Also, a survey was conducted on coordination of practices among service providers in the MDC, but there was no follow up for modifying practices leading to the implementation of a new health care model emphasizing the organization in networks of providers. Also, there was minimal improvement in the management of the health care system in the MDC mostly related to the implementation of new information and communication systems for emergency care. There were important achievements on the development of HIV-AIDS prevention and control strategies and activities in the MDC. An epidemiological surveillance system of second generation was designed and it is under implementation. Additionally, campaigns were developed to raise awareness among the population including posters and advertisements in metro stations. Also, a training program for health sector staff was carried out under the Project. 8.3 Efficiency Given the modest overall outcome of the Project, it was considered of no added value to update the cost/benefit analysis. At appraisal, Project management costs were estimated at US$3.1 million representing 5.1 per cent of the estimated total Project costs. Given the low level of implementation, total Project costs were only 16 percent of those originally estimated. The consequence was that, although Project management costs were substantially reduced at about 45 percent of the original estimate (US$1.4 million), they ended up representing about 14.5 percent 14 of total Project costs. It should be noted, however, that several activities originally included in the Project's scope were implemented by the same PCU with other financial resources not recorded in the actual Project costs. 8.4 Justification of Overall Outcome Rating Rating: MU As discussed in 8.2 above, the Project had a poor achievement of the DO established at appraisal. It had only a marginal contribution to expanding coverage in low income areas of the MDC. Regarding outcomes for Project components, the results were modest. Some primary care and hospital facilities were upgraded by improving infrastructure and providing equipments. Additionally, some 25 ambulatory centers for the BA program were financed and equipped under the Project. Also, participatory schemes and a new protocol for BA and for emergency situations due to disasters were implemented under the Project. The major contribution to outcomes was achieved by developing a new surveillance system, rehabilitating two outpatient centers, and carrying out training programs and dissemination campaigns for the HIV/AIDS component. Very little was achieved under the component aimed at the development of legal and organization instruments and models for the MDC. Consequently, the performance rating on outcomes is set at Moderately Unsatisfactory (MU). 8.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development N/A (b) Institutional Change/Strengthening N/A (c) Other Unintended Outcomes and Impacts (positive or negative, if any) N/A 8.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A 9. Assessment of Risk to Development Outcome Rating: S As noted earlier, the GOV has made a substantial effort to expand access to primary health care services under the BA program and by and large reached, through a parallel program, the target set for the Project. The GOV is currently starting to implement a plan to consolidate this phase and to establish a referral system strengthening secondary and tertiary systems. There are, however, two risks to this outcome. The BA program has been mainly staffed with Cuban personnel and it is unclear how long this effort can be maintained and whether the GOV would be capable of replacing this staff with Venezuelan personnel in due course. Moreover, should conditions change it is highly likely that the program would be dismantled. For these reasons, the risk to development outcome is rated Substantial (S). 15 10. Assessment of Bank and Borrower Performance 10.1 Bank (a) Bank Performance in Ensuring Quality at Entry Rating: MS It has been acknowledged, both by the GOV and the QSA, that the Project design was well thought for addressing problems affecting the health system in the MDC. It was basically founded on the experience gained under the Health Services Reform Project, which had implemented, successfully, critical reforms in the provision of health services at the state level. The Project also provided a critical instrument for achieving some of the objectives established in the interim CAS, under a scenario of a quite reduced lending program. However, some shortcomings remain: (i) although sector issues were well identified, the lack of more design work during preparation affected progress during early implementation; (ii) the flexible design used did not provide the Bank with sufficient leverage for addressing the implementation issues that arose later, leaving too much margin for discussions; and (iii) although Project risks were properly identified during appraisal, the mitigation factors were not adequate for addressing the difficulties that led to a considerable reduction in scope jeopardizing the achievement of the DO and key outcomes for most components. The Bank performance for ensuring quality at entry is rated Moderately Satisfactory (MS). (b) Quality of Supervision Rating:MU The QSA came to the conclusion that if the Bank decided to pursue the Project's original objectives, in spite of the difficult environment, a formal Project restructuring should have been done. It felt that canceling about 80% of the loan's proceeds without changing the DOs doomed the Project to failure. It argued that the lack of GOV support for the Project was evident from the outset and Bank missions received little cooperation as illustrated by long waits for missions to be approved. It concluded that the Bank should have considered three possible actions: (i) restructuring of Project components/activities; (ii) reassessment of the DO; and (iii) partial cancellation of funds. The QSA states that the Bank failed "to be more actively involved, to give stronger advice and mentoring, and to insist on clear documentation, [all of which] contributed to the less than adequate supervision of the Project". Consequently, the QSA rated the quality of supervision as Moderately Unsatisfactory (MU). On the other hand, supervision missions made a considerable effort to address issues preventing progress under a difficult environment. Under this scenario, restructuring was not considered a valid option since the dialogue with the borrower was not fluid, and Project design was sufficiently flexible to accommodate a wide range of changes without amending the Loan Agreement. Instead, the Bank opted for a strategy that refocused implementation during the last two years, supporting those areas where the GOV was requesting Bank assistance, particularly the BA program, and suggesting to the GOV to request cancellation of funds not required. Given the experience of the last two years, the Project would have had a better chance for a substantial achievement of the DO if implementation would have been extended by one or two years. [But the GOV did not want to consider requesting an extension of the closing date]. This Project did provide the Bank an instrument to keep the country dialogue open during its implementation. 16 However, under these circumstances, the supervision team could have recorded explicitly in its reports that an unsatisfactory Project was maintained in the portfolio purely for reasons related to country relations. The Quality of Bank Supervision is rated as Moderately Unsatisfactory (c) Justification of Rating for Overall Bank Performance Rating: MS Given the reasons stated above for the rating given to the Bank Performance for design and supervision, and because supervision took on a lot of weight given the changed environment the overall Bank Performance is rated as Moderately Unsatisfactory (MU). 10.2 Borrower (a) Government Performance Rating: MU The main factors that precluded the GOV from achieving an acceptable performance were, in the first place, the lack of political support to the Project shortly after implementation begun. Supervision missions were continuously delayed and the environment became diffucult at times. Furthermore, several agreed initiatives, between the MSDS' technical staff and the municipalities, could not be implemented because authorities failed to provide the needed support. Additionally, the GOV was unable, or unwilling, to resolve satisfactorily the difficulties faced with the participating municipalities for implementing the Project as designed. Moreover, the GOV made a shift in priorities during Project implementation, moving away from a decentralization policy previously adopted. This last factor seriously affected the Project, since a key design feature was based on delivering, on a decentralized manner, a comprehensive health care program through organizing networks of ambulatory centers, emergency services and outpatient departments of hospitals in the MDC. Secondly, additional time would have enabled making substantial progress towards achieving the Project's objectives and outcomes originally foreseen, since common grounds had been found to complete its implementation. Lastly, a high turnover of staff in the PCU and insufficient budgetary allocations and disbursement ceilings were constraints to a better Project implementation, particularly during the last two years. . . All these factors lead to rate the government performance as Moderately Unsatisfactory (MU). (b) Implementing Agency Performance Rating: MU Implementing Agency Performance Initially, the PCU started to work well with the municipalities involved. Unidad Criteria was agreed for designing schemes that would have led to an Coordinadora improvement in the efficiency and quality of services provided through the Proyecto Salud establishment of a network of providers operating with a well coordinated approach. However, when issues related to resource allocation and implementation procedures derived in a complex struggle between MSDS and 17 participating municipalities, the PCU lacked the leadership and political ability, or will, required to deal with them. As a consequence, protracted negotiations took place that led to a complete halt in Project activities. The only aspect that was implemented in a satisfactory manner was the component dealing with developing strategies and practices for HIV/AIDS prevention and control. It would appear that the GOV supported these strategies and practices because it did not have an alternative program to deal with these issues. For these reasons, the performance of the Implementing Agency is rated Moderately Unsatisfactory (MU). (c) Justification of Rating for Overall Borrower Performance Rating: MU Since both the Government and the Implementing Agency performance received the same rating, the overall rating is also Moderately Unsatisfactory (MU). 11. Lessons Learned An adequate political climate is crucial for Project success when there are many actors involved, particularly in a decentralization process. Although the Project had a good start mobilizing resources and support from the municipalities involved in developing a decentralized system, further steps for implementing it were stalled by a mix of different approaches to resource allocations, but most importantly by the political climate that took place in Venezuela during most of the 2002-04 period. Three of the mayors from participating municipalities and the MDC major were from the political opposition to the Chávez administration. Different positions on how to implement a decentralized program became insurmountable and the Project suffered from this struggle, leading to a sizable reduction in Project scope. Although the participation of many political actors was acknowledged during appraisal as an important risk factor, it was not possible to have foreseen that the political environment would make it impossible to implement a decentralized scheme as originally conceived. When supporting a decentralization process, Project implementation is best placed as low as possible within the available institutional structure. When this Project was appraised, the MSDS was meant to focus on taking a regulatory role and creating a new resource allocation mechanism. The provision of health services would be decentralized implementing a new health care model, emphasizing the organization of networks of service providers in the public sector. There were five autonomous municipalities involved, four of which were within the Miranda state which also had a role of its own within the health sector under the Venezuelan federal system. A recent reform had created the AM as an umbrella organization to play a coordinating role for autonomous municipalities within the city of Caracas (the MDC). At one point during Project implementation, it was proposed that the Project could be managed by the AM structure, since it was well in line with its coordinating role. Although the AM was eager to assume this role, the MSDS strongly opposed the proposal and eventually retained Project management. In retrospect, this would have been a better approach for implementing the Project since it was much closer to the municipalities and it fitted very well with the very nature of the AM. However, it lacked political support when it was proposed and the idea was abandoned. By that time, the GOV had a established a clear preference for a more centralized approach. What can or should the Bank do when a country environment jeopardizes Project 18 implementation? A poorly performing Project was used, to a great extent, as a vehicle to keep the country dialogue open. But this approach raised the issue of sacrificing the achievement of the Project's DO for a broader objective not related to Project performance. Assuming that the Government had lost interest in implementing the Project as originally conceived, the other option available was canceling. But such an approach would, most likely, have deteriorated even further the country dialogue. Whenever such a strategy is used, for valid reasons, it would be useful to make it explicit by regional management, and to record it as such in the Project files. 12. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Borrower's unedited evaluation report is attached in Annex 8. (b) Cofinanciers (c) Other partners and stakeholders N/A 19 Annex 1. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objectives of the Project are: · expand the access to and improve the quality and efficiency of ambulatory care services, including the outpatient services of hospitals, for 2.4 million poor people living in selected parishes and municipalities of the MDC; · improve the management of the health care system and support the decentralization process in the health system of the MDC; and, · develop HIV/AIDS prevention and control strategies and activities in the MDC. The Project is expected to contribute in the medium-term to the improvement of the health status of the most vulnerable population in the MDC. The Caracas Metropolitan Health Services Project followed up on some activities of the Health Services Reform Project (3823-VE) and the Endemic Diseases Control Project (3538-VE) Revised Project Development Objectives (as approved by original approving authority) N/A (a) PDO Indicator(s) Original Target Formally Actual Value Indicator Baseline Value Values (from Revised Achieved at approval Target Completion or documents) Values Target Years Indicator 1 : Increased coverage of health services for 2.4 million poor in the metropolitan district of Caracas Value (quantitative 1 million 2.4 million About 2.0 million or (rough estimate) Qualitative) Date achieved 06/26/2001 12/15/2006 There is not a reliable estimate for this indicator for the total coverage in the Comments MDC. There is however, ample consensus among the health sector community (incl. % that coverage has increased substantially. It should be noted that the Project only achievement) contributed to about ten percent of the increase in coverage. Indicator 2 : Adoption of policies and strategies for preventing and controlling HIV/AIDS. Value Not clear policies and (quantitative strategies for Policies and Policies and or preventing and strategies strategies established Qualitative) controlling HIV/AIDS established Date achieved 06/26/2001 Comments A sound strategy was adopted including the establishment of an improved (incl. % surveillance system, improved equipment for control centers, a dissemination achievement) campaign and training of health staff. 20 (b) Intermediate Outcome Indicator(s) Indicator Baseline Original Target Values Formally Actual Value Achieved Value (from approval Revised Target at Completion or Target documents) Values Years Indicator 1 : No intermediate outcome indicators were prepared for the Project. Value (quantitative or Qualitative) Date achieved Comments (incl. % achievement) 21 Annex 2. Restructuring (if any) Not Applicable 22 Annex 3. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Components Estimate (USD Estimate Actual/Latest Percentage of M) (USD M) Appraisal STRENGTHENING OF HEALTH CARE NETWORKS 44.07 INSTITUTIONAL DEVELOPMENT 6.89 DEVELOPMENT OF STRATEGIES AND ACTIVITIES FOR HIV/AIDS PREVENTION 2.74 AND CONTROL IN THE MDC DEVELOPMENT OF LEGAL AND ORGANIZATIONAL INSTRUMENTS AND 0.80 MODELS IN THE MDC PROJECT ADMINISTRATION AND EVALUATION 5.54 Total Baseline Cost 60.00 8.49 14.15% Physical Contingencies Price Contingencies Total Project Costs 60.00 Front-end fee PPF Front-end fee IBRD .30 0.30 100.00 Total Financing Required 60.30 8.79 14.58% (b) Financing Appraisal Source of Funds Type of Actual/Latest Percentage Cofinancing Estimate (USD M) Estimate (USD M) of Appraisal Borrower 30.00 4.43 14.76% INTERNATIONAL BANK FOR RECONSTRUCTION AND 30.30 4.36 14.38% DEVELOPMENT (c) Disbursement Profile 23 24 Annex 4. Outputs by Component Component Indicator Latest Estimate (2006) Strengthening of Health · % of women, in reproductive age, 92.4 % Care Networks who use contraceptive methods · % of children (under 1 year) 98 .4% immunized against measles · % of total population with medical 52.5% consultations · % of fertile age women assessed for 76.6% cervix cancer · % of pre-school age children 82.7% attending healthy child controls · % of TBC patients treated 21.0% · Ratio of prenatal care visits to no. of pregnant women under control 1.3 · % of pregnant women having prenatal controls during first 3 month 29.8% · % of women with postnatal control 14.8% · % of pregnant women with 15.1% HIV/AIDS test · % of pregnant women with first dose 32.7 of anti-tetanic shot · % of TBC patients that give up 1.0% treatment · Development of Strategies and · Ratio of total visits to first visits for 6.2 (average for 10 months) Activities for HIV/AIDS HIV/AIDS patients in the Vargas Prevention and Control in the Hospital MDC 25 Annex 5. Economic and Financial Analysis (including assumptions in the analysis) Given the modest overall outcome of the Project, it was considered of no added value to update the cost/benefit analysis. Additionally, it was not possible to differentiate the Project's contribution from other actions being implemented, notably the BA program. As mentioned in 8.2, it has not been possible to consolidate the statistical base used by the BA program with the system used by the MDSD. Moreover, there are no reliable estimates of the actual costs incurred by the BA program. 26 Annex 6. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/Specialty Lending Supervision/ICR Fernando Leonfanti Consultant LCSHH Marcelo Amador Osorio Consultant LCSPT Ana Pereda Consultant LCSHH (b) Ratings of Project Performance in ISRs No. Date ISR Actual Disbursements Archived DO IP (USD M) 1 12/19/2001 Satisfactory Satisfactory 0.22 2 06/11/2002 Satisfactory Satisfactory 0.53 3 12/11/2002 Satisfactory Satisfactory 0.67 4 06/17/2003 Unsatisfactory Unsatisfactory 0.67 5 12/12/2003 Unsatisfactory Unsatisfactory 0.67 6 06/03/2004 Unsatisfactory Unsatisfactory 0.67 7 11/29/2004 Satisfactory Satisfactory 0.76 8 04/26/2005 Moderately Moderately Satisfactory Satisfactory 1.26 9 06/06/2005 Moderately Moderately Unsatisfactory Satisfactory 1.38 10 11/22/2005 Moderately Moderately Unsatisfactory Unsatisfactory 3.05 11 10/27/2006 Unsatisfactory Unsatisfactory 4.38 (c) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands No. of staff weeks (including travel and consultant costs) Lending FY00 4 36.14 FY01 18 89.95 FY02 1 6.69 FY03 0.00 FY04 0.00 FY05 0.00 FY06 0.00 27 FY07 0.00 Total: 23 132.78 Supervision/ICR FY00 0.00 FY01 0.33 FY02 7 48.19 FY03 6 56.55 FY04 8 49.04 FY05 7 63.41 FY06 6 46.71 FY07 8 26.91 Total: 42 291.14 28 Annex 7. Detailed Ratings of Bank and Borrower Performance Bank Ratings Borrower Ratings Ensuring Quality at Entry: MS Government: MU Quality of Supervision: MU Implementing Agency/Agencies: MU Overall Bank Overall Borrower Performance: MU Performance: MU 29 Annex 8. Summary of Borrower's ICR and/or Comments on Draft ICR 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 Annex 9. List of Supporting Documents Project files in IRIS. 52