ICRR 12655 Report Number : ICRR12655 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/27/2007 PROJ ID : P062932 Appraisal Actual Project Name : Health Reform US$M ): Project Costs (US$M): 239.3 231.8 Program (first Phase: Mother And Child Insuranceand Decentralization Of Health Services) Country : Peru Loan /Credit (US$M): Loan/ US$M ): 80.0 27.0 Sector Board : HE US$M): Cofinancing (US$M): 95.0 28.0 Sector (s): Health (97%) Central government administration (3%) Theme (s): Other communicable diseases (25% - P) Child health (25% - P) Participation and civic engagement (24% - P) Decentralization (13% - S) Population and reproductive health (13% - S) L/C Number : L4527 Board Approval Date : 12/16/1999 Partners involved : IDB, DFID, OPEC Closing Date : 12/31/2003 06/30/2006 FUND Evaluator : Panel Reviewer : Group Manager : Group : Denise A. Vaillancourt Roy Gilbert Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The medium -term goals of Peru ’s 10-10 -year Health Reform Program (2000-2010) are to: (a) improve maternal and child health; and (b) help reduce morbidity and deaths of the poor from communicable diseases and inadequate environmental conditions. These goals will be achieved through increasing the access of the poor to, and improving the quality and efficiency of, health systems in Peru . This 10-year Program was to be supported by a series of three Adaptable Program Loans (APLs). The first APL in the series , the subject of this ICR review, was to provide financing for Phase I of the Program (FY2000-2004) with an IBRD loan of US$80 million. It aimed to contribute to the above medium -term Program goals by increasing the access of the poor to better quality health programs and services . The focus of Phase I was on empowering the poor, strengthening the demand side, while improving the quality of the supply side of health programs and services. At the operational level, this objective translated into the following subgoals : (a) reducing the economic barriers to utilization of health services , primarily through the implementation of Seguro Materno Infantil (SMI) (mother and child health insurance ); (b) enhancing the decentralization of the health system through: (i) increased participation of communities (through the community-managed health facilities – CLAS), municipalities and local health entities in planning, management and monitoring of health programs and services; and (ii) redefining the role of the Ministry of Health (MoH) including streamlining its mother, child and environmental health programs; and (c) adapting investments to local health problems, particularly to cultural aspects of health promotion and service utilization, and addressing communicable diseases and environmental health problems . Subsequent phases of the APL envisaged the following support : Phase II (FY-2004-2007, to be supported by an IBRD loan of US$ 50 million equivalent) was intended to support the continuation/adjustment of Phase I reforms and to tackle the issues of health insurance, health manpower, and autonomy of hospitals. Phase II (FY2007-2010, to be supported by an IBRD loan of US$ 50 million equivalent) was intended to consolidate the lessons from the previous two phases and to support public hospital reforms, continued strengthening of health care networks; unification of health insurance schemes, and chronic diseases . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (with estimated costs, including contingencies, and actual costs) 1. Strengthening Health Demand (US$149 149 .0 million or 62 percent of total estimated cost; US$ US$ 149. 149 .0 million or 64 149. percent of total actual cost ): Implementation of the Maternal and Child Insurance (SMI) to benefit the low income mothers and children, especially from the poorest rural areas of Peru . The loan was to finance: (i) the reimbursements for services provided by eligible health facilities under the SMI in the project areas; (ii) SMI administration at the central level (MoH) and at the local level (DISAs); and (iii) monitoring and evaluation of SMI implementation. 2. Strengthening Decentralization, Policy Development and Institutional Modernization US$ 22. 22 .8 million or 10 (US$22 percent of total estimated cost; US$ 22. 22 .8 million or 10 percent of total actual cost ): (a) technical assistance for decentralization , including (i) support for Community-Managed Health Facilities (CLAS) expansion and empowerment of communities (facilitation of democratic election processes; strengthening of CLAS management and of community capabilities to implement and monitor local health plans ); and (ii) strengthening capacity of municipalities and Regional Health Boards -- DISAs (formulation and implementation of regional health plans, elaboration of investment proposals and strengthening of municipalities ’ role in PHC administration); (b) reorientation of the role of MoH and public/private health providers , including improvement of normative role, streamlining of mother and child health programs, strengthening strategic planning and budgetary process; streamlining of environmental health programs; and development of studies, workshops and action plans related to second generation reforms (payment mechanisms and contracting of services between MoH and Social Security (ESSALUD) facilities, public/private insurance schemes articulation, health manpower skills mix and incentives for better focus on poverty); and (c) monitoring and evaluation: conversion of MoH’s information system as a M&E instrument that will track the distributional effects of health programs and services . 3. Improving the Quality of Health Programs and Services (US$49 49 .0 million or 21 percent of total estimated cost; US$ 49. US$49 US$ 49 million or 21 percent of total actual cost ): financing of subprojects in support of the regional /local health plans addressing the health priorities of the regional departments in four general areas : support for SMI (see component 1); nutrition; prevention and treatment of prevalent communicable diseases in the local area; and environmental health. 4. Project Coordination (US$8 US$ 8.0 million or 3 percent of total estimated costs; US$ 8.0 million or 3 percent of total actual costs ): partial financing of the operating costs of the Project Coordination Unit, PCU . The bulk of the APL’s first phase was supposed to finance the implementation of SMI (see component 1) and its corollary investment requirements in 10 geographic areas in 10 departments (Puno, Apurimac, Cajamarca, Cuzco, Lambayeque, Piura, Tumbes, La Libertad, Madre de Dios, Lima Norte and Lima Sur ), with planned Inter-American Development Bank (IDB) financing to provide similar support in 17 other departments. Technical assistance under component 2 was to benefit the national level and municipalities, local health departments and communities participating in the CLAS expansion . Changes to components: During project restructuring (2003) the number of departments in the project area was reduced, but the definition of the project area after restructuring is not clear . (On page 6 the ICR notes that the number of regions in the project area was reduced from 10 to 3, including four DISAs [Regional Health Boards]. On page 8 it is noted that the project's geographical scope was reduced to 8 regions). During project implementation, the maternal and child insurance (SMI), which was to be strengthened under the project, was transformed into the Integrated Health Insurance (SIS), which the project continued to support in its new form . Other changes due to restructuring included : the revision of the logframe to include five monitoring indicators to explain changes in maternal and peri -natal mortality; improved targeting of SIS to two poorest quintiles and a strategy to reduce linguistic and cultural barriers to access; consolidation of supply -side Components B and C and the targeting of maternal and peri -natal mortality instead of a broader set of diseases and environmental conditions that affect child health . In addition, the financing of reimbursements for each child and mother covered by the SIS was assumed by the Government under the project (instead of the Bank loan); and the investment component, originally designed as a demand -driven fund for communities, was amended to support a system of investment planning based on technical criteria developed by the World Health Organization (WHO) and to support complex referral systems . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Comments on Project Cost, Financing, Borrower Contribution, and Dates Total actual cost was 97 percent of the original estimate. The Bank and the IDB each financed only one third of their respective original loan amounts . This is due to Government’s decision to assume responsibility for financing the reimbursements for utilization of maternal and child health services under the insurance scheme (disbursement against results described in Section 3), which made up a substantial part of projects and was originally slated for Bank and IDB financing. As a result, the Government financed almost three times its original financing commitment . OPEC did not provide financing as originally planned . The project closed on June 30, 2006, two and one half years after the original closing date of December 31, 2003. The extension of the closing date was attributable to the very slow start and sluggish implementation through 2003, due to political changes, unsatisfactory physical and financial implementation progress, the fusion of two insurance schemes into one, turnover in PCU staff and decentralization challenges. 3. Relevance of Objectives & Design: Overall relevance is substantial. Relevance of Objectives . The latter were supportive of 1997 CAS, which aimed to reduce poverty and improve Peru’s human capital base. Medium-term sector reforms to be supported by the APL included : reconfiguration of the health system, promotion of integration of public and private services within a decentralized context, and reallocation of health expenditures to basic health care for the poor . The project is also consistent with World Bank ’s Strategy in the HNP sector (1997), aiming to: (i) improve the HNP outcomes of the Peruvian poor; (ii) enhance the performance of health care systems; and (iii) secure sustainable health care financing . The project is an operational follow -up to recommendations of the health sector report entitled, “Peru: Improving Health Care of the Poor� (Report No. 18549-PE) as well as Ministry of Health’s analytical work on the reforms needed in the health sector . The PDO continues to be relevant in present -day Peru. The new December 2006 Country Partnership Strategy notes the large acceleration in the rate of reduction of the infant mortality rate and proposes to sustain these gains . It states that the emphasis must be to continue improving results by improving quality and sustaining gains in coverage. Relevance of Design . The proposed project is closely linked to the health policy conditionalities in the SAL under preparation. The complementarity of the SAL and APL instruments and the utilization of the ESW results constituted a unique process for providing immediate and medium -term assistance to health sector reforms . The SAL also served to elicit Ministry of Finance ’s increased focus on social sector reform, presenting an opportunity for MoH . The choice of the APL was appropriate in that it accommodated both the flexibility and the incremental, medium -term perspective required for successful sector reform . Another feature of the design was its simultaneous strengthening of demand for services (through the SIS) and supply of services (through health investments). The original design of the project was innovative in its inclusion of a component that would disburse against results, reimbursing the government a certain amount for each child and mother who utilized health services covered by the SIS . However, the project design (a) was complex and included too many health -related issues to tackle at once; and (b) did not take into sufficient consideration the potential difficulties that MoH would have in obtaining information systems and human technical support for effective management of the intended reform process . The restructuring of the project in 2003 and its focus on maternal and peri -natal services was helpful in reducing project scope and complexity to more realistic levels. 4. Achievement of Objectives (Efficacy): Overall efficacy is substantial. (a) Reducing the economic barriers to utilization of health services : substantially achieved. The percent of pregnant women with four or more prenatal visits in the project area increased from 32 in 2000 to 57 in 2006, surpassing the 2003 target of 51, and increasing (by 78 percent) at a much faster rate than national averages ( an increase of 26 percent, from 60 percent in 2000 to 76 in 2006). The percent of deliveries attended by skilled health personnel increased in the project area by 84 percent from 28 in 2000 to 51 in 2005, surpassing the 2003 target of a 33 percent increase, and increasing at a faster rate than the national average, which increased from 55 percent to 71 percent. The percentage of children between 18 and 29 months vaccinated with DPT3 increased from 78 percent to 87 percent, but fell short of the 95 percent target. DPT3 was replaced by a more complex vaccine so that its coverage at the regional level was reduced in the last few years . (The Pentavalent, which includes DPT 3, Hepatitis B and Haemophilus Influenzae type b vaccines ) is more powerful but is more demanding in its application and logistics.) 28.1 million beneficiaries receive care in the primary facilities of MoH annually, exceeding the 2003 target of 17 million. The percent of deliveries attended institutionally by the Integrated Health Insurance (SIS) (which replaced the SMI – maternal-child insurance) increased 73 percent nationwide (from 38 percent to 65 percent); and, in the project area, it increased by 224 percent from 19 to 60 percent. The percent of deliveries financed by SIS in the two poorest quintiles of the population in the project area increased by 27 percent from 44 to 56 percent. The percent of newborn children covered by the SIS weighed within the first 24 hours of birth increased from 73 to 83 percent nationwide and, for the project area, increased from 64 to 74 percent , but fell short of the (nationwide and project area) target of 95 percent. The SIS was implemented nationally in 2002, ahead of the target of 2003. (b) Enhancing the decentralization of the health system and strengthening MoH : substantially achieved. All DISAs regularly sign annual Management Agreements with the MoH, reflecting commitments with respect to regional health priorities and indicators with quantitative goals for the period . One third of primary care clinics were administered by CLAS, which has strengthened the quality of services . CLAS have been shown to have greater productivity per employee, long hours of operation, higher levels of patient satisfaction and greater levels of community participation . CLAS employees have been given the status of civil servants due to a change in labor legislation and care will need to be given not to lose the quality and efficiency gains achieved through the “non-public� spirit of the CLAS. Fragmentation of the formerly “vertical� programs was reduced in part, through restructuring of care into age-based groups and the establishment of the Integral Health Model . In addition, the budget was integrated for all programs. However, the information systems still need to be unified through an integrated MoH monitoring system. An assessment of project results was completed by Government to produce lessons and recommendations for the formulation of Phase 2, which have been incorporated into the new design . This included an analysis of the incidence of health policies. (c) adapting investments to cultural aspects of health promotion and service utilization : substantially achieved (with a caveat) A Participation and Social Communication Strategy and an Indigenous People Plan were derived from a social assessment to detect and address relevant cultural patterns of users and health providers and gaps impeding the improvement and use of health services . While the intercultural strategy was designed only during the implementation phase and launched from mid -2004, the ICR notes that it has contributed significantly to improved utilization. The strategy has facilitated a better balance between the supply and demand sides of health services investments, highlighting the need to go beyond the provision of services by ensuring the effective and equitable use of these services . Over and above positive trends in service utilization and outcomes among the poor and among rural and disadvantaged rural populations documented in the ICR, the tracking of output and outcome indicators of indigenous populations would have provided added insight on the project's performance. Health outcomes: Nationwide infant mortality fell from a 2000 baseline 43 deaths per 1000 live births to 22.3 in June 2006, surpassing the 2003 target of 29.2. Likewise infant mortality in the project area fell from 48 to 28.3 in 2006, surpassing the 2003 target of 34.4. While infant mortality is the result of many factors, it is plausible to attribute this achievement in part to project investments in stimulating both the demand for services as well as its supply . The Borrower in its comments notes as well other investments in maternal and child health (European Union, Care, IDB and others). 5. Efficiency (not applicable to DPLs): Gains in efficiency were achieved through good targeting and increased coverage of the SIS (mothers and children in poorest income quintiles). The project incited greater demand for and utilization of services (generated by SIS) and achieved higher levels of productivity in previously underutilized facilities . SIS contributed to significant improvements in the targeting of public health expenditure . SIS improves the distribution of expenditures of other MoH facilities. By financing the co-payment allowing the poor entry to a hospital, it directs the subsidy to the facility to the poor beneficiary. Also the mother and child components of SIS have a relatively small cost (5 percent of public health expenditure or 0.07 percent of GDP), and this increase has been easily absorbed in the budget . Investment expenditures in 2005 were only 0.5 percent of public health expenditures and their recurrent cost implications are also minimal. The ICR refers to a benefit analysis of all the large social programs in Peru (carried out by RECURSO) which found SIS to be the most efficiently targeted program in health, noting that this program has a "concentration coefficient" of -0.08, compared with a "concentration coefficient" of 0.21 for expenditures in MoH hospitals . However, the meaning of this indicator is not explained . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: PDOs were substantially relevant and were achieved with substantial efficiency . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: SIS is strongly supported by the current government and recognized as a valuable service by the population . Government's strong political and financial support for maternal and child health services will be underpinned by continued Bank support through Phase 2 of the APL and a DPL. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: Quality at entry was good thanks to sound analytic work and a strong dialogue with government . Frequency of supervision missions was adequate and they were carried out by technically competent teams who were proactive during periods of major political turmoil and significant staff turnover . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Borrower and PCU performance were sound overall, despite political turmoil and staff turnover . The project was prepared simultaneously with the IDB -financed project. The Borrower's team was well qualified and made up of members who were familiar with IDB and Bank procedures and it was well coordinated by the Ministry of Health . Weighting the highly unsatisfactory performance of the Government up to 2003 (strongly linked to the political turmoil and resultant uncertainty about health policy and roles and responsibilities ) with the decisive and productive performance since November 2003, overall Government performance during project implementation is satisfactory. a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design . Monitoring. A M&E subcomponent was designed to measure the intermediate and final outcomes of the project, under the responsibility of the PCU . Coverage of health interventions were to be measured by administrative data generated by the existing information system, which was to be strengthened with project support . Annual coverage surveys were also planned to complement administrative data, by strengthening the social sector module of households. MoH (responsible for administrative data ) and the National Statistics Institute (responsible for the household surveys) were expected to sign an agreement satisfactory to the Bank to carry out a revised social sector module. Evaluation. Two Demographic and Health Surveys (a baseline in 2000 and an second one in 2003) were planned to measure progress in health conditions, including infant and maternal mortality . The National Statistics Institute was responsible for implementation. Annual ex-post evaluations of random samples of concluded subprojects and beneficiary assessments were planned to evaluate investment subprojects, including physical audits of a sample of evaluated projects. To measure subproject impact, a baseline and control group were to be established during the first year of implementation. With the assistance of DFID, a participatory monitoring and evaluation approach (PM&E) was also planned to complement other M&E activities and to foster fuller involvement of local stakeholders in the assessment of project activities . Implementation . While the launch was slow, towards the end of 2003, the project developed a monitoring system, based on MINSA information systems that allowed timely reporting of key project indicators in the eight DISAs and nationally for SIS indicators. The DHS became a continuous exercise (Continuous National Survey) starting in 2004, under the responsibility of the National Statistics Institute . The project also implemented a training program on monitoring and evaluation in the eight DISAs to develop local capacity in monitoring maternal and child health indicators. Utilization . The system was regularly used by Project staff to ensure that investments were targeted to critical areas within these jurisdictions. In addition, the system contributed significantly to monitor SIS targeting to the poorer quintiles. The Borrower, in its comments, noted that the project ’s M&E system influenced significantly the decision -making process in the project area . A particular attribute of the system was the fact that it used existing administrative systems to calculate indicators . Unfortunately, at the central level, this system was not valued as much as it was in the regions. The main reason for this is the large number of existing monitoring systems at the central level . Under the follow-on project the Government is requesting support for the unification of the various administrative systems in order to develop a solid M&E system at the MoH level . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low While IEG agrees with this overall Outcome : rating, an important caveat would be the challenge of sustaining the good performance of the CLAS, given that staff have now been given civil servant status. The original premise of CLAS was that public sector performance and accountability would improve with the involvement of the non-public sector, providing a counterforce to the public sector. Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Political volatility can have a detrimental effect on project implementation and impact . Frequent changes of political appointees often result in changes in priorities that lead to confusion in project teams and delay in project implementation. Use of a variety of lending and non -lending instruments can enhance dialogue and the implementation of critical sector reform. The use of a clear evidence -based causality model can strongly enhance the achievement of project development objectives. Strategies to document and address cultural barriers to health services access are vital for improving overall access to care when beneficiaries belong to diverse indigenous groups . 14. Assessment Recommended? Yes No Why? It would be interesting to study various aspects of what appears to be good practice for sector reform, including: focusing on both supply and demand at the same time; a logframe that includes indicators that link investments with outcomes; refined targeting of mothers and children in poorest two income quintiles; good synergies in the use of Bank instruments (health and non-health lending and ESW). 15. Comments on Quality of ICR: The ICR is well organized and well written, making good use of available data and substantiating the ratings and lessons well. There were only two points on which more clarity would have been beneficial : (1) The definition of the project area after project restructuring is not clear (Section 2.b). (2) On page 19 the CLAS are noted to have been very successful in improving the quality and efficiency of services and patient satisfaction. Their strength, indicated elsewhere in the ICR, is that the CLAS are made up of civil society actors and communities that keep a check on service delivery on behalf of the people . Yet the ICR notes that Government has changed the labor legislation, giving CLAS employees the status of civil servants . The ICR provides no assessment of the implications of this decision which run contrary to the original spirit of the CLAS reform (role of civil society and communities as a complement to, and check on, public sector services ). a.Quality of ICR Rating : Satisfactory