ICRR 12812 Report Number : ICRR12812 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 12/19/2007 PROJ ID : P072637 Appraisal Actual Project Name : Provincial US$M ): Project Costs (US$M): US$750 US$750 Maternal-child Health Sector Adjustment Ln. (PMCHSAL) Country : Argentina Loan /Credit (US$M Loan/ ): US$M): US$750 US$750 Sector Board : HE US$M ): Cofinancing (US$M): 0 0 Sector (s): Health (50%) Compulsory health finance (40%) Sub-national government administration (5%) Central government administration (5%) Theme (s): Child health (25% - P) Health system performance (25% - P) Population and reproductive health (24% - P) Law reform (13% - S) Indigenous peoples (13% - S) L/C Number : L7199 Board Approval Date : 10/28/2003 Partners involved : None Closing Date : 12/31/2005 03/31/2007 Evaluator : Panel Reviewer : Group Manager : Group : Gayle Martin Rene I. Vandendries Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The objectives of the Provincial Maternal Child Health Sector Adjustment Loan (PMCHSAL) were to (i) respond to the urgent health needs of the poor, particularly the uninsured mothers and children, and to (ii) assist the Government of Argentina to modify the incentive framework for financing and delivery of health care services starting in Argentina's poorest provinces . The objectives in the Program Document and Loan Agreement were consistent . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The loan contained 30 conditions spread over three tranches, respectively with 12, 10 and 8 policy actions. The conditions in the policy matrix and the Program's intended outcomes can be divided into five reform pillars, as discussed below: MCHIP ). The reforms needed for implementation of the MCHIP were to Maternal -Child Health Insurance Program (MCHIP) be developed and implemented. Examples include: a predefined health services package to uninsured mothers and children; restructuring the inter -governmental health financing arrangements to a more performance -based model; using differential reimbursement rates to create incentives for rural service delivery; and revision of the benefit package to increase cost -effectiveness. National -Provincial Health Policy Coordination . The role of the Federal Health Council (COFESA) was to be revitalized and strengthened to serve as an effective coordinating instrument for health policy across the country, and better aligning the national and provincial MoH actions . Essential Priority Health Programs . Essential public health programs threatened by the economic and financial crisis during the early 2000s were to be protected through 2004. Included here were: HIV/AIDS, tuberculosis, immunization and nutritional supplementation . Sexual and Reproductive Health . The sexual and reproductive health law was to be developed and implemented as a national program. This includes: sexual and reproductive health services delivered by provinces and inclusion of these services in the mandatory medical package (PMO) of the national health insurance system . National Health Insurance Regulation . Progress made during the past years in reforming the national health insurance system would be sustained and further consolidated, including the reduction of undesired cross -subsidies that affected the finances of provincial public hospitals, and the revision of capitation rates under the Solidarity Redistribution Fund (FSR). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The loan was fully disbursed in three tranches, and there was no requirement for partial or total waiver of any of the policy actions. The conditions for the first tranche (US$450 million) were met 9 days after board approval upon effectiveness in November 2003, and the second tranche (US$150 million) was disbursed in June 2004. The release of the third tranche (U$150 million) was delayed following suspension of the government's IMF program and the delay in meeting one of the 8 policy actions. Two project extensions were required, the last project extension requiring retroactive approval . The Program closed 15 months after the original closing date . 3. Relevance of Objectives & Design: Relevance of project objectives : The relevance of the objectives to the country and the Bank was substantial . The objectives were fully consistent with the government ’s Health Sector Reform Program, reflecting the sector priorities of the incoming administration (starting May 2003). The program provided urgent pro -poor support to the health sector, and was in direct response to the acute impacts in the poorest provinces following the economic crisis (starting late-2001). While the loan was not anticipated in the CAS (dated June 2000) nor in the CAS Progress Report (dated October 2001), the Program was aligned with the CAS priorities, and remains relevant in the new CAS (dated April 2004; prepared at the time of appraisal ). Relevance of project design : The relevance of Program design was substantial . The design reflected the conditions at appraisal, achieving a successful balance between the short -term actions needed to respond to urgent service delivery needs, and the longer term structural reforms . The quality of ESW and the Bank's detailed knowledge of the sector allowed for carefully chosen and reasonably sequenced conditions reflecting expected progress in policy development as well as the Borrower's financing needs . The design features also contributed to the program ’s pro-poor outcome, specifically: the geographic focus on poorest provinces; the emphasis of MCHIP on uninsured mothers and children; and the protection of expenditures on essential public health programs . 4. Achievement of Objectives (Efficacy): Each of the 30 policy actions in the policy matrix have been met and /or maintained by the end of the Program . Objective 1. Respond to the urgent health needs of the poor, particularly the uninsured mothers and children Health care access for uninsured mothers and children Process indicators : MCHIP was created by a Presidential Decree creating and following a national ministerial resolution MCHIP was launched and the MCHIP operation manual was approved . There was a high level of interest among the eligible provinces and in the first year of the project the organizational arrangements in place for implementing the insurance program in at least four of the nine eligible provinces . The beneficiary database was approved and piloted and by the release of tranche 3 MCHIP was fully functional in at least two of the eligible provinces . Outputs : MCHIP achieved 65% coverage of the eligible population (including 457,000 uninsured mothers and children in all nine targeted provinces, of which 12,700 beneficiaries are from indigenous groups ). This was well in excess of the target of 4,000 in at least two provinces. Outcomes : National infant mortality rate decreased from 18.3 per 1,000 live births in 2002 to 12.9 per 1,000 in 2007. The ICR reports that 9 northern provinces have "reversed their increasing (IMR) trends" (ICR, p16), although no evidence in support of this statement is provided . Urgent health needs Inputs and process indicators : Tranche conditions specified the level of financing for the identified essential public health programs (HIV/AIDS, tuberculosis, immunization and nutritional supplementation ). In all but one year, funding for essential programs was maintained well in excess of the minimum expected budget . No information on expenditure breakdown by program was provided to verify the continued funding of individual programs . The legal and institutional framework for improved sexual and reproductive service delivery was implemented, which includes: (i) the national Sexual and Reproductive Health Law published and corresponding regulations developed; and (ii) reproductive health services included in the mandatory medical package (PMO) of the national social health insurance system . Outputs : In 2006, 87% vaccination coverage was achieved in 19 of 24 (80%) provinces, surpassing the target of 70% of provinces. In 2006 country-wide measles vaccination coverage reached 97%, and 22 of 24 provinces reached the target of 87% measles vaccination coverage . In 2006 93.2% of HIV+ pregnant women received antiretroviral prophylaxis, exceeding the target of 90%. The condition requiring nutrition supplement (fortified milk) to 0.3 million children was met in 2004, but it is not clear from the ICR how many children received supplement by the end of the project . The national sexual and reproductive health program was implemented following the legal and institutional reforms. According to the ICR, more than 400,000 women in northern provinces were enrolled . No information on trends in utilization of family planning services was provided . Outcomes : According to the ICR, no measles case was reported in the country in 2004. Mother-to-child HIV transmission progressively decreased from 4.2% at in 2004 to 2.7% in 2006. No output and outcome data on tuberculosis was provided. Information on uptake of contraception (e.g., contraceptive prevalence rate ) would have been informative. The rating for this objective is substantial . Objective 2. Assist the Government of Argentina to modify the incentive framework for financing and delivery of health care services starting in Argentina's poorest provinces . Output and process indicators : At the start of the project consensus accord between national and provincial MoH was signed regarding the Federal Health Council’s (COFESA) role in leading and coordinating national -provincial health policy. The Council was fully functional throughout the project duration, exceeding the minimum meeting requirements set in the policy matrix. The achievements on National Health Insurance reform include : (i) updated beneficiary database functioning at health facility-level in at least three out of the five largest hospitals in at least five of the eligible provinces; (ii) a system for expediting cost recovery by public hospitals was developed and fully functional in at least five of the eligible provinces; (iii) a presidential decree was issued that eliminated the discretionary "institutional subsidies" by the Administration of Special Programs (APE) thereby increasing transparency and accountability; and (iv) after substantial delays an age and gender risk -adjusted capitation system in the Solidarity Redistribution Fund was introduced by Presidential Decree (in Dec 2006), thereby increasing transparency and reducing incentives for risk-selection by health insurers . Some of the achievements of the Maternal -Child Health Insurance Program mentioned under objective 1 had the effect of aligning incentives and facilitate improved and efficient service delivery to the MCHIP target group in the nine northern provinces . Examples include: the defining of a predefined benefit package of services for MCHIP beneficiaries; the signing of two thousand performance -based service contracts; by project closing 1,100 providers were billing the provincial MoH directly . Outcomes : An indication of the effectiveness of the rejuvenated COFESA is the fact that in 2004, 80% of total budget for milk and essential drugs from the national MoH to the provinces was allocated on the basis of the COFESA-agreed rates. The ICR remarks, although it is too early to measure the impacts of these National Health Insurance regulatory changes, their direction and content are technically appropriate . The rating for this objective is substantial . 5. Efficiency (not applicable to DPLs): Efficiency estimates are not required for adjustment loans, but a few comments on the efficiency of the reforms are worth noting. The chosen reforms explicitly addressed key sources of inefficiency in the health system and will likely contribute to improved technical and allocative efficiency . Examples are: performance-based service delivery contracts; defined health benefits package; improved efficiency of cost recovery systems; age and gender risk-adjusted per capita reimbursement . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Given the substantial relevance of the Program objectives, the substantial relevance of the Program design, and the substantial efficacy, the Program outcome is rated satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: At appraisal the Program faced external as well as internal risks . The external risks were mainly fiscal and related to the macroeconomic situation. The internal risks were two-fold: (1) historic difficulties in the national -provincial relationship; and (2) the implementation capacity of the provincial Ministries of Health . The macroeconomic picture stabilized, removing the serious fiscal concerns at the time of appraisal . Several of the internal risks have been mitigated upon effectiveness and during Program implementation . For example, (i) the accord signed between the national and provincial MoHs, and (ii) the fully functioning National Health Council with high level of provincial participation, did a lot to mitigate the first internal risk . The parallel lending operation (the Provincial Maternal-Child Health Investment Project APL-I and APL-II for US$135.8 million and US$300 million respectively) to a large extent mitigated the second internal risk . The history of difficult inter-governmental relationships in the health sector suggests that the national -provincial relationships will continue to impose some risk . The political economy of health sector reform is always challenging and a remaining challenge facing the Program is whether the enacted reforms will remain in place and continue to be operational. The delays in meeting the last condition was in part due to resistance from health insurers poised to lose from the reform. In the recent past there have been legal challenges to some presidential decrees (e.g., challenges to decrees pertaining to social security from players in the insurance industry ), suggesting at least some uncertainty regarding the future of the FSR risk -adjusted capitation reform. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at entry : According to the ICR, the appraisal time was short and was conducted under tense macroeconomic conditions. The quality at entry, rated highly substantial by QAG, was thanks to the high quality of ESW, detailed knowledge of the sector, the good relationship and policy dialogue with the sector leadership, and coordination during appraisal with key agencies (e.g., IMF and IDB). The Program objectives and design were fully owned by the borrower, again thanks to the quality of the policy dialogue . The conditions in the policy matrix were carefully selected and sequenced . Quality of supervision : The ICR commends the Bank’s frequency of supervision missions, and the technical quality of supervision, seeking to pre -empt blockages and give timely advice . The demand for the numerous follow-on operations is a testament to the project team's performance . at -Entry :Highly Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government performance : The letter of development policy carefully articulated the sector policy, an important input into the Program's design. The borrower carefully navigated the macroeconomic challenges and met the macroeconomic condition in the policy matrix, albeit after some delay . Implementing agency performance : Despite difficulties with one condition (the FSR risk-adjusted capitation reform), the Ministry of Health met, and in many instances exceeded, the conditions in the policy matrix . The successful revitalization of the Federal Health Council is largely due to the Ministry of Health leadership that emphasized transparency and effective provincial participation and ownership . The decision to introduce a new insurance program (MCHIP) and other structural reforms at a time when the country faced severe economic as well as political challenges was risky . Much of this risk was mitigated thanks to the direction provided by the national Ministry of Health leadership —noted in the ICR as being fundamental to the Program ’s success. The delay in achieving the FSR-related condition necessitated two Program extensions . Some weaknesses in monitoring and evaluation were noted in the ICR prompting the Bank to downgrade the project ’s IP rating to Moderately Satisfactory (between June 2005 and December 2006). a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: The Program Document included a carefully selected set of output and outcome indicators . The selection of indicators took into account the relatively short project duration (at appraisal the anticipated project duration was 26 months although actual duration was 41 months). The data collection for tracking the indicators was incorporated into existing information systems, thereby increasingly the likelihood of institutionalization . In the latter half of the Program there were some weaknesses in monitoring and evaluation (e.g., the delays in the baseline study ) resulting in the Program's Moderately Satisfactory IP rating . Overall, the Program indicators were tracked with reasonable diligence during Program implementation, providing the data to monitor outputs and evaluate outcomes, as evidenced in the ICR. a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The Program had a category C rating for environmental safeguard purposes . According to the ICR, there were no other reported fiduciary issues of significance . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Moderate Reasons: (i) the history of difficult Outcome : inter-governmental relationships and the critical importance of the progress in provincial-national relationships on the current MoH leadership, and (ii) the difficulty in achieving the FSR risk-adjusted capitation reform and uncertainties whether the enacted reforms will remain in place. Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The lessons below complement the lessons identified in the ICR : Ownership and leadership: As the ICR notes, crises provide windows of opportunity for reform . But, crises also carry a risk premium and under these conditions success depends critically on ownership at multiple levels of government and sustained and strong leadership by a reform champion . Selection of conditions: Good quality ESW and detailed knowledge of the sector are essential for careful selection of the conditions in the policy matrix in order not to be too onerous in the number of conditions imposed on the Borrower. Sequencing of conditions: The timing of conditions have to balance benefits and reforms in order to ensure that tangible benefits (protection of essential public health expenditures ) are felt early in the Program's lifespan by the stakeholders (in this case provincial governments ) whose support and cooperation was needed for Program implementation. 14. Assessment Recommended? Yes No Why? An assessment is recommended because the project design allowed the sector to address the short -term needs, while simultaneously investing in long -term reforms. Furthermore, this approach has been hailed by the 2007 HNP Sector Strategy (Box 6.2 in p74) as a way to achieve synergy between disease -specific (and mainly vertical) interventions and health system strengthening . An assessment is critical to understand how much of the Program implementation experience is generalizable . 15. Comments on Quality of ICR: The ICR does a good job at describing the Program achievements and provides a very positive picture of the implementation experience. While the ICR Review concurs with many of the conclusions, some reflection on the Program's areas for improvement would have provided greater opportunity for learning from the ICR . The ICR reports that IMR decreased from 18.3 to 12.9 per 1,000 live births between 2002 and 2007. Some discussion on the attribution of this phenomenal improvement to the Program inputs (as opposed to other factors, such as economic recovery ) would have been helpful. The ICR generally limited the data it reported to the indicators identified in the policy matrix . In several instances (as noted in Section 4) additional output and outcome data would have allowed the ICR to provide a clearer account of the Program's results chain . The decision to include the FSR risk -adjusted capitation condition can be viewed as risky, putting the release of the 3rd tranche at risk for what was a difficult condition . Alternatively, the importance of the 3rd tranche to the sector and the government, given the economic conditions, may have provided the leverage to bring about a difficult policy reform. Some insights on this critical issue could have been useful in the ICR . Minor comments: (1) A direct translation of the borrower's comments, rather a summary would have been useful . (2) Although this is not standard practice, it would have been helpful to cite the data sources for the results quoted in the ICR. a.Quality of ICR Rating : Satisfactory