ICRR 12481 Report Number : ICRR12481 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/21/2006 PROJ ID :P006059 Appraisal Actual Project Name :Ar-maternal And Child Project Costs 171.0 134.5 Health And Nutrition 2 US$M ) (US$M) (promin) Country :Argentina Loan/ Loan US$M ) /Credit (US$M) 100.0 100.0 Sector (s):Board: ): HE - Health (95%), US$M ) Cofinancing (US$M) 0 0 Central government administration (3%), Pre-primary education (2%) L/C Number :L4164 FY ) Board Approval (FY) 97 Partners involved : None Closing Date 12/31/2003 12/31/2005 Evaluator : Panel Reviewer : Division Manager : Division : Howard Nial White Kris Hallberg Alain A. Barbu IEGSG 2. Project Objectives and Components a. Objectives Objective one : substantially increase the provision of maternal and prenatal care services; Objective two : improve the provision of pediatric services for children aged 5 or younger; Objective three : transform a substantial number of existing child feeding centers and kindergartens into early childhood development centers; and Objective four : decentralize and strengthen social service arrangements . b. Components (or Key Conditions in the case of Adjustment Loans ): Original components Component A - Subprojects (Appraisal: US$ 77.9 million, 46 percent; Actual: US$ 71.8 million, 53 percent). Subprojects in the Provinces in the areas of Maternal and Child Health and Nutrition and Early Childhood Development, as well as the provision of technical assistance to support the preparation and execution of subprojects at the Provincial and Municipal levels . This project component included tasks in the following six large topics: (i) women’s reproductive health care; (ii) child health care; (iii) targeted food supplementation; (iv) health and nutrition education and promotion; (v) organization and delivery of basic package of maternal and child health and nutrition services; and (vi) improvement of local health service networks . The Early Childhood Development dimension included : (i) the transformation of pre-school feeding programs, child care centers and kindergartens into child development centers offering comprehensive care nutrition assistance for the development of children 5 years of age or younger; (ii) the training of teacher, mothers and other staff employed in child development centers; and (iii) the improvement or establishment of coordination between local health facilities and the child development centers and the promotion of outreach activities to strengthen linkages with household and the local community . Finally, the provision of technical assistance aimed at : (i) the implementation of integrated models of health care delivery by the Provinces and Municipalities; (ii) the establishment and consolidation of multisector teams (health, education and social action ) within the Provinces and Municipalities; (iii) the transfer of the methodology of the Project to other projects of MSAS, especially in the areas of targeting, intersectoral coordination, preparation, management, monitoring and evaluation of subprojects; and (iv) the improvement of the efficacy and efficiency of the implementation of the milk for pregnant women and malnourished infants program known in Argentina as Daci ón de Leche Program. Component B - Strengthening the statistical system of the Borrower ’s Ministry of Health and Social Action MSAS ). (Appraisal: US$ 5.0 million, 3 percent; Actual: US$ 1.3 million, 1 (Ministerio de Salud y Acci ón Social - MSAS) percent). The strengthening of MSAS’s Statistical System included upgrading of the system design, provision and use of computer hardware and software and training of personnel at the provincial and national levels . Component C - Project Administration . (Appraisal: US$ 5.4 million, 3 percent; Actual: US$ 10.6 million, 8 percent). The coordination and supervision of the carrying out of Components A and B mentioned above at the national, provincial and municipal levels. A Project Coordination Unit (PCU) within the MSAS was established at the national level and provincial (UEP) and municipal (UEM) executing units were created in the participating jurisdictions . Two components added in the wake of the crisis (but objectives not revised ) Component D - National studies (Appraisal: US$ 0, 0 percent; Actual: US$ 3.9 million, 3 percent). To (i) undertake a nutrition survey at the national level; (ii) carry out a baseline and impact evaluation studies; and (iii) carry out a special maternal, child health and early childhood development studies . Component E - Protection of the most vulnerable (Appraisal: US$ 10.4 million, 6 percent; Actual: US$ 34.6 million, 26 percent). Support for priority public health interventions, including immunization, disease control and ensuring services related to women’s reproductive health care and prenatal controls, such as anemia prevention, child delivery, newborn reception, pot -partum and child health care, through the provision of pharmaceutical products, medical supplies and iron-fortified milk Note: Percentages do not add to 100% as there were US$ 20 million unallocated at appraisal, and recurrent expenses (US$ 52.3 million at appraisal and US$ 12.3 million actual) were not allocated to specific components . c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The Bank's loan of US$ 100 million was fully disbursed. However, the total project cost was US$ 135 million, compared to the appraisal estimate of US$ 171 million. The reasons for the shortfall were (1) devaluation reducing the dollar value of recurrent expenditures, (2) government fiscal constraint with the onset of the crisis, and (3) some of the counterpart costs shifting to the United Nations Development Fund . Two extensions of one year each were granted to allow full utilization of funds . These were allowed in recognition of the fact that the crisis constrained implementation and the successful use of project resources in crisis response . 3. Relevance of Objectives & Design : The objectives were stated at the levels of outputs rather than outcomes . However, the original project design matrix linked these outputs to maternal and child health outcomes . These targets were very relevant to the country's health needs, and remain so to the extent that there are still gaps in coverage . The design was appropriate, including the focus on restoring the health data systems, which was meant to have taken place under PROMIN I, and the emergency response to the crisis . 4. Achievement of Objectives (Efficacy) : Objective one : substantially increase the provision of maternal and prenatal care services : Achieved . The number of pregnant women in the poorest income deciles receiving maternal prenatal care increased from 44 percent at the start of the project to about 85.8 percent in 2005 (compared to the appraisal target of 60 percent) Objective two : improve the provision of pediatric services for children aged 5 or younger : Achieved (i) The number of poor children aged 0-6 years-old in the lower income deciles receiving health care increased from 51 percent at the end of the nineties to 72 percent in 2005 (compared to the appraisal target of 60 percent) (ii) The number of poor children aged 0-6 years-old in the lower income deciles fully vaccinated increased from 65 percent at the start of the project to 90.8 percent in 2005 (compared to the appraisal target of 60 percent). Objective three : transform a substantial number of existing child feeding centers and kindergartens into early childhood development centers : Achieved The project financed 1,362 health facilities and child development centers (compared to appraisal target of 1,250). The subprojects financed 367 civil works, of which 119 were for early childhood development centers . Objective four : decentralize and strengthen social service arrangements . Partially achieved (i) A health delivery model was adopted in PROMIN facilities covering supervision, training, referrals and follow -up. This model has been adopted in some non -PROMIN facilities. (ii) The project strengthened the Provincial Maternal and Child Health Directorates of participating jurisdictions . (iii) The national and provincial health statistics systems were revived and are now fully operational . 5. Efficiency : The project objectives were achieved, with targets being surpassed, within budget . However, whether these were efficient estimates or not would require cost effectiveness analysis (or better still cost benefit analysis ) of at least a subsample of subprojects. There is nonetheless evidence that the project was less efficient than it could have been, e.g., on pp. 12-13, "...less than desirable provision of appropriate health equipment ..." and "lack of a policy, strategy, and resources at both the national and provincial levels for preventative and corrective maintenance to ensure quality service...". 6. M&E Design, Implementation, & Utilization: The PAD included a set of clearly defined indicators, with outputs from project data and outcomes which it should have been possible to monitor through the surveys supported by the project . However, these indicators were not being reported on in the Project Status Reports until June 2003, at which time the indicators were reformulated . Under this reformulation, selected outputs from the original set of KPIs were elevated to the status of outcome, and no actual outcome measures were retained . This change may be defensible on two grounds . First, the project objectives themselves referred to outputs, but this is a shortcoming in the statement of objectives . Second, monitoring systems should go to the output level, and that outcomes are a matter for impact evaluation . The ICR states that its findings are based on the impact evaluation studies, but also says that "no proper impact evaluation could be conducted". Hence the M&E system allows for outcome monitoring, but most likely does not permit attribution. Projects such as this one should include cost -benefit analysis of a sample of sub -projects, which was not done. 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Effective crisis response is possible, and can be more rapid if built upon existing institutions and on -going portfolio projects . Working within the framework of existing organizational and institutional arrangements and programs such as PROMIN, VIGIA and UFI-S allowed for the quick consolidation of a knowledgeable and skilled team and the improvement of institutional capacity to face unforeseen challenges that occur in such a major crisis . The selection of key personnel able to understand the intricacies of various governmental agencies and work effectively with all Bank regulations and guidelines was central to the achievement of the objectives of the program . Set up appropriate evaluation criteria at the outset . PROMIN II lacked an appropriate impact evaluation scheme to make inferences about the contribution or causal link between project activities and outcome indicators . It is highly recommendable in future operations to ensure from the onset of project implementation : (i) a statement of expected developmental objectives; (ii) agreement with the Borrower on a set of impact indicators; (iii) agreement on impact evaluation methodology; (iv) a baseline data; (v) agreement on a collection protocol and an algorithm to analyze and periodically update the impact indicators information; (vi) agreement on the periodicity of updating the information; and (vii) agreement on a strategy to disseminate impact findings . 10. Assessment Recommended? Yes No Why? Crisis response. 11. Comments on Quality of ICR: The ICR is weak in its treatment of the objectives and the reporting of indicators : (1) The objectives are not dealt with in a systematic manner . Indeed, the section discussing objectives makes no explicit mention of the fourth objective, and deals only indirectly at best with the third objective . In addition: (2) The ICR is incorrect to state that the ERR is inapplicable to this project : it can and should be calculated for a sample of subprojects. (3) The ICR is contradictory about impact evaluation, stating that its findings are based on these studies, but also that no proper impact evaluations were conducted .