ICRR 12569 Report Number : ICRR12569 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/09/2007 PROJ ID : P035827 Appraisal Actual Project Name : Women & Child US$M ): Project Costs (US$M): 422.4 433.8 Development (WCD) Country : India Loan/ Loan /Credit (US$M ): US$M): 300 287 Sector Board : HE US$M ): Cofinancing (US$M): 0 0 Sector (s): Health (91%) Pre-primary education (4%) Sub-national government administration (3%) Central government administration (2%) Theme (s): Nutrition and food security (23% - P) Child health (22% - P) Gender (22% - P) Education for all (22% - P) Participation and civic engagement (11% - S) L/C Number : CN042 Board Approval Date : 06/29/1998 Partners involved : None Closing Date : 09/30/2004 03/31/2006 Evaluator : Panel Reviewer : Group Manager : Group : Howard Nial White Kris Hallberg Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: 1: Improve the nutrition, health and psycho -social status of children, 0-6 years of age, with particular emphasis on preventing malnutrition in under threes, and improve child care practices at the household level; 2: Improve the nutrition and health of women, particularly pregnant and breastfeeding mothers, and adolescent girls; and 3: Empower adolescent girls, through increased awareness, to take better care of their own health and nutrition needs. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components (or Key Conditions in the case of DPLs, as appropriate): US$ 218. A. Service Delivery (appraisal : US$218 52 %; actual : US$245 218 .2 million, 52% 245 .61 million, 69%): US$ 245. 69 %): (1) Service Quality Improvement: Improving the quality of Integrated Child Development Services (ICDS) with particular emphasis on (a) the provision of supplementary feeding and nutritional supplements to children and pregnant and lactating women and the regular monitoring to detect malnutrition and promote growth; (b) immunization against six childhood diseases; and (c) promoting overall psycho-social development of children . (2) Adolescent Girls' Empowerment: Activities included the provision of iron supplements and deworming treatment . (3) Staffing and Infrastructure Development including construction, equipping and staffing of over 14,000 Anganwadi (village) Centers (AWCs). US$ 37. B. Program Support (appraisal : US$37 US$ 36. 36 .09 million, 10%): 37 .8 million, 9%; actual : US$36 10 %): (1) Management and Institutional Development: including study tours, workshops, quality improvement and preparation of new projects . (2) Capacity Building for Training: (a) training of staff of both new and old blocks; strengthening /establishing training centers and teams at the state and district levels; and (c) development of training materials. (3) Information, Education, and Communication (IEC). US$ 166. C. Central Component (appraisal : US$166 39 %; actual : US$76 166 .4 million, 39% 76 .7 million, 21%): US$ 76. 21 %): (1) Management Information System (MIS): including a computerized MIS with emphasis on monitoring and evaluation . (2) Nutrition Surveys: Activities included carrying out a baseline and endline nutrition surveys in the project states . (3) Reviewing Operational Research Priorities of the project states . (4) National Training for all states and union territories . As part of the restructuring of the project in December 2003 the Sub-component A.1 Service Quality Improvement was revised to improve targeting of children in the 0-3 age group and of pregnant and lactating women of the poor households and to strengthen the provision of regular nutrition and health counseling . Also, selected service delivery activities and construction of model AWCs were to be provided for additional states . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Both project start-up and closing were delayed by 18 months, the initial delay being caused by the time taken for government approval and ensuring adequate project staffing . At mid-term review there had not been much progress, and large savings were possible from low implementation and the devaluation of the rupee . The government proposed a restructuring to utilize these funds to strengthen ICDS service delivery in six other states . In consequence, the credit agreement was amended in December 2003, effective retroactively from November 1, 2000, for the project to include six additional states (Madhya Pradesh, Chhattisgarh, Bihar, Jharkhand, Orissa, and Uttaranchal) for the ICDS delivery and program support, and nine additional states and union territories (Karnataka, Gujarat, West Bengal, Haryana, Himachal Pradesh, J & K, Punjab, Andaman & Nicobar Islands, Pondicherry ) for construction of model AWCs buildings for the ICDS . As the close date approached project outcome was rated unsatisfactory on both implementation and development outcomes. However, management decided to waive the policy of not extending unsatisfactory projects, giving an initial 9 month extension subject to an action plan being prepared and a further 9 months conditional upon satisfactory implementation of that plan . There were two other amendments to the credit agreement : By an amendment made in June 2001, the project also supported the earthquake reconstruction activities in the state of Gujarat in the amount of SDR 46.4 million, of which SDR 16.5 million was cancelled from this credit and assigned to Tsunami -related reconstruction. Finally, by another amendment to the credit agreement in August 2004, SDR 17.7 million was allocated to the purchase of polio vaccines, which was supervised under a separate project . 3. Relevance of Objectives & Design: The overall goal of improving health and nutrition amongst women and young children was highly relevant given the low value of social indicators in much on India . Strengthening the quality of the existing ICDS program was the most sensible, i.e. relevant, approach to doing this, given that it already covered about 2/3 of the development blocks in the country at the time the program started . However, since the program included both poor and well - performing states, it was not entirely targeted to the most needy areas . 4. Achievement of Objectives (Efficacy): Assessment of the achievement of objectives is complicated by the lack of coverage of the outcome data . Not only were data not collected related to all three objectives (i.e. not at all for objective 3, and only for children aged 0-3 for the first objective which targeted those aged 0-6), but the data referred only to the five original focus states, not those included in the project at restructuring . Note that the funds re-allocated to earthquake reconstruction were supervised under a separate project and so are not covered by these ratings . 1: Improve the nutrition, health and psycho -social status of children, 0-6 years of age, with particular emphasis on preventing malnutrition in under threes, and improve child care practices at the household level : substantial . The indicator used to assess this objective is the prevalence of malnutrition among 0-36 month old children in original five focus states. The project target was for a 2 percentage point annual decline, equalling a 10 percentage point change over the project (this target was changed at MTR, the appraisal target being a 25% decline). Baseline (2000) and final (2006) values were UP: 58.1 down to 44.4; Rajasthan: 39.1 down to 35.4; Maharashtra: 51.8 down to 42.6; Tamil Nadu: 50.9 down to 39.5 ; Kerala: 10.8 down to 3.5. So the target was achieved in two of the states, and very nearly achieved in a third . Achieving the target in Kerela implied the virtual eradication of malnutrition, so its reduction to 3% can be taken as also achieving the target . Hence, only one state, Rajasthan, failed to substantially achieve the target . Data on outputs establish a plausible contribution of the project toward these outcomes : 85% of the planned 20,690 AWCs were constructed; 99% of AWCs (old and new) provided supplementary feeding and 97% received scales to carry out growth monitoring. Over 8,000 handpumps were installed (68% of the target number). The proportion of children consuming Vitamin A rich food increased from 43% to 73% during the project, and the proportion of women initiating breast feeding within 2 hours of birth increased from 35% to 64%. However, the government continued to attempt universal coverage, rather than the more targeted approach proposed by the Bank. Not much more than half of AWCs actually conducted growth monitoring (58% up from 52% at baseline). 2: Improve the nutrition and health of women, particularly pregnant and breastfeeding mothers, and adolescent girls : modest . The proportion of children born with low birth weight dropped slightly from 20.6% in 2000 to 19.0% in 2005, a reduction in less than 8% compared to the target of 25%. However, the quality of data is questionable (less than half of children are weighed at birth), and the indicator was dropped at the MTR . Additional outputs supporting this objective included an increase in the proportion of women consuming at least 100 IFA tablets in the last pregnancy rising from 9 to 33%; 85% of women with at least three ante-natal visits (baseline data not available); 3: Empower adolescent girls, through increased awareness, to take better care of their own health and nutrition needs : not rated . There was no outcome indicator linked to this objective . The project design summary in the PAD included the following two indicators: (a) Proportion of adolescent girls covered by awareness program, and (b) % of girls marrying after 18th birthday; but these are not reported on in the PAD, nor are any associated outputs . 5. Efficiency (not applicable to DPLs): In the end the project expenditure was slightly greater than planned, but implementation levels fell short of those targeted, despite the local savings made available by devaluation of the rupee . However, the ICR states the cost of construction and of handpump installation were competitive by international standards . On the other hand, efficiency was to be improved by targeting supplementary feeding to poor and growth faltering children, but this was not done, feeding being provided to all children aged 0-6 attending the AWC. The ERRs were calculated for the supplementary feeding component only (20% at appraisal and in the ICR). Its cost share cannot be stated as it was merged with other components . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: As noted in section 4, outcome indicators were limited in both scope and coverage . Those indicators which were collected showed uneven progress . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Government's commitment to the ICDS program is strong as demonstrated by the 30 years' expansion and the increased funds for the program in the next Five -Year Plan to finance both a sustained expansion of the program and half the cost of the Supplemental Nutrition Program . Community support is also growing as evidenced in the provision of land for the construction of AWCs and organization of mother's groups adolescent clubs in some blocks . ICDS figures prominently in the preparation of the next Five -Year Plan (2007-2011). Moreover, the DWCD of the Union Ministry of Human Resource Development was recently upgraded to a full -fledged Union Ministry of Woman and Child Development (MWCD). However, the development outcome achievement of the current program structure is modest . There are risks to sustaining and more importantly accelerating outcomes achieved under the program . Key areas of attention are further strengthening of institutional capacities and essential reforms to the ICDS delivery mechanism . There may be a sharp trade-off between the expansion of the number of ICDS blocks, the commitment to an integrated program of child health and nutrition and early childhood education; and making the program more effective in addressing malnutrition. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Bank performance at the early stages of project preparation assistance was very good . It engaged the GOI and participating states in policy dialogues, and encouraged their active participation and ownership of the project through social assessments . However, Bank performance at appraisal and negotiations and in assuring project readiness was unsatisfactory . The nation-wide training component, which accounted for about a third of total project costs, was included in the project after the return of the appraisal mission and without proper appraisal . The baseline data were not assessed and therefore targets for the monitoring and evaluation indicators and Project Implementation Plans could not be defined at negotiations . Such actions delayed Borrower's completion of the Project Implementation Plans, project effectiveness, and early start up of the project implementation . Regarding supervision, the project can be divided into three periods . The first was before restructuring at Mid-Term Review but after project effectiveness . The second was between MTR restructuring and just prior to the original closing date. The final period was just prior to and during two project extensions, in which implementation was dramatically improved. In the first two periods, the Bank carried out supervision missions regularly every six months with well balanced staff-specializations. However, there was a long delay in project startup and significant weaknesses in promoting effective M&E and innovative activities . Bank supervision did not adequately respond to these challenges so that the initial phase can be characterized as unsatisfactory . In the last period, the Bank not only intensified its supervision efforts, but took a risk -taking and professional approach, which can be characterized as satisfactory . at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: During project preparation, the central government's leadership and state governments' active participation was exemplary. Ever since the beginning of the project preparation, GOI, in particular the Department of Woman and Child Development (DWCD), took the ownership in the processing of the credit and implementation of the project by guiding and stimulating potentially interested states . However, the Central Government's implementation record was marred by the late start -up of the project. Completion of the Project Implementation Plans took longer than planned, DWCD obtained Cabinet approval of the project nine months after Board approval, and the administrative order for activating the Central Project Management Unit (CPMU) was issued more than two years after Bank approval. During the period, State Project Management Units (SPMUs), which were established earlier, became inactive and required more time to reappoint staff or reactivate them . The first half of the project implementation suffered from other inefficiencies as well . CPMU never reached full staffing, project funds to states were not released on time, procurement and disbursement problems were rampant in all project states, supplementary feeding was not targeted, and the health and nutrition counseling was not conducted regularly . Most of those problems could have been prevented or resolved by learning lessons from the previous two ICDS (WCD) projects of the same nature and modality . Moreover, staff turnovers at both central and state levels were so frequent that project implementation was disrupted and delayed . During the second half of project implementation, GOI made extraordinary efforts to complete the project efficiently. Funds were released more regularly, and monitoring and evaluation systems were established at both central and state levels, However, it has never been able to make up for the time lost in the first half of the project life, and the project was completed with an 18-month delay. The only comfort is that most of the project outputs and outcomes set at appraisal were achieved . Both CPMU and the SPMUs performed satisfactorily during this second period. a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: A. Design. Design of the indicators to monitor the progress of project's implementation and evaluate the achievem ent of project's development objectives was defective from the beginning . Monitoring and evaluation indicators were defined mostly in terms of percentage improvements from the baseline, and the baseline data had yet to be obtained through surveys after the credit became effective . Most indicators selected were to measure either inputs or processes (such as percentage of pregnant women registered ). Therefore, the linkage between the input and process indicators and the development outcome indicators has to be made without much help of output indicators . The situation improved considerably after the project restructuring when some output indicators were adopted . However, the outcomes monitored provided only partial coverage of the objectives . B. Baseline Survey. The baseline data survey in each of the five original project states was carried out only in the period 2000-2001. Some selected monitoring indicators were regularly collected by AWC workers and were reported upwards throughout the ICDS administrative hierarchy . However, the compiled data were rarely analyzed or disseminated for policy-making or operational uses. Data collection did not take place in the states added to the project at restructuring. C. Endline Survey. The endline survey was carried out in each of the five original project states by October 2005. Hence, the project ultimately produced both baseline and endline data during the project life, a good practice . However, no control groups were selected in the baseline surveys for comparison with non -project beneficiaries. D. Quarterly Progress Reports (QPRs). After the substantial delays and weak monitoring and evaluation activities at the earlier stage of the project implementation, the CPMU devised a useful format for monitoring SPMUs' project implementation and reporting it to the Bank on a quarterly basis . Although the report placed more emphasis on the physical and financial operations of the project, it also contained information on a few important inputs, processes, and outputs of the project. This report also served as a useful communication tool from states to the central government and to the Bank. However, the QPR lacked important information on the costs and financing of the project by project component and by the source of financing . Another important weakness of the QPR was the lack of monitoring on ICDS's coverage of the targeted women and children living in each block, district and states and on the progress made in the behavioral changes made by the women, adolescent girls, and children who received health and nutrition educational/counseling services. The same statement can be made on the coverage of the preschool component . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Project implementation suffered from financial management, and often was not in compliance with the Bank fiduciary requirements. This is one reason why the project was flagged as a problem project . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR draws the following general lessons : A. The Bank and the government should agree on the timing of government approval of a Bank -assisted project. The Bank should not go to the Board before the Cabinet approves the project proposal, so that the project can become effective soon after Board approval . B. Lessons learned from previous projects, both positive and negative, should be rigorously analyzed and applied to the subsequent project design, and a new initiative should be pilot tested on a small scale . The health and nutrition aspects of the ICDS program have not been adequately implemented under the current project due mainly to the one worker and one helper model of the AWC, although alternatives have been demonstrated in other projects. Also, this project, like ICDS I and II projects, suffered from serious problems with project funds release, procurement, and disbursement . GOI and the Bank should have anticipated such problems since this project was a third operation of the same nature and modality in the same WCD sub -sector but on a larger scale . C. The quality at entry of the project profoundly affected efficiency of project implementation . The inclusion of a nation-wide training component without proper appraisal and the lack of defined monitoring and evaluation indicators at negotiation delayed the start -up of the project and adversely affected the subsequent project implementation processes. Baseline data should be collected early so that it can serve as a basis for preparing and agreeing on Project Implementation Plans . E. In addition to input and process indicators, more output indicators (e.g. proportion of mothers who changed their child caring behaviors) should be used, so that the input and process indicators can be more persuasively linked to project outcome indicators . F. The weak financial management and disbursement documentation were important causes of project implementation delays. Some audit disallowances have still not been resolved . An in-depth review of the processes, streamlining them, and making arrangements for training of local staff in these aspects are essential before continuing with another Bank operation . G. A specific investment project is not an optimal tool for meeting the varied needs and capacities of different states in a large federated nation as India . Once the Government's project preparation and implementation capacity is increased and experience accumulated, a program loan approach would be preferred with different roles played by the central government, states, and the Bank . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR gives a balanced and comprehensive review of the project . More information on the following aspects would have been useful: (a) the extent to which audit observations were related to malpractice and, if so, what actions were taken, (b) although critical of project indicators, the ICR could be yet more critical of the failure to collect outcome data related to all objectives and to collect M&E data for the project in the new states added at restructuring . a.Quality of ICR Rating : Satisfactory