ICRR 13899 Report Number : ICRR13899 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/13/2012 Country : Madagascar Project ID : P001568 Appraisal Actual Project Name : Second Community US$M ): Project Costs (US$M): 30.25 75.55 Nutrition Project L/C Number : C3060 Loan/ US$M ): Loan /Credit (US$M): 27.60 52.61 Sector Board : Health, Nutrition and US$M): Cofinancing (US$M ): Population Cofinanciers : Board Approval Date : 04/21/1998 Closing Date : 01/31/2004 07/31/2011 Sector (s): Health (70%); Primary education (19%); Central government administration (10%); Agricultural extension and research (1%) Theme (s): Nutrition and food security (40% - P); Participation and civic engagement (40% - P); Child health (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Judyth L. Twigg Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: According to the Development Credit Agreement (DCA, not paginated), the project’s objectives were “to improve the nutritional status of children under three, primary school children, and pregnant and lactating women, and ensure long-term sustainability of nutrition outcomes by improving the quality and quantity of food intake by children at home; and, in so doing, improve quality of life and productivity, decrease child morbidity and mortality rates, and support primary education.� The statement of objectives in the Project Appraisal Document (PAD, p. 2) contains only the first part of the objectives as worded in the DCA: “to improve the nutritional status of children under three, primary school children, and pregnant and lactating women in the project target areas . It also aims to ensure long-term sustainability of nutrition outcomes by improving the quality and quantity of food intake by children at home .� Because the latter part of the statement of objectives in the DCA (quality of life, productivity, morbidity and mortality rates, and education) involves longer-term goals to which nutritional status is only one contributor, this review will assess the objectives as stated in the PAD . The project was a follow-up to the Food Security and Nutrition (SECALINE) Project, US$ 21.3 million, 1993-1998, whose objectives were to reduce food insecurity and malnutrition in Madagascar ’s two most food-insecure provinces. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project contained four components (the costs provided for each component specify the Bank contribution only, as the ICR does not provide total project costs by component ): The project team explained that total project costs by component are difficult to assess because the Government had its own National Program, the elements of which did not have a one-to-one fit with the project components . 1. Community nutrition program (appraisal, US$ 14.97 million; supplemental financing of US$ 9.0 million; actual, US$ 20.15 million). This component was to reach 100% of the target population in districts with the highest malnutrition rates in the six project provinces . Nutrition centers were to be established in participating villages, which would elect a community nutrition agent (CNA) to be responsible for all nutrition interventions . The CNAs, assisted by social workers, were to be responsible for promoting community mobilization and developing community -based activities to improve nutrition, hygiene, and sanitation in their villages . The project was to support training for CNAs and social workers, growth monitoring and food supplementation for children, nutrition awareness and education sessions, vitamin A supplementation for children and lactating women, incentives for community -based activities that addressed nutrition issues, and awards to the most dynamic nutrition centers . 2. School nutrition program (appraisal, US$ 5.07 million; supplemental financing of US$ 1.66 million; actual, US$ 3.00 million). This component, implemented by the Directorate for Primary Education of the Ministry of Secondary and Basic Education (MINESEB), was to support distribution of micronutrients, anti -helminth treatment (for parasitic worms), teacher training, promotion of improved nutrition and hygiene practices among children and their parents, monitoring of consumption of iodized salt, incentives for school -based activities to improve nutrition and hygiene in the school environment, and awards to the most dynamic schools . 3. Intersectoral support (appraisal, US$ 0.68 million; supplemental financing of US$ 2.46 million; actual, US$ 1.84 million). This component, implemented by the Ministry of Health and the Food and Agriculture Organization (FAO), was to finance: (i) training on the Integrated Management of Childhood Illness (IMCI); and (ii) preparation of technical guidelines on improved agricultural techniques and their dissemination to a few selected districts where the Ministry of Agriculture, with the assistance of the FAO, was implementing a small -scale food security project . 4. Information, Education, and Communication (IEC), training, and project management (appraisal, US$ 6.88 million; supplemental financing of US$ 3.88 million; actual, US$ 27.63 million). This component was intended to : (i) design and implement an IEC and training strategy to strengthen IEC capabilities for community nutrition workers, social workers, teachers, and health staff; (ii) establish a management information system to monitor project results and outcomes; and (iii) support project management functions and institutional coordination . The components were not formally revised during implementation, but at approval of a second additional financing in 2006, some changes were introduced . Under Component 1, the age range of children eligible to benefit from the community nutrition program was raised from under three to under five . Under Component 2, responsibility for implementing the school nutrition program was transferred to the Ministry of Education . Under Component 3, financing of intersectoral activities was replaced by support for the institutionalization of a National Nutrition Policy (NNP) and was to be used for: (i) strengthening the National Nutrition Council ’s ability to carry out its policy setting and strategy definition role; (ii) improving the capacity of the National Nutrition Office (NNO) to perform its mandates of intersectoral coordination, monitoring and evaluation (M&E), research, and resource mobilization; (iii) increasing the capacity of selected executing agencies to define and manage nutrition prevention and mitigation activities; and (iv) developing and implementing the national M&E system for nutrition activities, as well as the National Food and Nutrition surveillance system, including the early malnutrition warning system . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost: Differences between costs at appraisal and actual costs are due to exchange rate gains between the SDR and the dollar. Substantially less than planned (of Bank funds) was spent on Component 2, and more than planned on Component 4. The ICR does not explain these differences between planned and actual Bank costs, nor does it provide total project costs by component . The project team explained that less than planned was spent on Component 2 because of the transfer of the school nutrition program to the Ministry of Education, and that more than planned was spent on Component 4 because of an increase in coverage of sites that entails higher costs for training of CNAs and for M&E . Financing: The original credit was modified on December 1, 2003, to provide a supplementary credit of US$ 10 million intended to strengthen fiduciary arrangements, support the scale -up of nutrition interventions to additional districts, and extend the project’s closing date by one year to December 31, 2005; in December 2005, the closing date was again extended by one year to December 31, 2006. Importantly, this supplemental credit was also intended to finance a transition period needed to transfer project activities to a national program, with the introduction of program budget support in place of project financing . On November 22, 2006, a second additional financing of US$ 10 million was approved to further extend the project ’s geographic focus to all 110 districts, expand coverage to include children under five (rather than children under three), and adapt the program to the government ’s new harmonized national community program . This additional financing also extended the project ’s closing date from December 31, 2006 to December 31, 2008. Following civil unrest that began in December 2008 and a change in government in March 2009, the Bank’s portfolio in Madagascar was subject to OP /BP 7.30 (as of March 17, 2009), and disbursements were put on hold until November 2009, when they resumed for certain key projects (including this one) on humanitarian grounds. At this time, a retroactive extension of the closing date was approved on an exceptional basis, from December 31, 2009 to July 31, 2011. The ICR makes reference to co-financing by the World Food Program (WFP), but the only detailed information is contained in Annex 8 of the ICR (Comments by Co-financiers), where WFP describes a US$ 12.2 million contribution from 1998-2004 in the form of fortified blended food for distribution to underweight children ages 6-36 months and pregnant/lactating mothers. The project team clarified that WFP was not a formal co -financier of the project, but instead financed parallel activities . According to the WFP document provided by the project team, WFP provided a total of US$ 19 million in food supplies from 1999-2003. Borrower Contribution: There was a US$ 1.67 million planned Government contribution . The ICR’s Annex 1 on Project Costs (p. 34) cites an actual contribution of US$ 21.96 million, but it uses unexplained acronyms when citing this contribution; the sources and uses of this financing are not clear . The project team referred to Annex 10 (p. 58) of the ICR for the accurate total Government contribution : 8.1 million Ariary (US$ 3.87 million). Dates: The project’s closing date was extended multiple times, as described above . 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial . The objectives were substantially relevant to country conditions at appraisal . At that time, Madagascar had some of the highest rates of chronic and severe malnutrition in Africa . These rates were linked to poverty, insufficient food production, inadequate knowledge of feeding practices, and high prevalence of infectious diseases such as diarrhea. Micronutrient deficiencies were also highly prevalent, including iron deficiency anemia, Vitamin A deficiency, and helminthes. The objectives are also substantially relevant to Madagascar ’s development strategy for 2007-2012, the Madagascar Action Plan (MAP), which contains an explicit goal of reducing malnutrition in children under age five, and the Bank ’s Country Assistance Strategy (CAS, FY 2007-2011), current at project closure, which explicitly includes child nutrition under its second pillar of improving access to and quality of services . The objectives also substantially support the country ’s National Nutrition Policy (2004) and National Plan of Action for Nutrition Policy in Madagascar (2005). b. Relevance of Design: Substantial . The project’s results chain (PAD, pp. 21-23) plausibly connects the project ’s development objectives to its planned activities, outputs, and outcomes . For example, the community nutrition workers elected and trained under Component 1 would be expected to oversee nutrition interventions at the village nutrition centers; at those centers, children under three would be monitored annually, rehabilitation would be provided for those who were severely undernourished, and supplemental feedings would be provided to those not gaining weight appropriately . Through these Component 1 interventions, the percentage of children under three who were underweight – a key performance indicator – would be reduced, meeting the objective of improving the nutritional status of children under three. Potential exogenous factors that could interfere with this results chain, such as resistance to behavior changes due to strong cultural beliefs, were adequately anticipated and mitigated (in the case of cultural factors, through subcomponents involving sensitization of local village authorities and incentive programs ). 4. Achievement of Objectives (Efficacy): Though this Review is not assessing progress on the longer -term goals toward which nutritional status is only one contributor, the ICR did report data on some of these indicators . Infant mortality fell from 84/1,000 in 2001 to 49/1,000 in 2009, and under-five mortality fell from 159/1,000 in 1997 to 72/1,000 in 2008. Improve the nutritional status of children under three is rated Substantial . In 2006, the target population was modified from children under three to children under five, but the project development objectives were not revised . Outputs : 1,337 communities implemented nutrition-related activities, exceeding the target of 1,200. The number of community nutrition agents (CNAs) and social workers trained annually in nutrition techniques and IEC increased from 969 in 1998 to 6,330 in 2011, meeting the target. 17 brochures, 15 posters, and 17 sets of training materials were developed and disseminated by 2001, meeting the target. The ICR notes that the ability levels of the CNAs decreased after 2005, resulting in less effective use of training materials and lower satisfaction levels among beneficiaries (p. 23). The number of children under three enrolled in a growth monitoring and promotion program increased from 182,000 in 1998 to 955,004 in 2010, partially reaching the target of 970,000. The number of women involved in growth monitoring and promotion and IEC activities increased from under 100,000 in 2000 to 879,294 in 2010, exceeding the target of 679,700. 7,257 severely malnourished children were adequately rehabilitated at hospitals and health centers in 2010, exceeding the target of 2,700. Outcomes : The percentage of children under age three who are underweight decreased from 36.1% in 1999 to 17.8% in 2011, exceeding the target of a 20% decrease. The percentage of children under age two who are underweight decreased from 25.5% in 2006 to 15.8% in 2011. The percentage of children under age five who are underweight decreased from 26.5% in 2006 to 18.5% in 2011. Impact evaluations comparing communities supported and not supported by the project, as well as before and after initiation of the project, allowed for attribution of observed results to project -financed interventions. This study showed a decline in moderate underweight prevalence in children under age three from 47.9% in 1997 to 34.2% in 2007 for project sites, against a less marked decline from 44.2% in 1997 to 39.6% in 2007 for non-project sites. Econometric analysis in the evaluations indicated that 26-37% of the observed decline in malnutrition rates between 1997 and 2004 could be attributed to the project . The methodology of these impact evaluations appears to be sound . The number of children under age three receiving Vitamin A supplementation increased from 54,616 in 2000 to 1.01 million in 2010, exceeding the target of 670,000. However, 603,000 children received Vitamin A supplementation in 2011; the ICR does not explain this decline . The percentage of young children receiving Vitamin A supplementation increased from 4.2% of children 6-35 months in 1997 to 72.2% of children 6-59 months in 2008. This indicator is a proxy for Vitamin A deficiency, which was considered to be too costly and impractical to measure . Improve the nutritional status of primary school children is rated Modest . Outputs : 27,084 schoolteachers were trained in nutrition syllabi and IEC in 2005, exceeding the target of 6,000. From 2004-2006, 106 school-based activities annually were designed and implemented to improve nutrition, hygiene, and sanitation, not reaching the target of 1,000/year. The percentage of schools involved in school nutrition activities increased from 48.7% in 2003 to 100% in 2010. The number of schools testing iodized salt used in the households of school children was 11,222 in 2007, exceeding the target of 9,000. Outcomes : The project’s results framework does not include outcome data related to the nutrition, hygiene, and sanitation training and activities for teachers and schools serving primary -age children. Also, there are many factors outside the activities of this component that could impact the nutritional status of school -age children. The number of school children receiving weekly iron supplements increased from 136,069 in 2000 to 1,200,675 in 2009, partially reaching the target of 1.5 million. This indicator is a proxy for iron deficiency anemia, which was considered by the project to be too costly and impractical to measure . The number of children ages 3-14 (both enrolled and not enrolled in school ) receiving anti-helminth treatment increased from 595,777 in 2000 to 2,653,982 in 2020, partially reaching the target of 3 million. This indicator is a proxy for helminth infection, which was considered to be too costly and impractical to measure . Improve the nutritional status of pregnant and lactating women is rated Substantial . Outputs /Outcomes : The number of lactating women receiving Vitamin A supplementation increased from 11,220 in 2000 to 127,045 in 2011. This did not meet the target of 430,000, but the ICR Data Sheet (p. iii) states that this target was unrealistic, particularly when compared to the analogous Ministry of Health target of 110,000 women/year. The percentage of lactating women receiving Vitamin A supplementation increased from 19% in 2004 to 43.1% in 2008. This indicator is a proxy for Vitamin A deficiency, which was considered to be too costly and impractical to measure . According to the DHS, the percentage of pregnant women receiving iron supplementation increased from 33.0% in 2004 to 43.0% in 2008. The percentage of pregnant women suffering from anemia declined from 46.0% in 2004 to 35.3% in 2008. Ensure the long -term sustainability of nutrition outcomes is rated Substantial . In addition to data provided above on community nutrition-related activities and school -based activities: Outputs : The National Nutrition Office (ONN) and National Nutritional Council (CNN) became fully operational, together with 22 Regional Nutritional Offices. All have adequate human, material, and financial resources to carry out their mandates. A National Nutrition Monitoring and Evaluation System was operational by 2008 and continues to function. 111 trainers and 303 health staff were trained in IMCI, partially meeting the targets of 160 and 384. Outcomes : The most recent impact evaluation indicates that groups participating in project -sponsored activities scored significantly higher than a “never participant� group on enhanced knowledge of nutrition practices, use of exclusive breastfeeding for the first six months, and knowledge of proper meal preparation practices (ICR, p. 24). The ICR does not provide specific data on these indicators . 5. Efficiency: Substantial . The PAD (pp. 15, 31-36) contained an economic analysis supporting the statement that “cost-effectiveness considerations have played an important role in the selection of the proposed interventions and their target groups .� A detailed study of the cost per intervention and cost per beneficiary was conducted before appraisal (summarized at PAD, p. 35). Micronutrient supplementation was selected because it is among the most cost -effective of all health programs, costing less than US$ 50 per disability-adjusted life-year (DALY) gained. Treatment of helminth infections among children ages 3-14 is seen as cost-effective because it reduces infection rates not only among those directly treated, but also among the rest of the population by reducing transmission . Food supplementation, a relatively costly intervention, was provided to children under three and pregnant women rather than children under five because of cost-effectiveness considerations (PAD, p. 15). The ICR (p. 42) cites a 2007 analysis completed by the Bank estimating the net present value of reduced cost of illness and productivity gains of reduced stunting and increased ability at US$ 377 (at discount rate 5%) or US$ 653 (at discount rate 3%). The ICR (p. 42) also cites a finding by a 2004 impact evaluation of the benefit -cost ratio for the community nutrition program at 1.7 for children ages 0-5 and 2.7 for infants less than one year old . The community nutrition program efficiently focused on promoting behavior changes among entire communities, rather than only on nutritional supplements for individual children . As behavior is changed, feeding programs become less necessary, reducing long -term costs. According to the ICR (p. 26), efficiencies were also achieved through : (i) the organization of nutrition sites by community to avoid overlapping presence with non -governmental organizations; (ii) the application of relatively straightforward measurement concepts to help mothers understand the progress of growth of their children; (iii) the use of spot-mapping to help communities visualize their participation in project activities; and (iv) the introduction of mobile phones for CNAs to provide them with continual answers to nutrition questions . It is likely that delays and challenges in the transition from project to program financing, as well as the political instability since 2009, have contributed to inefficiencies . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project’s objectives were substantially relevant to country conditions at the time of appraisal, and they remain relevant to current country strategy and Bank strategy . The project’s design was also substantially relevant, with a results chain plausibly linking activities, outputs, desired outcomes, and achievement of development objectives . Three of four development objectives were substantially achieved, with progress on nutritional status among children under age three and pregnant /lactating women plausibly linked to project interventions, and long -term sustainability of nutrition outcomes substantially enhanced under the project . The ICR, however, provides only modest evidence of improvements in nutritional status among primary school -aged children. The project’s interventions were efficiently chosen on the basis of their cost -effectiveness, and they were efficiently implemented . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: Institutional risk is significant . While the roles and functions of the National Nutrition Committee, National Nutrition Office, and National Community Nutrition Program have been agreed on and key structures are functional, some issues have not been entirely resolved (ICR, pp. 15-16). According to the ICR (p. 29), there will “almost certainly� be institutional conflicts in the future involving leadership among different actors and choice of operational strategies . Linked to the debates over operational strategies are continued methodological differences over the most effective and efficient program approaches and choice of target groups, indicators, and geographic areas for intervention . However, some institutions created under project appear likely to be sustained and enhanced through application to other issue areas; for example, the Bank -supported HIV/AIDS project in Madagascar has used the nutrition project ’s sites and CNAs as channels for delivering HIV /AIDS prevention messages, and mother -support groups inaugurated under the nutrition project are now starting to be used also as conduits for supply of basic health products such as bed nets, water purification, etc . (ICR, p. 28). Non-governmental organizations supported by the project built sustainable capacity to the point that they are now bidding for new contracts with other projects (including Bank-financed projects). Financial risk is also significant . According to the ICR (p. 29), the Government’s ability to continue to finance the program at the level achieved during the project period is questionable . The broader, related contextual risk of Madagascar’s continued political and economic situation is also significant . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: Lessons learned from previous Bank -financed projects in Madagascar, as well as the implementation experience of nutrition projects in other developing countries, were taken into account during preparation : (i) communities should actively participate in the design and implementation of project interventions; (ii) project preparation and implementation should be based on strong coordination and consensus -building mechanisms; (iii) projects managed outside traditional government entities tend to be more efficient and flexible in serving communities; and (iv) community and household participation are cornerstones for success of nutrition programs . A Nutrition Technical Committee, with participation of a wide range of stakeholders, was set up to help prepare the project; this body became the project Advisory Committee during implementation . The decision to separate health and nutrition interventions was judged as appropriate by both the Bank team and the Government partners. A series of risks was identified and mitigated (PAD, pp. 18-19), with only one risk, poor collaboration with the Ministry of Health, rated as substantial . According to the ICR (p. 7), this risk assessment was “very cursory,� particularly given the intensive internal review of proposed strategies and components during preparation . The PAD does not mention the possibility /probability of natural disaster and its potential impact on implementation, nor does it analyze the risk associated with long -term sustainability of project outcomes . In addition, institutional arrangements for project management were not adequately detailed (which delayed effectiveness ). at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Supervision was characterized by appropriate regularity of supervision missions and timely, high -quality aides-memoires. The Mid-Term Review “was extremely well organized, and ten years on remains a valuable source of information� (ICR, p. 30). A June 2008 Quality Enhancement Review concluded that the attempt to shift from project to program funding threatened to undermine achievements to date (due to insufficient Government capacity and commitment), recommending a series of proactive corrective measures including securing political support at the highest level for institutionalization of the project ’s activities, assembling a donor coalition supportive of the project’s aims, and dissemination of the technical lessons of the project both nationally and internationally. The Bank team responded with a series of clear action plans, including the hiring of a consultant in the country office to help move the program forward . Since the political developments of early 2009, there has been limited opportunity for the Bank to influence the situation . Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Government commitment was in evidence during preparation, through continuous support for the previous SECALINE project under the leadership of the Prime Minister, and the elaboration of a National Plan of Action for Nutrition and a Food Security Strategy . The relevant ministries (Agriculture, Education, and Health ) successfully fulfilled project-related responsibilities. Throughout, there was Government support for strengthening project coordinating and implementing bodies at the central, provincial /regional, and local levels, along with a willingness to rely on the implementation capacity of non -governmental and community-level organizations. However, there were delays in adoption of the legislative framework supporting the effective functioning of these institutions . There were also political issues and delays concerning the approval of appointments, including to the National Nutrition Council, that impeded implementation . The Government did not optimally manage the transition from project to program financing, and it has not established a sustainable basis for financing the overall nutrition program. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: Implementation responsibilities were shared among national nutrition institutions, government ministries, and locally recruited NGOs. The distribution of responsibilities at the national level passed through three phases : from effectiveness through 2004, the project was managed by a project implementation unit; from 2004 through 2007, the project was formally under the management of the National Nutrition Office, though the lack of an operational structure there left the project management structure relatively intact; and from 2007 on, the project management structure became the operational arm of the National Nutrition Office with responsibility for the national community nutrition program. Although performance overall has been satisfactory, there have been periods of uncertainty and ambiguity in the respective roles of the National Nutrition Office and the national community nutrition program. Local implementing NGOs exhibited high levels of capacity and commitment, particularly during periods when funding was intermittent (during the political crisis). They promoted some innovative interventions (such at the “spot map�) and initiated others (the production of local “artisanal� weaning food with the help of local women ’s groups). Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The key features of the project ’s M&E plan were: (i) surveys conducted at district level prior to implementation, at mid-term, and at completion; (ii) routine data monitoring at the provincial level; and (iii) standard quarterly, semi-annual, and annual reviews. Two Policy and Human Resources Development Fund (PHRD) grants strengthened the analytical underpinnings of the operation, financing : (i) an anthropometric survey to assess rates of acute and chronic malnutrition in children under five; (ii) a study on the prevalence of parasitic infections among school-age children; (iii) a beneficiary assessment of communities ’ perceptions of malnutrition; (iv) an assessment of the prevalence of anemia and Vitamin A deficiency among pregnant women; and (v) a qualitative and quantitative analysis of food consumption habits . The PAD does not provide detailed results of these studies, discussion of proposed M&E activities and arrangements, or adequate baseline data and targets . Some indicators were expressed in terms of absolute numbers, while others used proportions, without an evident logic guiding those choices. The timing and financing of periodic project evaluation studies was not explicitly determined, resulting in some studies not being carried out . b. M&E Implementation: Demographic and Health Surveys (DHS) were conducted in 1994, 1997, 2003-2004, and 2008-2009, and anthropometric surveys in 1998, 2004, 2007, and 2011. At the mid-term review, some follow-up studies identified in the PAD were conducted, contributing to a thorough and comprehensive MTR . Additional data collection instruments and studies were introduced, including sentinel site studies for iron and Vitamin A deficiencies . Data monitoring for the project took place routinely, including monthly information on key inputs, outputs, and outcomes for each community nutrition site. Consultants were hired in 2001 to develop a new information system . However, the Summary of the Borrower’s ICR (p. 51) cites challenges in the form of delays in or lack of reporting of data, as well as “fictitious data.� With the change from project to program implementation, further M&E challenges emerged, including delays in establishing the M&E system at the national level, and failure to provide for operating costs at the community level; these challenges were overcome by the end of the project period . Issues with transmission of data from the community to the national level were addressed through the implementation and scale -up of a pilot program using mobile phones. c. M&E Utilization: Through 2004, M&E data and analysis were discussed regularly and contributed to a successful MTR . However, the institutional mechanisms for a broader dialogue on nutrition were not sufficiently operational . During the latter half of the project period, technical and financial difficulties, as well as the prevailing political situation, impeded analysis and utilization of information . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was Environmental Category C . According to the PAD (p. 17), the project was likely to have a positive impact on the environment, since it was to contribute to improved hygiene and sanitation in schools and villages . b. Fiduciary Compliance: For both financial management and procurement, problems identified early in the project persisted throughout implementation (ICR, p. 14). There were two principal financial management concerns throughout implementation : the timely replenishment of accounts, and organizational arrangements . To remedy the former situation, the Bank increased the amount of the Special Account, but there were still instances where Government funds were systematically reduced and /or disbursed late. Challenges with organizational arrangements stemmed from the transition from project to program financing; during transition periods, several different accounting systems were used, resulting in delays and failure automatically to produce consolidated financial statements reflecting activities from all funding sources . As a result, financial monitoring reports were generally produced manually, resulting in issues around the accuracy of calculations, data presentation, and possibly data quality (ICR, p. 15). Annual audits were carried out on time and were without qualification. There were recurring concerns about the organization and monitoring of the procurement plan, the conformity of bidding with Bank procedures, and the completeness of tendering and contracting documentation . There were also procurement complexities related to the importation of food and micronutrients for communities, and dosage of specific drugs for deworming and for iron supplementation in schools . After the establishment of the National Nutrition Office, an internal auditor was recruited for the project . c. Unintended Impacts (positive or negative): None. d. Other: None. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Satisfactory The project’s objectives were Satisfactory substantially relevant to country conditions at the time of appraisal, and they remain relevant to current country strategy and Bank strategy . The project’s design was also substantially relevant, with a results chain plausibly linking activities, outputs, desired outcomes, and achievement of development objectives. Three of four development objectives were substantially achieved, with progress on nutritional status among children under age three and pregnant /lactating women plausibly linked to project interventions, and long-term sustainability of nutrition outcomes substantially enhanced under the project. The ICR, however, provides only modest evidence of improvements in nutritional status among primary school-aged children. The project’s interventions were efficiently chosen on the basis of their cost-effectiveness, and they were efficiently implemented . Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: These lessons are drawn from the ICR, with adaptation : Premature shifts away from sector investments toward program and budget support can jeopardize the effectiveness and credibility of human development sector operations . In this case, the move to program support took place before the necessary institutional and financial prerequisites were in place, putting achieved outcomes at risk. Initial and sustained community involvement is critical for the success of nutrition interventions . The CNAs and local implementing NGOs in this project were the backbone of its effectiveness and sustainability, and their contribution now extends beyond nutrition to other areas . The roles and relationships among coordinating and implementation institutions must be clarified and agreed to from the start. In this case, overlaps in function between the project implementation unit, the National Nutrition Office, and other agencies led to delays and inefficiencies . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provides a clear and comprehensive narrative describing a lengthy and complex project implementation period. It effectively employs data from the project and other sources to assess outcomes supporting the project development objectives. The ICR does not adequately describe and assess the contributions of the World Food Program and of the Borrower. a.Quality of ICR Rating : Satisfactory