IEG Report Number: ICRR14752 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 06/30/2015 Country: Moldova Project ID: P143202 Appraisal Actual Project Name: Emergency Project Costs (US$M): 10.00 7.51 Agriculture Support Project L/C Number: Loan/Credit (US$M): 10.00 7.51 Sector Board: Agriculture and Rural Cofinancing (US$M): Development Cofinanciers: Board Approval Date : 03/19/2013 Closing Date: 09/01/2014 09/01/2014 Sector(s): Other social services (46%); Crops (46%); Public administration- Agriculture; fishing and forestry (7%); Animal production (1%) Theme(s): Other rural development (52%); Social safety nets (48%) Prepared by: Reviewed by: ICR Review Group: Coordinator: Katharina Ferl Soniya Carvalho Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: According to the Financing Agreement of May 29, 2013 (p. 5), the objective was “to mitigate the negative effects of the 2012 drought by helping to restore the corn and wheat production and preventing livestock destocking in the most affected districts.” The objective in the Project Paper was the same. This Review will use the PDO in the Financing Agreement as the basis for the validation. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project contained three components: Component 1: Cash transfer to affected farmers (appraisal estimate US $ 9 million, actual US$ 6.53 million, 72.6% of appraisal estimate ): This component was to finance cash transfers to targeted farmers in areas that were most affected by the drought in 2012. Farmers would receive the cash transfer ex-post based on the area that they planted with wheat and corn. Component 2: Awareness and training (appraisal estimate US $ 0.35 million, actual US$ 0.35 million, 100% of appraisal): This component was to finance the dissemination of information on the program to create awareness and incentives for affected farmers to plant seeds despite receiving the cash transfer ex-post. Component 3: Project Management (appraisal estimate US $ 0.65 million, actual US$ 0.63 million, 96.9% of appraisal estimate): This component was to finance project implementation by the Agency for Interventions and Payments in Agriculture (AIPA) within the Ministry of Agriculture and Food Industry (MAFI). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost: At appraisal, the project was estimated to cost US$ 10 million. Actual cost was US$ 7.51 million (75.1% of appraisal estimate). Financing: The project was financed by a US$ 10 million IDA credit, of which US$ 2.3 million was cancelled. Borrower Contribution : The borrower contributed approximately US$13.5 million through a budget support grant by the European Union. Dates: The project was restructured on March 20, 2014 as follows: i) US$ 2.3 million was cancelled as requested by the Borrower; (ii) US$ 177,000 was reallocated from category 1 (cash grants) to category 2 (Goods, non-consulting services, consulting services, training, incremental operation costs); and (iii) the eligibility of districts for wheat compensation was expanded from districts where losses were more than 35% to districts where losses were more than 30%. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High: The objective of the project was highly relevant. Between 2011 and 2012, Moldova experienced severe drought conditions which led to a 23% decline in value added in the agriculture sector. The corn and grain harvest decreased by 50%, resulting in a US$ 78 million loss, 9.5% of agriculture Gross Domestic Product. The Bank responded to Moldova’s request to provide financial and technical support to farmers in the most affected areas. The objective was in line with the “National Development Strategy Moldova 2020” which includes measures to reduce climate change and disaster events. Also, the project was in line with the Bank’s Country Partnership Strategy (2009-2012) which aimed to improve economic competitiveness, minimize social and environmental risks, build human capital, promote social inclusion, and improve public sector governance. Furthermore, the objective was in line with the Bank’s current Country Partnership Strategy (2014-2017) which focuses on enhancing human capital, minimizing social risks, and promoting a green, clean and resilient Moldova. b. Relevance of Design: High: The design of the project was kept simple and the planned activities to mitigate the negative effects of the 2012 drought included cash-transfer to corn and wheat farmers in the most affected districts. In addition, information dissemination activities on the cash-transfer program, to avoid elite capture and encourage affected farmers to grow corn and wheat, were planned. Also, activities included the training of farmers on animal feeding. In order to avoid the distribution of cash to ineligible farmers, a grievance system was developed. The causal relationships between these interventions and underlying assumptions about how conditional cash transfers, training and awareness activities would lead to an increase in the production of corn and wheat production and restore the farmers' ability to feed their animals and, therefore, reduce the risk of destocking, was logical and properly laid out. 4. Achievement of Objectives (Efficacy): Mitigate the negative effects of the 2012 drought by helping to restore the corn and wheat production and prevent livestock destocking in the most affected districts . The project provided cash-transfers to affected corn farmers in 31 districts and for affected wheat farmers in 18 districts. Moldova has a total of of 34 districts. Modest Outputs  60,085 beneficiaries, of whom 39% were female, benefited from the project, surpassing the target of 22,000 beneficiaries of whom 15% should have been female.  In 2013, in 31 selected districts 76% of the area, with plots larger than 0.5 hectare, was planted with corn, not achieving the original target of 300,000 hectares.  In 2013, in 15 selected districts 120% of the area, in plots larger than one hectare, was planted with wheat, surpassing the target of 80%.  98% of the beneficiaries were small farmers, surpassing the target of 70%.  US$ 6.7 million were distributed via cash transfers. The target of US$ 9 million was not achieved due to the cancelling of US$ 2.3 million during the restructuring of the project.  841 awareness workshops were conducted, surpassing the target of 830.  A grievance mechanism was established and is functioning.  35 days of training on feeding practices, livestock dietary requirements and nutritional value/quality of different kinds of feeds were provided, achieving the target of 35.  149 broadcasting events took place, surpassing the target of 120.  A small scare corn silage feasibility study was completed. Outcomes According to the ICR (p.10), 54% of the country's planted wheat area was covered by funds financed by the Government and the European Union. Also, the ICR (p.11) notes that the 2013-2014 agricultural season benefited from very favorable climatic conditions. The datasheet in the ICR states the following outcomes:  The average national production of wheat increased by 33% compared to the 2009 to 2011 average national production, surpassing the target of a 20% decrease. The baseline value was a 39% decrease.  The average national production of corn increased by 6% compared to the 2009 to 2011 average national production, surpassing the target of a 20% decrease. The baseline value was a 61% decrease.  The average national number of pigs increased by 60%, surpassing the target of a 30% decrease. The baseline value was a 34% decrease.  The average national number of chickens increased by 6%, surpassing the target of a 17% decrease. The baseline value was a 19% decrease.  The average national number of cattle decreased by 14%, surpassing the target of a 16% decrease. The baseline value was a 18% decrease. The ICR does not discuss the extent to which these national level outcomes can be attributed to the project, especially for wheat production, which only covered 18 of Moldova's 34 districts, and was also supported by the Government and the European Union. Also, the ICR (p. 5) states that there was a systemic problem of reliable data at the MAFI due to lack of capacity and shortage of staff. 5. Efficiency: Substantial The Emergency Project Paper (p.51) included an economic analysis. The analysis compared the costs of the project with the assumed benefits resulting from the increased production of corn and wheat. Those benefits included an increase in the availability of food for human consumption and animal fodder/feed and, therefore, could have contributed to a reduction in incidence of destocking. The assumptions on destocking were based on data from the 2007 drought. The analysis used a discount rate of 10%. The estimated internal rate of return (IRR) was 30% and the estimated Net Present Value (NPV) was US$ 940,981. The ICR applied the same analysis model. However, the analysis reflects the reduction from US$ 10.0 million to US$ 7.5 million in project costs. For the estimation of benefits, actual data for revenues from wheat, corn, beef, chicken, pork meat, eggs and milk in 2013, and if available for 2014, was used. The analysis applied a discount rate of 10% and an attribution rate of 5%, resulting in an IRR of 31% and a NPV of US$ 499,167. However, the ICR does not provide any details why this attribution rate was chosen. According to the task team of the project, the significant difference in NPV is due to lower project costs, and the area for corn being 40% smaller than originally estimated. However, in the absence of information on new technology, better extension service, and improved irrigation or drainage, it is difficult to determine if the analysis is robust. There are questions about the extent of attribution but even a low rate of attribution results in a high rate of return. Overall efficiency was Substantial. a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of objective is rated high given the urgent need to mitigate the negative effects of the 2012 drought. Relevance of design is rated high as the project's activities were logically and plausibly linked to anticipated outcomes. The objective was modestly achieved due to the difficulty of attributing the outcome data that is provided to the project. Efficiency is Substantial. Therefore, the overall outcome rating is rated Moderately Satisfactory. a. Outcome Rating: Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The government is committed to decreasing the vulnerability of its agriculture sector to natural disasters and is working with the Bank to strengthen the country’s capacity in forecasting, preparing for and responding to severe weather events and natural disasters. Also, Moldova is committed to making the agro-food sector more competitive by improving the food safety management system and supporting market access for farmers. However, it is critical for Moldova to improve its data collection and monitoring of the agriculture sector. The risk to development outcome is moderate. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: Within a period of six months between the government requesting the Bank’s support and project appraisal, the Bank prepared an emergency project that was responsive to the needs of Moldova. The project design was kept simple and the project was based on lessons learned from previous Bank involvement in agriculture projects and from a similar drought relief program funded by the European Union, which was prepared at the same time. Lessons learned from the project by the European Union were that a target mechanism to reduce elite capture by developing an accurate beneficiary list and an information campaign and grievance mechanism to increase transparency and social accountability are critical. Among these lessons were that cash transfers allow for a faster implementation by not having to procure inputs, and cash transfers have a positive impact on individual choice and ownership while minimizing market distortions. The start of project implementation was delayed due to political instability and issues related to opening a designate account. However, according to the ICR (p.15) the Bank tried to solve these issues as fast as possible. During project preparation, the Bank identified the distribution of cash to ineligible farmers as the key risk. In order to mitigate this risk the Bank developed a grievance mechanism to report and investigate perceived irregularities. Also, an awareness campaign was expected to improve transparency and reduce the risk of targeting ineligible farmers. Quality-at-Entry Rating: Satisfactory b. Quality of supervision: Within a year of project implementation, the Bank conducted three supervision missions to provide technical support. The project benefited from a Task Team Leader who worked in the local Bank office and could provide regular support. According to the Bank team, the Bank worked with all relevant project implementation partners to strengthen the robustness of the M&E and produce relevant information. The ICR (p.7, paragraph 31) states that no verification of beneficiary lists was conducted by AIPA and Posta Moldovei -- a critical mitigation effort to prevent corruption in any conditional cash transfer program. Given all these shortcomings, supervision is rated Moderately Satisfactory. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Due to political instability the project effectiveness and start of program implementation was delayed by four months. After the situation stabilized, the government was committed to achieving the objectives of the project. However, due to the initial delays the funds for affected farmers only became available in the autumn of 2013, significantly after the corn planting season in spring 2013. The government mitigated this delay by announcing the cash-transfer program in advance to raise awareness among eligible farmers. The Ministry of Agriculture and Food Industry (MAFI) performed the overall coordination and supervisory functions and successfully set up all necessary institutional arrangements for the implementation of the project. Also, the government delivered critical elements of the program despite tight time and resource constraints. However, according to the ICR (p. 15) MAFI lacked sufficient capacity, and had not paid sufficient attention in collecting and monitoring relevant statistics on corn producers, and hectars of land owned/rented by corn producers. Therefore, the data on drought affected areas and amount of potential beneficiaries on which the project was based was inaccurate. Also, there was a discrepancy between the project paper, opera tional manual and a decree by MAFI in terms of the responsibilities of AIPA. While the project paper and operational manual stated that AIFA was responsible for “documentary and field controls, additional controls and post-payment checks”, the MAFI decree only stated that AIPA is responsible for collecting and submitting consolidated lists to the Central Ministerial Commission. The ICR does not comment if M&E data influenced decision making and if the government coordinated with other donors, such as the European Union which implemented another drought relief project at the same time. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: The project was implemented by two implementing agencies, the Agency for Interventions and Payments in Agriculture (AIPA) and the Agency for Consultancy and Training in Agriculture (ACSA). Component 1 was implemented by AIPA while component 2 was implemented by ACSA. AIPA sub-contracted with Posta Moldovei, the Moldavian Post, to make cash-transfers to the few farmers who did not have a bank account. According to the Bank team both, AIPA and ACSA effectively cooperated to facilitate quick delivery of cash transfers to drought affected farmers. However, an operational review, conducted by an independent consultant, found that AIPA’s internal control system had some minor shortcomings. Both, AIPA and Posta Moldovei, did not conduct any verification of the accuracy of the beneficiaries’ lists but only relied on local commissions for the accuracy of these lists. Verification of beneficiaries is critical in any conditional-cash transfer in order to prevent corruption. According to the Task Team Leader AIPA generally processes between 1,200 and 1,800 payments on an annual basis. However, in this project they had to process over 60,000 payments significantly surpassing the original target of 22,000. In order to be able to do so, AIPA developed a management system that supported the disbursement of a significantly larger amount of payments. Due to the delays at the beginning of the project ACSA used its own funds to conduct trainings and raise awareness for the cash-transfer program even before the project became effective. ACSA also successfully developed and managed a grievance mechanism. Also, ACSA raised awareness about live-stock management practices under conditions of drought. According to the ICR (p.6) 500 grievances were submitted and 15 complaints were identified for review. The Bank team stated that ACSA conducted a review in 2013-2014, which concluded that none of the complaints were satisfied, proving that the targeting and payment decisions were correct. The Consolidated Agricultural Project Management Unit (CAPMU) was responsible for carrying out the fiduciary functions of the project. CAPMU had extensive experience in supporting agricultural/rural Bank-financed project in fiduciary aspect and provided financial reports and annual audits satisfactorily according to the ICR (p.7). Due to the timely reporting, corrective actions could be taken such as the cancellation of US$ 2.3 million before project closing. Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The objective to mitigate the negative effects of the 2012 drought by helping to restore the corn and wheat production and preventing livestock destocking in the most affected districts was clearly defined and reflected in the selected indicators. The Results Framework in the Emergency Project Paper (p.33) included four PDO indicators and nine intermediate outcome indicators. The indicators were measurable in terms of numbers. However, the design of the M&E had a major shortcoming. The estimations for number of direct beneficiaries and amount of area planted with corn by 2013 were based on inaccurate data provided by the MAFI. Also, PDO indicator 1 ("percentage change over the 2009-11 average national production of wheat") measures an outcome, which can not solely be attributed to the project since 54% of the country's planted wheat area was covered by funds financed by the Government and the European Union. b. M&E Implementation: AIPA and ACSA were responsible for tracking indicators, collecting data and monitoring results. Progress reports on the two components were submitted to CAPMU on a quarterly basis. Based on those reports CAPMU developed semi-annual progress reports. The project also funded an operational review of the cash transfer program and an impact assessment which were conducted by an independent consulting firm. However, the reports were only submitted to CAPMU briefly before the project closed which did not give stakeholders enough time to provide feedback. The ICR does not comment if the data was reliable and of good quality and if the M&E was owned by various stakeholders. Also, national data was not disaggregated in order to measure outcomes in project areas. c. M&E Utilization: The M&E measured system outputs as opposed to outcomes. The ICR does not comment to what extent M&E data was used to inform decision making. M&E Quality Rating: Modest 11. Other Issues a. Safeguards: The project was rated as category C since it was not expected that project activities would have any significant negative environmental and social impact. b. Fiduciary Compliance: Financial Management Fiduciary risk was rated substantial during project appraisal due to large amounts of money that needed to be disbursed within a short period of time. This could have led to delays and/or the mismanagement of funds. However, a financial management review, conducted by the Bank in April 2014, showed that the project always complied with the legal financial management covenants. CAPMU and AIPA, the two agencies responsible for financial management, applied internal control procedures and kept accounting records in integrated accounting software appropriately. Also, their teams included qualified accounting staff. The final audit report was unqualified and satisfactory according to the ICR (p.9). However, as mentioned before (ICR p. 7), the project did not verify the beneficiary lists -- a critical function to prevent corruption in any conditional cash transfer program. Procurement CAPMU was responsible for the project’s procurement activities which included the procurement of goods, non-consulting and consulting services. CAPMU had extensive experience in implementing Bank-funded projects and employed two qualified full-time procurement specialists. According to the ICR (p.8) CAPMU provided good support, which led to a fast completion of the project’s package. A final project review found that procurement plans, which were developed during project preparation and implementation, were appropriate. Document filing which allowed for easy tracking was also appropriate. c. Unintended Impacts (positive or negative): No d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Satisfactory Moderately There are questions about the extent of Satisfactory attribution. Risk to Development Moderate Moderate Outcome: Bank Performance: Satisfactory Moderately Weak M&E, weak evidence of Satisfactory efficiency, lack of attributable outcomes, and lack of measures to ensure that the intended beneficiaries were reached and not others. Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR: Unsatisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR (p. 16) identifies some valuable lessons including the following:  Reliable in-country data is critical for the preparation of an emergency project when there is not sufficient time to collect and analyze data before project implementation starts. In this project, MAFI was lacking capacity for the continuous collection and monitoring of data in the agriculture sector. Therefore, the data for the amount of beneficiaries and hectares of land owned/rented by those beneficiaries was significantly underestimated and did not allow for optimal planning.  If countries are likely to experience climate change and disaster events more frequently, a more systematic approach to address these events is critical. Even though Moldova had experienced another drought and heavy flooding quite recently, the project could not rely on an institutionally embedded national natural disaster management system. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provides a good overview and is concise but is overly output focused, and neglects to report sufficiently on outcomes and their attribution to the project. The ICR does not provide detailed information on critical areas such as Bank supervision, and data quality. The ICR is sometimes inconsistent. For example, while it says on page 6 that compensation payments arrived in October/November 2013, it says on page 10 that they arrived in July/August 2013. The ICR does not comment if the data was reliable and of good quality. The ICR does not provide any information on the candor and quality of performance rating and the Bank's supervision of fiduciary aspects. The ICR does not provide any information on the drought relief project which was implemented by the European Union and the Government at the same time. Also, the ICR does not disaggregate outcome data to measure outcomes in project areas and reports increases in the national wheat and corn production, which cannot be solely attributed to the project given that 54% of the country's planted wheat area was covered by funds by the European Union and the Government. a.Quality of ICR Rating : Unsatisfactory