ICRR 13222 Report Number : ICRR13222 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 10/30/2009 PROJ ID : P005750 Appraisal Actual Project Name : Agricultural Support US$M ): Project Costs (US$M): 35.46 24.46 Services Project Country : Tunisia Loan /Credit (US$M): Loan/ US$M ): 21.33 24.97 Sector Board : ARD US$M): Cofinancing (US$M ): Sector (s): Agricultural extension and research (65%) Agricultural marketing and trade (19%) Animal production (15%) Central government administration (1%) Theme (s): Rural markets (50% - P) Small and medium enterprise support (25% - S) Rural policies and institutions (25% - S) L/C Number : L7063 Board Approval Date : 06/26/2001 Partners involved : Closing Date : 06/30/2007 12/30/2008 Evaluator : Panel Reviewer : Group Manager : Group : Hassan Wally John R. Heath Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: "The project is the first phase of a longer term (ten year) program that aims at improving production quality, competitivity [sic] and market access, particularly for smaller and medium scale producers . To this end, the project objectives are to: 1. Develop, on a pilot basis, organizational structures for producers that represent their needs and interests . 2. Improve the institutional capacity and quality of agricultural services delivered by public and private institutions and producer organizations . 3. Improve the flow of information for all sector stakeholders " (Project Appraisal Document, p. 2). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Component 1. Building the capaci ty of agricultural producers and inter -professional organizations . (Appraisal Cost US$ 5.26 million, Actual Cost US$ 2.31 million ). Support a pilot project to strengthen the capacity of local and regional producer organizations to demand, manage and provide services . Provide support to national inter-professional organizations (GIPs) to improve product quality and develop new markets and brand images for Tunisian produce. Component 2. Strengthening the supply of research, training and farming advisory services . (Appraisal Cost US$ 10. US$1010 .03 million, Actual Cost US$ 9.28 million ). Strengthen and regionalize agricultural research including the introduction of a selective or competitive grant system, creation of a unified research institute and regional research centers, user participation in setting the research agenda, and creation of an information clearing house on agricultural technology. Also, support a pilot project to improve producer access to training and farming advice through setting up a demand-driven advisory service in six pilot areas, reinforcement of women's extension; and strengthening of agricultural training . Component 3. Livestock and animal health . (Appraisal Cost US$ 8.74 million, Actual Cost US$ 4.57 million ). Support the improvement of services in animal production and health including : diagnostic services, animal identification, training, further transfer of services to the private sector; and increasing the share of producers in paying for certain services . .( Appraisal Cost US$ 6.44 million, Actual Cost US$ Component 4. Plant protection and seed and plant certification .(Appraisal 4.97 million ). Strengthen the capacity for plant protection and seed certification by expanding the pesticide residue and seed testing capacity, development of monitoring and surveillance capability on trade in plant products; and improving awareness about cost effectiveness and sustainability of integrated pest management (IPM). Component 5. Strengthening public interest services and project management support . (Appraisal ( Appraisal Cost US$ 4.98 million, Actual Cost US$ 3.33 million ). Increase the availability of agricultural statistics and information for all sector stakeholders; and provide support for project management . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The project closing date was extended by 18 months. The ICR (p. 7) points to procurement delays among other factors as the main reason for the extension . In March 2006, 11% of the original loan amount was cancelled due to cost savings related to exchange rate fluctuations and an overestimation of technical assistance and training costs at appraisal (ICR, p. 4). The Government of Tunisia (GOT) was expected to provide US$ 16.69 million counterpart funds but, by project completion, it had supplied only US$ 5.8 million (ICR. p.28). 3. Relevance of Objectives & Design: Objectives . (Rating : Substantial ). At the appraisal stage the project objectives were in line with the priorities of the Bank's Country Assistance Strategy (CAS) and a continuation of support to the agriculture sector development in Tunisia. The project was intended to implement the integrated and coordinated reform process envisaged in the Government of Tunisia (GOT) action plan (ICR, p. 1). The project objectives are also relevant to the current Bank's Country Assistance Strategy (2005-08) which states that in order to strengthen the business environment to support the development of a more competitive, internationally integrated private sector and improve competitiveness of the Tunisian economy six outcomes need to be achieved including "improved competitiveness of agriculture while ensuring that social and environmental concerns are properly addressed " . ( Rating : Modest ) The project design simultaneously tackled infrastructure and capacity building gaps, Design . (Rating following up on earlier agricultural service operations . There were a number of persistent problems with the farming incentive regime, resolution of which lay beyond the scope of this project but could perhaps have been addressed through a complementary development policy operation . In the absence of this complementary operation it could be argued that the various parts of the project would not be sufficient to achieve the overarching objective of enhanced competitiveness. Also, the design did not make adequate provision for outreach to small and medium farmers . The Bank did not conduct any agricultural sector work between 1986 and 2006, suggesting perhaps that the analytical foundations for the project were insubstantial . The ICR (pp. 5-6) identifies several other design failings . First, the pilot launched in support of the first objective under estimated local conditions and made "unrealistic assumptions". Second, some of the key performance indicators were "too broad" and others were "not well related to the PDOs". Third, the project was hampered by the lack of a "stand-alone M&E component". Fourth, the project was arguably too complex, increasing the likelihood of implementation delays. Fifth, the PAD did not contain " a very substantial definition of the activities to be carried out under each component". The design shortcomings were considerable (as reflected in the Quality at Entry rating in Section 8a below); considerable enough to outweigh the substantial relevance of objectives . Overall, therefore, relevance is rated modest. 4. Achievement of Objectives (Efficacy): The overarching objective was to improve production quality, competitiveness and market access, particularly for smaller and medium-scale producers. Achievement is rated substantial , based on progress made toward specific objectives (particularly the second of these which accounted for 75 percent of project costs). The project laid the necessary public sector foundations for an eventual private sector response. Particular achievements were the strengthening of agricultural research infrastructure, enhanced controls for plant and animal health, and the development of improved procedures for product certification and traceability . However, the ultimate impact on small and medium-size producers was unclear when the loan closed . 1. Develop, on a pilot basis, organizational structures for producers that represent their needs and interests, (Rating : Modest ). The strengthening of the Groupement interprofessionnel (GIP) GIP ) was the first activity under this objective. The ICR (p.10) explains that the quality specialists received substantial training in Tunisia and Europe on what constitutes a quality product . Also, 11 quality seals were approved and a decree was published specifically addressing wine origins. However, some of the quantitative targets that measure performance (ICR, p. 10) were not fully achieved. For example, training targets were only about two -thirds realized. The second activity was a pilot operation to support representative producer organizations (POs) POs ) by providing economic and technical assistance to existing cooperatives by private consulting firms . The ICR (p.11) states that technical assistance was provided to 44 cooperatives and 32 Groupement de Developpement Agricole (GDAs) to help them establish accounting and financial controls, financial diagnostics, general management advice and strategic investment planning . However, the TA was not useful as most of the cooperatives did not genuinely represent the needs and interests of producers, many cooperatives were "not financially viable" and "not adequately staffed" to be able to benefit from the TA provided (ICR, p. 11). Also, the financial and managerial advice provided did not respond well to the need and priorities of most cooperatives and GDAs (ICR, p. 11). Moreover, the progress and results of the pilot were not monitored and evaluation was only carried out towards the end of the project . The ICR (p. 11) notes that the pilot focused more on technical training without enough attention to actual activities of cooperatives, and concludes that the pilot "did not substantially address the wider question of whether and how to provide TA to expand cooperatives for various products, and to offer a wider array of services useful to farmers ." 2. Improve the institutional capacity and quality of agricultural services delivered by public and private institutions ( Rating : Substantial ). Institutions and services targeted under this objective were and producer organizations . (Rating agriculture research and extension services, plant and seed protection and certification, and livestock and animal health. Agriculture research was successfully decentralized and regionalized with research planning responding better to farmers' needs (ICR, p.12). Research results are better documented and posted on the intranet of MOA and shared with producer groups to get their input on research priorities . The ICR (p. 12) notes that number of consultations per year with the research database was 40,000 (target: 50,000) and the number of research results included in the research achievements library was 104 (target:104). Two regional research centers were established with financial and budget autonomy and multi -disciplinary teams responding to local needs (ICR, p. 31). There is local buy-in: the GIP and the Regional Commissariats for Agricultural Development (CRDAs) participate in setting the research agenda and contribute to funding research (ICR, p. 31). The ICR (p. 12) notes that 149 research actions were initiated (target: 149). On the other hand, extension pilots for the development of private extension advisory services were " poorly designed and executed and had little impact " (ICR, p.12).On the positive side, the ICR (p.12) highlights that "training of extension agents and farmers improved services significantly, and the pilot to strengthen extension services to women farmers worked well ." That said, the lack of adequate M&E makes it difficult to assess the impact of the training and extension services . Plant protection capabilities were improved with a new phytosanitary station at the container port of Rades, the extension of analytical laboratories for pesticides and the provision of equipment for three regional stations (ICR, p.13). Also, the project activities established epidemiology systems for controlling pests in various horticultural crops, trained experts in integrated pest management (IPM) and facilitated a decree that established user fees (ICR, p.13). "Most of the lab capability and related work originally planned was completed" (ICR, p. 35). Some of the mid-project investments were not finished at closing but the prospect of enhanced access to EU markets provides government with a strong incentive to finish the job . Furthermore, following passage of a law in April 2008, labs can now charge for their quality control services, enhancing the prospects for sustainability of these investments and reducing dependence on the government budget (ICR, p. 35). With regard to veterinary control systems, the ICR (p.14) states that "the diagnostic capacity of MOA has been greatly enhanced through the completion of the animal identification network for dairy cows and poultry, through improved computerization and communication of urgent problems and through expanded laboratory analytical capability." As a result of the project activities reliable identification systems were put in place for traceability and food safety for fish, poultry and dairy and about 80% of the cows were identified with ear tags and records (ICR, p. 34). However, there were problems in establishing a traceability system for red meat given that "half of meat animals were slaughtered outside of the official system " (ICR, p.34). The project provided training for 313 (target 300) agents for animal identification, 231 (target 225) technicians for artificial insemination and 247 (target 240) milk system controllers (ICR, p.15). 3. ( Rating : Modest ). The project activities supported the . Improve the flow of information for all sector stakeholders . (Rating creation of websites containing a variety of agricultural information including agricultural statistics and agricultural research as well as information from different offices at the Ministry of Agriculture . The ICR (p.15) argues that the information flow could have been better so as to reach more stakeholders, particularly private entrepreneurs who could benefit from detailed information on commodities and could provide feedback on analysis and proposals . Also, contrary to expectations, the 23 commodity specific studies funded under the project were not posted on the internet . If they had been posted on the website of the Agricultural Investment Promotion Agency (APIA), they would have provided invaluable information for foreign or domestic investors considering opportunities in the agriculture sector . 5. Efficiency (not applicable to DPLs): The PAD provides no specific ERR analysis . Such analysis is not appropriate for an institutional development project of this type (although, under the private extension pilot, it would have been useful to compare the cost effectiveness of private and public provision of agricultural services ). By loan closing total costs were 69 percent of appraisal estimates and yet delivery of expected outputs (infrastructure, goods and training ) averaged about 90 percent, warranting a substantial rating for efficiency. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: A modest rating for relevance combined with a substantial rating for efficacy and efficiency yields an outcome rating of moderately satisfactory. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The continuing failure of the Tunisian government to address price distortions and other weaknesses in the incentive regime may undermine achievement of the overarching objective of the project . Also, the sustainability of the research facilities and laboratories might be at risk if the Ministry of Agriculture is subject to budget cuts, although this not so far a significant concern . The progress with product certification and traceability has a high payoff in terms of improved access to European markets and seems likely to be maintained . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry . The Bank successfully identified a project that is in line the with the Bank's CAS and reflects priorities of the Tunisian government for the agriculture sector . However, there were shortcomings in ensuring quality at entry where the projects first PDO "paid insufficient attention to existing private sector farmer -to-market value chains and made unrealistic assumptions about the pace of development of representative producer organizations" (ICR, p.20). This could partly be caused by the lack of sector work addressing Tunisian agriculture: the TTL explained that when the project was appraised in 2001 the most recent agricultural sector review was dated 1986. Supervision . The ICR (p.21) notes that each supervision mission had 8 to 9 experts. However, there was a lack of follow up on social aspects due to the absence of social development specialists or sociologists among the supervision team. Also, pilot evaluations were not carried out except towards the end of the project . The Bank supervision team should have given earlier attention to M&E, rather than waiting till the mid -term review. The supervision team should have modified the content of the TA to assist GDAs since a 2004 law restricted their ability to market agricultural products . The ICR (p. 22) states that the project should have been restructured early on given the disbursement shortfalls and limited progress toward (poorly identified) targets. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government . The ICR (p.22) notes that the Tunisian government was committed to the project and reforms in the agriculture sector. In general the government seemed supportive of the project activities, but slow to act in some cases to adopt certain legal texts . The ICR (p.22) notes that establishing new laws proved to be "time-consuming extending beyond the loan closing date in some cases ". Hiring restrictions by the Ministry of Finance restricted the ability of the Ministry of Agriculture (MOA) to recruit specialized staff which in turn led to implementation delays (ICR, p.23). Also, there were shortfalls in counterpart funding (see Section 1d above). Implementing agencies . The ICR (p.23) describes the management and staff of the implementing agencies as "enthusiastic, energetic, and knowledgeable about implementing their respective portion of the project ". But procurement difficulties delayed project implementation . The ICR (p. 23-24) provides details about the activities executed by each implementing agency and problems or shortcomings were candidly discussed . In particular, the project management unit (PMU) was " too low in the MOA hierarchy to help implementing agencies sort out problems". The ICR (p.24) states that the PMU had "too few staff" and did not have staff "specialized in procurement or M&E." a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: Design . M&E design received limited attention given the complexity of the project with five components and 12 executing agencies. M&E lacked quantitative indicators and baseline data making it difficult to assess progress towards meeting the PDOs or decide whether corrective measures are needed . The ICR (p. 7) suggests that M&E could have been a separate component of the project . Implementation . The ICR (p.7) points out that MOA failed to hold a five day M&E workshop that the Bank recommended with the support of an international and a national M&E consultant . The later adjustments to the M&E system "do not tell the extent to which the project achieved its overall PDOs and what the outcomes /impacts of the project where" (ICR, p. 8). The ICR (p.viii) notes that 8 original outcome/impact indicators in the PAD were not monitored or evaluated. Outreach to small and medium-size farmers was not satisfactorily quantified . Utilization . The ICR (P. 8) concludes that M&E was not well developed and it is not possible to evaluate the current utilization of M&E. a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards . The project was classified as environmental category "B" (PAD, p.19). An environmental impact study (EA), carried out during preparation found that the project would positively contribute to environmental sustainability and public health (ICR, p.8). Diagnostic and research laboratories were constructed according to environmental regulations and individual environmental assessments addressed the risks and the proper handling of laboratory waste. Also, no pesticides were procured under the project . The project supported and encouraged integrated pest management, the use of biological control methods and helped improve consumer food safety (ICR, p. 8). Fiduciary . Although financial management was reportedly sound, implementing agencies took longer than they should have to report to the PMU what contracts they had approved . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Building partly on the lessons in the ICR the following lessons are emphasized : Project design must include a sound M&E system.system. In complex projects with many components and multiple implementing agencies the presence of a sound and measurable monitoring and evaluation indicators is key towards successful management and implementation . Projects should be formally restructured when there are major obstacles to achieving objectives . At the objectives. start, design flaws held up the progress of this project. A formal restructuring might have provided renewed impetus and allowed objectives to be more fully achieved . Privatization of agricultural extension requires a design that accommodates local conditions, establishes appropriate incentives and involves provision by specialized firms firms.. The substitution of public by private extension services is often difficult because farmers are often reluctant to pay for extension. The introduction of a private service should be demonstrated first through pilots in which farmers interested in increasing productivity are chosen and appropriately organized. The extension service should be provide by specialized firms with experienced advisors who understand local needs. Finally, the government has to provide the proper incentive for the sustainability of private extension service until the system is established. This could be in the form of temporary subsidies to the service providers during a transition period . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is well organized and clearly written . It provides thorough information on all components of the project and is candid about shortcomings. a.Quality of ICR Rating : Satisfactory