ICRR 12986 Report Number : ICRR12986 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/25/2008 PROJ ID : P076961 Appraisal Actual Project Name : Horticultural Exports US$M ): Project Costs (US$M): 6.6 4.4 Promotion And Technology Transfer Project Country : Jordan Loan/ US$M ): Loan /Credit (US$M): 5.0 3.9 Sector Board : ARD US$M): Cofinancing (US$M ): Sector (s): Agricultural marketing and trade (45%) Agricultural extension and research (35%) Central government administration (20%) Theme (s): Export development and competitiveness (33% - P) Trade facilitation and market access (17% - S) State enterprise/bank restructuring and privatization (17% - S) Rural markets (17% - S) Technology diffusion (16% - S) L/C Number : L4668 Board Approval Date : 06/26/2002 Partners involved : Closing Date : 12/31/2006 12/31/2007 Evaluator : Panel Reviewer : Group Manager : Group : John R. Heath Fernando Manibog Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: The development objective of this Learning and Innovation Loan (LIL) is "to improve horticultural export marketing by pilot testing: (i) a system of outgrower farming between large and small /medium scale farmers (henceforth called outgrowers) in order to achieve 'critical mass' or 'bulk volumes' demanded by the target markets and improve the income of participating farmers; and (ii) the building of the technological capacity of farmers, especially the outgrowers, to improve crop husbandry practices and their produce quality to satisfy the requirements of target markets and reduce the rejection rates of exported consignments " (PAD, p. 3). This is consistent with the statement of objectives in the Loan Agreement . Objectives were not formally revised during implementation. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (i) Horticulture Export Promotion (Estimated cost, US$2.3 million; Actual cost, US$1.37 million). Includes creation of a specialized department in the Jordan Export Development and Commercial Centers Corporation (JEDCO) and support to the Ministry of Agriculture's Agricultural Marketing Department (AMD). Related activities are: providing price information to farmers, collecting statistics, and lobbying for export -friendly policies. (ii) Strengthening Technology Support Services (Estimated cost, US$2.4 million; Actual cost, US$1.4 million). Includes helping farmers to adopt high -value crop varieties, support to large growers to mentor small and medium outgrowers, and recruitment of technology transfer specialists . (iii) Quality Testing and Export Certification Services (Estimated cost, US$1.1 million; Actual cost, US$0.7 million). Includes targeting of farmers with a dependable farm -to-airport cold chain and assistance to farmers seeking to meet export quality standards (particularly those wishing to export to the European Union ). (iv) Project Coordination Unit (Estimated cost, US$0.8 million; Actual cost, US$0.9 million). This was to be housed in the Ministry of Agriculture. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The project closing date was extended by twelve months to allow more time for adapting contract farming models to the needs of small and medium producers . 3. Relevance of Objectives & Design: Objectives . The project aimed to ease three constraints to the export marketing of fresh produce : (i) the innate large-producer preference of importers, based on the difficulty of organizing small farmers to meet volume, quality and timeliness requirements; (ii) small farmers' limited information about markets and lack of know -how about how to raise productivity and product quality; and (iii) delays associated with produce testing and export certification laboratories, slowing the responsiveness to the changing requirements of export markets . The project's objectives are consistent with the current Country Assistance Strategy (FY06-FY10), which supports strengthening the investment environment for a skill -intensive and knowledge-based economy. In the discussion with the project team it emerged that Jordan is possibly less suited to outgrower development than other countries in the Region, given the position of large relative to small farmers (see discussion in Section 13 below). Design . The design of the project was partly inspired by similar attempts at building institutions for marketing and quality control that the Bank had supported in Hungary and Chile, "two middle income countries comparable to Jordan" (PAD, p. 4). This was a learning-centered, pilot operation which aimed to develop private -sector expertise to substitute for the recently abolished public sector Agricultural Marketing Organization (a semi-independent entity in the Ministry of Agriculture). Implementation entailed cooperation between the Ministry of Agriculture (which housed the Project Coordination Unit) and a Horticultural Export Promotion Department, which the project temporarily set up in the export development corporation (JEDCO), pending its ultimate takeover by a suitably equipped group of farmers and exporters. Criteria were developed for selecting the large and small farmers that would partner each other in new outgrower schemes. Based on findings from periodic standardized interviews with farmers, designed to capture changes in farm practices, product quality and incomes, the project would allow for lesson learning, consistent with the purpose of LILs (PAD, p. 5). The design assumption was that by improving the supply of public goods (market information, quality norms), export agriculture would be enhanced; but it did not address the lack of incentive for large farmers and exporters to work with smaller producers --the outgrower principle, which was the whole purpose of the project. There were three design flaws. First, project preparation made only limited use of beneficiary assessment to gauge interest in the outgrower model and to better understand the constraints faced by small and medium-scale farmers. Second, project objectives were broadly defined and not sufficiently linked to outcome-centered performance indicators, limiting the scope for learning from the project . Third, key institutions were not in place when the project became effective (ICR, p. 16). Even if these shortcomings had been averted, project outcomes would still have been constrained by the lack of credit for small and medium -scale farmers (ICR, p. 15): in this respect, the package of inputs available to poorer farmers remained incomplete . Also, in a meeting with IEG, the project team stated that there was insufficient understanding of the costs involved at each step in the supply chain--information that needed to be factored into the project design . The overall rating of relevance is modest . 4. Achievement of Objectives (Efficacy): (1) Pilot testing of outgrower farming between large and small /medium scale farmers ('outgrowers' ) in order to achieve 'critical mass' or 'bulk volumes' demanded by the target markets and improve the income of participating farmers . (Rating : Modest ). Although outgrower development was an explicit part of the Project Development Objective, this ended up as a minor part of the operation and there was little operationally -relevant learning about how to forge contractual ties between small -scale and large-scale operators. The design of the project did not allow for piloting of the sort of contract that would have facilitated small farmer involvement : there was no provision for credit. There is no information about the assets and socioeconomic profile of the farmers who joined outgrower schemes sponsored by the project : how small-scale these farmers were and how their scale and profile compared to farmers not participating in outgrower schemes . Nor is there any information about the change in the incomes of participating farmers between project start up and completion; or how this compared to income changes for nonparticipants. Without this information it is impossible to assess the potential of the outgrower schemes . The ICR notes that the project overlooked support to the private sector, failing to address the challenge of linking small farmers to large exporters (p. 13). On the positive side, the project confirmed findings from the literature that agrarian structure influences the scope for developing outgrower schemes . (2) Building of the technological capacity of farmers, especially the outgrowers, to improve crop husbandry practices and their produce quality to satisfy the requirements of target markets and reduce the rejection rates of exported consignments . (Rating : Modest ). By project close, the area farmed by beneficiaries that complied with European-standard "Good Agricultural Practices" (EUREPGAP) was 250 hectares--the target was 300 hectares (ICR, p. iv). By project closing 16 farmers had been EUREPGAP certified and 34 were in the process of becoming so (ICR, p. 11). This is a small share of the farmers covered by the project --by project close there were 192 contract farmers and 750 farmers had participated in trade fairs and other promotions . The ICR provides no significant detail about the other outcomes associated with this objective (research, extension, laboratory testing for chemical residues, export certification ). 5. Efficiency (not applicable to DPLs): Rating : Modest . No ERR was estimated at appraisal. The ICR does estimate an ERR but, for a LIL, the rating of efficiency should be based on whether substantial, operationally -relevant knowledge was generated in a cost-effective manner. The failure of the M&E unit to collect the necessary data limited the scope for learning . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate Yes 33% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The overall rating of relevance is modest . Combined with modest ratings for efficacy and efficiency this yields an outcome rating of unsatisfactory . This downgrade is partly justified by the lack of key information in the ICR, which suggests that there was a shortcoming in attaining the learning objective of this LIL (see footnote to Section 12 below). a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Technical factors . The ICR says that small and medium-scale producers are less able than large producers to meet product quality standards (p. 15). This is to be expected. The key point is that it is not clear if the project developed an approach that would allow smaller producers in contract farming schemes to meet these standards . Financial factors . The project does not appear to have generated any significant learning about how to tackle small farmer financing constraints. The ICR does not say whether the terms of the outgrower schemes ensured that participating small farmers secured adequate finance . The ICR acknowledges that the lack of finance for hardware (particularly, cold storage) would continue to limit export expansion; although provision of such hardware was beyond the project's scope. Environmental factors . It is not clear to what extent the project tested out ways for increasing the efficiency of water use in smallholder agriculture. Water scarcity is a significant risk . Governance factors . There is a history of government intervention in markets leading to reduced returns for small-scale farmers. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Quality at entry was low, as reflected in the design flaws discussed in Section 3 above, the failure to make adequate provision for M&E, and the inadequacy of the institutional framework for project implementation . In the course of supervision, despite sustained efforts, the Bank was unable to rectify the M&E deficiencies and, in the absence of adequate survey data about project effects, the project did not generate the learning expected from a LIL. Nevertheless, during the latter part of project implementation, the Bank team supported the establishment of a Horticulture Export Fund to assist farmers with business services and a Business Development Unit to assist farmers and exporters to help prepare business plans (both of these initiatives were positive steps beyond the initial project design). at -Entry :Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: The government--particularly the Ministry of Agriculture --was not committed to achieving project objectives . It was slow to provide the clearance needed to train laboratory technicians and it did not respond to Bank requests that staffing of the M&E unit be strengthened . Cumbersome government procurement procedures limited the incentive for private sector participation . The Ministry of Agriculture and JEDCO did not coordinate their activities with each other or with the Project Coordinating Unit . The Project Coordinating Unit was ineffective and lacked qualified personnel, especially in M&E. a. Government Performance :Unsatisfactory b. Implementing Agency Performance :Unsatisfactory c. Overall Borrower Performance :Unsatisfactory 10. M&E Design, Implementation, & Utilization: Design . At appraisal, the outcome indicators were "increase in the volume and value of exports to target markets " and "increase in the number of small and medium scale farmers producing and being linked to export markets ". These indicators are broadly appropriate but there was no attempt to build a baseline capturing pre -project production and export levels and the number of poorer farmers participating . No outcome or output targets were set, making it impossible to assess the project's level of achievement (PAD, pp. 17-18). Building a baseline would not have been easy, however, given the information vacuum : at the time of preparation there was "not a single institution with a mandate to collect and assemble farmgate, wholesale and export prices and estimate marketing margins on a timely and regular basis" (PAD, p. 53). Implementation . Four indicators bearing on the project development objective were identified after the loan became effective: the aim was to record the value of ten horticultural crops exported to target markets, comparing the rate of growth achieved by all project beneficiaries with the rate of growth experienced by those beneficiaries engaged in contract farming; the number of contract farmers and the area of land under improved farming practices was also measured. The ICR includes baseline and target values for these indicators (pp. iii-iv). These were pulled together by the Bank in the final supervision mission (ICR, p. 8) because the M&E unit was totally ineffective : it "was unable to carry out baseline and impact surveys of randomly selected farmers in both project and non -project areas" ( p. 7), it failed to collaborate with JEDCO, and it failed to gather data . Use .There was no real-time M&E data to help guide project implementation and, in any event, the participating Use. agencies were not committed to M&E. "Implementing partners did not regard the M&E exercise as a learning process but instead conducted their promotion activities without consulting or collaborating with the M&E unit . Relationships between the M&E unit and the Project Coordination Unit Director were tense and lacked mutual trust " (ICR, p. 7). a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project did not trigger any safeguard policies . Audits were delivered late but no other fiduciary concerns are raised in the ICR. No unintended impacts are recorded . 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Unsatisfactory Contrary to the intentions of a LIL and Satisfactory the explicit focus of the Project Development Objective, the project generated little, operationally-relevant knowledge with respect to the potential for involving small farmers in outgrower schemes; although it did some confirm some findings in the literature about the role of agrarian structure in influencing the scope for such schemes (see Section 13 below). Risk to Development Moderate Significant The ICR underestimates the significant Outcome : technical and financial constraints to smallholder participation in outgrower schemes. No lessons may be drawn from the project about how to tackle those constraints. Bank Performance : Moderately Moderately The rating reflects weaknesses in M&E Satisfactory Unsatisfactory at the design phase which, despite Bank efforts, were not rectified during supervision, compromising achievement of the leaning outcome that is expected from a LIL. Borrower Performance : Moderately Unsatisfactory The government was not committed to Unsatisfactory the project and the implementing agents performed ineffectively . Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: There are substantial obstacles to the successful involvement of small farmers in outgrower schemes . When the project team met with IEG they suggested that Jordan was less suited to outgrower development than other countries (e.g. Morocco) because large farmers controlled the bulk of export production, effectively squeezing small producers out of the supply chain . This confirms a finding that has been cited in the research literature : large-scale producers and exporters have no incentive to work with small farmers when the latter do not account for a significant share of high value crops --where the distribution of farm holdings is highly concentrated outgrower schemes are less likely to flourish . The ICR says that the project -sponsored experiments with contract farming worked best where the exporter rented land from small farmers, and co -managed it. By themselves, forward contracts proved to be ineffective as a means of eliciting the desired volume and quality of produce from small producers, although training of the farmers led to some improvements . Low volume, poor quality and untimely and inflexible supply response--these are the hurdles that small farmers need to overcome if they are to participate successfully in outgrower schemes . Small farmers need to be organized to bargain effectively with traders . Well-trained producer associations are essential if farmers are to improve farming and marketing practices . The expansion of outgrower schemes may be held up by the lack of investment in physical infrastructure . Pre-cooling units, packing houses and refrigerated transport are essential parts of this infrastructure and when this capacity is created it may be easier to promote the formation of producer groups . The ICR notes that a Bank-supported export promotion project in Senegal (P051610) showed how infrastructure investment could enhance marketing opportunities . Outgrower projects involving small farmers will only succeed if the private sector is adequately engaged . This project failed to address the substantial disincentives for large processing and exporting enterprises to work with small producers. There was no effective rapport between the government and the private sector, partly because of cumbersome government procedures . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is candid about the failure of this project to meet the learning expectations of a LIL . Owing to the substantial weaknesses in M&E it is not able to adduce the information needed to show what approaches may best help to alleviate the constraints on small farmer participation in outgrower schemes . The ratings proposed are generally more positive than the impression derived from reading the text . a.Quality of ICR Rating : Satisfactory