ICRR 12186 Report Number : ICRR12186 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 09/21/2005 PROJ ID : P073911 Appraisal Actual Project Name : Second Agriculture Project Costs 8.91 8.73 Rehabilitation Project US$M ) (US$M) Country : Timor-Leste Loan/ Loan US$M ) 8.00 /Credit (US$M) 8.00 Sector (s): Board: RDV - General Cofinancing agriculture fishing and US$M ) (US$M) forestry sector (34%), Agricultural extension and research (33%), Central government administration (11%), Other social services (11%), Roads and highways (11%) L/C Number : Board Approval 02 FY ) (FY) Partners involved : Closing Date 12/31/2003 06/15/2005 Prepared by : Reviewed by : Group Manager : Group : Nalini B. Kumar Ridley Nelson Alain A. Barbu OEDSG 2. Project Objectives and Components a. Objectives The project was the second phase in a program of three projects and aimed "to improve the food security of rural families and to increase agricultural production in selected areas of Timor -Leste." The original Development objective remained unchanged throughout the project's implementation period . b. Components The project had four components : (i) Pilot Participatory Development and Natural Resource Management (appraisal estimate US$0.89 million, actual US$0.77 million) The component was to pilot an approach to strengthen the resilience of poor farming communities by helping them improve the management of their natural resource base and diversify their income . (ii) Rapid Infrastructure Rehabilitation (appraisal estimate US$2.71 million, actual US$2.92 million) The component sought to increase agricultural production in irrigated areas rehabilitated by the Project and stimulate off -farm employment in selected rural areas . (iii) Services to Farmers (appraisal estimate US$3.10 million, actual US$2.68 million) The component was to provide essential services to farmers to help them bridge transitional difficulties associated with lack of information, unavailability of production inputs, shortage of cash and poorly working markets . (iv) Program Management (appraisal estimate US$ 2.21 million, actual US$2.36 million) The component was to help the new Government evaluate key policy options, help upgrade core skills of agriculture staff and oversee the implementation of the agriculture program . c. Comments on Project Cost, Financing and Dates At appraisal government counterpart funding was to be US$ 0.91 million. Actual government funding was US$ 0.73 million. The tasks set out in the original design were ambitious given the low capacity in the Ministry and the short time frame (27 months) and the project required several date extensions . The project ultimately closed on June 15, 2005. 3. Achievement of Relevant Objectives: The project contributed to increased agricultural production though incremental production was less than envisaged at appraisal. The expectation at appraisal that double cropping in most of the irrigated areas would be combined with the use of fertilizer and high-yielding varieties by the end of the project was not met . The project contribution to food security appears to have been limited as component 1, which was to address long-term food security, did not take the needed broader approach towards improving livelihood systems . There are concerns regarding efficiency of resource use since the weak implementation capacity at the design stage was addressed by involving a considerable number of international advisors that absorbed a large amount of project funds.Over 40 percent of the project costs were used for consulting services . Further, though a rate of return of 20 percent was estimated for the project at appraisal, the ICR concludes that the originally envisaged quantifiable economic benefits were too high for all but the livestock component . At the ICR stage, recalculation of a rate of return was carried out for the irrigation and vaccination investments . The recalculated rate of return for irrigation investment was 6 percent compared to 20.6 at appraisal. For the light to medium damaged irrigation schemes it was 3 percent, compared to 35 percent at appraisal, and 11 percent for the community-based schemes compared to 26 percent at appraisal. The investments in the Pilot Agricultural Service Center did not show positive rates of return . For the vaccination program however the rate of return is reported to be about 23 percent compared to 28 percent at appraisal. 4. Significant Outcomes/Impacts: A total of 3,908 ha in 46 community-based schemes (186 percent of the appraisal target ) were rehabiliated. The project helped set up a functioning livestock vaccination service built on a country -wide network of trained Village Livestock Workers (VLWs). Challenges in the delivery of highly perishable vaccines through a cold chain system were successfully addressed; 123 km of farm-to-market roads (123 percent of the appraisal target ) were rehabilitated; The project made a significant contribution to capacity building in the new Ministry of Agriculture, Forestry and Fisheries (MAFF); The project was instrumental in the development of important laws and legislation for fisheries and quarantine regulation; 5. Significant Shortcomings (including non-compliance with safeguard policies): Monitoring and evaluation was poor and hence scope for learning from the experience of the pilot participatory development component is limited (ICR page 6); Capacity development in MAFF remained at the central level with little impact at the district level (ICR page 5); The implementation of component I was largely confined to plantation of trees and construction of fish ponds and a broader approach towards improving livelihood systems and food security was only partly undertaken; The services to farmers component had several weaknesses related to appropriateness of type of media chosen, the messages and timeliness of delivery (ICR page 8); The Pilot Agriculture Services Centers were not able to function satisfactorily because of management problems and difficult conditions; WUAs remain weak and need considerable support to develop the necessary community organizing skills (ICR page 7); There are concerns about efficiency of resources used (section 3 above). 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately While the project was implemented in a Unsatisfactory challenging post-conflict environment, there were some 'major' shortcomings as noted in section 3 and 5 of the ICR review. OED rates a project moderately unsatisfactory when it is expected to achieve its major relevant objectives but with 'major' shortcomings. Institutional Dev .: Substantial Substantial Sustainability : Likely Non-evaluable Significant capacity has been built in MAFF and ARP III is expected to help consolidate achievements under ARP II . However WUAs continue to be weak and need considerable support to develop the necessary community organizing skills as the basis for the development of functional operation and maintenance systems. A clearer delineation of the VLW's role in treating animals (a paid service) and in protecting animals through vaccination (a free public service) is still needed through intensive information campaigns. Further, infrastructure investment still lacks appropriate O&M arrangements. Bank Performance : Satisfactory Satisfactory But only marginally so. The project was ambitious, there were several design weaknesses and the pressure to implement negatively affected project outcomes. The ICR itself acknowledges that "A better understanding of farming systems and livelihood opportunities and a better analysis of the most severe pockets of food insecurity and its causes in both the upland and lowland areas could have led to an improved design or an early re-design of some of the project interventions and would have improved the projects overall outcome." Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. NOTE: 7. Lessons of Broad Applicability: Based on the ICR key lessons are : (i) Post-conflict situations provides no holiday from the exigencies of good practice within a reasonable time frame . A better understanding of the socio economic environment and integration of these findings into project design along with a more realistic implementation schedule could have contributed to a better project outcome . (ii) Project design has to be in keeping with local capacity . Foreign expertise can help but must not take over local responsibilities and should be use with a well -defined exit strategy. (iii) The building of adequate community participation requires time, resources and explicit attention to incentives for the stakeholders. Setting up WUAs is just the first step . More time and resources are needed to strengthen and consolidate gains so that their sustainability over the long run is assured . (iv) Clearly defined result-based indicators that are easy to monitor facilitate project implementation . For example, the project design and later implementation could have benefited from a clearer definition of food security at the project design stage together with a set of monitorable results -based indicators. 8. Assessment Recommended? Yes No Why? (i) to assess the outcome, institutional development and sustainability impact; (ii) to provide input to OED's LICUS study.. 9. Comments on Quality of ICR: The ICR is satisfactory.