ICRR 12317 Report Number : ICRR12317 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/10/2006 PROJ ID :P059930 Appraisal Actual Project Name :Id-decnt. Project Costs 23.57 19.8 Agricultural/forestry Ext US$M ) (US$M) Country :Indonesia Loan/ Loan US$M ) /Credit (US$M) 18.0 16.5 Sector (s):Central ): government US$M ) Cofinancing (US$M) administration; Sub-national government administration; Other social services L/C Number :C3280; L4510 FY ) Board Approval (FY) 00 Partners involved : Closing Date 03/31/2004 03/31/2005 Evaluator : Panel Reviewer : Division Manager : Division : Nalini B. Kumar Christopher D. Gerrard Alain A. Barbu IEGSG 2. Project Objectives and Components a. Objectives The project objective was to assist the Borrower in enhancing farmers' capacity to participate in extension activities and in strengthening the capacity of the district -level integrated agricultural and forestry extension system which would promote economically feasible, environmentally sustainable, and socially acceptable farming practices and increased farmers' income. Project objectives were not revised during implementation . b. Components (or Key Conditions in the case of Adjustment Loans ): The project had three components : (i) Enhancement of farmers' capacity to participate in and to lead extension activities (appraisal estimate US$6.19 million, actual US$7.4 million) The component was to support the revitalization of farmer groups and organization of farmers' networks; building of capacity of farmers to participate in and to lead extension activities; and promotion of participatory extension methods and provision of media and technology support . (ii) Strengthening the district extension system (appraisal estimate US$ 8.20 million, actual US$ 7.8 million) The component would support the introduction of institutional and management reforms at district level; building the capacity of extension staff in the participating districts; and strengthening the extension support and delivery systems in participating districts. (iii) Provision of central extension policy and project management support (appraisal estimate US$ 3.61 million actual US$ 4.6 million) The component was to support the improvement of extension policy and conduct of special studies; strengthening the central extension support systems, including degree and non -degree training; and provision of technical assistance and project management support . While components were not changed there was considerable restructuring of the project through amendment to the Development Credit Agreement (DCA). In April 2001, the number of project districts was reduced from 20 to 16 when the District Center for Information and Extension offices were abolished in four of the pilot districts . Because of delays in initiating project activities the concept of having 10 pilot districts in five provinces in year 1 followed by 10 partner districts in a further five provinces in year 2 was abandoned and districts were thereafter referred to as "participating" districts. The DCA was further amended in June 2002 to allow the direct transfer of farmer managed activity grants comprising a Village Extension Plan to farmer groups through a Village Integrated Extension Management Unit, a committee of three persons elected by registered farmer groups; and to increase the proportion of Bank financing of farmer managed activities from 80 to 100 percent. In May 2004, the number of participating districts was increased to 21 as a consequence of three participating districts being sub -divided. c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The actual project cost was 84 percent of the project estimate . At appraisal government funding was to be US$ 5.6 million. Actual government support was for US$ 3.3 million. Based on the request of the government the project closing date was first extended from March 31 to December 31,2004 and then later to December 31, 2005. However the project was terminated abruptly at the instruction of the new Minister of Agriculture who assumed office in November 2004 as part of his effort to rationalize the role of externally -funded projects within the Ministry of Agriculture program. This created a lot of uncertainty within the Executing Agency as well as Bappenas and Ministry of Finance which was not resolved till early March 2005 when the official notification from the Ministry of Finance to the Bank proposed a revised closing date of March 31, 2005. 3. Relevance of Objectives & Design : Project objectives were substantially relevant and consistent with the government's policy toward increased decentralization, increased effectiveness of public services, encouraging self -reliance among rural communities and withdrawal of government support where it is not needed or where it can be provided by the private sector . Promoting decentralization of public services such as agricultural extension is also a major thrust of the Bank's current rural development strategy. 4. Achievement of Objectives (Efficacy) : Project efficacy is rated modest . Project objectives were achieved but with some major shortcomings. The project demonstrated the effectiveness of an integrated extension service at the sub -district level and the benefits to both farming communities and front-line extension workers of participative, farmer -led approaches to extension . But, as noted by the ICR itself (pp. 4 and 8), progress towards institutional change to support these reforms was patchy and less evident. Part of the problem was that the project was implemented in a challenging institutional environment . Other shortcomings were: The project was a program of targeted intervention, however, there were some high income /high cost activities (e.g. orchids, gurami fish etc) to which large proportions of the resources were allocated benefits from which went to a few better-off households (ICR, p. 5); At the district level an inter-agency district extension committee was to provide guidance to the extension centers. However most district extension committees remain dominated by the government and failed to realize their potential (ICR, p. 8); An effective system of disseminating relevant information to farmers through printed, visual and audio media has yet to be established (ICR, p. 10); Studies supported under the project have had limited impact on policy development and project operation (ICR, p. 11); Project M&E was poor and was not able to provide reliable data to assess project performance and failed to inform management or assist in extension planning and decision -making (ICR. p. 11). 5. Efficiency : Project efficiency is rated modest . The ICR notes that the cost-effectiveness analysis at the appraisal stage was an over-simplification of the economic impact of the project and was based on unrealistic and erroneous assumptions . Since the M&E was poor the ICR presents the economic benefit of the project in terms of villages and families directly impacted which was considerably less than envisaged at appraisal . The number of farmer groups who benefited was 59 percent and the number of villages was 75 percent of the revised (reduced) target. 6. M&E Design, Implementation, & Utilization: M&E in the project was ineffective and did not generate good data . Project performance indicators were not systematically tracked during the project implementation period . Even the physical progress has not been systematically aggregated and documented at the project level despite having such information at village and district level. 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): None. The project was classified as a Category "C" project. 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately IEG rates a project's outcome as Unsatisfactory moderately unsatisfactory when it is expected to achieve its major relevant objectives with major shortcomings . Institutional Dev .: Modest Modest Sustainability : Likely Non-evaluable There is a follow on project and the government has begun supporting farmer -led extension through its own resources . While farmer-led extension activities have a tremendous potential, sustainability of efforts begun under the project is dependent on the technical and financial support from local governments. Further external support and clearer definition of roles and responsibilities for both the district center for information and extension and district extension committees is essential to consolidate progress. Budget constraints at the district level can hinder support for activities begun under the project. Further, progress towards institutional change to support these reforms has been patchy . The National Center for Agricultural Extension Development has prepared a draft policy statement of objectives and strategies for decentralized, farmer -driven agricultural extension incorporating the project's principles which is yet to be formally adopted. Bank Performance : Satisfactory Unsatisfactory The ICR itself rates quality at entry unsatisfactory. Supervision was also weak. 13 supervision missions were undertaken. All of them rated overall implementation progress as satisfactory even though the Borrower had failed to put in place an effective MIS system . The ICR itself acknowledges that "monitoring of project progress by the Bank was weak, and this affected the quality of supervision. Supervision missions confined their interest to assessing physical achievements but no outcome or results-based indicators were monitored and the Bank could have improved the poor outcome on the MIS/M&E component through providing additional technical assistance" (ICR page 19). Borrower Perf .: Satisfactory Unsatisfactory While the PMU diligently managed the implementation of the project several factors raise concerns about Borrower ownership: (i) the abrupt early termination of the project by the new Minister of Agriculture resulted in great uncertainty which negatively impacted project implementation; (ii) delays in budgetary releases every year reduced the effective working period of the project by 15 months and negatively affected all project activities but most severely the entire sequence of farmer-managed activities; (iii) the Ministry of Agriculture did not establish an effective MIS system at the central PMU or in the district PIUs to guide project management and to capture project impacts. Neither were the capacity and skills of M&E staff upgraded to perform the expected role; (iv) reorganization of districts and agricultural support services following decentralization and regional autonomy laws introduced during 2001 necessitated the redesigning of the project and amendment of the Credit/Loan Agreement including reallocating of categories, which delayed implementation. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Based on the ICR the key lessons are : (i) Continuous monitoring and evaluation of project processes and impact with the active participation of beneficiaries and key stakeholders are essential for influencing policy and institutional change; (ii) Timely budget flows from central and local governments to the project implementing agencies down to the field level should be ensured; (iii) To ensure high adoption rates among farmers, extension activities should focus on introducing and /or improving low-cost, highly profitable, market-led production and processing activities; (iv) Management structures such as the district extension committees, which are set up to coordinate extension activities at the district level, need both political and financial backing to survive beyond the life of the project . 10. Assessment Recommended? Yes No Why? To verify the ratings and to draw lessons of experience . The decentralization of agricultural extension to the local government level is one of the major strategies that the Bank is adopting to revitalize extension in member countries. 11. Comments on Quality of ICR: The ICR is satisfactory and provides a candid account of the project's implementation experience . However there is inconsistency between the text and most of the ratings assigned .