ICRR 11830 Report Number : ICRR11830 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 07/29/2004 PROJ ID : P006880 Appraisal Actual Project Name : Agriculture Technology Project Costs 84.0 85.9 Development Project US$M ) (US$M) Country : Colombia Loan/ Loan US$M ) 51.0 /Credit (US$M) 46.8 Sector (s): Board: RDV - Agricultural Cofinancing extension and research US$M ) (US$M) (54%), Sub-national government administration (38%), Central government administration (8%) L/C Number : L3871 Board Approval 95 FY ) (FY) Partners involved : Closing Date 12/31/2001 12/31/2003 Prepared by : Reviewed by : Group Manager : Group : John R. Heath Christopher D. Gerrard Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives "The Project aims to achieve the following objectives : Introducing a new approach, in support of the National System for Agricultural Technology Development and Transfer, including with measures to build institutional capacity, which would promote the development and testing of technical packages especially appropriate for small farmers, and to divulge new technology; Promoting private sector participation in agricultural research; Improving the quality and efficiency of agricultural reserach by introducing a demand -driven, competitive selection system that would cofinance projects with universities and private and public sector entities; Supporting decentralization of agricultural reserach and extension activities by strengthening regional centers, training agricultural extensionists, and disseminating information; and Revitalizing underused assets, both physical and human, that presently exist within Colombia's research services system. Research funded under the Project would ultimately help to reduce rural poverty, increase rural employment, and improve the competitiveness of Colombia's agricultural producers ". (Staff Appraisal Report) b. Components (i) Technology Development and Adaptation (Estimated cost, US$53.6 million; Actual cost, US$46.3 million). Comprises demand-driven investments in applied and adaptive research projects and technology transfer; (ii) Institutional Development (Estimated, US$30.4 million; Actual, US$39.6 million). Includes: establishment of Coordination Unit in Ministry of Agriculture and of a National Panel to provide technical assistance; set up of Regional Panels to assist in subproject selection; training; and establishment of an Agricultural Technology Information System. c. Comments on Project Cost, Financing and Dates The actual cost breaks down into Bank (US$46.8 million), government (US$9.1 million) and a balance of US$30.0 million subscribed by various public and private entities, universities and NGOs in the form of co -financing for research subprojects. The difference between the estimated and actual cost of the Technology Development and Adaptation component reflects reallocation of US$ 15 million from the original loan amount to help fund emergency disaster relief in response to the 1999 earthquake; this was partly offset by an eventual increase in counterpart funding but the reallocation was sufficient to cause concern that the project would cease to be viable, causing it to be rated temporarily as "at risk". The two-year delay in closing is not explained in the ICR but probably reflects interuptions in counterpart funding, possibly aggravated by the earthquake . 3. Achievement of Relevant Objectives: The original development objective statement may be collapsed into three elements, all of which were highly relevant. (i) Improve research and extension by increasing competition between service providers (Achieved ). The system of competitive funding that was introduced expanded the range of suppliers participating in publicly -financed research helping both to raise the quality of subprojects and to make them more demand -driven. The share of research spending co-financed by the various participating public and private entities was 60 percent, substantially higher than appraisal expectations (45 percent). (ii) Raise poor farmer incomes by developing and transferring appropriate technology (Achieved ). The project achieved the appraisal target of reaching 350,000 small farmers. A total of 635 technology transfer subprojects were financed, 98 percent of which were rated successful . Of a sample of small farmers surveyed, 71 percent showed a high level of acceptance of the technologies developed by the project . The actual nationwide adoption rate for technologies averaged 32 percent, which is deemed reasonable for this sort of project . Among farmers that adopted the technologies, average annual household income rose by US$ 1,533, reflecting increases in productivity and employment. (iii) Make the research and extension system more responsive to regional priorities (Achieved ). The project strengthened the decentralized planning process for technology development by setting up regional nodes, establishing a network of stakeholders, and strengthening extension agencies in the municipalities . Five administrative regions were involved in the process of subproject approval, the largest share of approvals being made by Center-East (30 percent) and West (29 percent). The decentralized project model has since been formally endorsed by government, helping to underpin the Colombian National System of Agricultural Science and Technology. 4. Significant Outcomes/Impacts: The overall institutional development impact was substantial . The competitive funding model has now been incorporated into the Ministry of Agriculture's planning framework . The following are examples of project results with a sustainable and system-wide impact: Development of methods for monitoring and evaluating research subprojects; Preparation of a guide for formulating municipal agricultural programs; Development of a directory of experts from which national and regional advisory panels were formed; and Development of a web-based Information System, specially adapted to meet the needs of local technical assistance providers. 5. Significant Shortcomings (including non-compliance with safeguard policies): Private sector co-financing is still relatively modest; The potential for integrating universities into the research system has not yet been fully realized; Local NGOs generally lacked the necessary capacity to prepare quality subprojects; There was a lower than expected dissemination of results between subprojects and regions; and There was some unecessary duplication of activities between subprojects and a lack of long -term strategic vision. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory The performance of the implementing agency was highly satisfactory . It coordinated the various elements of this complex project very well and developed a monitoring and evaluation capacity that was not built into the original project design. The agency was given an award of excellence in a competitive review by the Colombian government. Quality of ICR : Satisfactory NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. NOTE: 7. Lessons of Broad Applicability: This competitive funding model of reserach worked because it was founded on principles of transparency, inclusion, quality and efficiency, helping to enhance its legitimacy in the eyes of beneficiaries and financing agencies. Demand-driven research tends to favor the treatment of short -term problems and dispersion of funds may result in a lack of critical mass; some share of resources needs to be set aside for addressing longer -term, strategic issues--these would probably need to be allocated on an administrative rather than a competitive basis . Attention must be given to ensuring that the results from dispersed subprojects are adequately disseminated, helping to avoid duplication. Continuous monitoring and evaluation is critical to ensure that this complex, demand -driven and decentralized process produces and transfers high -quality research results. 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR is data rich reflecting the contribution from two detailed impact evaluation reports . But there are some flaws. The treatment of costs and benefits (Annex 3) presents a series of tables but with no discussion of assumptions and confusing terminology: separate reference is made to the "IRR" and the "social IRR" which might be taken to refer respectively to the financial and economic rate of return --but Section 4.4 says that a financial rate of return was not calculated. There could have been more detail on how the competitive process of sub -project selection actually worked. The rating of private sector development as modest (Annex 5 ) could have been fleshed out in the text, given that private-public partnership is such a central part of the project design . The sections on Bank and Borrower Performance were not properly developed . The reasons why the project closed two years later than expected may be inferred from references to counterpart funding delays, but are not explicitly stated .