Document of The World Bank FOR OFFICIAL USE ONLY Report No. 18686 IMPLEMENTATION COMPLETION REPORT PAKISTAN AGRICULTURAL RESEARCH II PROJECT (CREDIT NUMBER 2154-PAK) December 11, 1998 Rural Development Sector Unit South Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS US$1.0 Rs. 21.95 in 1990 (at appraisal), Rs. 23.84 in 1991, Rs. 25.14 in 1992, Rs. 28.13 in 1993, Rs. 30.64 inl994, Rs. 31.68 in 1995, Rs. 36.13 in 1996, Rs. 41.19 in 1997, Rs. 44.27 (at completion) WEIGHTS AND MEASURES (Metric System) FISCAL YEAR OF BORROWER July 1 - June 30 ABBREVIATIONS AND ACRONYMS AERU Agricultural Economics Research Unit MIS Management Information System ARI Agricultural Research Institute MOU Memorandum of Understanding ARP-I Agricultural Research I Project (Cr. 1158- MSU Management Support Unit PAK) MTR Mid-term Review ARU Adaptive Research Unit NARC National Agricultural Research Center ASIP Agricultural Sector Investment Project NCRP National Coordinated Research Program AUF Agricultural University, Faisalabad NMARP National Master Agricultural Research AUP Agricultural University, Peshawar Plan AZRC Arid Zone Research Center NWFP North West Frontier Province AZRI Arid Zone Research Institute PARC Pakistan Agricultural Research Council BARB Baluchistan Agricultural Research Board PARB Punjab Agriculture Research Board CGS Competitive Grants System PC-I Planning Commission Form No. I DCA Development Credit Agreement PCRWR Pakistan Council of Research in Water ECNEC Executive Committee of the National Resources Economic Council PIU Project Implementation Unit GIS Geographic Information System PRMP Provincial Research Master Plan GOB Government of Balochistan PSC Project Steering Committee GONWFP Government of North West Frontier SAR Staff Appraisal Report Province SARO Sindh Agricultural Research Organization GOP Government of Pakistan SAU Sindh Agricultural University GOPUN Government of Punjab TAIPAN Transformation and Integration of GOS Government of Sindh Provincial Agricultural Network Project ICB International Competitive Bidding UM University of Minnesota IDA International Development Association UNE University of New England ISNAR International Service for National USAID United States Agency for International Agriculture Research Development M&E Monitoring and Evaluation MINFAL Ministry of Food, Agriculture and Livestock 77 7 ~~~~Vice President: Mieko Nishimnizu .. < :: - a ; ~Country Director:'t Sadiq Ahmed : Sector Managers: Ridwan Ali and Michael Baxter 20 S . ~~~~~~Task Leader: Ivar Serejski- FOR OFFICIAL USE ONLY Table of Contents Preface .........................................................i Evaluation Summary ......................................................... ii Part 1: PROJECT IMPLEMENTATION ASSESSMENT .......................................................... I A. Project Objectives ..........................................................1 B. Achievement of Objectives ..........................................................1 C. Major Factors Affecting the Project ......................................................... I I D. Project Sustainability ......................................................... 13 E. Bank Performance ......................................................... 13 F. Borrower Performance ......................................................... 14 G. Assessment of Outcome ......................................................... 14 H. Future Operation ......................................................... 14 1. Key Lessons Learned .........................................................1 5 PART II: STATISTICAL TABLES Table 1. Summary of Assessments ................................................... 16 Table 2. Related Bank Loans/Credits ................................................... 18 Table 3. Project Timetable ................................................... 19 Table 4. Loan/Credit Disbursements; Cumulative Estimated and Actual ........................ 19 Table 5. Key Indicators for Project Implementation ................................................... 20 Table 6. Key Indicators for Project Operation ................................................... 20 Table 7. Studies included in the Project ................................................... 20 Table 8a.Project Costs ................................................... 21 Table 8b. Project Financing ................................................... 22 Table 9. Economic Costs and Benefits ................................................... 22 Table I Oa. Status of Legal Covenants (Original Agreements) ......................................... 23 Table I Ob. Status of legal Covenants (Amended Agreements) ........................................ 25 Table I 1. Compliance with Operational manual Statements ............................................ 26 Table 12. Bank Resources: Staff Inputs ................................................... 27 Table 13. Bank Resources: Missions ................................................... 28 APPENDICES 1. Mission's Aide-Memoire 2. Borrower's Comments MAP IBRD Map No.21279 This document has a restricted distribution and may be used by recipients onlly in the perfornnance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. i AGRICULTURAL RESEARCH 11 PROJECT (Cr. 2154-PAK) Preface This is the Implementation Completion Report (ICR) for the Agricultural Research 11 Project in Pakistan for which Cr.2154-PAK in the amount of SDR44.40 million (US$57.3 million) was approved on 27 July, 1990 and made effective on 2 August, 1991. The IDA Credit was closed on June 30, 1998. A total of SDR4.0 million (US$5.16 million) was cancelled. The Credit was not fully disbursed at completion, with SDR12.33 million (US$17.282 million) remaining unspent which has been cancelled effective October 31, 1998, which is the final date of payment under the Credit. The ICR was prepared by a mission from the FAO World Bank Cooperative Program, which visited Pakistan from 9-27 June, 1998. It is based on material in the project files, field investigations and discussions with staff of the World Bank, Pakistan Agricultural Research Council (PARC), Project Management Units of PARC and the provincial governments, and research institutes and centers of PARC and North West Frontier, Baluchistan, Sindh and Punjab provinces associated with the project. Representatives of the above agencies contributed to the preparation of the ICR by submitting data required, arranging field visits and meetings with research staff concerned with the project, and providing comments reflected in both the mission's Aide Memoire and the ICR. The Borrower's comments are appended to the ICR. 1 Drs. D. A. Ivory (Mission Leader) and P. J. Smith (Research Institutions Specialist, Consultant). ii IMPLEMENTATION COMPLETION REPORT PAKISTAN AGRICULTURAL RESEARCH II PROJECT (Cr. 2154-PAK) Evaluation Summary Introduction I. The Govemment of Pakistan (GOP) requested assistance from the International Development Association (IDA) to further develop and strengthen its agricultural research services following prior support (first Agricultural Research Project; Cr II 58-PAK) to strengthen the federal research system to enable it to guide and co-ordinate research throughout Pakistan. Project Objectives and Components 2. The original Development Credit and Project Agreements of July 27, 1990 were revised twice, first by the Association's letter of April 28, 1993 and for the second time, in July 25, 1995, as a result of the mid-term review (MTR) of February 1995. The original objectives were modified only slightly. The revised objectives were to: (i) improve andc strengthen research capabilities at the provincial level through the establishment of a basic inistitutionalfranaework within which a better research program may be developed, (ii) support priority agricultural research programs both at the federal and the provincial level, (iii) consolidate and complement the achievements already made in improving and strengthening agricultural research capabilities at the federal level, and (iv) improve and strengthen the linkages between agricultural research and agricultural extension. The project supported: the creation, equipping and training of staff of Management Support Units (MSUs) for these bodies and the provision of additional staff (research scientists and staff for MSUs), research and administrative buildings, vehicles and laboratory equipment, human resource development (78 places for Ph.D.and 34 places for M.Sc./M.Phil. courses overseas and 1,200 months of local and 300 months of overseas short-term training); and 327 months of international and 96 moniths of national technical assistance (TA) in management and in research. In support of the second objective, the project provided for the production of provincial and national research master plans; infrastructure and operational support for selected priority research programs; and introduction of a Competitive Grants System (CGS), following the project mid-term review. The third objective to further strengthen the federal research system was to be achieved through: construction of scientific and administrative buildings for PARC/NARC and its distant institutes/centers; laboratory equipment; techinical assistance (127 months of international and 30 months of national TA) and human resource development (places for 32 Ph.D. and 6 M.Sc./M.Phil. courses overseas and 372 months of local and 161 months of foreign short-term training); and support to two Agricultural Universities, mainly to strengthen their social sciences disciplines. The project aimed to attain the fourth objective through joinlt diagnostic studies, creation of Adaptive Research Centers, and strengthiening of the Agricultural Economics Research Units (AERUs). 3. Special Covenants and Agreements. Major covenants concerned: reorganization of PARC Planning Directorate; the assignment of overall coordination responsibility for agricultural research (in the wide sense) to newly established provincial autonomous research bodies, and delegation to them of powers to establish their own rules; establishing MSUs; provision of funds in support of the twinning of Sindh Agricultural University (SAU) and Agricultural University Faisalabad (AUF) with overseas iii universities; and preparation of provincial and national research master plans. Following the project mid- term review, it was agreed that the CGS would be initiated, and that the provinces would assume financial responsibility for continuing this work following project closure. 4. Evaluation of Objectives. The four main objectives are clear, relevant, and build on earlier bank lending. Implementation Experience and Results 5. Project Success. While there was significant variation between provinces in project implementation, overall the project outcome was unsatisfactory, as it failed to fully achieve each of its four main objectives. While the project has met or slightly exceeded some of its physical targets by Credit closure (such as awards of overseas training fellowslhips, TA support, procurement of vehicles and equipment and most of the civil works), all of these were achieved very late, thereby minimizing the potential benefits. Many activities, for example the development of priority research programs, were severely limited by the very delayed preparation of national and provincial master plans and the slow/delayed release of funds. These delays arose from deficiencies of the Government's institutional systems and procedures, inappropriate choice of strategies to achieve the project objectives, and interactioin of these factors. Overall, there was little significant improvement in research/extension linkages, although there was variation between provinces. 6. Costs and Financing Arrangements. The final total project cost (following completion of Credit disbursement by October 31, 1998) is estimated at Rsl,925 million (US$55.023 million), against the original estimated cost of Rsl,770 million (US$81.933 million). The large devaluation (120%) of the Rupee against the Dollar during the project period meant that while total project expenditure was 9% greater in Rupee terms it was 33% less in Dollar terms. Of the total IDA Credit of SDR44.40 million (US$57.30 million), SDR28.070 million (US$40.123 million) had been disbursed to the GOP by final Credit closure oni October 31, 1998. A total of SDR4.0 million (US$6.219 million) of the Credit was cancelled during the project period. The undisbursed Credit at October 31, 1998 was SDR12.330 million (US$17.282 million). The use of the Credit was not only greatly reduced, but also considerably delayed with only 20% of the Credit disbursed in the first four years. 7. Implementation Timetable. There was no detailed activity timetable prepared in the SAR. Implementation of most components was greatly delayed. TA was expected to be in place in 1991 (but was actually in place in 1993); the research master plans were expected in 1992/93 (actually becoming available in 1997); and the long-term overseas training (Masters and PhDs) was expected to be initiated by June 1991 and completed by June 1998 (but was not started until 1994 and will only be completed by June 1999, with fellowship costs for the post-project closure period pre-paid, with special permission from the Bank). Factors Affecting Achievement of Major Objectives. 8. The failure of the project to fully achieve its major objectives was a result of the inability of the provinicial governments to complete the institutional changes envisaged in the research system and the delayed establishment of an effective project management system. The failure to achieve project objectives was associated with the following factors: (a) Project Designi. While the SAR identified the risk arising from the expected difficulty in reorganization of the research services, there was seemingly insufficient consultation with the iv relevant government departments to obtain the ownership and commitment to the basic concept of the major institutional innovation and the detailed administrative and legal changes required. Both the proposed innovation itself and the reliance on project covenants to obtain compliance were based on the incorrect assumption that formal structures and procedures alone determine organizational effectiveness. In addition, insufficient time was allocated for the administrative processes required, in drawing up the implementation plan and the phasing of project activities and expenditure, with the result that the expected timeframe for achievement of implementation targets/components was over-ambitious. (b) Decision Making Process. The government administrative system is quite centralized, and many of the prescribed processes require repeated scrutiny of matters already decided. The prevailing cumbersome bureaucratic procedures resulted in significant delays in: approval of the umbrella PC-I by the Executive Committee of the National Economic Council (ECNEC); establishment of provincial project MSUs and their staffing; procurement of equipment; contracting of TA services; and placement of overseas trainees. (c) Management. There has been a rapid turnover of senior management staff within all provincial government agencies associated with project implementation, resulting from the exercise of personal influence over the placement of preferred staff members, the pursuit of attractive postings and, at the higher levels, political changes. In addition, in many instances political and highly personalized policy decisions have been made which cut across the intended functional unity of the agricultural research system. Such factors have destabilized management, broken continuity and commitment, and dissipated the benefits of training. Furthermore, the reduced status of PARC has affected its autonomy and performance, and vitiated the project objective of consolidating the achievements of ARP 1. Finally, the highly directive style of management suppressed the contribution of middle-managers and scientists to improving institutional performance, limited the extent to which project staff were able to discuss delays and problems with senior staff, and in many cases cut off most attempts at establishing informal professional links between Ministries/ Departments. (d) Financial. Budget policies and financial release and distribution procedures of the federal and provincial governments (due largely to deteriorating fiscal imbalances of the successive regimes) resulted in delays in utilization of the Credit and reductions and delays in actual counterpart budget releases in each financial year. 9. Performance of Bank. The perceived deficiencies in project design noted above should have received deeper attention at Appraisal. The Bank participated in eleven supervision missions, and conducted the mid-term review of the project; suggestions for improvements were comprehensive, and appreciated by the Borrower. However, the Bank should have been more sensitive early in project implementation to the major problem encountered in establishing the autonomous provincial agricultural research bodies. Even before the MTR, the Bank should have thought to undertake an intensive, joint examination of the underlying institutional and political issues, with a view to finding a mutually- acceptable solution and agreed timebound framework for implementing the necessary institutional framework reforms. Following the MTR, the Bank had to threaten to use its remedies twice. In both cases, however, the GOB took some actions that led IDA to believe that all the reforms would be implemented as agreed. However, this did not materialize. The Bank was mild in giving sufficient emphasis in its supervision to project activities that would strengthen research-extension linkages. v 10. Performance of Borrower. There was a general lack of continuing ownership and commitment to the project concept of autonomous research bodies at the higher levels of government, which created difficulties in project implementation at the lower levels. Administrative, political, managerial and financial constraints all contributed to an overall unsatisfactory performance of the Borrower. Many covenants of the Development Credit Agreement were not fulfilled by the Borrower in due time, leading to serious delays in project activities. The approval of the umbrella PC-I at the federal level was unnecessarily delayed by up to one year, resulting in a delay of up to two years in the appointment of the project director and key support staff in each province. This consequentially adversely affected the implementation of training programs, contracting TA, preparing master plans and initiating high prioirity research programs. Future Operations, Sustainability, and Key Lessons 11. Future Operations and Sustainability. Benefits from training and improved research facilities are probably sustainable. The sustainability of the research planning and prioritization processes are more dependent on future funding and support as many of the research management innovations (MSUs and their associated management tools) have not been institutionalized into a comprehensive, functioning system. Of the autonomous research boards, the Sindh Agricultural Research Organization (SARO) is unlikely to survive; the Punjab (PARB) and Baluchistan Agricultural Research Boards (BARB) will probably continue, though with a narrowly agricultural research mandate, while the proposal to replace the University-led integrated research system in NWFP with an autonomous provincial body is still uncertain. Adequate funding from governments will be critical to the survival of autonomous research bodies beyond Credit closure. The CGS established under the project is likely to survive in Punjab, and in NWFP and Baluchistan if support from the proposed World Bank Agricultural Sector Investment Project materializes, but not in Sindh. 12. Key Lessons Learned. The main lessons learned from the project are concerned with issues associatecl with project design, implementation planning, institutional change, management performance, and project performance: (a) Project Design. Over-ambitious project design and objectives should be avoided. In particular, a national level project should be avoided where project design is complicated, project scope is extensive and implementation difficult thus causing difficulties in providing adequate supervisory support to all four provinces and also the federal level. (b) Imiplementation Planning. An implementation timetable should take into account the aciministrative realities of the processes for appointing key project staff, mobilizing counterpart funds, and initiating activities with long lead times (particularly appointing key staff, procurement, recruitment of TA, and legislative components of major institutional changes) and the major deficiencies in Government administrative and financial systems. The possibility of dealing with these matters in a relatively long pre-project period where appropriate covenants must be met before project funding commences should be considered. The use of computer-based methods to prepare realistic implementation schedules should be considered and phased expenditure forecasts should be based on these analyses. (c) Institutional Change. Where institutional change is needed, it is essential to evaluate the options of either (i) creating new bodies, regulations and administrative procedures or (ii) providing the resources and human skills required to enable staff to function better within the existing structure. Where substantial institutional change is envisaged, it is necessary to thoroughly analyze existing vi systems as a basis for designing these new institutional structures, rules, procedures, management strategies and tactics, and for designing programs of change to bring them into effective existence. It is essential that these processes are carried out in collaboration with all the parties involved on the Government side. (d) Management Performance. It is essential that designated individuals be given responsibility for the management of the project and each of its major components, and that they retain these positions throughout the project in order to achieve commitment, vision and managerial stability. (e) Unsatisfactory project performance. The Bank should be less compromising and terminate projects or components of a project when there is a persistent unsatisfactory performance. IMPLEMENTATION COMPLETION REPORT PAKISTAN AGRICULTURE RESEARCH II PROJECT (Cr. 2154-PAK) PART I: PROJECT IMPLEMENTATION ASSESSMENT A. PROJECT OBJECTIVES I. Project Objectives and Scope. The project's original main objectives were to: (i) improve and strengthen research capabilities in the provinces, (ii) consolidate and complement the advances achieved under the first Agricultural Research Project (ARP I) at the federal level, that is with the Pakistan Agricultural Research Council (PARC) and National Agricultural Research Center (NARC), (iii) support high priority research programs at the provincial and federal level, and (iv) improve and strengthen research/extension linkages. The major thrust of the project was to preferentially strengthen research capacity in livestock, soils, horticulture, integrated pest management (IPM) and social sciences, as well as research management, as research capacity in these areas was identified as being much weaker than in other research disciplines, both at the provincial and federal levels, thus building on the achievements of ARP 1. 2. Project Changes after Appraisal. The Development Credit Agreement (DCA) and Project Agreements were revised twice: first, by the Association's letter of April 28, 1993 and later, by the letter of July 25, following the mid-term review (MTR) in February 1995. The original objectives were only modified slightly. The revised objectives were to: (i) improve and strengthen research calpabilities at the provincial level through the establishment of a basic institutionalframework within wthich a better research program m7ay be developed, (ii) support priority agricultural research program7)s both at the federal and the provincial level, (iii) consolidate and complement the achievements alreadyv made in improving and strengthening agricultural research capabilities at the federal level, and (iv) improve andcl strengthen the linkages between agricultural research and agricultural extension. 3. Evaluation of Project Objectives. The objectives were clear, realistic, and built on foundations laid with the help of an earlier IDA Credit. B. ACHIEVEMENT OF PROJECT OBJECTIVES Improve and Strengthen Research Capabilities in the Provinces 4. Staffing. In the four provinces, a total of 231 additional staff were proposed to be recruited for (i) the Management Support Units (MSUs) to be established under the project, and (ii) identified scientific positions. While the allocated positions were filled in North West Frontier Province (NWFP) and Baluchistan province, many positions remained vacant at the end of the project in Punjab province (47%) with fewer vacancies remaining in Sindh province (9%). Many of 2 the proposed incremental staff were, in fact, transferred from elsewhere in the government system, often without replacements. There appear to be no permanent staff in the provincial agricultural research boards and their associated MSUs, as staff are reluctant to be permanently assigned to these bodies because of concerns over job security and pension rights. The appointment of key project staff was delayed by the procedures required to provide local budget for them, which, in turn, was delayed by the late approval of the umbrella (federal) Planning Commission Form I (PC-I), the local project implementation document. 5. Civil works. A limited amount of construction of new facilities was proposed under the project. Facilities constructed included administrative buildings, library and seminar halls, green and shade houses and garages in Sindh province, and an administrative block (not completed due to contractor dispute) and shade house in Baluchistan province. No facilities were proposed in NWFP, and in FPunjab province the proposed administrative building was not started because of a land dispute. 6. Equipment and vehicles. Vehicles were purchased by all provincial governments to improve mobility of administrative and research staff. While in NWFP and Baluchistan, these were purchased early in the project, in the other provinces many vehicles were only purchased towards the end of the project. Some strategic equipment was purchased for laboratories in all provinces. Most equipment purchased was for soils research (soil physics and soil chemistry), horticultural research and livestock research. Considerable delays occurred in the purchase of all equipment requiring International Competitive Bidding (ICB) (refer para. 43). A second major tranche of equipment prepared for purchase during the last two years of the project could not be ordered and purchased before Credit closure because of the administrative delays, equipment price escalation and complicated bidding procedures. 7. Long-term Staff Training. Major emphasis has been given to human resource development. The original proposal was to provide 75 Ph.D. and 25 M.Sc./M.Phil. fellowships for long-term training in selected disciplines, for staff of provincial research institutes/centers, based on a needs assessment at the time of project preparation. All Ph.D. and M.Sc./M.Phil. fellowships were undertaken at overseas universities. It was proposed that the provinces would identify the first group of trainees by March 31, 1991 and that all long-term training fellowships would be taken up by June 1994. Placemnent of all fellowships was considerably delayed. Fellowship targets were exceeded, with a total of 78 Ph.D. and 34 M.Sc./M.Phil. fellows placed overseas. The last fellowships were only approved and placed overseas late in 1995, which has meant that while 79% of M.Sc./M.Phil. candidates have returned, only 6% of the Ph.D. candidates have returned.2 Some eight candidates dropped out for various reasons.' The major problem with this program was that the selection processes did not commence early enough and the responsibility for placement was assigned to the technical assistance (TA) contractors (except in NWFP). Thus the delays in establishing the project managers and MSUs for the provincial components of the project (para. 15) and recruitment of the TA (para.9), inevitably caused serious delays in the staff training program. Except in the case of Discipline areas chosen for post-graduate studies were in line with the major thrust of the project which was to preferentially strengthen research capability in soils, horticulture, IPM, livestock and social sciences and research management. 2 The Banik has approved the provision and allocation of sufficient funds with the relevant technical assistance contractors to finance the completion of the outstanding period of unreturned fellows. Seven students failed and one student has deceased. 3 Baluchistan province, where there was limited local experience in overseas student placement, the weak structure of the apex bodies with persistent staff shortages was probably the main cause to assign this responsibility to the TA contractors in Sindh and Punjab provinces. 8. Short-term Staff Training. An extensive program of both overseas and local short-term training was also provided. It was proposed that a total of 300 months of overseas and 1,200 months of local training would be provided for both scientific/technical and managerial staff to meet critical training needs in horticulture, livestock, soil science, IPM and social science and for research management and organization. In addition, to this more specific training, some local training funds were used for attendance at local seminars, workshops and conferences. The short-term training program was underutilized, with a total of 225 months and 1,007 months used for overseas and local training, respectively. There has been no assessment of the value of this short-term training. 9. Technical Assistance (TA). TA was seen as essential for planning and implementation of parts of the project. A total of 423 months of technical assistance was proposed for the four provinces, 327 months for expatriate assistance and 96 months for local consultants. Recruitment of TA was, however, delayed by more than two years due to extremely slow processing of the TA bidding and contractual arrangements. This caused two major problems. Firstly, the long-term training program was delayed by more than two years, as responsibility for student placement at overseas universities was given to TA contractors in Baluchistan, Sindh and Punjab provinces. Secondly, the Provincial Research Master Plans (PRMPs), which required a major input from TA, were completed more than three years after the original scheduled completion date. However, once started, the TA input has generally been appropriate and well appreciated by those closely involved with the consultants (but not perhaps generally), except in Baluchistan where seemingly some aspects of the performance of consultants were criticized. Many valuable reports were produced by the consultants, but relatively few of the recommendations to improve research management systems, research programs and research methodologies and practices have been implemented or followed up. 10. Institutional Reorganization. A major objective of the project was to establish autonomous apex research bodies in Sindh, Punjab and Baluchistan provinces to manage research in agriculture, livestock, fisheries and forestry. Different institutional arrangements were foreshadowed in each province. 11. North West Frontier Province. In NWFP the project objective was to maintain the existing institutional relationship where the Government of NWFP, assisted by the USAID-financed Transformation and Integration of Provincial Agricultural Network Project (TAIPAN), had transferred responsibility for crop and livestock research from the Department of Agriculture to the Agricultural University, Peshawar (AUP) in the late 1980s. Project implementation was therefore the responsibility of the Vice Chancellor of the University, who, through a Director General of research, had responsibility for managing research in research institutes/centers and stations in agriculture and livestock production, but not in animal health, fisheries and forestry. The revised Project Agreement of July 1995 provided for the submission of a time-bound action plan to address key institutional issues identified during the MTR as well as the merger of AUP and agricultural research. A draft plan was prepared but never implemented. 12. Baluchistan Province. In Baluchistan province the project supported the establishment of the Baluchistan Agricultural Research Board (BARB) which was proposed as an autonomous body to 4 control allocation of research resources in agriculture, livestock, fisheries and forestry. While BARB was finally established and new rules of business promulgated, after much delay, research in forestry and fisheries has not been included. Throughout the project duration, a major part of horticulture research, and all of livestock research was outside the control of the Department of Agriculture., BARB now appears to command genuine support. 13. Sindh Province. In Sindh province the project aimed to support the establishment of a semi-autonomous research organization, to be called the Sindh Agricultural Research Organization (SARO) which would merge the research activities of the Sindh Department of Agriculture (crops, livestock and fisheries). SARO was established after considerable delay and with little commitment (as indicated by the use of a legally weak Administrative Order). However, major institutional restructuiring occurred during the project, which further prejudiced the establishment of a functioning autonomrous research organization. The previous Ministry was divided into two new ministries, a Ministry of Livestock and Fisheries and a Ministry of Agriculture, Science and Technology (MAST). Subsequently, MAST was further divided with agricultural research being virtually the sole activity of a separate Department under a Secretary for Science and Technology, cutting it off from agricultural extension. Forestry is in a separate Ministry, although this structure is under review. It is unlikely that SARO will continue to exist after Credit closure. 14. Puni(ab Province. In Punjab province, the project supported the establishment of the Punjab Agricultural Research Board (PARB) which was proposed as an autonomous body to control research resource allocation in agriculture, livestock, fisheries and forestry. The establishment of PARB has been a protracted. Initially it was established under Notification in 1991, with subsequent strengthening of its position by the issue of an Ordinance in September 1996, which was renewed at the requiired 3-month intervals until the passing of the Punjab Agricultural Research Bill in October 1997. From 1995 until 1997, PARB was transferred from the Agriculture Department to the Planning and Development Department as its attachment, because of provincial dissatisfaction with its progress. However, immediately after the Credit closing, it was again transferred back to the Department of Agriculture. PARB's rules of business and conditions of service have yet to be changed., Presently PARB is responsible for agriculture, livestock and fisheries research, but not forestry research. In its present form, PARB is legally secure, and it is understood from the Planning and Development Department that agricultural research is likely to receive increased financial support in the near future from the Agricultural Income Tax. 15. Management Support Units (MSUs). After significant delays, MSUs were established in each province to advise on and assist with research planning, resource allocation, programming, monitoring and internal evaluation. In Sindh province the MSU was established as a secretariat of SARO. In NWFP the MSU was attached to the office of the Agricultural University of Peshawar's (AUP) Director General of Research and as a secretariat for the AUP's Research Coordination Board. In Punjab and Baluchistan provinces, MSUs were established and functioned as secretariats for PARB and BARB, respectively. In addition, in Punjab smaller MSUs were also established for the Director General's of Agricultural and Livestock Research. In-service training was provided for staff of the MSUs as part of the international technical assistance to develop systems and skills for planning and project budgeting, and to establish functional management information systems and The horticultural section has become incorporated into the Department of Agriculture under BARB since project closure. 5 monitoring and evaluation procedures. The skills and systems functionality achieved seems variable, with some MSUs apparently unable to operate the systems piloted/established. In some cases, this is attributed to a shortage of computer-literate staff, but there may be other reasons, such as managerial commitment and non-evolution of work program for the limited institutionalization of the skills and tools provided under the project. 16. Management Information Systems (MIS). The International Service for National Agricultural Research (ISNAR) "INFORM" methodologies have been introduced in most provinces, using database software, as well as the introduction of spreadsheet software for preparation of budgets, with support of the overseas technical assistance provided. It seems in many cases, however, that fully functional systems oriented towards decision making have not been realized where the relevant sections of the MSUs of the management bodies established under the project use such systems routinely for management purposes. 17. Monitoring and Evaluation (M&E). The project provisions for establishing M&E systems in provincial management units, although delayed, were ultimately adequate, and fairly fully implemented with TA support. The systems established focus on the flow of core technical activities of the project: planning and conduct of research, etc. However, no provision was made for installation and training of a CPM-based project management system, to give staff the means to prioritize, plan and control project implementation activities (recruitment, contracting TA and training services, procurement, and administrative processes connected with budget, program approval, etc.). This contributed to the delays. Initially, project staff were often unable to identify the most critical tasks for immediate attention; later, delays were not identified until they had grown into major problems. Support Priority Research Programs at Provincial and Federal Levels Provincial Level 18. Research master plans. All provinces finally completed PRMPs after delays of three to four years. The plans were based on information obtained from diagnostic studies and internal reviews and discussions. Significant assistance from the international TA was utilized in finalizing the documentation. The PRMPs produced are very comprehensive and generally achieved the objectives of determining priority programs and projects for research, manpower requirements, training needs, facilities, equipment and budget requirements for operations and maintenance and capital and recurrent requirements for a five-year period for each province. Indicative additional requirements for five years were also determined. Some work remains to be done. Plans have been made on the assumption of some level of increased resources, but post-project financial resources may be reduced. An effort is still needed to adjust the plans to current realities and to provide activity timetables for the guidance of planning units. Furthermore, a final stage of development would be required to identify and prioritize individual research studies within the often broad topics defined, as this ultimately defines the quality of the scientific output. Within these limits, PRMPs are already providing important resource documents for planning the development of provincial research systems. Because these plans have only been completed recently there has been no opportunity to update them on a regular basis as originally envisaged. The National Master Agricultural Research Plan (NMARP) was prepared simultaneously and the information contained in the PRMPs was consolidated into an input to the NMARP. 6 19. Priority research programs. A major objective of the project was to orient the research systems to only support research that was identified as being of high priority. Some high priority research programs were identified at appraisal and additional priority research programs were to be identifiecl during the PRMP preparation process. This was a major objective of the development of PRMPs. But because of the delays in preparing the PRMPs, support for additional research programs was only provided during the last two years of the project. Thus, although two groups of high priority research programs were supported, overall budget support for this activity was greatly under-utilized. However, in line with the objective of strengthening research in thematic areas which were considered to have been poorly supported previously, priority has been given to research programs concernedl with soils research (soil fertility and soil physics), integrated pest management, horticulture (vegetables, fruits and ornamentals), livestock (animal nutrition) and in social sciences. Although necessary, there has been no internal or external assessment of the quality and potential impact o-f the priority research programs. A superficial assessment indicates variability in quality with some very good programs (e.g., IPM in Sindh province) initiated and generally weak programs in animal nutrition in all provinces, with the exception of Punjab. However, in some instances changes in institutional arrangements have adversely affected some programs. For example, in Balochistan province the fruit research program was taken outside the project following transfer of this research unit to the Department of Agricultural Extension. Also in Punjab province an effective IPM program has been stymied by the establishment of an IPM Institute, separately from an existing Plant Protection Institute and separate Departments of Entomology and Plant Pathology, as well as a lack of staff and functional laboratory for the IPM Institute. A common complaint from scientists, at least in the provincial research systems, was that their priority research programs were adversely affected by delays and reductions in budget allocations. 20. Competitive Grants System (CGS). CGSs were established in all four provinces, as a recommendation of the project's mid-term review. The CGS was introduced with a view to, inter alia, (i) increase the allocative efficiency of funds for agricultural research, (ii) diversify the funding away from block grants to government institutions and tap underutilized resources in the wider research and development community, and (iii) quickly shift resources to address specific, high national research priorities and farmer identified problems. Guidelines were established to set up a system of advertising, submission and selection of proposals, which would be open to all research agencies (both public and private) and universities, to be considered for funding of high priority research. Fully functional CGSs were set up in NWFP, Baluchistan and Punjab provinces. In Sindh province, however, although proposals were received and selections of research projects made, no research was contracted to the applicants. Because of the relatively short time available for implementation of the CGS within the remaining project period, this was considered a pilot scheme and guarantees were obtained from the provinces that they would continue to fund these contracted projects beyond Credit closure. Because most programs have only been undertaken for one year it is too early to assess the effectiveness of the program. Support for the approved projects and the program may be provided for in Punjab, Baluchistan and NWF provinces under the proposed Agricultural Sector Investment Project (ASIP). The continuation of CGS program is, however, problematic in Sindh province, which has not submitted proposals for funding under ASIP. Federal Level 21. Research in Priority Areas. The project: (a) supported research work in NARC in soil fertility, chemistry and soil physics, and provided a new laboratory, Geographic Information System 7 (GIS), plotter, and digitizer to strengthen the Soil Survey of Pakistan program; (b) supported research into improved water use efficiency, drainage and water recharge and water quality, and provided a building, computer, and equipment to strengthen the Pakistan Council for Research in Water Resources; (c) supported fruit, vegetable, mushroom and floriculture research in the Horticulture Research Institute at NARC; (d) supported the National Coordinated Research Programs in buffalo breeding and dairy cattle cross-breeding; (e) supported livestock research at the Arid Zone Research Center (AZRC), Quetta; (e) supported rangeland research and long-term training at the Arid Zone Research Institute (AZRI), Umerkot; and (f) supported research programs for development of agriculture in upland and mountain areas at Karakorum Agricultural Research Institute, Jagalot and the Himalayan Agricultural Research station, Kaghan. All centers/institutes/stations indicated that research work was constrained by both delays and reductions in budget releases and by equipment not being purchased and delivered from PARC. Despite these constraints, a superficial assessment of the programs indicates that all have produced useful and relevant work, but less than anticipated. There has been no internal or external assessment of the quality and potential impact of the priority research programs, even though this would be useful. 22. Competitive Grants Scheme (CGS). The CGS initiated from PARC attracted 101 applications for research grants, of which 49 recommended projects were submitted for the Chairman's approval. However, approval was not given to these projects because of the shortage of local funds from the GOP. The establishment of a functional CGS at the federal level will depend on obtaining funding support under ASIP. 23. Collaborative Research with Overseas Institutions. It was envisaged that PARC would operate a small (US$0.5 million) collaborative research program between eminent overseas research institutions and federal or provincial research institutes doing similar work. Only one such program was initiated, although very late in the project period. PARC contracted a small collaborative research program between IRRI and PARC/Provincial Governments which could not be fully completed at Credit closure. Consolidate and Complement Previous Support at Federal Level Support to Pakistan Agricultural Research Council, National Agricultural Research Center and Other Associated Institutes 24. Institutional. Federal institutional restructuring during the project resulted in PARC's incorporation under the Ministry of Food, Agriculture and Livestock (MINFAL) and has been accompanied by a deterioration of its funding and functioning. The Council now receives allocations from the Ministry, which are normally less than required or approved, and these releases are normally late, and/ or piecemeal. At the same time, PARC's ratio of fixed cost: operational cost has deteriorated to about 80:20. The Council acknowledges that this is partly the result of staff being transferred to its non-development budget after each of a series of projects, a normal procedure. Internally there has been a restructuring of PARC's Planning Directorate to constitute four-units, including Planning and Programming, Review and Evaluation, Research Inventory and Implementation and Monitoring Units. The new Directorate itself is functioning well, and made a substantial contribution to the production of the National Master Agricultural Research Plan (NMARP). 8 25. Civil Works. Major infrastructure development was envisaged within the research institutes and centers of PARC. A new crop science building for the Plant Science Division of NARC was constructed and facilities at other distant locations were upgraded/expanded, including the Northern Areas Research Center at Jagalot and Mountain Research Center at Kaghan. A new laboratory and ancillary complex (42,000 ft2) was approved by IDA in 1995 for six programs (animal health research, fisheries and aquaculture research, range and environment research, farming systems research, technology transfer and seed sciences and technology), was also completed. Houses for junior staff and an additional water supply system have also been constructed. All have been completed, except at Gilgit where it is expected to be finished by the end of December 1998. This has all been handled effectively by PARC's own engineers, in contrast to the poor experience with contractors elsewhere in the project. 26. Procurement of Vehicles and Equipment. Approximately 75% of the approved budget has been spent, mainly on vehicles and scientific equipment. The preparation/bidding and ordering of a second major tranche of equipment was very delayed and was cancelled because of impending Credit closure (refer also para. 6). 27. Long-term Staff Training. The objectives of long-term training for staff of the PARC and its centers and institutes was the same as for the provincial research system (para. 7). Fellowships were orig;inally proposed for 25 Ph.D. and 6 M.Sc./M.Phil. slots. With Bank agreement, these were increased to 44 Ph.D. and 13 M.Sc./M.Phil. slots. This represents a substantial knowledge and skills asset when all fellows return. However, because of delays in the implementation of the training program, 5 Ph.D. and 5 M.Sc./M.Phil. students had not returned by Credit closure. These students will complete their studies during 1998-99 and will be given further financial support by the Bank beyond Credit closure. 28. Short-term Staff Training. The utilization of overseas short-term training by staff of PARC and its research centers and institutes also greatly exceeded original proposals. A total of 219 months of-overseas short-term training was undertaken, against 161 months allocated. However, local short-teriTI training was greatly under-utilized with only 128 months utilized out of 372 months allocated. 29. Technical Assistance. Some 102 months of local and 188 of foreign TA were used, mainly on the preparation of NMARP, and on specific scientific tasks connected with research. Both exceeded the allocation of 30 and 127 months in the SAR. However, local TA remained within budget. because of savings on rates. Both categories of TA appear to have performed well. 30. Social Sciences. Special emphasis was given to strengthening economic and social sciences research within PARC. The Social Sciences Division and its existing five Agricultural Economic Research Units (AERUs), which are located in each province, were strengthened through training and TA, and the provision of computers, vehicles and office equipment. An additional AERU was opened in Azad Jammu Kashmir. In total some 200 studies were completed. In Baluchistan, their future operations are indicated to be constrained by inadequate budget allocation from PARC. 9 Support to Agricultural Universities 31. Twinning agreements were signed between the Sindh Agricultural University (SAU) and the University of Minnesota (UM), U.S.A; and between the Agricultural University, Faisalabad (AUF) with the University of New England (UNE), Australia to strengthen the two agricultural universities in social sciences disciplines. These provided for both long- and short-term training and for technical assistance provided by staff of overseas universities on short-term visits to the host universities. 32. Long-term Training. The objectives of long-term training for staff of SAU and AUF were the same as for the provincial research system (para. 7). Fellowships were originally proposed for one Ph.D. slot for AUF and six Ph.D. slots for SAU. With Bank agreement, these were increased to 6 and 11 Ph.D. slots, respectively. Only one student from each university had completed their studies by early June 1998. The remaining students will only complete their studies during 1998-99 with support already provided by the Bank beyond Credit closure. 33. Short-term Training. SAU greatly exceeded its allocated overseas short-term training (66 months used vs. 18 months allocated) while the 12 months of local short-term training were fully utilized. AUF greatly under-utilized its allocated short-term training, with only 18 out of 30 months of overseas training used and none of the 72 months of local training used. 34. Both twinning agreements were delayed by more than two years due to the prolonged selection process and contractual disagreements. SAU experienced problems with equipment procurement, some visa arrangements and use of some more junior faculty staff of UM than originally proposed, because of serious delays in initiation of the twinning agreement. Despite this, the project provided strong support in strengthening the social sciences. Additional support in veterinary sciences was provided outside the twinning agreements. The twinning agreement of AUF with UNE was very satisfactory, resulting in a substantial number of genuine research collaborations, as well as in considerable strengthening of the social sciences. Some difficulties with equipment procurement were experienced. Improve and Strengthen Research/Extension Linkages 35. A major objective of the project was to improve and strengthen research-extension linkages. Extension staff were involved with agricultural research scientists and social scientists in the several diagnostic surveys carried out in all provinces to analyze and describe the agro-ecological and socio-economic conditions that affect farming systems and from which priorities for research were defined in the PRMPs. The establishment of Adaptive Research Coordination Units (ARUs) at Agricultural Research Institutes in Punjab (Faisalabad), Baluchistan (Sariab), NWFP (Tarnab) and Sindh (Tandojam) provinces and the strengthening of Agricultural Economic Research Units (AERUs) at these same locations were envisaged as additional mechanisms to strengthen research- extension linkages. However, while the ARUs undertook some adaptive research there was little or no involvement of their staff with Subject Matter Specialists and staff of the Adaptive Research Farms within the provincial Agricultural Extension Departments. Linkages between research staff and AERUs also appeared limited. The concerns of the continued weak linkages between research and extension are being addressed by the GOPUN with the submission of a recent concept paper entitled "Feasibility Study for Restructuring of Punjab Agricultural Research and Extension System". 10 Project Financing and Costs 36. Project costs. Total project cost is estimated at RsI,925 million (US$55.023 million), which is 9% above the original estimated project cost of Rsl,770 million (US$81.933 million), but because of a 120% devaluation of the Rupee against the Dollar during the course of the project, is 33% bellow the original estimated project cost in Dollar terms. Project expenditure was considerably delayed with only 6% of the estimated project costs expended in the first three years of the project and only 22% expended after four years (in Rupee terms). 37. Credit disbursements. Of the total IDA Credit of SDR44.40 million (US$57.30 million), SDR28.07 million (US$40.123 million) was disbursed to the GOP by final Credit closure on October 31, 1998. A total of SDR4.0 million (US$6.219 million) of the Credit was cancelled during project implementation. Only 20% of the Credit was disbursed in the first four years of the project. The cancelled undisbursed Credit at October 31, 1998 was SDR 12.33 million (US$17.282 million). 38. The funding support proposed under the project was for 59% of the funds to go directly to the provinces and 41 % to PARC. Of the 41 % of funds allocated to PARC, 15% was to be used for strengthening PARC and NARC with a further 26% of funds flowing through PARC to the two agricultural universities (5% each) and to indirectly strengthen provincial research (16%) through coordinated research programs, collaborative research, expansion of Agricultural Economic Research Units and expanding PARC's remote research facilities. In fact, PARC consumed 44% of the IDA credit disbursed to Pakistan, with proportionately more (about 8% each) going to the universities than proposed, but the distribution of funds for PARC/NARC and those earmarked for indirectly strengthening provincial research could not be determined. While the actual disbursement of IDA Credit to the provincial governments (56%) was less than proposed, the funding support for NWFP (10.7%) and Baluchistan (9.4%) provinces was proportionately about the same as proposed, but Sindh province (19.6%) received greater support while Punjab province (16.6%) received lesser support than planned. Overall IDA funding support for training (49%) and TA (23%), particularly TA, was much greater than proposed, while for incremental research operating costs (13%) was less. Project Benefits 39. The serious delays in implementation of almost all components of the project have meant that few benefits have been realized within the period of Credit disbursement. The major potential benefit of'the project beyond Credit closure is expected to come from the upgrading of qualifications, skills and knowledge of a significant number of researchers in the provincial research system. Also, the introduction of the innovative CGS has potentially paved the way for expanding the role of the private sector, universities and farmers' organizations and for fostering public-private partnerships. This, together with improved research facilities and improved focus of research on priority areas should ultimately result in the development of improved technologies for agricultural production. Because of the delayed start, there has been no attempt by any research institution to assess the impact of its research on agricultural development. However, initial results from research in developing new crop varieties, cropping patterns, crop diversification, IPM procedures for a range of important crops, etc., give positive indications of the longer term potential benefit from funding priority areas of research. The realization of these potential benefits is, however, conditional on the continued financing of these research programs. It appears that except in Punjab province, expected reductions in research budgets will reduce funding for research programs. II C. MAJOR FACTORS AFFECTING THE PROJECT 40. By the end of the project many of the physical targets had been achieved, but all were severely delayed, thus adversely affecting a significant beneficial impact on research and research system development within the time-frame of the project. The major achievements include (i) the allocation of fellowships for long-term overseas training, (ii) the knowledge and skill transfer that occurred from the TA provided, (iii) improved staff mobility and greater capacity and effectiveness of research as a result of the acquisition of vehicles and equipment, (iv) the preparation of appropriate guidelines for future research and system development as a consequence of the production of provincial and national research master plans, and (v) the establishment of a substantial amount of reasonably well-focused research following the funding of priority research programs and the introduction of the competitive grants system. It has to be stressed, however, that these achievements were only obtained towards the end of the project after three to four years of delayed implementation performance. Overall, the project is considered unsatisfactory, as three of the major project objectives were only partially achieved and the fourth objective of improving and strengthening research/extension linkages was only minimally achieved. This unsatisfactory outcome was due to many constraints associated with project design, bureaucratic, political, financial and managerial factors which occurred during project implementation. Factors not Under the Control of Government 41. Problems were experienced with local contractors and entrepreneurs. In Baluchistan province, the contractor for the construction of the BARB office building defaulted. In NWFP, the supplier of project vehicles decamped after supplying only one vehicle out of eight, but with a majority of the total purchase price paid. In Punjab province, there was a dispute over the acquisition of land on which the PARB offices were planned to be constructed. Factors Under the Control of Government. 42. Continuity of Management. The rapid rotation of senior staff had a very damaging effect on the project, as it contributed to lack of follow-through on decisions already taken. This appears to have resulted from the exercise of personal influence over the placement of preferred staff members, the pursuit of attractive postings, and - at the higher levels - political changes. However, all levels were affected by this turbulence. Rapid staff rotation is destabilizing, breaks continuity and commitment, and dissipates the effects of training. In many cases, major sections of the agricultural research structure have never been incorporated into a single unit, or in others, have been moved outside of it, such decisions being taken in a non-transparent manner. In the case of Sindh province, there were major institutional changes made that divided ministries on political grounds rather than for reasons of improved efficiency and effectiveness, which adversely affected the project's attempt to create an autonomous research body. Furthermore, the Ministry of Agriculture was further divided by a policy decision to create a new post of Secretary for Science and Technology, responsible for agricultural research (in the narrow sense, excluding livestock, fisheries, etc.) and training facility for on-farm water management, although this arrangement is now under review. The changed institutional arrangement for PARC has also had serious adverse effects on its autonomy and 12 functioning, as well as on the project objective of consolidating and complementing the advances achieved under ARP I at the federal level. 43. Administrative. Many of the decisions involved in implementation are centralized (e.g., procurement, confirmation of nomination of overseas trainees). Others are subject to repeated scrutiny of matters already decided (e.g., the issue of actual staffing levels can be, and was, re-opened during the administrative approval phase of release of funds) or require an excessive number of steps in preparation, proposal and approval (e.g., procedural system for finalization, placement and approval of equipment lists under ICB). At the same time, the unavoidable problems inherent in the administrative processes in Pakistan were not taken into account in drawing up the implementation plan or the phasing of the expenditure, with the result that both were over-ambitious in the early years. Major groups of actions which got delayed by this complex of deficiencies were: the approval of the umbrella PC-I by ECNEC; the establishment of a core management group of key full-time staff for implementation of the project in each province; vehicle and equipment procurement; contracting of TA services; and placement of overseas trainees. In addition, the existence of multiple levels of control appears to have been the main cause of the serious divergence between the SAR and the individual provincial PC-is that were incorporated into the umbrella PC-I. 44. Financial. The release of counterpart funds and utilization of the Credit were greatly delayed throughout the project. Not only were the amounts released much less than the proposed and allocated budgets, but releases were invariably delayed until the latter part of each financial year. This had serious implications both in planning support for priority research and in undertaking research, which is often seasonal in nature. Factors Subject To Implementation Agency Control. 45. The deficiencies at this level were mainly managerial. The top-down, bureaucratic style of management adopted by most senior managers is inappropriate to a professional service and has suppressed internal debate where this was necessary. In Sindh, for example, project staff were unwilling to raise the issue of the establishment of SARO because they believed that this would incur the then Secretary's displeasure. The delay in PC-I approval and completion of subsequent steps leading to Credit effectiveness, combined with the inherent length of the administrative processes resulted in late appointment of the core project managerial staff in each province and the lack of any project mianagement tools after they were appointed, allowing priorities to be overlooked and emerging difficulties to be ignored. An apparent lack of effective feedback of problems encountered in project implementation from M&E units to senior management also hindered implementation effectiveness. Factors Associated With Project Design 46. The single most important design factor which has adversely affected the performance of the project has been the great difficulty experienced in making institutional changes in the research system and establishing autonomous research bodies for each province. This was a central feature of design on which the success of most of the project components was dependent. Altogether the Bank and preparation teams spent about two years discussing proposed institutional changes with the involved gyovernment departments. The proposed institutional changes were in line with the 1988 Report of the National Commission on Agriculture. However, differing opinions/statements have 13 been provided as to the desirability, feasibility and commitment of such institutional changes at the time of project preparation. In the end though, all governments agreed to such institutional changes which were then translated into the original Development Credit and Project Agreements. What is clear is that the potential difficulties and complexities in implementing the process of such change were not adequately considered. Greater consultation with a wider audience (participatory workshops) during project preparation would have provided a clearer understanding and appreciation of factors which control vital steps of the process of change and the means to achieve this change. D. PROJECT SUSTAINABILITY 47. While the benefits obtained from more knowledgeable and skilled staff and improved research facilities are expected to be sustained, the sustainability of research programs and the institutional and managerial improvements achieved are at serious risk. The institutional changes at the federal level have weakened the autonomy and effectiveness of PARC. The general financial crisis in Pakistan will mean that all government departments and agencies will receive restricted budget allocations for the immediate future years. Some exceptions may occur as optimism has been expressed regarding future funding of research in Punjab province. Funding shortages will result in greatly restricted availability of funds for operational and maintenance expenditures in agricultural research. This would have adverse effects on the realization of benefits from project investments in human resource development, upgrading of research and support facilities and continued implementation of identified priority research programs. Overall, the project has failed, though only partially in the case of Punjab and Baluchistan, to establish the organizational and managerial systems envisaged that were identified as necessary to ensure the efficiency and effectiveness of highly integrated and relevant research in agriculture, livestock, fisheries and forestry. Progress has been made towards achieving functional flexibility of the apex bodies established in Punjab and Baluchistan provinces. However, fully functional apex bodies are not yet in place in these two provinces. In Sindh province little progress has been achieved, whilst in NWFP the project has failed to address the post-merger issues identified during the project mid-term review. Sustainability of these apex bodies will greatly depend on adequate govemment commitment and financing. E. BANK PERFORMANCE 48. The Bank participated in eleven supervision missions, including an in-depth review, and conducted the mid-term review of the project as well. The MTR and the second in-depth review missions undertook a thorough analysis of the institutional reforms needed to ensure functional flexibility of the research system. The suggestions given for improvement in project performance were very comprehensive and seemed generally well appreciated by the Borrower. The continued and close association of the resident Bank staff and the continuity of appraisal staff with project implementation was also commendable. However, given the key importance of institutional changes proposed to the provincial research systems, the Bank should have been more sensitive early in project implementation to the difficulties arising from limited local commitment to the establishment of autonomous provincial agricultural research bodies. Even before the MTR, the Bank should have sought to undertake an intensive examination of the underlying institutional and political problems, and engaged in participatory workshops with a view to finding a mutually acceptable solution. Following the MTR, the Bank had to threaten to use its remedies twice. In both cases, however, the GOB took some actions that led IDA to believe that all the reforms would be implemented as agreed. However, this did not materialize. Some of the reason for delays in purchasing equipment also 14 appears to be due to the time taken by the Bank to process documents associated with the ICB proceduires. The Bank was also mild to give sufficient emphasis to project activities that would strengthen research-extension linkages, given that this was a major objective of the project. F. BORROWER PERFORMANCE 49. While project implementation performance improved significantly with time, overall Borrower performance is considered unsatisfactory. Whilst, given the inherent difficulties of the government system, there may have been unreasonable expectations in the SAR of implementation progress, much could and should have been done to speed implementation progress. Many covenants of the DCA were not fulfilled by the Borrower in due time leading to serious delays. The serious delay in the appointment of a project director and establishment of an implementation support unit in each province was unnecessary and had serious consequential implications for many other important activities (training programs, contracting of TA, preparation of master plans, etc.). Even in the early years of the project, when the financial situation was more favorable, insufficient local funds were made available and Credit disbursements were very slow. A significant amount of project resources have been used for supporting ongoing operational expenses rather than funding new project interventions. For example, project staff are largely transferees from other posts rather than new incremental staff and a substantial share of the research financed by the project appears to have been for ongoing experiments and studies rather than new priority research. In a few cases, it was reported that project staff and vehicles had been assigned to other uses outside the project. G. ASSESSMENT OF OUTCOME 50. The overall assessment is that the project outcome is unsatisfactory. The four main project objectives were only partially achieved, mainly because many of the implementation targets were not achieved until very late in the project. While the proposed overseas training program was exceeded, the national and provincial research master plans were prepared, the TA provision was fully utilized and a substantial portion of the civil works was completed, all of these were achieved very late in the project, thereby prejudicing their beneftts. Many other activities were not completed satisfactorily even after eight years of project life, such as vehicle and equipment purchases and incremental staffing. Most importantly, however, the research systems show only limited improvement. They have certainly not achieved the project concept of integrated, autonomous research systems with significantly improved research management, in which the research environment and terms and conditions of employment should have been greatly improved, and where all research is focused and adequately funded and supported. H. FUTURE DEVELOPMENT 51. Punjab province has recently organized a workshop on agricultural research and extension with the aim of restructuring agricultural research and extension in the province. Baluchistan province has established a committee to examine the further development of their research system in which it is planned to establish a number of research stations in agro-ecological zones. In NWFP a task force was constituted in 1997 to consider the future organizational arrangements for agricultural research in the province and although a report was submitted in June 1997 no further developments have occurred. It is generally believed that physical resources are adequate and need relatively little strengthening. The continued scarcity of funds for operations, 15 maintenance and capital development, however, does not seem to have engendered emphasis on the potential for cost savings through rationalization of the size of the research system or to explore mechanisms for tapping alternative sources of finances (both public and private). Further external assistance, such as ASIP, is required to achieve effective research systems. However, this must be done after thorough appraisal to ensure commitment at the highest levels to institutional change and necessary changes in government bureaucratic procedures and research funding to achieve fully functional, autonomous and integrated agricultural research systems. 1. KEY LESSONS LEARNED 52. Key Lessons Learned. The main lessons learned from the project are concerned with issues associated with project design, implementation planning, institutional change, management performance, and project performance: (a) Project Design. Over-ambitious project design and objectives should be avoided. In particular, a national level project should be avoided where project design is complicated, project scope is extensive and implementation difficult thus causing difficulties in providing adequate supervisory support to all four provinces and also the federal level. (b) Implementation Planning. An implementation timetable should take into account the administrative realities of the processes for appointing key project staff, mobilizing counterpart funds, and initiating activities with long lead times (particularly appointing key staff, procurement, recruitment of TA, and legislative components of major institutional changes) and the major deficiencies in Government administrative and financial systems. The possibility of dealing with these matters in a relatively long pre-project period where appropriate covenants must be met before project funding commences should be considered. The use of computer-based methods to prepare realistic implementation schedules should be considered and phased expenditure forecasts should be based on these analyses. (c) Institutional Change. Where institutional change is needed, it is essential to evaluate the options of either (i) creating new bodies, regulations and administrative procedures or (ii) providing the resources and human skills required to enable staff to function better within the existing structure. Where substantial institutional change is envisaged, it is necessary to thoroughly analyze existing systems as a basis for designing these new institutional structures, rules, procedures, management strategies and tactics, and for designing programs of change to bring them into effective existence. It is essential that these processes are carried out in collaboration with all the parties involved on the Govemment side. (d) Management Performance. It is essential that designated individuals be given responsibility for the management of the project and each of its major components, and that they retain these positions throughout the project in order to achieve commitment, vision and managerial stability. (e) Unsatisfactory project performance. The Bank should be less compromising and terminate projects or components of a project when there is a persistent unsatisfactory performance. 16 PART II: STATISTICAL TABLES Table 1: Summary of Assessments A. Achievement of objectives Substantial Partial Negligible Not Applicable (4) () (4) (4) Macro policies EL El El El Sector policies El El El ED Financial objectives El El El El Institutional development E ml E] El Physical objectives ml El El El Poverty reduction l El E E Gender issues El El El Other social objectives EL E l Environmental objectives E El Public sector management E E E E Private sector development E E E E Other (specify) E El E E B. Project sustainability Likely Unlikely Uncertain (l) (4) (4) Highly C. Bank performance satisfactory Satisfactory Deficient (4) (4) (4) Identification E E E Preparation assistance E E E Appraisal E E [ Supervision E E E 17 Highly D. Borrower performance satisfactory Satisfactory Deficient (4) (v) (4) Preparation Z I O Implementation F z 2 Covenant compliance Operation (if applicable) E Highly Highly E. Assessment of outcome satisfactory Satisfactory Unsatisfactory unsatisfactory (v) () (4) (4) Li Li Ei L 18 Table 2: Related Bank Loans/Credits Loan/credit title Purpose Year of Status approval Preceding operations 1. Punjab Agricultural To introduce "Training and Visit" (T&V) 1978 Closed in June Extension and Agricultural system of agricultural extension in five core 1985 Development Project districts of Punjab Province (Cr.813-PAK) 2. Sindh Agricultural To introduce T&V system in five core 1979 Closed in June Extension and Adaptive districts of Sindh Province by increasing 1985 Research Project (Cr. 922- extension staff, providing regular training, PAK) establishing research farms and two soil testing laboratories. 3. Agricultural Research To upgrade Pakistan's agricultural research 1981 Closed in Project (Cr. I 158-PAK) system by supporting establishment of PARC, December 1987 providing technical assistance, staff training, research program support at NARC, contract research system for provincial institutes and improving support services. 4. Baluchistan Agricultural To strengthen T&V system in four districts of 1985 Closed in June Extension & Adaptive Baluchistan province to achieve increases in 1992 Research Ill Project (Cr. sustainable agricultural production and 1533-PAK) widespread adoption of improved farming practices. 5. Punjab and Sindh To achieve sustained increases in agricultural 1987 Closed in June Agriculture Extension and production in Punjab and Sindh Provinces 1994 Adaptive Research Project through strengthening of agricultural (Cr. 1762-PAK) extension and adaptive research activities not covered in the Phase I provincial projects above. Following operations NIL' " During 1997-98 consideration has been given to the preparation of two follow-up projects, Agricultural Support Services F'roject (ASSP) and Agricultural Sector Investment Project (ASIP) which would further support research development in Pakistan. Subsequently, the preparation of ASSP has been discontinued while ASIP is now in an advanced stage of preparation. 19 Table 3: Project Timetable Steps in project cycle Date planned Date actuaUl latest estimate Identification February 1986 Preparation April 1986 9 March-9 April 1987, September 1987, December 1987 " Appraisal 28 Sept. 1986 9 Oct.-7 Nov. 1988, 12-16 February 1989 and 11 -26 May 1989 2/ Negotiations 27 March 1989 12-17 April 1990 Board presentation 18 May 1989 12 June 1990 Signing June 1989 27 July 1990 Effectiveness 29 October 1990 2 August 1991 Midterm review (if applicable) 31 January 1994 4-26 February 1995 Project completion 30 June 1997 30 June 1997 Loan closing 30 June 1998 30 June 1998 Footnotes "GOP identified and prepared project with the assistance of the FAO/IBRD Cooperative Program and the ISNAR. ' Appraisal missions by IBRD. Table 4: Loan/Credit Disbursements: Cumulative Estimated and Actual (US$ millions) 1/ FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 Appraisal 4.00 7.00 14.00 25.00 37.00 46.00 53.00 57.30 estimate Actual - 2.1(0 4.42 7.32 15.45 24.11 32.20 40.12 Actual as % of 0.0 30.0 31.6 29.3 41.8 52.4 60.8 70.0 estimate Date of final October 31. 1998 disbursement "The project financing plan and disbursement schedule in the SAR is given in US Dollars, not SDRs. Comment: Closing date for disbursement given at June 30, 1998. The Bank allowed disbursements to continue until October 31, 1998. The remaining Credit of SDR12.330 million has been cancelled. Appraisal estimate used an exchange rate of SDRI .0 = US$1.29 while actual estimates are based on historic exchange rates. 20 Table 5: Key Indicators for Project Implementation 1/ 1.' Key implementation Appraisal Actual Actual indlicators in SAR/ Unit Estimate Completed Percent President's Report Completed 1. Overseas Long-term Training Ph.D. No. 113 122 44 2' 36 M.Sc./M.Phil. No. 32 47 35 D 74 2. Short-term training Overseas Person months 462 462 427.5 93 Local Person months 1572 1572 1085 69 3. Incremental Staff No. 435 435 312 72 4. Technical Assistance International Person months 454 454 513 113 Local Person months 126 126 209.5 166 " Key indicators were not set in SAR for project implementation, but some of the more readily identifiable physical targets are assessed. v Seven students failed and returned and one deceased Table 6: Key Indicators for Project Operation (This table is not applicable to this project) Table 7: Studies Included in Project Purpose as defined Study at appraisal/redefined Status Impact of study 1. Diagnostic To introduce a farming systems perspective into A number Provided basic data Surveys in all research planning and programming, promote on- of surveys and information for Provinces farm research and improve research/extension completed preparation of linkages the project supported multidisciplinary in all Provincial teams of research and extension staff to provinces Research Master undertake diagnostic surveys within each Plans province's various agroecological zones 21 Table 8A: Project Costs Appraisal estimate (US$M) Actual/latest estimate (US$M) Item Local Foreign Total Local Foreign Total costs costs costs costs 1. PARC 4.5.3 5.3 9.8 10.848 12.892 23.740 2. Punlab 6.2 8.8 15.0 4.351 5.179 9.530 3. Sindh 4.1 7.4 11.5 5.539 5.206 10.745 4. Baluchistan 1.6 4.2 5.8 1.595 3.475 5.070 5. North West Frontier 2.0 5.0 7.0 1.989 3.950 5.939 6. Federal Support to 5.4 5.5 10.9 Provinces 7. Ag. Univ. Faisalabad 0.7 2.5 3.2 8. Sindh Ag. Univ. 0.3 2.8 3.1 Baseline Costs 24.8 41.5 66.3 Physical Contingencies 0.8 3.6 4.4 Price Contingencies 4.5 6.7 11.2 TOTAL 30.1 51.8 81.9 24.322 30.701 55.023 Comments: 1. Costs are estimated on the basis of average historic annual exchange rates between Rupee and Dollar. 2. Estimated actual expenditure under items 6, 7 and 8 have not been separated and are included in total expenditures by PARC. Moneys for operation of university programs were given directly to universities, but payment for technical assistance, short- and long-term training and equipment for universities was paid directly by PARC. 3. Total estimated project costs includes US$6.9 million taxes and duties, but actual taxes and duties not available. 4. Estimated actual total project cost in Rupee terms as of October 31, 1998, was Rs1,925 million or 109% of the appraisal estimate of Rsl,770 million. Due to devaluation of the Pakistan Rupee against the US Dollar the estimated actual project cost was US$55.023 million or 67% of the appraisal cost estimates of US$81.937 million. 22 Table 8B: Project Financing Appraisal estimate (US$M) Actual/latest estimate (US$M) " Source Local Foreign Total Local Foreign Total costs costs costs costs IDA 5.5 51.8 57.3 11.114 30.701 41.815 Govt. of Pakistan 11.0 - 11.0 5.450 0.0 5.450 GOPunjab 5.4 - 5.4 2.577 0.0 2.577 GOSindh 4.3 - 4.3 2.567 0.0 2.567 GOBaluchistan 1.8 - 1.8 1.143 0.0 1.143 GONWFP 2.1 - 2.1 1.471 0.0 1.471 LTOTA 30.1 51.8 81.9 24.322 30.701 55.023 " Estimated actual project financing in US$ is at final Credit closure (31 October, 1998). Comment: The estimated actual expenditure of the IDA Credit ($41.815 million) is slightly higher than the actual IDA disbursement (SDR28.070 million equivalent to US$40.123 million) because the estimated expenditure (in Rupees) of the IDA Credit by the governments of each province and PARC (GOP) is based on estimated average annual exchange rates (the best data available) between the Rupee and Dollar. Table 9: Economic Costs and Benefits There was no net present value (or internal economic rate of return) estimated, as such measures are not required for this type of project. 23 Table 10a: Status of Legal Covenants of Original Agreements Pakistan Second Agricultural Research Project Original Revised Covenant Present ftulfillment fulfillment Description of Agreement Section class status date date covenant Comments DCA Prov. 3.01 (a) 5 CP Undated Commitment to the project objectives Chronic delays in meeting 2.01(a&b) and performance of all obligations in key covenants and remedial accordance with the provisions of PAs measures threatened on number of occasions DCA Prov. 4.01 (b) 4 CD 31/12/92 Project accounts audited and audit Audit reports submitted with 3.01 (iii) reports submitted to IDA annually delay DCA Prov. 4.02 (a) 4 CD Undated Adequate capital and operational Commonly a shortfall of 3.01 (a) funds funds for PARC, Punjab and Sindh PARC 2.04 (b) 5 CP Undated Inform Association of conditions Punjab Board and Sindh interfering with progress of Parts Al, Agricultural Research B I, B3, C and D Organization not operational and staffing situation unsatisfactory PARC Prov. Sched. 1. 5 CD 30/12/90 31/3/93 Employment of consultants by PARC Fulfilled Section 11 and other implementing entities (a&b) PARC Sched. 2. 5 C 30/12/90 21/08/91 Reorganize Planning Directorates Fulfilled para. 3 PARC Sched. 2, 5 CD 30/11/90 31/05/91 Define national research priorities and Fulfilled para. 4 establish guidelines for allocating resources PARC Sched. 2, 5 CD 30/12/90 31/03/93 Enter into arrangements for making Fulfilled para. 5 funds available to AUF and SAU PARC Sched. 2. 5 CD 30/06/91 31/03/93 Cause AUF and SAU to enter into Fulfilled para. 5 twinning arrangements with Universities PARC Sched.2, 5 CD 30/11/91 31/03/93 Annual review Fulfilled para. 7 (a) (ii) Prov. Sched. 2, 5 CD 30/11/91 31/03/93 Annual review Fulfilled para. 5 PARC Sched.2, 5 CD 31/01/94 15/12/94 Mid-term review of the project Fulfilled para. 7 (a) progress (ii) PARC Sched. 2. 5 CD 30/06/93 Terms of reference (TOR) for mid- Fulfilled para. 7 (b) term review (MTR) to be fumished to IDA PARC Sched. 2 5 C 28/02/91 Every Progress report OK para. 8 and every semester semester starting 28/02/92 Prov. Sched. 2 5 C 28/02/91 Every Progress report OK para. 6 and every semester semester starting 28/02/92 24 Original Revised Covenant Present fulfillment fulfillment Description of Agreement Section class status date date covenant Comments Prov. Sched. I C Undated Open and maintain in dollars special Fulfilled 2.01 (c) accounts Prov. Sched.2, 5 CD 30/06/91 30/06/93 Overall responsibility in Sindh be SARO established but para. I (a) exercised by SARO structure presents serious (ii) weaknesses Prov. Sched.2. 5 NC 30/06/91 30/06/93 Overall responsibility in Punjab be PARB not fully functional para. 7 (a) exercised by PARB (i) Prov. Sched. 2, 5 CD 30/06/91 30/06/93 Overall responsibility in NWFP be Fulfilled, but structure para. 7 (a) exercised by AUP presents serious weaknesses (iii) Prov. Sched.2, 5 CD 30/06/91 30/06/93 Overall responsibility in Baluchistan Fulfilled para. I (a) be exercised by BARB (iv) Prov. Sched.2. 5 CD 31/12/90 31/12/92 MSUs will be established para. 7 (h) Prov. Sched.2, 5 C 31/10/90 31/10/91 Workling parties para. 3 (a) Prov. Sched.2, 9 CD 31/08/92 Fumish PRMPs to IDA for review and para. 3 concurrence (b&c) Prov. Sched. 2. 9 CD 31/03/93 Fumish to IDA updated PRMPs every para. 3 (d) year Prov. Sched.2. 5 NC 30/06/91 31/12/93 Punjab and Baluchistan will modifyl para. 4 Public Service regulations Status: C: Covenant complied with CD: Compliance after delay NC: Not complied with SOON: Compliance expected in reasonably short time CP: Complied with partially NYD: Not yet due Covenant Class: I Accounts/audit 2 Financial performance/generate revenue from beneficiaries 3 Flow and utilization of project funds 4 Counterpart funding 5 Management aspects of Project or of its executing agency 6 Environmental covenants 7 Involuntary resettlement 8 Indigenous people 9 Monitoring, review and reporting 10 Implementation 11 Sectoral or cross-sectoral budgetary or other resource allocation 12 Sectoral or cross-sectoral regulatory/institutional action 25 Table lOb: Status of Legal Covenants of Amended Agreements Pakistan Second Agricultural Research Project Original Revised Coven Present fulfillment fulfillment Description of Agreement Section -ant status date date covenant Comments type DCA Prov. 4.01 (b) (ii); F PART 31/12 each Fumish to IDA audit reports Audit reports fumished but 3.01 (b) (iii) year often delayed DCA Prov. 4.02 (a); F PART Adequate capital and operational funds Frequent shortfall of 3.01 (a) operational funds and funds to pay taxes for procurement PARC 2.04 (b) T PART Inform Association of conditions PARCs autonomy diluted by interfering with progress of Parts Al, abolishing ARD; it lacks B I, B3, C and D authority to interact with Provinces PARC Sched. It, r PART 30/11/90 not Detine national research priorities and NMARP completed after para. 4 formally establish guidelines for allocating considerable delay advised resources PARC Sched. ll. T OK 31/12/95 PARC shall submitto IDA forapproval para. 6 the procedures for using the Competitive Grant System PARC Sched. lI, T PART 30/09/95 PARC shall submit a satisfactory Plan submitted but not para. 7 stafling plan and implement it thereafter implemented satisfactorily PARC; Sched. 11, M RVS 28/02/91 PARC/Provinces shall submit semestral Irregular and untimely. Prov. PA para. 9 (rev); progress reports to IDA Sched. ll, para. 6 Prov. 3.02 (a) M PART Delegate to AUP, SARO, PARB, and SARO and PARB not been BARB the necessary authority to delegated with required establish their own rules authority Prov. Sched. 11. M PART Establishment and operation of SARO, Institutions established late in Part A, para. PARB and BARB for integrated project, but not operating as 2 research planning and resource envisaged allocation Prov. Sched. ll. M PART 30/09/95 Punjab shall establish PARB as an Subsequently reattached to para. 2 (a) autonomous body attached to P&D Agricultural Department (i) Dept. Prov. Sched. ll. M PART 31/12/95 Punjab shall empower PARB as an Subsequently reattached to para. 2 (a) autonomous body attached to P&D Agricultural Department (i) Dept. Prov. Sched. ll. M PART 30/09/95 Sindh shall amend notification of SARO Notification issued but fell para. 2 (b) to restructure its composition and short of the covenant's establish its rules of business requirement for SARO to carry out its objectives independently Prov. Sched. ll. M PART 31/12/95 Baluchistan shall amend notification of New rules of business para. 2 (c) BARB to include Rules and Functions promulgated but research in forestry and fisheries has not been included Prov. Sched. ll M PART 31/12/95 NWFP shall submit to IDA a time Merger never achieved para. 2 (d) bound plan to address institutional issues regarding the merger of research into AUP 26 Original Revised Coven Present fulfiilment fulfillment Description of Agreement Section -ant status date date covenant Comments type Prov. Sched. ll, M OK 31/12/95 Provinces shall submit to IDA for para. 4 (b) approval the procedure for using the Competitive Grant System Prov. Sched. ll. M PART 31/12/95 Punjab and Baluchistan will modify Baluchistancompliedalthough para. 6 service rules for scientific staff to rules for scientists in Livestock provide flexibility and agility in not properly modified; Punjab recruitment and promotion yet to comply Status: OK: Covenant complied with PART: Partial compliance SOON: Compliance expected in reasonably short time NYD: Not yet due ACT: Needs use of formal remedies to bring about compliance RVS: Needs to be revised Covenant Class: I Accounts/audit 2 Financial performance/generate revenue from beneficiaries 3 Flow and utilization of project funds 4 Counterpart funding 5 Management aspects of Project or of its executing agency 6 Environmental covenants 7 Involuntary resettlement 8 Indigenous people 9 Monitoring, review and reporting 10 Implementation 11 Sectoral or cross-sectoral budgetary or other resource allocation 12 Sectoral or cross-sectoral regulatory/institutional action Table 11: Compliance with Operational Manual Statements Compliance with all OMs. 27 Table 12: Bank Resources: Staff Inputs Stage of Planned Revised Actual project cycle Weeks US$ Weeks US$'000 Weeks US$O000 Preparation to n.a. n.a. n.a. n.a. 118.0 91.3 appraisal . Appraisal n.a. n.a. n.a. n.a. 82.0 155.0 Negotiations through n.a. n.a. n.a. n.a. 14.3 13.5 Board approval Supervision n.a. n.a. 167.35 425.84 187.5 380.4 Completion TOTAL 417.4 893.3 401.8 640.2 Comments: I. Data from WB MIS: staff weeks given includes all associated staff inputs (in Bank as well as field) as at June 2, 1998. 2. Planned staff weeks and costs only available from FY 1996. 28 Table 13: Bank Resources: Missions Performance rating Stage of Number Specialized Implement- Develop- project cycle MonW of staff skills ation ment Types of e Mycar persons field represented status 2' objectives problems v Through appraisal n.a. n.a. n.a. n.a. Appraisal through n.a. n.a. n.a. n.a. Board approval Supervision I Nov.-Dec. 1 21 EC 2 1 M 1990 Supervision 2 Oct.-Nov. 2 19 A (2) 2 1 M 1991 Supervision 3 July 1992 2 26 A (2) 3 3 F, M Supervision 4 Jan.-Feb. 2 20 A (2) 3 3 F. M 1993 Supervision 5 Sept. 1993 1 15 A 3 3 M Supervision 6 Jan.-Feb. 2 22 A (2) 3 3 M 1994 Mid-term Review Feb. 1995 441 23 A (2). RIS (2) 5 U M Supervision 7 Oct. 1995 2 23 A. RIS S U M Supervision 8 May-June 4 25 A (2). EC. FS S U M 1996 Supervision 9/in-depth April 1997 5 22 A (3). EC. RIS S S 6e M review Supervision 10 Oct. - 3 18 A(3) S S 6 M Nov. 1997 Supervision II and June 1998 4 51 A (3). RIS U U M Completion . A =- Agriculturist; EC = Agricultural Economist; RIS = Research Institutions Specialist; FS = Financial Specialist 2/ Summary objectives ratings: I (HS) - highly satisfactory; 2 (S) - satisfactory; 3 (U) - unsatisfactory; 4 (HU) - highly unsatisfactory ' T= Technical; F = Financial; M = Managerial/Institutional 4/ Two additional representatives of GOP also participated in Mid-Term Review 5' Two representatives from FAO/CP 6/ Rated marginally satisfactory Appendix I Page I of 12 PAKISTAN: Agricultural Research II Project (Credit 2154-PAK) FAO/World Bank Cooperative Programme ICR Mission 9-27 June 1998 Appendix 1: Mission Aide-Memoire A. INTRODUCTION I. At the request of the World Bank (WB), an FAO/CP Mission' visited Pakistan from 9-27 June 1998 to meet with officials of the Pakistan Agricultural Research Council (PARC) and the Project Management Unit (PMU), the government research services in North West Frontier, Baluchistan, Sindh and Punjab provinces and Agricultural Universities in Sindh and Punjab provinces to obtain relevant information for the preparation of the Implementation Completion Report (ICR) for this project. Field visits were made to the main research institutions/centres in each province to discuss with scientists the research assistance and impact under the Agricultural Research 11 Project (ARP 11). The mission overlapped with the WB supervision mission2 during its last week. 2. The mission wishes to thank all staff of PARC, the PMU, the provincial departments of agriculture and the institutes/centres visited for giving generously of their time, information and assistance with the meetings, field visits and preparation of the ICR for this project. The preliminary findings and views presented in this Aide-Memoire reflect those of the FAO/WB missions, based on discussions held with federal and provincial staff of departments/centres/institutes visited and from prior consultation of project documents at World Bank. The Aide-Memoire has been amended, to take into account points raised in a final wrap-up meeting, held under the chairmanship of the chairman of PARC on June 27, 1998. B. BACKGROUND 3. The Government of Pakistan (GOP) requested assistance from the International Development Association (IDA) to further develop and strengthen its agricultural research services following prior support (first Agricultural Research Project; Cr 1158-PAK) to strengthen the federal research system to enable it guide and co-ordinate research throughout Pakistan. The second Agricultural Research Project (ARP 11) was mainly directed towards upgrading provincial research services as well as providing some further strategic support to PARC. C. THE PROJECT 4. The project was prepared during 1987 by the FAO/IC, assisted by the International Service for National Agricultural Research (ISNAR) and appraised by the World Bank during 1988 and follow- up missions in February and May 1989. The eight-year project was approved on July 27, 1990 with credit effectiveness set for October 29, 1990 and project closure scheduled for June 30, 1998. Project effectiveness was delayed, however, until August 2, 1991 due to delays in the GOP fulfilling the ' D.A.lvory (Mission Leader) and P. J. Smith (Institutions Management Specialist, Consultant) 2 1. Serejski (Task Leader) and Rashed ul Qayyum (Agriculturist) Appendix I Page 2 of 12 conditions for credit effectiveness. A mid-term review (MTR) of the project was held in February, 1995, one year later than scheduled. 5. The project's main objectives were to: (i) improve and strengthen research capabilities in the provinces, (ii) consolidate and complement the advances achieved under ARP I at the federal level, that is with PARC and NARC, (iii) support high priority research programmes at the provincial and federal level, and (iv) improve and strengthen research/extension linkages. The Development Credit Agreement and Project Agreement were revised twice: firstly, by the Association's letter of April 28, 1993 and secondly, following the MTR in July 1995. The main thrusts to achieve the revised objectives were to be through: assistance for the provinces to (i) reorganise their research systems through the establishment of a basic institutional framework within which a better research programme may be developed, and (ii) improve research planning, resource allocation, monitoring and evaluation, and assistance for identified priority research programmes. In particular, upgrading research programmes in horticulture, livestock, soil science, integrated pest management, and social sciences, and funds to support aidditional provincial priority research identified during the preparation of Provincial Research Master Plans (PRMPs). One-third of these funds were to be used following the Competitive Grants System (para. 20). D. MISSION FINDINGS Project Implementation Physical Resource Development 6. Staffing. Some 434 additional staff were proposed to be recruited for the Management Support Units established under the project, as well as additional scientific staff. While the allocated positions were filled in NWFP and Baluchistan, many positions remained vacant in PARC (61% vacant), Punjab (47%) and Sindh (9%). Many of the proposed incremental staff have, in fact, been transferred from elsewhere in the system, often without replacement. There appear to be no permanent staff of the provincial agricultural research boards, as staff are reluctant to be permanently assigned to these bodies because of concerns over job security and pension rights. The appointment of key project staff was delayed by the procedures required to provide local budget for them, which, in turn, was delayed by the late approval of the PC-I. 7. Civil works. A limited amount of construction of new facilities was proposed under the project. Facilities constructed included administrative buildings, library and seminar halls, green and shade houses in Sindh province, an administrative block (not completed due to contractor dispute) and shacde house in Baluchistan province. No facilities were proposed in NWFP and in Punjab province the proposed administrative building was not started because of a land dispute. Within PARC, rnajor infrastructure development was envisaged. A new crop science building for the Plant Science Division of NARC was constructed and facilities at other distant locations were upgraded/expanded, including the Northern Areas Research Centre at Jagalot and Mountain Research Centre at Kaghan. A new laboratory and ancillary space area for six programmes (animal health research, fisheries and aquaculture research, range and environment research, farming systems research, technology transfer and seed sciences and technology) was also completed. Houses have also been constructed. All have been completed except at Gilgit which is expected to be finished by the end of June 1998. Appendix I Page 3 of 12 8. Equipment and vehicles. Vehicles were purchased by all provincial governments and PARC to improve mobility of administrative and research staff. While in some provinces such as NWFP and Baluchistan these were purchased early in the project in others, such as Punjab, many vehicles were only purchased towards the end of the project. Some strategic equipment was purchased for laboratories in all provinces and for PARC. Most equipment purchased was for assisting soils research (soil physics and soil chemistry), horticultural and livestock research. Considerable delays occurred in the purchase of all equipment requiring International Competitive Bidding (ICB). Much less equipment has been purchased than originally proposed and at present there is a large consignment of equipment under ICB that will not be paid for and received before Credit closure. Human Resource Development 9. Long-term training. Major emphasis has been given to human resource development. The original proposal was to provide 107 Ph.D. and 31 M.Sc./M.Phil. fellowships for long-term training for staff of research institutes/centres and universities in selected disciplines, based on an identified needs assessment at the time of project preparation. All Ph.D. and MSc./M.Phil. fellowships were undertaken at overseas universities. It was proposed that PARC and the provinces would identify the first group of trainees by March 31, 1991 and that all long-term training fellowships would be taken up by June 1994. Placement of all fellowships was considerably delayed. Fellowship targets were greatly exceeded, particularly by PARC, with a total of 138 Ph.D. and 52 M.Sc./M.Phil. fellows placed overseas. The last fellowships were only approved and placed overseas late in 1995, which meant that while 77% of M.Sc./M.Phil. candidates have returned, only 19% of the Ph.D. candidates have returned. Some eight candidates dropped out for various reasons. The major problem, with the exception of NWFP, with this programme was that the responsibility for placement was assigned to the TA contractors. Thus the delay in establishing the core staff of the Provincial segments of the project and recruitment of the TA (para. 11), inevitably caused problems. Except in the case of Baluchistan, where there was limited local experience in overseas student placement, it is not clear why NARC, Sindh and Punjab decided to assign this responsibility to the TA contractors. The WB has approved the provision and allocation of sufficient funds with the relevant TA contractors to finance the completion of the outstanding period of unreturned fellows. Institution/Province Ph.D. programme MSc/M.Phil. Original Sent abroad Returned a/ Original Sent abroad Returned a/ PARC/NARC/AZRI 25 44 39 6 13 8 Punjab 20 20 0 6 16 15 Sindh 23 23 4 6 6 2 NWFP 17 17 0 5 5 2 Baluchistan 15 18 0 8 8 8 AUF, Punjab 1 6 1 0 0 0 SAU, Sindh 6 11 1 0 2 2 Grand Total 107 139 45 31 50 37 a/ Includes seven students that failed and one student deceased Appendix I Page 4 of 12 10. Short-term training. An extensive programme of both overseas and local short-term training was also provided. It was proposed that a total of 510 months of overseas and 1,656 months of local training would be provided for both scientific/technical and managerial staff to meet critical training needs in horticulture, livestock, soil science, IPM and social science and for research management and organisation. The short-term training programme was less than proposed, particularly local training with a total of 189 awards, 134 for overseas and 55 for local training. There has been no assessment of the value of this short-term training. Some local training funds were used for attendance at local seminars, workshops and conferences. Technical Assistance It. Technical assistance (TA) was seen as essential for planning and implementation of parts of the project. Some 580 months of technical assistance was proposed, 454 months for expatriate assistance and 126 months for local consultants. Recruitment of TA was, however, delayed by more than two years due to extremely slow processing of the TA bidding and contractual arrangements. This caused two major problems. Firstly, the long-term training programme was delayed by more than two years and secondly, the Provincial Research Master Plans (PRMPs) were completed more than three years after the original scheduled completion date. However, once started, the TA input has generally been appropriate and well-appreciated by those closely involved with the consultants, except in Baluchistan where seemingly some aspects of the performance of consultants was criticised. Many valuable reports were produced by the consultants, but relatively few of the recommendations to improve research management systems, research programmes and research methodologies and practices have been implemented or followed up. Except for very specific scientific inputs, longer inputs by fewer consultants would be more useful. Support to Agricultural Universities 12. Twinning agreements were signed between the Sindh Agricultural University (SAU) and the University of Minnesota (UM), U.S.A. and the Agricultural University, Faisalabad (AUF) with the University of New England (UNE), Australia to strengthen the two agricultural universities through both long- and short-term training and through technical assistance provided by staff of overseas universities on short-term visits to the host universities. Both twinning agreements were delayed by more than two years due to prolonged selection process and contractual disagreements. SAU experienced problems with equipment procurement, some visa arrangements and use of some more junior faculty staff of UM than originally proposed because of serious delays in initiation of the twinning, agreement. Despite this the project provided strong support in strengthening the social sciences and veterinary sciences particularly. The twinning agreement of AUF with UNE was very satisfactory resulting in considerable strengthening of the social sciences particularly. Some difficulties with equipment procurement were also experienced. Institutional Reorganisation 13. A major objective of the project was to establish autonomous apex research bodies in Sindh, Punjab and Baluchistan provinces to manage research in agriculture, livestock, fisheries and forestry. Different institutional arrangements were foreshadowed in each province. 14. North West Frontier Province. In NWFP the project objective was to maintain the existing institutional relationship where the Government of NWFP had transferred responsibility for crop and livestock research from the Department of Agriculture to the Agricultural University, Peshawar Appendix I Page 5 of 12 (AUP) in the late 1980s as assisted under the USAID-financed Transformation and Integration of Provincial Agricultural Network Project (TIPAN). Project implementation was therefore the responsibility of the vice-chancellor's office of the University, who through a Director General of research, had responsibility for managing research in research institutes/centres and stations in agriculture and livestock production, but not animal health, fisheries and forestry. The revised Project Agreement of July 1995 provided for the submission of a time-bound action plan to address key institutional issues identified during the MTR and regarding the merger of AUP and agricultural research. A draft plan was prepared but never implemented. 15. Baluchistan Province. In Baluchistan province the project supported the establishment of the Baluchistan Agricultural Research Board (BARB) which was proposed as an autonomous body to control research resource allocation in agriculture, livestock, fisheries and forestry. While BARB was finally established and new rules of business promulgated, research in forestry and fisheries has not been established and part of horticulture research is outside the control of the Department of Agriculture. 16. Sindh Province. In Sindh province the project aimed to support the establishment of a semi-autonomous research organisation, to be called the Sindh Agricultural Research Organisation (SARO) which would merge the research activities of the Sindh Department of Agriculture (crops, livestock and fisheries). SARO was established after considerable delay but with little commitment to such a body and when combined with a number of institutional changes that have separated agriculture, livestock and fisheries and forestry into separate ministries, has never been a significant entity. Its continued existence must be unlikely after Credit closure. 17. Punjab Province. In Punjab province, the project supported the establishment of the Punjab Agricultural Research Board (PARB) which was proposed as an autonomous body to control research resource allocation in agriculture, livestock, fisheries and forestry. Although PARB and its Executive Committee was established under notification in December 1991, with further notifications to give autonomous status and attachment to the Planning & Development Department in November 1995, the establishment of PARB under a govemment ordinance was delayed until September 1996. Subsequently, the PARB ordinance was re-issued every three months until the passage of the Punjab Agricultural Research Board Act of October 24, 1997. Recently the PARB has been re-attached to the Agriculture Department and the composition of PARB, comprising representatives from amongst the members of the provincial assembly, PARC, the agricultural university, progressive farmers and agribusiness, has been notified recently. The rules and regulations of business have yet to be changed to improve the working conditions of scientists. PARB presently is responsible for agriculture, livestock, fisheries and forestry research. Research Programme Development 18. Research master plans. All provinces finally completed PRMPs after delays of three to four years. The plans were based on informnation obtained from diagnostic studies and intemal reviews and discussions and had significant assistance from the intemational TA in finalising their documentation. The PRMPs produced are very comprehensive and generally achieved the objectives of determining priority programmes and projects for research, manpower requirements, training needs, facilities, equipment and budget requirements for operations and maintenance and capital and recurrent requirements for a five-year period with an additional indicative five years for each province. The PRMPs are providing important resource documents for planning the development of provincial research systems, but have not been updated as originally intended since completion. The inclusion of activity timetables for development of the research systems in future revisions would provide greater assistance Appendix I Page 6 of 12 to planning units. The National Master Agricultural Research Plan (NMARP) was prepared simultaneously and the information contained in the PRMPs was consolidated into an input to the NMARP. 19. Priority research programmes. A major objective of the project was to orient the research systems to support research that was only of high priority. Some high priority research programrnes were identified at appraisal, other additional priority research programmes were to be identifiedl during the PRMP preparation process. This was a major objective of the development of PRMPs. But because of the delays in preparing the PRMPs, support for additional research programrnes was only provided during the last two years of the project. Thus two groups of high priority research programes were supported, but overall, the provisional budget support for this activity was greatly under-utilised. 20. Competitive Grants System (CGS). CGSs were established in all four provinces, as a recommendation of the mid-term project review. Guidelines were established to set up a system of advertising, submission and selection of proposals, which would be open to all research agencies (both public and private) and universities, to be considered for funding of high priority research. While fully functional CGSs were set up in NWFP, Balochostan and Punjab provinces, in Sindh province, although proposals were received and selections of research projects made, no contracted research wvas with the applicants. Because of the relatively short time available for implementation of the CGS within the remaining project period, this was considered a pilot scheme and guarantees were obtained from the provinces that they would continue to fund these contracted projects beyond Credit closure. Support for the approved projects and the programme may be provided for in PARC, Punjab, Baluchistan and NWF provinces under the proposed Agricultural Sector Investment Project (ASIP). The continuation of CGS programmes is, however, problematic in Sindh province, which has not submittedl proposals for funding under ASIP. Research Management Support 21. Management Support Units (MSUs). After significant delays, MSUs were established in each province to advise on and assist with research planning, resource allocation, programming, monitoring and internal evaluation. In Sindh province the MSU was established as a secretariat of SARO. In NWFP the MSU was attached to the office of the AUP's Director General of Research and as a secretariat for the AUP's Research Coordination Board. In Punjab and Baluchistan provinces, MSUs were established and functioned as secretariats for PARB and BARB, respectively. In addition, in Punjab smaller MSUs were also established for the Director General's of Agricultural and Livestock Research. In service training was provided for staff of the MSUs as part of the international technical assistance to develop systems and skills for planning and project budgeting, establish functional management information systems and monitoring and evaluation procedures. The skills and systems functionality achieved seems variable, with some MSUs seemingly unable to operate the systems piloted/established. 22. Management Information Systems (MIS). The ISNAR INFORM methodologies have been introduced in most provinces, using database software, as well as the introduction of spreadsheet software for preparation of budgets, with support of the overseas technical assistance provided. It seems in many cases, however, that fully functional systems have not been realised where the relevant sections of the MSUs of the management bodies established under the project use such systems routinely for planning and management purposes. Appendix I Page 7 of 12 23. Monitoring and evaluation (M&E). The provisions provided for assistance in establishing M&E systems in provincial management units were adequate, and fairly fully implemented with TA support. The systems established focus on the flow of core technical activities of the project: planning and conduct of research, etc. However, no provision was made for installation and training of a CPM-based project management system, to give staff the means to prioritize and control the setting-up of project implementation activities (recruitment, contracting TA and training services, procurement, and administrative processes connected with budget, programme approval, etc.). These contributed to the delays: initially, project staff were often unable to identify the most critical tasks for immediate attention, and, later, delays were not identified until they had grown into major problems. Research - Extension Linkages 24. A major objective of the project was to improve and strengthen research-extension linkages. Extension staff were involved with agricultural research scientisits and social scientists in the several diagnostic surveys carried out in all provinces to analyse and describe the agro-ecological and socio-economic conditions that affect farming systems and from which priorities for research were defined in the PRMPs. The establishment of Adaptive Research Units (ARUs) at Agricultural Research Institutes in Punjab (Faisalabad), Baluchistan (Sariab), NWFP (Tarnab) and Sindh (Tandojam) provinces and the strengthening of Agricultural Economic Research Units (AERUs) at these same locations were envisaged as additional mechanisms to strengthen research-extension linkages. However, while the ARUs undertook some adaptive research there was little or no involvement with Subject Matter Specialists and Adaptive Research Farms within the provincial Agricultural Extension Departments. Linkages between research staff and AERUs also appeared limited. Major Factors Affecting Project Implementation 25. By the end of the project many of the physical targets had been achieved, but all were severely delayed, thus adversely affecting a significant beneficial impact on research and research system development within the timeframe of the project. The major achievements were the filling of all fellowships for long-term overseas training, though with three years delay, the knowledge and skill transfer that occurred from the TA provided, the improved staff mobility and greater capacity and effectiveness of research support as a result of acquisition of vehicles and equipment, the preparation of appropriate guidelines for future research and system development as a consequence of the production of provincial and national research master plans, and the establishment of a substantial amount of reasonably well-focused research, following the funding of priority research programmes and the introduction of the competitive grants fund. It has to be stressed, however, that these achievements were only obtained towards the end of the project after three to four years of poor implementation performance. Overall, the project is considered unsatisfactory, as three of the major project objectives were only partially achieved and the fourth objective of improving and strengthening research/extension linkages was only minimally achieved. This unsatisfactory outcome was due to many constraints associated with project design, bureaucratic, political, financial and managerial factors which occurred during project implementation. 26. Design. The major institutional innovation of autonomous research bodies for each province were proposed (and accepted by GOP) without adequate consideration of the complexity of the change, and without full consultation of all the Government agencies involved at Province level (e.g. S&GAD, Law Dept) which might have contributed advice and control vital steps of the process. Appendix I Page 8 of 12 In at least one province (Punjab), there is a strong feeling that the authorities were pressured into this change, to which they were not committed. This affected the degree of local ownership of the proposals, and consequently, the extent to which provincial staff were willing or able to pursue issues through the necessary administrative processes, and this aggravated the problems listed in the next section. 27. Bureaucratic. Many of the decisions involved in implementation are unnecessarily centralized (e.g., procurement, confirmation of nomination of overseas trainees). Others are subject to repeated scrutiny of matters already decided (e.g., the issue of actual staffing levels can be, and was, re-opened during the administrative approval phase of release of funds). At the same time, the unavoidable problems inherent in the administrative processes were not taken into account in drawing up the irnplementation plan, or the phasing of the expenditure, with the result that both were over- ambitious in the early years. Major groups of actions affected by this complex of deficiencies were: the delayed approval of the umbrella PC-I by the Executive Committee of the National Economic Council (ECNEC); the establishment of a core of key full-time staff at each provincial centre of the project; procurement; contracting of TA services; and placement of overseas trainees. In addition, the existence of multiple levels of control appears to have been the main cause of divergence between the SAR and the various PC- Is. 28. Political. The rapid rotation of senior staff has had a very damaging effect on the project, as it has contributed to lack of follow-through on decisions already taken; it appears to result from the exercise of personal influence over the placement of preferred staff members, and the pursuit of attractive postings. This happens at all levels; it breaks continuity and commitment, and dissipates the effects of training. In many cases, major sections of the agricultural research structure have never been incorporated into a single unit, or have been moved outside of it, such decisions being taken in a non-transparent manner. In the case of Sindh, a policy decision has been made to create a new post of Secretary, Science and Technology, responsible for agricultural research (in the narrow sense, excludingy livestock, fisheries, etc.) and on-farm water management, although the Mission understands that this arrangement is now under review. The changed institutional arrangement for PARC has also had serious adverse effects on its autonomy and functioning as well as the project objective of consolidaiting and complementing the advances achieved under ARP I at the federal level. Frequent changes in higher level management are also politically controlled which obviate the stability required in management of research organisations. 29. Financial. The release of counterpart funds and utilization of the Credit was greatly delayed throughout the project. Not only was the amount released much less than the proposed and allocated budgets, but releases were invariably delayed to the latter part of each financial year. This has serious implications both in planning support for priority research and undertaking research which is often seasonal in nature. 30. Managerial. The top-down, bureaucratic style of management adopted by most senior managers is inappropriate to a professional service, and it has suppressed internal debate where this was necessary (in Sindh, for example, project staff were unwilling to raise the issue of the establishrnent of SARO, because they believed that this would incur the then Secretary's displeasure). The delay in PC-I approval, combined with the inherent length of the administrative processes resulted in a late establishment of the core staff of each Provincial centre of the project, and the lack of any project management tools, allowed priorities to be overlooked, and emerging difficulties to be ignored; the authoritarian has limited the extent to which project staff have been able to progress- chase matters which have become stuck at a senior level in many instances. The design, political, Appendix I Page 9 of 12 bureaucratic, and management deficiencies of the situation have combined to delay or disrupt all the main thrusts of the project. Project Costs and Financing 31. Project costs. Total project cost at May 1998 is estimated at Rsl,702.399 million (US$49. 306 million) (refer table 6.2), which is 4% below the original estimated project cost of Rsl,769.742 million (US$81.933 million) even with more than a 100% devaluation of the Rupee against the Dollar during the course of the project, and is 40% below the original estimated project cost in Dollar terms. Project expenditure was only 6% of the estimated project costs in the first three years of the project. 32. Credit disbursements. Of the total IDA Credit of SDR44.40 million (US$57.30 million), SDR26.55 million (US$38.08 million) had been disbursed to the GOP by May 31, 1998. A total of SDR4.0 million (US$5.16 million) of the Credit has been cancelled. Only 20% of the Credit was disbursed in the first four years of the project (Refer Table 6.1). The undisbursed'Credit at May 31, 1998 was SDR1 3.85 million (US$14.06 million), which will not be fully disbursed by Credit closure on June 30, 1998, or following processing of outstanding claims up to October 31, 1998, for expenditure incurred prior to Credit closure. 33. The funding support proposed under the project was for 59% of the funds to go directly to the provinces and 41% to PARC. Of the 41% of funds allocated to PARC, 15% was to be used for strengthening PARC and NARC with a further 26% of funds flowing through PARC to the two agricultural universities (5% each) and to indirectly strengthen provincial research (16%) through coordinated research programs, collaborative research, expansion of Agricultural Economic Research Units and expanding PARC's remote research facilities. In fact, PARC has consumed 54% of the IDA credit disbursed to Pakistan, with proportionately more (about 8% each) going to the universities than proposed, but the distribution of funds for PARC/NARC and those earmarked for indirectly strengthening provincial research not yet determined. While the actual disbursement of IDA Credit to the provincial governments (46%) was less than proposed, the funding support for NWFP and Baluchistan was proportionately about the same as proposed, but Sindh province received greater support while Punjab received lesser support than planned. Project Benefits 34. The serious delays in implementation of almost all components of the project have meant that few benefits have been realised within the periods of Credit disbursement. The major potential benefit of the project beyond Credit closure is expected to come from the upgrading of qualifications, skills and knowledge of a significant number of researchers in the provincial research system. Also, the introduction of the innovative CGS has paved the way for expanding the role of the private sector, universities and farmers' organisations and for fostering public-private partnerships. This, together with improved research facilities and improved focus of research on priority areas should ultimately result in improved technologies and agricultural production. Because of this situation there has been no attempt by any research institution to assess the impact of its research on agricultural development. However, results from research in developing new crop varieties, cropping patterns, crop diversification, IPM procedures for a range of important crops, etc., give positive indications of the longer term potential benefit from funding priority areas of research. The realisation of these potential benefits are however conditional on the continued financing of these research programmes. Appendix I Page 10 of 12 Project Sustainability 35. While the benefits obtained from more knowledgable and skilled staff and improved research facilities is expected to be sustained, the sustainability of research programmes and the institutional and managerial improvements achieved is at serious risk. The general financial crisis in Pakistan will mean that all government departments and agencies will receive restricted budget allocations for the immediate future years. Some exceptions may occur as optimism has been expressed in future funding of research in Punjab province. Funding shortages will result in greatly restricted availability of funds for operational and maintenance expenditures in agricultural research with consequent adverse effects on the realisation of benefits from project investments in human resource developrnent, upgrading of research and support facilities and continued implementation of identified priority research programmes. Overall, the project has failed, though only partially in the case of Punjab and Baluchistan, to establish the organisational and managerial systems envisaged that were identified as necessary to ensure the efficiency and effectiveness of highly integrated and relevant research in agriculture, livestock, fisheries and forestry. Progress has been made towards achieving functional flexibility of the apex bodies established in Punjab and Baluchistan. However, fully functional apex bodies are not yet in place in these two provinces. In Sindh little progress has been achieved, whilst in NWFP the project has failed to address the post-merger issues identified during the MTR. Bank and Borrower Performance Bank Performance 36. The Bank participated in ten supervision missions as well as conducted the mid-term review of the project. The suggestions given for improvement in project performance were very comprehensive and seemed generally well appreciated by the Borrower. However, given the key importance of institutional changes proposed to the provincial research systems, the Bank should have been more sensitized early in project implementation to the difficulties arising from limited local commitment to the establishment of autonomous provincial agricultural research bodies and sought to undertake an intensive examination of the underlying institutional and political problems, with a view to finding a mutually-acceptable solution, rather than threatening the suspension of Credit. The Bank also failed to give sufficient emphasis to project activities that would strengthen research-extension linkages, given this was a major objective of the project. Borrower Performance 37. While project implementation performance improved significantly with time, overall Borrower performance is considered unsatisfactory. Whilst there may have been unreasonable expectations in the SAR of implementation progress, given the inherent difficulties of the government system, rnuch could and should have been done to speed implementation progress. Many covenants of the Development Credit Agreement were not fulfilled by the Borrower in due time leading to serious delays. T he very delayed appointment of a project director and establishment of an implementation support unit in each province was unnecessary and had serious consequential implications to many other important activities (training programmes, contracting of TA, preparation of master plans, etc.). Even in the early years of the project, when the financial situation was more favourable, insufficient local funds were made available and Credit disbursements were very slow. A substantial amount of project resources have been used for supporting ongoing operational expenses, such as: project staff are largely transferees from other parts of the relevant Department; a substantial share of the research financed has been for ongoing experiments and studies rather than new priority research; and in a few Appendix I Page II of 12 cases, it was reported that project staff and vehicles had been assigned to other uses outside the project. Future Operational Plans 38. Punjab province has recently organised a workshop on agricultural research with the aim of restructuring agricultural research in the province. Baluchistan province has established a committee to examine the further development of their research system in which it is planned to establish a number of research stations in agro-ecological zones. In NWFP a task force was constituted in 1997 to consider the future organisational arrangements for agricultural research in the province and although a report was submitted in June 1997 no further developments have occurred. Most agree that physical resources are generally adequate and need relatively little strengthening. The continued scarcity of funds for operations, maintenance and capital development, however, does not seem to have engendered emphasis on the potential for cost savings through rationalization of the size of the research system or to explore mechanisms for tapping alternative sources of finances (both public and private). Lessons Learned 39. The main lessons that have emerged from the implementation experience of this project are: (a) There is a need to consider how far it makes sense to lend (or borrow) for technical investment projects in a situation in which there are widespread deficiencies in the Government system, without a parallel effort to address these deficiencies. (b) The problems of the Borrower in managing the cashflow problems of its ongoing, project-related activities in the agricultural sector should be recognized, and the possibility of loan support for these examined. (c) Where technical projects involve substantial institutional change, it is essential that formulation missions provide the professional expertise required to analyse existing systems, design new arrangements, and design programmes of change (i) in collaboration with all the parties involved on Government side, and (ii) using tools and concepts appropriate to situations in which there is a plurality of objectives and strategies, and no strong central control. (d) Where such change is needed, it is essential to evaluate both of the main options: (i) structural and procedural measures, and (ii) providing the resources and skills required to enable units within the existing structure to function better. (e) In the interests of providing realistic schedules as standards against which actual implementation progress can be measured, project formulators should be required to use CPM-based methods to prepare implementation schedules, and to base phased expenditure forecasts upon these. (f) It should be a requirement (supported with resources and training) that modem project management techniques be applied to the setting-up phase of projects, i.e., the provision of new or improved facilities and assets, whether those be hardware, improved administrative procedures, or staff with enhanced capability. (g) Key technical assistance should be selected before project start-up. Appendix I Page 12 of 12 (h) Key institutional reforms should be in place prior to the project start-up. (i) Long-term training programmes can be managed by Project Management Units, as demonstrated in NWFP, and shouldn't be managed by consultants. E. FOLLOW-UP 40. The mission will complete the preparation of the ICR for this project for submission to World Bank by late July 1998. Final project costs and Credit disbursements will have to be finalised by the World Bank with assistance of FAO/CP after the four month period for reimbursements following Credit closure. The GOP must prepare their final assessment of the project according to the guidelines provided by the mission and submit this to the World Bank by August 31, 1998. The mission Aide- Memoire will be amended as required, based on discussions in the wrap-up meeting, and will be attached to the ICR prepared by the mission for submission to the World Bank. Appendix 2 Page 1 of 6 No. 1(17)IDA-1187. Nfi M s GOVIERNMENT OF PAITAN MINISTRY OF FINANCE -AND ECONOMIC AFFAIRS (ECONOMIC .AFF.IRS DWIISION) Tcleg: ECONOMIC Telex: ECDIV:O05-i3 Islamabad, the 24th November, 1998 Fax: 9210734 & 9205971 SECTION OFFICER ab x6B A Phone No.9206556: 2 5 NOY 1 Subject:- PAKISTA.N: -Aricultural Research Proiect (Cr.2154-PAKh): RECEIED Comments on Draft Implementation Completion Report (ICR' i)ear \lr. Cx)avvum. Please refer to vour letter dated November 12. 1998 regarding the above subject. A opy of consolidated comments on Implementation Completion Report (ICR) received from L o Food. .Vriculture & Livestock is forwarded for further processing the case at your end. With re-zarcls. Yrours sincerely. (Sharafat Hussain Niazi' Mr. Rashed ul QawunL Agriculture Specialist. World Bank Resident Missiont 1sdamabid. Appendix 2 PAKISTAN - Agricultural Research II Project Page 2 of 6 (Cr. 2154-PAK) Implementation Completion Report Appendix 2: BORROWER'S COMMENTS A meeting to finalize comments on the subject Implementation Completion Report(ICR) was held on 26.10.1998 at PARC headquarters, wherein representatives from federal and all the four provincial components participated. The meeting was chaired by Dr. Kauser Abdulla Malik, Chairman, PARC. 2. The Project Directors/representatives of all the components reiterated that the project designing and implementation was faced with the following major obstacles. The participants unanimously resolved that the following views should be prominently depicted in the ICR report. 2.1. The project was appraised in 1989, while the World bank approved the project in July, 1990 and it was physically started in July, 1993-94. There was a duration gap of five years in the project: appraisal and physical implementation. The time bound action plan should also have been prepared/revised. 2.2. The local currency allocation during the first two financial years 1990-91 and 1991-92 was zero in NWFP, Balochistan and Punjab while GOP and the Govt. of Sindh made token allocation; which averaged at less than 1% of the PC-1 allocations. The appointment of consultants, as per requirement of the World Bank, was also delayecd. The effective project implementation was five years insteacl of eight years as originally scheduled; because during first three years only 6% of the provisions was provided/spent. The project cost utilization both IDA and local, in-term of US$, during the first three years was 0,1 and 5% in PARC; 0,6 and 16% in Sindh; 0,0 and 4% in Punjab; while both in NWFP and Balochistan the utilization was 0,0 and 3%. Thus implementation of activities was delayed and shifted to years 4-8. 2.3. It was consensus that the report has highlighted weakness-of the borrower and its implementation; while the achievements have been masked; thereby declaring the project un-satisfactory. This is.unfair; project should be declared as satisfactory because the various; component targets in term of civil works, procurement of --machiner,, equipment, vehicles, technical assistance and-traiaing. have been fully implemented. In some of the components the targets were exceeded. The restructuring of the research system was included without consultation of sta 6 holders and identification of repercussions. The inclusion of the restructuring/autonomy, an Appendix 2 Page 3 of 6 o6ge-old complexity, should have been given indepth thought/study. The Bank too was relaxed on these conditions till last two years of th project. In fact the issue should have been analyzed thoroughly, time schedule fixed, the slippage should have been negotiated with the stake holders. Similarly preparation of the, National Agricultural Master Research Plan (NMARP) was delayed till last 2 years and the consultants were busy in placing the students in the universities abroad; and no attention was given to preparation of NMARP during the first five. years. This again was due to first three years slippage of activities. 2.4. The research activities particularly high priority research & Competitive Grant System (CGS) scheduled to be initiated three years earlier, could not be undertaken because of initial 3 years slippage. The preparation of NMARP was also delayed. Neither borrower no(' donor gave a second thought to. revise the planned targets and preparation of the schedule of activities. The credit disbursement went on as per schedule and was closed in June, 1998. Irrespective of the above considerations inspite of the strong need, the project was not extended to overcome the slippages. -2-.5. The multiple and bureaucratic control of the'projeEctt;0 loss-Of autonomy by PARC, duality of control in NWFP eroded the autonomy of research sygtem. The restructuring of research system by the Bank was never agitated, except in papers, during early phase of the project. 2.6. Inspite of all the bottlenecks, the research system both at federal and provincial level was effectively able to support the existing research system through ARP-II. Significant achievements particularly research were made, which needs to be depicted more prominently in the report. 2.7. Inspite of very complicated bureaucratic procurement system both of the World' Bank and in the provinces, a significant percentage(70%) of the equipment was procured. There is strong need of simplifying the procedure for procurement of machinery, equipment both by the donor and the borrower. 2.8. It was generally felt that technical assistance without considering utilization strength of the stake holders was built in the project. Although consultancies were implemented' but utilization of most of the consultancies was not upto the required standards and most of the reports/recommendations have been shelved for want of funding. There is a little possibility/hope that these- recommendations will be implemented; thereby substantial part of the credit being wasted. Appendix 2 Page 4 of 6 _2.9. It. was consensus of the program implementation leadership that the report is unilateral in highlighting weaknesses of the borroweirs; while strong points including achievements particularly of the implementing agencies although mentioned but without effectively highlighting them. Thus the report gives unrealistic picture of the implementation of the project. 2.10. We strongly urge that the report should be re- itten in the light of above comments of all the implementing tnors at federal and provincial level. .i . . i Appendix 2 (#'nIL7Sub: Pakistan Agricultural Research Project(ARP-11)/CR.2154-PAK/ Page S of 6 &' X^/ Draft Implementation Completion Report(ICR) - PARC Comments The following para-wise comments/corrections are suggested before the report is finalized: Title/Para No. Acronyms: MINFAC be changed to MINFAL USAID stands for United States Agency for International Development Table of Contents Table-6: The title may be changed to "Key Indicators for Project Operation". Preface Para 2 line-5: PARC may be inserted before the North West Frontier. Evaluation Summary Para-5: The word "unsatisfactory" may be changed to "satisfactory" because the overall targets in termns of physical progress except restructuring of research system were achieved. This was consensus of all the participating units. Para-6 & subsequent: The financial figures for expenditure may be changed in the light of attached updated expenditure statement. Para-7(a): The project achievements may be more prominently depicted; the changing scenario should have been thoroughly discussed and vigorously pursued. Part-I, Objective-I Para 6: The purchase of second lot of equipment during the last two years in all the units including PARC was delayed because of complicated financial bidding process. The last lines of he paragraph may be amended that "the equipment could not be ordered/purchased because of complicated financial bidding process and credit closure date". Para-7: The figures for PARC M.Sc. and Ph.D. fellows returned may be changed as follows: Degree Sent Completed Remaining Ph.D. 44 39 05 M.Phil 06 03 03 M.Sc. 07 05 02 Total: 57 47 10 Para-7, Last two lines: The hiring of TA was obligatory condition of the agreement from World Bank and subsequently NWFP Agriculture University, Peshawar was able to manoeuver exemption. PARC was in much stronger position to handle TA of their own as well as Provinces. In fact PARC had the experience of doing it in larger projects of USAID in the past. Para-9: It was strongly felt that the TA provision in the PC-I was generally higher and it could not be appropriately utilized; except those who were given specific research assignments. The wording "TA input has generally been appreciated" may be deleted. Para-12: The horticulture research component has now become part of the BARB. This para may be modified. End of Para-19: Last sentence conceming delayed and reduced budget allocations is not correct. In fact in PARC, the budget could not be utilized by coordinating Appendix 2 Page 6 of 6 units and there was a balance to the end of the year(s). The statement may be modified. Para-21: The purchase of over 70 per cent of the machinery, equipment and vehicles(100 per cent) was made and delivered to the units. About 30 per cent equipment purchases could not be made mainly because of complicated procedure for procurement both of the Governmnents and IDA. The price escalation also would not allow purchase of all the equipment. Para-23: PARC contracted International Rice Research Institute(IRRI), Philippines for collaborative research programme between PARC/Provincial Governments, and a very useful interaction did occur. Para-24, line two: The word Department of Agriculture may be substituted with MINFAL. Para-26: It is pointed out that the last consignment of equipment could not be ordered due to procedural difficulties both at PARC/ Provinces and World Bank. The last sentence may be modified accordingly. Para-27: The figures of training may be revised for PARC as per para-7 above Objective-4 Para-36: The figures on financial utilization may be revised in the light of attached updated expenditure statement(s). Para-39: The project achievements and benefits should be more prominently highlighted. Para40: Para concerning performance of the project declaring it unsatisfactory on the basis of 3 project objectives is not logical. The performance should be evaluated in terms of achievements, component-wise rather than objective-wise. The progress in improving research system was the only objective which was not fully achieved. In fact fourth objective concerning research/extension linkages; although evaluated in the ICR did not appear in the evaluation mission reports, plans of implementation and no specitic financial provisions wvere maMe-for this- objective. Therefore, we strongly dis-agree with the performance evaluation on the basis of objectives particularly objective-4 outlined because they did not tally with the outlined project activities/financial provisions. The performance may be declared satisfactory. Para-46/47: PARC also need to be mentioned/commented. Para-52(b & c): The paragraphs may be elaborated particularly on pre-conditions and project design, schedule of activities with target dates. It is extremely important that financial provisions for follow-up of the post project period sustainability of activities is included in the Project Document. A schedule of post project follow-up is agreed upon by the Implementing Agency and donors. This has paid a very high dividend in some of the bilateral country projects. Table-3: Project completion plan and actual was 30th June, 1998. Similarly loan closing date was 31st October, 1998. Table-5: The actual completion figures for training may be revised. Table-8(a &b): Unit e.g. million etc. may be indicated. AGRICULTURAL RESEARCH 11 PROJECT U S _ CHINA MAIN AGRICULTURAL RESEARCH INSTITUTIONS AND AGRO-ECOLOGICAL ZONES , r RESEARCH INSTITUTIONS: < R Parc and jottitftes t\ -Cr,,o.'r. * Plnt, soils or forestry ? Vll A Li-estck -nd fisheries * Agriciutr-l ni-ersitfis2 Agro-esologicor boundoriet .0 0<, 3|Mbo l e O, AGRO-ECOLOGICAL ZONES: f/ NV / f Indus delta Os y , ; Southern irrigated plain M ° / 111 Sandy desort o&b| V IV N.rtOrn irrig.tnd plamin l&b) < t 0 °- r,ru0 v Ba--i lads tdrt JAA14MU VI Wet mAurtains rO V AND VIl Nathe-n dry monetains r % d vD KASHM IR VAII Wete- dry noantos IX Dry ester- plateau X Salo-n piedmont 13 Coral irrigated lands I-hlsyts in mr a. A Natial -apital M 3PreoAee capitals Other cities and lawns _Surerrt d s, L ~~~~~Risat _.. -...-..- Proolnee b -ndais > S K III I ._._ Internotional boundories , - { A g g / R~IlIbVri,roAr > 29 for zon~oeoo- |/MIhf °bd AHORE / r ~~~ ~ ~ ~~~~ ~ ~~~~~~~~Vlill /AVa ,- A F G H A N I S T A N U A / / ./ / OUE~~~~~~~~~~~~~1TTIA U olN zP f:/ / /~~~~~~~~~~~~~~~~~~~~~~~ \\ / 2 '/ | / 7 ~~~~~~~~~~~I N D I A x B~~~~~~ A Z U C /5 T A ISLAMIC REP. OF IRAN 1 I x . N ' \ . / D -- o s0 100 50 0<0 2<0 300 350 4(0 f E ' k \\2> ** ~~~~~~~~~~~~~<, J 9 50 CO~~I t 2 50 ZCO 000 As24 R B 'A N' * F A;v/- ,o> '2' w 0 i-~~~~~~~~~~~~~~~~~~~~- 0,0 'rp rs reetrro,rO 4