Report No. 19523 World Bank Agricultural Extension Projects in Kenya An impact Evaluation June 30, 1999 Operations Evaluation Department Document of the World Bank Currency Equivalents (annual averages) Currency Unit = Kenya Shillings (Ksh) 1982 US$1.00 Ksh. 10.92 1983 US$1.00 Ksh. 13.31 1990 US$1.00 Ksh. 22.92 1991 US$1.00 Ksh. 27.51 1997 US$1.00 Ksh. 58.73 1998 US$1.00 Ksh. 62.15 Acronyms and Abbreviations ABA Actionaid (Kenya) Beneficiary Assessment ASIP Agricultural Sector Investment Program ATD Africa Technical Department BA Beneficiary assessment CF Contact farmer CVM Contingent valuation method DEA Data Envelopment Analysis FEW Frontline Extension Worker GDP Gross domestic product IDA International Development Association IFAD International Fund for Agricultural Development Ksh. Kenya shilling MALDM Ministry of Agriculture, Livestock Development and Marketing NCF Non-contact farmer NEP National Extension Project OED Operations Evaluation Department PPA Participatory Poverty Assessment RHBS Rural Household Budget Survey SAR Staff Appraisal Report T&V Training and visit WTP Willingness to pay Fiscal Year July 1 - June 30 Director-General, Operations Evaluation Mr. Robert Picciotto Director, Operations Evaluation Dept. Ms. Elizabeth McAllister Manager, Sector and Thematic Evaluations Group : Mr. Gregory Ingram Task Manager Mr. Madhur Gautam The World Bank Washington, D.C. 20433 USA. Office of the Director-General Operations Evaluation MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUIBFECT: World Bank Agricultural Extension Projects in Kenya: An Impact Evaluation Attached is an evaluation of the impact of World Bank supported projects for agricultural extension services in Kenya. The Training and Visit (T&V) system of management was introduced in Kenya by the World Bank in 1982 as a pilot, financed by the Second Integrated Agricultural Development Project (Cr. 959-KE). It has since been supported by the National Extension Project (Cr. 1387.-KE, henceforth referred to as NEP-I), approved in 1983, and the Second National Extension Project (Cr. 2199-KE, henceforth NEP-II), which became effective in 1991. NEP-I closed in 1991, three and one-half years behind schedule. NEP-II closed in 1998, three months behind schedule. The Bank projects had two objectives: institutional development of the extension service and sustained increases in agricultural productivity. The effectiveness of the extension approach adopted by the projects has been a subject of debate because of the perceived high cost and an apparent lack of impact on agricultural production. This debate has been part of a broader disagreement on the effectiveness of the T&V approach to extension. The arguments have focussed on its efficacy relative to alternative mechanisms for delivering extension advice. While it is generally agreed that the T&V system is costly, the controversy centers on the impact on agricultural production. Despite the intensity of the debate, however, there have been very few attempts to rigorously establish the impact of T&V projects. This evaluation adopted a theory based approach to gather a credible body of empirical evidence to rigorously establish the impact of the projects. Following a results-based management framework, key indicators are evaluated at the various stages of the results chain to assess the performance of the Kenyan extension system. The evaluation finds that the projects have had limited institutional development impact. They have had some beneficial impacts in terms of increased geographical coverage, improved research- extension linkages (albeit belated) and improved staff quality through training. Overall. however, this evaluation found the current extension system to be ineffective and inefficient in delivering the needed services to the farmers. The institutional design has lacked a focus farmer empowerment. As such, inappropriate incentives have resulted in a lack of accountability or responsiveness to the clients' needs. Most importantly, the system is not financially sustainable. A distinction needs to be maintained between the relevance of extension services per se and the relevance of the projects' design. The rationale for providing extension services is still relevant; however, several features of the projects' design proved to be inappropriate. The available evidence suggests that the extension approach applied in Kenya was not efficacious. The physical impact of the extension services can also not be established with the current data. While it is likely that there was a positive 2 impact on farmer productivity and efficiency in the initial years of NEP-I, the benefits appear to have been short-lived as even the 1990 data do not indicate any significant impact. This evaluation, thus, could not establish a significant impact for either the key "outcome" or the "results" indicators; nor could it establish a positive rate of return to the current expenditures on agricultural extension. The main lessons emerging from this evaluation are: (i) There is a need for more efficient targeting of extension services to focus on groups and areas where the potential for growth is highest. This in turn calls for a more flexible and a "smart" system that can identify gaps between average and best practices, and allocate scarce resources more rationally. (ii) To guide the "smart" system and to target better, it is necessary to have timely flows of relevant management information and continuous evaluation to provide in-time feedback. Hence, there is a need for a reliable monitoring and evaluation system. (iii) Commensurate with the pace of technology generation and the demand for advice, the intensity of the extension service needs to be suited for particular circumstances. In some areas, a leaner and less-intensive presence (in terms of extension staff per farm households) would allow a wider geographical coverage and may be more cost-effective. This calls for a more demand-driven and responsive delivery system. (iv) A blanket approach using a uniform methodology in all circumstances is also not likely to be effective. A more pluralistic approach that exploits the synergies between the extension service, low-cost modern communications, demonstrations, printed media and partnerships with the civil society and the private sector needs to be developed. (v) Finally, the service delivery mechanism needs to fully incorporate client focus. This requires considering alternative options such as cost-sharing, farmer organizations, decentralization, etc. as an integral part of the delivery mechanism. Attachment Contents Preface .............................................................................................................................................iii Sum m ary y......................................................................................................................................... v . Introductin ............................................................................................................................. 1 Objectives........................................................................................................................... 2 The National Extension Projects....................................................................................... 3 Evaluation Strategy and Outline........................................................................................ 3 2. Institutional D evelopm ent ...................................................................................................... 5 Before ... ............................................................................................................................. 5 ... and after ......................................................................................................................... 6 Conclusions......................................................................................................................... 8 3. Beneficiary Assessm ent ......................................................................................................... 10 4. Efficacy: Q uantity and Q uality of Contact.........................................................................12 Program D esign................................................................................................................ 12 Findings............................................................................................................................ 12 Conclusions....................................................................................................................... 16 5. Outcom es: Aw areness and Adoptio ................................................................................... 16 Findings ............................................................................................................................ 17 Conclusions....................................................................................................................... 20 6. Results I: Farmer Efficiency and Productivity Change.....................................................20 Findings............................................................................................................................ 21 Conclusions....................................................................................................................... 24 7. Results II: Production Effects of Extension........................................................................24 Findings............................................................................................................................ 25 Conclusions....................................................................................................................... 26 8. Client Focus: Farmer's Valuation of Extension Benefits..................................................26 Findings............................................................................................................................ 27 Conclusions....................................................................................................................... 28 9. Conelusions ............................................................................................................................ 29 R e ferences ..................................................................................................................................... 30 Annex A. Im pactM odel .............................................................................................................. 33 ii Annex B. Background .................................................................................................................. 35 A nnex C. The Design of NEP-I and II........................................................................................ 37 Annex D. Institutional Features .................................................................................................. 38 Annex E. Change in the access and quality of services over the past 10-15 years.................42 A nnex F. Briefing ......................................................................................................................... 43 Annex G. Awareness and Adoption of Extension Messages....................................................45 Annex H. The Contingent Valuation Method (CVM) and its Application.............................46 Annex I. Borrower Comm ents .................................................................................................... 47 Annex J. OED Response to Borrower Comments ....................................................................51 Working Papers (Available on request): 1. The Efficacy of the T&V System of Agricultural Extension in Kenya: Results from a Household Survey 2. Awareness and Adoption of Extension Messages 3. Reconsidering the Evidence on Returns to T&V Extension in Kenya 4. Farmer Efficiency and Productivity Change in Kenya: An Application of the Data Envelopment Analysis 5. Production Impact of Extension in Kenya 6. The Willingness to Pay for Agricultural Extension Services in Kenya: An Application of the Contingent Valuation Method This report was prepared by Madhur Gautam (Evaluation Officer). Peer reviewers were Gershon Feder, David Nielson, and John Heath. William Hurlbut provided editorial assistance. Helen Claverie, Maisha Hyman, Constance Frye and Janet Wyse provided administrative assistance at various stages of the evaluation. iii Preface This is an evaluation of the impact of World Bank supported projects for agricultural extension services in Kenya. The training and visit (T&V) system of management was introduced in Kenya by the World Bank in February of 1982, as a pilot project, financed through the Second Integrated Agricultural Development Project (Cr. 959-KE). The pilot was implemented in two high-potential districts, Nandi and Kericho. Based on the brief experience of the pilot, the T&V system was expanded to 30 of Kenya's then 41 districts over a 3-year period with the support of the National Extension Project, approved in 1983 (Cr. 1387-KE). The project (henceforth referred to as NEP-I) was the first phase of a longer term institution building effort. Subsequently, the Second National Agricultural Extension Project (Cr. 2199-KE; or NEP-II), became effective in 1991. NEP-I closed in June 1991, three and one-half years behind schedule. Total project costs estimated at appraisal were US$28.6 million, of which IDA was to provide US$15.0 million. The project was cofinanced by the International Fund for Agricultural Development (IFAD) in the amount of US$6.0 million. The final costs were US$24.5 million, with IDA contributing US$14.5 million and IFAD US$5.5 million. The remainder of the IDA credit, US$0.5 million, was cancelled at closing. NEP-II closed in June 1998, three months later than planned. Total costs at appraisal were US$47.9, of which IDA was to provide US$24.9 and IFAD US$6.0 million. The final costs were US$24.8 million. Total IDA disbursements were about US$16.0 million. The IFAD contribution was US$0.7 million. The remainder of IDA credit was cancelled at closing. Following standard OED procedures, a draft report was sent to the government for comments. The comments are attached as Annex I. OED's response to the comments is attached as Annex J. This report presents the main findings of the evaluation. The detailed analyses are contained in six working papers which are available on request. The overall analysis was done by Madhur Gautam, with contributions from Jock Anderson (working paper 3), Jonathan Alevy (working paper 4), and Amy Gautam (working paper 6). The household survey and data processing were undertaken by the Tegemeo Institute of Egerton University under the overall supervision of Gem Argwings-Kodhek. The team included Francis Karin, Thomas Awuor (supervisors), Betty Kanunga, Bridget Ochieng (assistant supervisors), Joshua Ariga (editing), Lucy Wanyanga, Daniel Ouma, Alex Wanjohi, Geoffrey Oganga, Isaack Maritim, Milka Wangare, Salim Mgallah, Tony Sisule, Joshua Kerima, and Stephen Irungu (enumerators). The staff survey and data analysis was done by Willis Oluoch-Kosura (University of Nairobi). The inventory of technology and extension messages was compiled by Lutta Muhammad (Kenya Agricultural Research Institute, Katumani Research Station). The time series data on extension services in the study districts were compiled by Martin Mutuku and Allan Musamali of the Ministry of Agriculture. The Africa Technical Department household survey data for 1990 were graciously made available by Robert Evenson and the Rural Household Budget Survey data for 1982 by Steven Block. Many individuals in Kenya have contributed to this evaluation and it is not possible to acknowledge all of them individually. The contributions of the many Kenyan farmers who patiently participated in the household survey and its pre-tests, and those subjected to the many field visits made during the course of this evaluation, have been invaluable. The support and assistance of the Ministry of Agriculture, particularly of the extension staff at Kilimo House, in the study districts, and in other districts visited, has been instrumental in conducting this evaluation. Generous contributions were made by many staff of the Kenya Agricultural Research Institute and the International Research Centers in Nairobi, and members of the donor community and the civil society in Kenya. Finally, the support provided by many colleagues in the resident mission and at Bank Headquarters is gratefully acknowledged. V Summary The Kenyan extension service adopted the training and visit (T&V) system of management, in 1982 and has since been supported by the World Bank through the First and Second National Extension Projects (NEP-I and II). The projects had two objectives: institutional development of the extension service and sustained increases in agricultural productivity. The effectiveness of the extension approach adopted by the projects has been a subject of debate because of its perceived high cost and an apparent lack of impact on agricultural production. This debate is part of a broader disagreement on the effectiveness of the T&V approach to extension. The arguments have largely focussed on its efficacy relative to alternative mechanisms for delivering extension advice. While it is generally agreed that the T&V system is costly, the controversy centers on the returns to the high levels of investments by the borrowing countries, and hence on the impact on agricultural production. Despite the intensity of the debate and the high volume of lending by the Bank, there have been very few attempts to rigorously establish the impact of T&V projects. This evaluation adopted a theory based approach to systematically gather a credible body of empirical evidence to establish the likely impact of the projects. Following a results-based management framework, the evaluation sought to relate the observed results in the farmers' fields to the projects' inputs. In addition, intermediate output and outcome indicators are measured to assess the performance of the extension system along the results chain to confirm the potential for impact. The main findings and conclusions are: Institutional development: The institutional development impact of NEP-I and II has been limited. NEP-I introduced T&V as "first and foremost a management system". However, after fifteen years, there appears to have been no appreciable improvement in the effectiveness of the extension services: there is a lack of a strategic vision for the future development of the extension system; and the management continues to be weak, with virtually non-existent information systems. The prolonged ineffectiveness of the extension services has led to recent efforts by the Government of Kenya, with the help of other Bank projects and donors, to rationalize its extension services using alternative approaches. * The projects established a national system organized along the T&V lines of management. * The main benefits of the projects have been increased geographical coverage, improved research-extension linkages (albeit belated) and improved staff quality through training. * The institutional design, reflecting the projects' objectives, has lacked a focus on the critical issue of farmer empowerment. As such, inappropriate incentives have resulted in a lack of accountability or responsiveness to the clients' needs. * The hierarchical structure has been a disincentive for innovation, partnerships, and efficiency, and the extension system has been top-down, supply driven and non-participatory. Sustainability: The system is neither financially sustainable nor cost-effective. * The current system is significantly more costly, and no more efficient, than the one it replaced. * Government allocations to extension, as for other public expenditures, continue to decline, and the system is heavily dependent on donor funds. vi * An overwhelming proportion (80%) of the operational budget is consumed by staff salaries. As a result, many problems that limited the effectiveness of the previous system have persisted, and staff have reverted to the methods of dissemination that were used earlier. * The approach taken-a high intensity of contact with a limited number of farmers-has been costly and unwarranted given the inadequate stock of messages for dissemination and the slow pace of new technology generation. Relevance: A distinction is needed between the relevance of extension services per se and the relevance of project design. The rationale of providing extension services to smallholders is still relevant. However, several features of the projects' design proved to be inappropriate. * There is an unmet demand for extension services, and the farmers value the access to such service enough to be willing to pay for it. * There are still few alternatives to government-provided extension, although alternative providers are emerging. * The relevance of the biweekly, or even a monthly, visit schedule is questionable considering that most farmers, including the contact farmers, do not want to meet the extension agent very often. The staff themselves are wary of the repetitiveness and ineffectiveness of the visits. * A blanket cover of the majority of production areas using a single approach and standard messages proved to be inefficient and unproductive. The limited experience from some pilot initiatives under NEP-II confirms the potential usefulness of alternative and more responsive approaches. Efficacy: The farmers did not have adequate access to extension advice in 1982, and appear not to have adequate access now. All methods using the available data indicate that the current institutional arrangements are ineffective in delivering the service. This reflects, in part, the poor enabling environment in which the extension service has operated in recent years. However, the poor performance is not entirely a function of external factors. Available evidence suggests that the current situation is not much different from that in 1990. * Both the qualitative and quantitative assessments indicate that the relevance of the advice delivered has been limited. The extension system has not been responsive to the types of information farmers want, and the relevance of the advice to the needs of a broad range of farmers is questionable. * The focus of the extension service has remained on disseminating simple agronomic and maize-related messages. Extension activities under NEP-I and II have had little influence on the evolution of the patterns of awareness and adoption of recommendations. * The failure to take advantage of the face-to-face extension approach to deliver more advanced and context-specific advice has reduced the cost-effectiveness of the project design. This is particularly true for NEP-II, at the start of which most farmers were known to have already adopted the simpler maize messages. * A significant finding is that a very high proportion of those who are aware of the messages, on even the more complex practices, have adopted them. Thus, although non-extension related factors, including the often cited lack of financial resources or access to credit, may be important in preventing farmers from adopting certain complex practices (e.g., fertilizers and pesticides), it is evident that the lack of information continues to be an important constraint. * The progress on gender issues has been mixed. The earlier bias against women farmers has been rectified, but some bias persists in the selection of contact farmers. The proportion of female field-extension agents has remained largely unchanged since 1982. vii * Although the coverage has increased, access to existing services is limited, especially for the poor and the less educated. The availability of information and the quality of the services provided are reported by the farmers to have declined since the early 1980s. * The level of outreach is well below the anticipated levels, with only about 7% of the contact farmers (and 2% of all farmers) meeting with extension agents on a regular basis and in a setting prescribed by the project design. Efficiency: Overall, a positive rate of return to the current expenditures on extension cannot be established. It is likely that NEP-I had some early beneficial impacts. However, these benefits appear to have been short-lived as the data do not indicate any significant impact even by 1990. * There has been some improvement in the technical efficiency of farmers since 1982, but the overall efficiency continues to be low. Their low economic efficiency indicates the potential for farmers to achieve significant savings by simply moving to a more economical mix of inputs under current market conditions. * The data show a very small positive impact of extension services on the level of technical efficiency, but the level of confidence in the result is low. The data also indicate that extension services have no discernible impact on the level of economic efficiency. * A significant impact of the supply of extension on productivity at the farm level cannot be established from the current data. * What can be established is that the allocation of extension resources has been inefficient. Extension services have been poorly targeted. The growth in agricultural production has been higher in the previously less productive areas, whereas the placement of extension staff has favored the more productive areas. While extension possibly had an early positive impact in the spread of simple technological messages to the previously less productive and under- served areas, this impact cannot be conclusively established with the current data. * A significant proportion of farmers are willing to pay for extension services. This reflects that the farmers value the advice when they receive it. The perceived benefit, as reflected in the amount that farmers are willing to pay, however, is well below what the government currently spends on extension services per farm. The main recommendations follow from the lessons emerging from this evaluation, and are relevant for the design of future agricultural extension projects. Targeting: The first lesson is the need for more efficient targeting of extension services to focus on areas and groups where the impact is likely to be the greatest. This calls for a more flexible and a "smart" system that can identify the gaps between existing best practice and average: practice and allocate the scarce resources more rationally. Further, the selection of farmers to interact with should be more representative of the local socio-economic environment so that more relevant advice can be delivered to different categories of farmers. Information systems: Targeting in turn calls for appropriate flows of timely and reliable information, and hence monitoring and evaluation (M&E). An important lesson emerging from the Kenyan experience is the need to identify farmer demands and tailor the service to suit local technological and economic conditions and circumstances. M&E also is critical for identifying the gaps and guiding the "smart" system as needed for a more efficient targeting of services. Intensity: Reflecting their experience, the farmers currently do not want to see the extension agent too often; and there are not enough new technological recommendations to sustain a high intensity of visits. It would thus be more cost-effective to establish a leaner and less-intensive presence but with wider coverage. It is conceivable that with improved quality of service, the viii demand will increase commensurately. To account for such a circumstance, it is imperative to have a responsive and dynamic delivery system as discussed above under targeting. Pluralism: A blanket approach, using a single or uniform methodology to deliver standard messages is likely to limit the effectiveness and efficiency of extension services. Younger, more educated farmers are taking over from their parents; radio programs are popular among farmers; and alternative providers are beginning to emerge in rural Kenya. It would be advisable to adopt a more cost-effective strategy that exploits the synergistic effects of low-cost modem communications, demonstrations, printed media, and partnerships with civil-society and the private sector. This would leverage the resources to increase outreach, and is likely to have a greater impact with the same or lower demand on government resources. Client focus: The central focus of the institutional design should be on empowering the client, the farmer. An effective way to incorporate client focus is to consider alternative options which give a voice to the farmer, such as cost-sharing, farmer organizations, decentralization, etc., as an integral part of the delivery mechanism. In particular, cost recovery (even if only partial) offers several advantages: it provides appropriate incentives, addressing the issues of accountability and quality control; it makes the service more demand-driven and responsive; it provides some budgetary respite; and it encourages alternative providers. Such institutional arrangements remain unexplored in Kenya. 1 1. Introduction The focus on agricultural extension in Kenya is rooted in agriculture's strategic importance for development. With 70% of the population living in rural areas, almost half in absolute poverty (World Bank 1998), the centrality of rural development to any strategy for poverty alleviation is obvious. Critical to rural development is the performance of agriculture, both because of its direct contribution to the national economy (28% of GDP, 60% of export earnings, and 80% of national employment) and because of its indirect multiplier effects. Within agriculture, 81% of the farmers are smallholders (owning less than 2 hectares), and hence the imperative is to improve smalIholder productivity. Accordingly, the Government of Kenya has long had agricultural extension on its development agenda. As part of its growth strategy, Kenya adopted the training and visit (T&V) system of management for its extension services in 1982 with the support of the World Bank. The extension system has since been supported by two Bank-financed projects, the National Extension Project (NEP-I) and the Second National Extension Project (NEP-II). The performance of the Kenyan extension system, however, has been controversial. This debate is part of a broader disagreement on the cost-effectiveness of the T&V approach to extension. The arguments have largely focussed on the efficacy of the mechanisms for delivering extension advice (Picciotto and Anderson, 1977). It is generally agreed that the T&V system is costly; the controversy, however, centers on the returns to the high levels of investment by borrower countries in the T&V system, and hence on its impact on agricultural production (Purcell and Anderson 1996). The nature of the general debate is briefly discussed in Annex B. Despite the intensity of the debate, the important role of agricultural extension in the Bank's development strategy for Africa, and the large volume of investments made, there have been very few attempts at rigorously measuring the impact of T&V extension. In the Kenyan context, the debate has been elevated by the estimate of very high returns to T&V extension by an Africa Technical Department (ATD) study (Bindlish and Evenson 1993, 1997) on the one hand, and the lack of any visible "results on the ground," on the other In light of the poor performance of agriculture in recent years, declining budgetary resources, and efforts to rationalize the structure of the Ministry of Agriculture, Livestock Development and Marketing (MALDM), the effectiveness of the service increasingly has been questioned? The estimated high marginal return to extension by the ATD study notwithstanding, questions about the effectiveness of the T&V approach in Kenya were raised in the Performance Audit of NEP-I by OED (World Bank 1996). The audit concluded that NEP-I had some beneficial aspects, but it had several operational deficiencies and, most important, was not financially sustainable. With little evidence to suggest any significant impact on agricultural growth to justify the high fiscal costs of T&V, the audit questioned both the "appropriateness of the extension format developed as a uniform approach throughout Kenya's cropping areas" and the overall outcome of the project. The ensuing discussions with the Africa region led to a final OED rating of the project outcome as marginally satisfactory. The rating was contested by the Africa region. The disagreement has persisted, pending this impact evaluation of the Kenyan agricultural extension projects. I The Kenyan study is one of the three notable exceptions that have attempted to rigorously estimate the returns to T&V investments, as noted in annex B. However, as discussed in section 7, the estimated high returns are not robust. 2. A public expenditure review for agriculture revealed that in 1996-97 extension claimed 61% of the development resources allocated for the MALDM's core services. Extension also accounts for about 45% of MALDM's total expenditure and over half of its staff. 2 The discontent with the current system of extension is not limited to the Bank. The criticism is sharpest among the donor and NGO communities, who seek reforms toward a demand-driven system that is more accommodating of alternative approaches to reach farmers. The government is also wary of continuing with the large allocation of resources to extension given the poor performance of agriculture since the start of the 1980s. Does the poor performance of agriculture necessarily reflect on the performance of extension? The answer is not clear, since it is possible that agriculture could have done worse were it not for extension. The difficulty in establishing the impact of extension from the aggregate performance of the sector, much like the casual observations from field visits, reflects the lack of an appropriate counterfactual. The problem can be demonstrated with maize yields? Between 1970 and 1989, the annual growth rate for maize yields in Kenya was 4.7%. Accounting for rainfall, the rate increases to 5.4%.4 Considering the pre- and post-1982 (i.e., T&V) periods, the growth slowed from 6.3% to 5%, with the difference being statistically significant. However, accounting for rainfall, the difference between the two periods is no longer statistically significant. Given the strong and divergent opinions on the perceived performance of NEP-I and II, and lack of evidence to establish their impact, this evaluation takes an objective empirical approach' Most of the conclusions are based on the results of an OED household survey and a survey of the extension staff, supplemented as needed by secondary data and information from several recent studies conducted by the MALDM. The OED household survey covered the same population surveyed for the 1990 ATD study, which in turn used a subsample of the Rural Household Budget Survey (RHBS) of 1982. In 1998, interviewers revisited as many of the respondents as could be contacted in the clusters sampled by the ATD study. Thus, the evaluation has the advantage of some baseline data, albeit limited. To dissociate the survey from the government extension service and the World Bank, the household survey was implemented by the Tegemeo Institute of Egerton University in Kenya. Objectives The objective of this evaluation is an empirical assessment of the impact of the NEP-I and II projects in Kenya. Following a theory based evaluation approach, a mix of qualitative and quantitative methods are used to arrive at a consensus on the likely impact of the projects. In so doing, it seeks to provide an independent review of the earlier findings on the impact of extension in Kenya. To draw appropriate policy conclusions, a clear distinction needs to be maintained between the impact of a particular system, and the impact of extension per se. This has implications for interpreting the results in terms of the three key aspects of evaluation: the relevance, efficacy, and efficiency of the extension system. In the Kenyan case, the task is complicated by the fact that the T&V system was introduced on a national scale, preventing a with-without comparison. The system was also introduced rapidly (over the course of 3 years), now some 15 years ago, and while the data allow some before-after comparisons, they are limited. 3. Aggregate crop statistics are notoriously poor in Kenya. Given the importance of maize in Kenyan agriculture and the attention it has received from all quarters, the data on maize are likely to be the most accurate. Nevertheless, the accuracy of these data cannot be vouched for. 4. Average growth between 1970-96 is estimated at 2.7%, reflecting a deterioration of yields in the 1990s. The rainfall- controlled growth rate cannot be calculated from 1990 onwards for lack of rainfall data. The rainfall data used are average annual millimeters of rain from 14 stations from the south, southeast, central, and western parts of Kenya. 5. Most opinions are based on anecdotal evidence from field visits. See Box in Annex B on the difficulty in interpreting such "data." 3 The National Extension Projects The traditional system of extension in Kenya suffered from several weaknesses (World Bank 1983).' To overcome these problems, the T&V system of extension was introduced with the intent of providing "competent, well-informed village-level extension workers who will visit farmers frequently and regularly with relevant technical messages and bring farmers' problems to research" (Benor and Baxter 1984). The features of the T&V system as implemented by NEP-I and II are described in Annex C. T&V was introduced as a brief pilot project in two districts in 1982. Starting in 1983, it was rapidly expanded to cover about 90% of Kenya's arable land. The objectives of NEP-I were to achieve sustained increases in agricultural production in 30 of Kenya's 41 districts, covering all medium- and high-potential arable areas. The instrument was the reorganization and strengthening of extension services by the adoption of T&V over a period of three years, combined with an improvement in the link between research and extension. NEP-I was considered an institution-building project and the Bank expected to provide external assistance for 10-15 years to ensure that the needed institutional reforms and improvement in staff skills were made (World Bank 1983). In 1991, NEP-II succeeded NEP-I. The objectives of NEP-Il were to stimulate the development and adoption of technical packages that would enable smallholders to increase their productivity and incomes. In addition to continuing to support the work of NEP-I, the project introduced T&V to six new areas, and with the splitting of four of the original districts, T&V extension came to be applied in 40 out of Kenya's 45 districts. The project aimed to consolidate and fortify the gains made under NEP-I, increase direct contact with farmers, improve the relevance of extension information and technologies, upgrade skills of staff and farmers, and introduce innovations into the extension system on a pilot basis (World Bank 1990). Evaluation Strategy and Outline The measurement of the impact of the projects focuses primarily on their key objectives. Both NEP-I and II had two main objectives, institutional development and sustained increases in agricultural productivity. While important economic, social and environmental impacts are implicit in the rationale for these projects, they are not explicitly stated as major objectives. Nevertheless, the key social issues, gender and distribution of benefits, are naturally addressed because of the large proportion of female and smallholder farmers in Kenya. A theory based evaluation approach is adopted to establish the impact of NEP-I and II using the stylized causal "flow" model given in Annex A. The model reflects the design of NEP-I and II, which relied on the delivery of time-bound "messages" (indicated by solid arrows) by extension agents from researchers to farmers The feedback from farmers to researchers (indicated by broken arrows) was to be through the extension workers. As implemented, the program was decidedly non-participatory in that there was no pretense of involving farmers in technology development, in the development of messages, or in deciding which topics they might be interested in (MALDM 1997a). The impact of the projects can be assessed at several points along the continuum from the knowledge complex to final welfare change. Following the results-based management 6. These problems, of course, were not specific to Kenya (Feder, Lau, and Slade 1985). 7. As the perception of the roles and functions of extension has evolved, so have the models used to capture the interactions between research, alternative extension providers and methodologies, and farmers. The simplistic model here is meant to reflect the interactions as assumed for NEP-I and II. 4 framework, this evaluation sought to relate the observed results in the farmers' fields to the projects' inputs. In addition, intermediate output and outcome indicators are measured to assess the performance of the extension system along the results chain to confirm the potential for impact. The key indicative measures at each stage are given in the boxes below the chart. The evaluation strategy was to use a mix of qualitative and quantitative methods to measure and analyze the key indicators. No attempt is made to study the impact on household welfare as it is likely to be affected by a number of factors beyond the scope of extension activities. The layout of this report follows the approach adopted for evaluation: Institutional development: The focus of institutional impact is on the left half of the model in Annex A, specifically on the institutional arrangements, or "inputs," used by NEP-I and II to deliver their "output." The analysis is presented in Section 2, and combines secondary data with the findings of several recent reviews of extension in Kenya and results from the staff survey. Beneficiary assessment: A key element of the strategy was to analyze participatory beneficiary assessments (BA) of extension service. Although the BAs for this evaluation were not conducted, findings from two recent participatory assessments in Kenya are pertinent and these, along with the farmers assessments from the OED survey, are discussed in Section 3. Efficacy: Quantity and Quality of Contact: The debate on the effectiveness of T&V in Kenya has centered mostly on the "real" side of the equation, that is, on its impact at the farm level on agricultural productivity. To evaluate the projects' impacts, the strategy pursued follows the hypothesized flow in Annex A. The first of these is an analysis of the outreach and quality of interaction between extension agents and farmers-discussed in Section 4. Outcomes: Next in the causal chain, prior to a quantitative assessment of the actual physical impact of NEP-I and II, are the proximate outcomes of extension efforts indicating its potential for impact. These are measures of farmer awareness and adoption of extension outputs (i.e., the technological recommendations or "messages"). These measures, and how they relate to the supply of extension services, are discussed in Section 5. Results: Determining the physical impact, or the "results on the ground," requires relating the supply of extension services to changes in productivity and efficiency at the farm level. To study these impacts, two lines of inquiry are followed. One is a non-parametric measurement of farmer efficiency in production and how farm productivity has changed over time, discussed in Section 6. The second approach uses econometric methods to measure the impact of extension on farm production. Described in Section 7, these estimates reveal the returns to investment in extension and are needed to establish the efficiency of extension. Willingness to pay: The analysis goes back to the farmer in Section 8, with a new methodological tool. The contingent valuation method is used to elicit directly the farmers' willingness to pay for extension services. The findings from this exercise have implications for the relevance of the design of the past projects, and they provide insights for future project design. 5 2. Institutional Development A key objective of NEP-I and II was the development of organizational and institutional arrangements for efficient and effective delivery of agricultural services to Kenyan smallholders. After 15 years of development, as anticipated in 1982, the system is expected to be fully mature. It is natural to ask how the current system is performing relative to the one it replaced in 1982. The following discussion is based on reviews undertaken or commissioned by the Ministry of Agriculture over the past 2-3 years in an attempt to develop a national policy and a framework to rationalize its current extension service! These are complemented by conclusions from discussions with extension and ministry staff, focus group and individual discussions with district-level staff, discussions with former extension staff familiar with the early days of NEP-I, and findings from the staff survey conducted by OED for this evaluation. To review the current, or the after picture, it is instructive to put it in perspective with a review of the previous, or before picture. Before... Agricultural extension in Kenya dates back to the early 1900s, and several approaches had been tried. These included individual visits, group methods, unified extension, a farm management approach, an integrated development approach, and specialized commodity extension programs. Except for the last, none have endured. However, one area where extension had been highly successful was in the dissemination of hybrid maize technology. Starting in 1965, through a nationwide program of demonstrations and field days, hybrid maize was promoted to the extent that by 1977, 50% of even smallholders were growing hybrid maize (Johnson, et al. 1979). In 1982 when NEP-I was introduced, several disparate and uncoordinated donor supported efforts were operating. The various extension arrangements lacked a consistent national strategy and as of 1982 were essentially ad hoc project components. This proliferation of extension services was viewed as being expensive, inefficient, and largely ineffective. Nevertheless, the extension service had a well-defined line of command from the Director of Agriculture down to the field-level Frontline Extension Worker (FEW). The staff numbers were deemed adequate, but the service was judged to be performing well below its potential (World Bank 1983, 1990).' There were several reasons for this: the junior staff (FEWs) were not well trained there was a lack offield emphasis, and FEWs visitedfewfarms, and most visits were to progressive farmers, who represented at the time about 10% of all smallholders. Other problems included insufficient operational funds, which limited mobility (except for projects and programs funded by donors), resulting in lax supervision and support of the field staff; allowances for the FEWs were generally inadequate, which resulted in them travelling on foot and unable to cover a large area. The pre-project extension system concentrated its efforts on male farmers, although almost one- third of the farmers or farm operators were female. There was a proliferation of women's groups, 8. The key reference reports are the "Staffing Norms Study" conducted by the Ministry of Agriculture (MALDM 1997c); a study on "Alternative Approaches to Agricultural Extension" (MALDM 1997b) undertaken in preparation for the Kenya Agricultural Sector Investment Program, a multi-donor effort; a study on the "Proposals to Improve the Effectiveness of the Agricultural Extension Services in Kenya" (Kandie 1997); and the ministry's own reform proposal for a new "National Agricultural and Livestock Extension Programme" (MALDM 1997a). 9. Later, however, it turned out that due to a miscalculation of the number of farm families, the farm to staff ratio was much higher than the 500:1 ratio judged to be adequate by the Staff Appraisal Report. 6 and recognizing their importance, the government had started to integrate women into the extension service. Research extension linkages were weak although it was recognized that the research staff were more specialized and better equipped to analyze farmer problems than the specialists working with extension. The main vehicles for disseminating technical knowledge to extension staff were barazas,"o field days, research station bulletins, and Agricultural Information Center publications on recommended practices. The SAR, however, was realistic in that it warned against expectations of major increases in production as in the past (at that time) because of a lack of "new technological developments available" to promote rapid growth. The focus was thus to rely on smallholder intensification and effective resource conservation. ... and after A more detailed discussion of some specific institutional features is presented in Annex D. A summary analysis of the approach to agricultural extension adopted by NEP-I and II, from the Kenyan policy makers' perspective, is presented in MALDM 1997b (p.14) and is reproduced in Table 1. A brief prepared for a focus-group discussion by the staff of one of the districts visited for this evaluation also summarizes the generally held views about NEP-I and II in Kenya. This brief is reproduced as Annex F. The main conclusions that emerge for the institutional impact are described below. Table 1. NEP-I and II: Local Perspective on Approach Weak Points Strong Points / Very broad objective / Wide coverage / Top-down planning still strong / Cover all types of farmers / Not specific target / Strong staff training / Farmer participation weak / Development of professionalism at district / Staff motivation low officer level / M&E weak / Strong FEW present / Too much supply driven messages / Procurement of transport equipment and / Donor dependent office accommodation / Low flexibility / Low accountability Source: MALDM (1997b): Alternative Approaches to Agricultural Extension Organizational aspects. NEP-I and II succeeded in putting in place an integrated national system, but the management of the system is weak, an outcome of the poor project implementation arrangements. Kenya is also still in the process of developing a national policy on agricultural extension. As shown in Table 1, the system has some strong points and a number of weak points. While a similar qualitative analysis of the extension approach before NEP-I is not available, the institutional characteristics of the previous system, as summarized in the preceding section, provide some scope for comparative analysis. On the positive side, even though the organizational structure was maintained from the headquarters down to the field level, the influx of large sums of development and operational funds in the early years of NEP-I infused an unprecedented level of energy in the system. Increased staff training, new vehicles and office equipment, and a new paradigm had a positive initial impact in lifting the morale of the field staff. The links with research were weak, but sufficient to ensure an adequate flow of 10. A baraza is a public meeting called by the Chief, the location administrative officer, or by the local extension staff. 7 simple agronomic messages to farmers. The increased activity is generally believed to have had a beneficial initial impact, and training improved staff skills. The focus was on maize and on simple agronomic messages. While initially this is likely to have had some positive impact on areas previously not covered by extension, the continued narrow focus over time reduced the incremental benefits. With sufficient funds flowing, the service expanded rapidly with large increases in staff numbers, which in turn helped increase outreach to previously uncovered areas and categories of farmers." This helped reduce some of the biases of the previous system, namely those against women and younger farmers, and those living far from access roads. At the same time, however, new biases entered, in favor of the more educated, and more productive areas. The allocation of resources has also favored areas with lower levels of poverty, and there does not appear to have been any improvement in staff productivity in terms of increased contacts per staff member. With staffing rising to unsustainable levels and the high costs entailed by the extension approach adopted, operational budgets proved to be inadequate. As a result, at the end of NEP-II, a number of the problems afflicting the previous system have continued and are significantly affecting the effectiveness of the of service. The staff survey shows that less than a third of those who were in service in 1982 feel that the system is more effective now than it was in 1982." Over half consider it to be less effective (the rest think it has remained the same). The system is also more expensive now, but no more efficient. Insufficient operational funds mean limited transport facilities and inadequate staff allowances, restricting mobility and reducing supervision. The lack of new messages, because of slow progress on technology generation and limited adaptive research,, has made training sessions and field visits repetitive and unproductive. As a result, staff motivation and morale have declined substantially since the initial boost in the early years of NEP-I. Institutional aspects. The institutional aspects of extension also have been poorly developed. As implemented, both NEP-I and II adopted a top-down, supply driven approach to extension, with a lack of focus on the critical issue of farmer empowerment. While this approach may have been useful in the formative early years, the changes have been few and slow, even during NEP-Il despite its explicit design intent. More importantly, the primary client, the farmer, still has little or no voice. While most staff believe that both they and the farmers have a say in the development of messages, a majority also note that the topics of the training sessions are decided by the district officers. There is a mismatch between the type of information farmers want (advice on complex practices) and what is effectively delivered by extension (simple agronomic messages). There is also a mismatch in the methodologies preferred by the farmer (demonstrations) and the extension agents (home visits). These disconnects manifest the lack of client focus and responsiveness of the extension service. As a consequence of the poor functioning of the contact-farmer and the contact-group approach, together with inadequate messages and lack of operational funds, the extension agents have gone back to the old system of disseminating messages through public meetings or barazas. A large number of field staff are also working with alternative providers, NGOs, the private sector, and other projects. However, the extension service has not yet attempted to institutionalize these linkages to make the system more effective and efficient. 11. Here, outreach is simplistically defined as any type of contact between farmers and extension; however, as discussed in Section 4, this measure can be poor measure of effectiveness. It is also not consistent with the "contact farmer" or "contact group" as advocated in T&V. However, for reasons discussed below, field staff are using alternative methods, particularly public meetings (barazas), to increase their outreach. 12. About 26% of those in service in 1990 think the current system is more effective now than in 1990; 49% think it is less effective and 25% think it has remained the same. 8 In the institutional design of NEP-I and II, a key feature, the incentive structure, has not been given adequate attention. The FEW has no accountability to the farmers. The control mechanism adopted in NEP-I and II, per the standard T&V approach, is designed to ensure that the FEW follows a prescribed route to regularly meet with a fixed number of contact farmers. Supervision is thus reduced to ensuring compliance with the specified route. There is no focus on the quality of the FEW-farmer interface and the FEW is, for the most part, a messenger. Financial aspects. The most problematic feature of the current Kenyan system is its financial unsustainability. Compared to the previous system, the system now is significantly more expensive. In 1982, Kenya was spending an equivalent of US$3.92 per household in 1991 constant dollars (Bindlish and Evenson 1993, estimates for their study districts). Current estimates (using national level-data, which do not appear to be much different from estimates for the same study districts) suggest an expenditure of US$13.29 in 1991 dollars ($15.11 in 1997 dollars). That is, in real terms extension expenditures have gone up over 300%, while at the same time, increasing fiscal difficulties have led to declining operational budgets for extension services. This rise in costs reflects the current design of the system, with its focus on a high frequency of field visits and training sessions, and a blanket cover of most of the country's arable area using a standard FEW:farm ratio. While it may be argued that the high costs are due to increases in staff numbers, and the increase in staff was not due to NEP-I or II, it should be noted that the current staffing levels are below the standards deemed acceptable by the SARs for both NEP-I and II. Even at the current levels, however, the operational budget is highly inadequate. Conclusions The limited impact of the two projects on the institutional development of the extension services in Kenya is evident from the fact that, at the end of NEP-II, the Kenyan extension service lacks a strategic vision; there appears to have been no appreciable improvement in the effectiveness of the service; and the management of the service is weak. The management information system is virtually non-existent. The benefits include wider coverage, improved research-extension linkages and improved staff quality through training. While the underlying general principles of T&V are relevant for any institutional design," the highly structured extension approach adopted by NEP-I and II has been neither effective (as discussed in a later section) nor sustainable. There has been little flexibility in the approach adopted, even though the design of NEP-II sought to introduce some plurality to improve the functioning of the system. From an institutional perspective, responsiveness and accountability are aspects that usually can not be bureaucratically imposed. The NEP-I and II design failed to incorporate mechanisms to ensure appropriate incentives for effective service delivery. The prolonged ineffectiveness and fiscal unsustainability of the current system have led to some recent efforts by the Kenyan government to rationalize its extension services using alternative approaches. Some recent steps towards the adoption of a farming systems approach to extension, improved research-extension links, a rationalization of the deployment of the frontline extension staff, and experimentation with alternative modes of delivery of extension services suited to localized circumstances, are important developments. In conjunction with other donors, the bulk of this work has been supported by the Bank, indicating that the Bank is also being responsive. The resulting changes on the ground are, however, yet to be seen. 13. Professionalism, single line of command, concentration of effort, time-bound work, client orientation, and regular training. 9 In considering the alternatives for the future, appropriate "exit" mechanisms need to be incorporated into the institutional design. To do this, the nature of the benefits, the nature of the services provided, and the efficiency of the mechanism for delivery need to be considered. Most agricultural information, whether it be embodied as a technological characteristic of a good (e.g., the quality or new type of fertilizer) or disembodied (e.g., information to improve the management skills of the farmer), yields benefits-private returns-to the farmer!' Several types of information also have benefits external to the farmers (social returns) as others learn by indirect information or observation. Nevertheless, so long as there are private benefits, the farmers should be willing to pay for the information, suggesting scope for cost recovery, and the findings in Section 8 reveal that this is the case. On the nature of the service provided, the types of information that can be associated the most with the extension service are simple agronomic messages (Section 5). These messages have genuine public goods characteristics, as they can and do get easily passed around. But these are also the types of information that can be delivered more through more cost-effective means such as radio broadcasts, pamphlets, or public meetings. Printed media, in particular, have significant potential as more educated younger farmers are taking over from their parents. Given the nature of the information they have most to offer, the decision of most extension agents to return to the baraza method of dissemination appears to be rational. For other types of extension information, more detailed and personal advice is required, e.g., addressing specific pest problems or the optimal type and quantity of fertilizer for localized soil conditions. Since the returns to such information are also largely private, there is significant scope for cost recovery or private provision. Further, since the increased use of such inputs also benefits the private input suppliers, they have an incentive to deliver such information. That this trend may already be under way, albeit slowly, is also evident from the findings in Section 5. There are a number of institutional options which can be used to make the system more client focussed, including decentralization, using farmer organizations and civil society, outsourcing of extension services, cost-sharing, etc. The central focus of the institutional design should be on the empowerment of the farmer, which requires incorporating appropriate incentives in the design of the system. A direct way to address the issue of incentives is to build in some degree of commercialization. It is often argued that extension services are a public good and that most farmers, particularly the poor subsistence farmers, may not be willing to pay for the service. This hypothesis is difficult to test, however, because public extension historically has been provided for free, which has tended to crowd out alternative providers. Still, in the Kenyan communities where extension is not reaching or is inadequate, mostly low potential and poorer areas, NGOs are active and some are also charging a fee, albeit indirectly as a membership fee to a group or a club. Also, commercialization does not necessarily imply a direct cash payment for advice or doing away with public extension altogether. The latter may be neither feasible nor desirable, at least in the foreseeable future (in the interest of equity and broader coverage). Creating partnerships with the local suppliers, providing training sessions for them (although they are also likely to be able to afford fees for training), and improving the infrastructure for improving input delivery are also ways of increasing commercialization of information delivery. Cost recovery through direct charges may not be as a far-fetched as is often believed; and farmers appear to be willing to pay for extension advice. Even if only nominal or partial, direct charges have several advantages in correcting for the shortcomings of the current system. They provide 14. Most institutional analyses focus on the characteristics of the goods or services from the suppliers perspective. Thus, the subtractability and rivalry of benefits from the goods and services provide guidelines for the optimal delivery mechanism. However, even for a public good, viewing benefits from the demand side helps conceptualize efficiency gains from cost recovery, even if cost recovery is only partial. 10 the right incentives for the agent to deliver the advice that the farmer wants, it makes the agent accountable to the client and builds in a genuine quality control mechanism, it alleviates the budgetary constraints (at least somewhat) and it encourages other, notably private, provision of services. Note that commercialization does not necessarily mean totally privatizing the service. There are various options of introducing commercial elements into the public service provision and through public-private and public civil society partnerships that can be profitably exploited. Such partnerships already are emerging in Kenya, as discussed above. 3. Beneficiary Assessment The design of this evaluation included a participatory beneficiary assessment (BA) to hear directly from the farmers about their access to extension services, the quality and relevance of the advice, and their suggestions for the future." In the spirit of collaboration, it was agreed that the study would be conducted by the MALDM, with the active involvement of OED and the Africa Region. After the initial preparations, however, MALDM unilaterally decided not to pursue the BA for reasons unknown. This evaluation draws instead on a BA done for another study by Actionaid Kenya (Actionaid 1997, henceforth ABA). The ABA included a component designed to obtain the views of users and potential users of Kenyan extension services and hence some of the findings are relevant to this evaluation. In addition, a Participatory Poverty Assessment (PPA) was conducted in Kenya in 1994 (Narayan and Nyamwaya 1995).6 Some results from this PPA are also pertinent to the current evaluation. The findings of the ABA and PPA complement those from the OED survey. Welfare and productivity. The ultimate goal in NEP-I and II was to improve farmers' welfare, primarily through increased agricultural productivity. Hence, at the start of the OED survey, farmers were asked to assess their welfare (defined as a self-assessed standard of living) and farm productivity relative to the situation 10- 15 years ago. A majority (66%) think their welfare is lower now than before; only 25% think it is better. As for agricultural productivity, over 72% think it is lower now, while 25% think it has improved. Similar sentiments on welfare were expressed in the 1994 PPA-most respondents thought that life was better 8-10 years before the survey. In the 1996 PPA, about 70% of the participants thought that poverty was worse (relative to 5 years ago). The ABA also presents a similar picture. Access to services. The OED survey asked farmers about the change in access to, and quality of, 19 social and infrastructure services, including extension services. The results are presented in Annex E. A vast majority (75%) think that access to extension is the same as 10-15 years ago. Very few think quality has improved (11%), and most (39%) think it has deteriorated. Relative to other services, extension appears to be generally worse off in terms of perceived quality. It is noteworthy that for a number of services, the decline in public services is being compensated by private provision. For example, veterinary services have been privatized as a matter of policy, and this is reflected in the improved access to and quality of private extension services. Other public services experiencing significant deterioration are roads and public health. Nature of interaction. Results from the PPA and the ABA also reveal the lack of access to information, particularly by the poor. The assessments found extension services to be sporadic or 15. Besides complementing the quantitative, survey-based findings for this evaluation, the intention was also to compare the quantitative results with those from a more qualitative and participatory approach. 16. Even though these were "poverty" assessments, a substantial number of people belonged to land classes that are comparable to the OED survey households. 11 irregular, and generally targeted toward the "rich" or large landowners, bypassing the poor. Interactions with extension were noted to be low and service to be generally inconsistent and inadequate." Most farmers report the chief's baraza, the radio, or friends and neighbors as the "most consistent" source of information. The few demonstrations that are held are on the wealthier or more progressive farmers' fields, who are the ones that extension agents generally visit. The ABA also found that residents of lower potential zones have little access to extension and are being serviced by NGOs. There is no effective interaction between extension and NGOs. Relevance of advice. The differing perceptions of the needs of the poor are an important issue for the effectiveness of extension services. The extension staff target the wealthy farmers in the hope of faster adoption of technology, as the wealthy are more likely to be able to afford the new technologies. At the same time, however, the intent and hope is that the demonstrations will encourage the other, relatively poorer farmers, to adopt the same high input-high cost technologies. This contradiction is an important factor limiting the effectiveness of the current approach to extension. The current "non-users" or those who do not have access, would like to get information on crops that the larger farmers do not grow (specifically, crops other than maize or coffee). They also seek advice on less costly technologies, marketing, and diversification. These are not the types of information they get on demonstration plots, which are usually maize demonstrations. At the same time, the "users," i.e., the few that do have access, consider the technical advice provided to be of high quality and relevant. Farmers'priorities. Among the services that farmers would like to see improved, if they were to pay for the improvements, about 5% picked extension as their top priority item for improvement. In terms of ranking, however, extension ranks sixth among the 19 services ranked." The services ranked higher are also the ones for which the farmers report a deterioration in the quality over time." The services that have generally improved over time were ranked lower. This is consistent with the perceived deterioration of extension services. Conclusions. The BA and the survey are consistent in finding that welfare has declined over time. The BA findings also show that most farmers, especially the poor, have little access to extension advice. This result should, however, be put in perspective. Both the PPA and ABA note that those who have access, or the current users of extension services, recognize the quality of the advice rendered. Extension is also not expected to reach all farmers and hence the need for selectivity and reliance on farmer-to-farmer dissemination. The observation that extension presence is weak in low-potential zones is also consistent with the focus of extension, especially in NEP-I, on the relatively higher-potential zones. The key insights from the ABA and the PPA point to the reasons for the limited effectiveness of the current extension system. The poor quality of the interaction with the vast majority that are poor and smallholder farmers, and the relevance of advice rendered to suit their needs suggests poor targeting and responsiveness. If extension has to be selective, it should select a more representative set of farmers, so that the advice delivered is relevant to a broader range of farmers. It is also clear that the needs of the farmers, particularly the small farmers, are diverse and go well beyond the traditional crops such as maize and coffee. 17. On average, once a year in low-potential zones, twice in medium-potential zones, and often in higher-potential zones. 18. The ranking uses a simple tabulation of the reported first choice. Considering that basic preferences are likely to take priority, alternative rankings were tried using the top three and five choices, using the assigned ranks as weights for aggregating across observations. These results were consistent with extension retaining the fifth or sixth ranking. 19. The only exception being electricity. In declining order, the rankings are water, public health services, dry season access roads, electricity, and tarmac roads. 12 4. Efficacy: Quantity and Quality of Contact Efficacy refers to the extent to which the design of the projects was successful in delivering extension advice. For NEP-I and II, efficacy can be viewed both as the overall effectiveness of the extension system in terms of its outreach, as well as the effectiveness of the extension approach adopted. In either case, efficacy is assessed in terms of the output indicators for the projects: the extent and nature of contact between farmers and extension. The detailed findings are presented in Working Paper 1 and are summarized here. The analysis uses data from the OED survey on the farmers' access to information on agricultural enterprises, including specific questions relating to the nature and extent of contact with the public extension service. Program Design It is generally believed that the system in place before NEP-I was ineffective and inefficient. It favored the more "progressive" farmers, especially the larger, more educated, and male farmers. NEP-I and II sought to rectify these biases. The design of NEP-I and NEP-II is summarized in Annex C. Both followed the standard T&V principles, and used contact farmers, later contact groups (henceforth, both individual and group contact farmers are referred to as CFs), as the point of interaction with the farming community. Using the CF approach, advice was to be provided to about 10-15% of the farmers on a biweekly schedule, with up-to-date information on practices best suited to their specific conditions. The extension agent was to work mainly with the CFs, but was to involve as many other farmers as possible in the demonstrations and discussions. Extension workers would visit farmers regularly and receive systematic training and technical support from research staff. The initial focus under NEP-I was to be on simple messages, concentrating on a few important crops and the most important aspects of crop production activities. The focus was also on low- cost improvements, which the majority of farmers could afford. The implicit assumption being that once there was an increase in productivity and revenues, the farmers could graduate to more costly technological components with the additional income generated. Findings Information access. Public extension has historically been an important source of Figure 1. Availability of Information information in rural Kenya. The "normal" (relative to 10-15 years ago) source of information on agricultural activities for most farmers is the 60 - government extension service, indicating 50 that few alternatives are available. Almost 40 - half of the farmers, including CFs, also 30 - think that information is less available 20 . now than it was 10-15 years ago (Figure 10 1); less than 30% think it is more available 0 now than before. While this does not More Less Same Not Sure provide an accurate pre-project picture, it does indicate that the availability of CF NCF information has declined over the period Source: OED survey of the two projects. 13 The CF approach. The data show that CFs make up about 9% of the sample. While this is a bit lower than the desired 1(-15%, the proportion is reasonable. However, the data also show that the more educated have a significantly higher probability of being selected as CFs; being a female farner lowers the probability, although the result is only weakly significant (at the 10% level). This result is consistent with the findings of the ABA and PPA in the previous section, and also shows that at least some of the biases from the previous system have continued. The poor quality of the contact is reflected in the fact that 22% of even the designated CFs meet even monthly (as opposed to the prescribed biweekly meeting). Less than a third of the CFs normally meet the extension agent either on their own or on their neighbors' fields (Figure 2). The most striking finding is that (even on a monthly basis) only about 7% of the CFs meet extension as planned (regularly, either on their own fields or on others' fields, and at least once a month). Following the projects' design, using monthly meetings as the norm, about 20-30% of the population should be in regular contact with extension. However, in the entire sample, Figure 2. Normal Meeting Place and Frequency of CFs only about 2% of the farmers regularly meet with extension agents. Considering that extension was concentrating on 30 a few selected farmers of 25 choice, this low level of 20 contact is highly unsatisfactory. 10 - The low frequency of farmer- 5. -: extension contact is not a 0 statistical artifact of the current Ownfield Others Baraza Co-op Other survey or of current times. The field Society 1990 ATD survey obtained almost identical results. Even W12+ per year E 6-11 per year M 1-5 per year O Otherlirregular in 1990, before the start of Source: OED survey NEP-II, about 2% of all farmers were meeting extension agents as planned (only 3% of all farmers were meeting extension agents on a monthly basis in any setting in 1990). These results indicate the poor efficacy of both NEP-I and II in delivering extension advice in Kenya. The efficacy of the CF approach also depends on the indirect dissemination through demonstration and spread effects. The "T&V plots" established on the CFs' fields were expected to have strong demonstration effects on other farmers. However, this effect is likely to have been limited considering that less than a third (31%) of the non-contact farmers (NCFs) even know of a CF in their neighborhood.20 An alternative mechanism is the verbal spread-effects, from CFs to NCFs. These effects are also likely to have been limited because of apparently poor communication between CFs and NCFs. Among the few NCFs who know of a CF, 58% report ever having received advice from the CF and only 22% (i.e., 8% of all NCFs) receive such advice 20. It is possible that some NCFs may not know that their neighbor is a designated CF, and consequently may report their source of information as "friends and neighbors." However, it is unlikely in a small community, especially where group activities are reportedly high, that farmers would not observe the regular and frequent visits of the extension agent or other farmers to one particular farm. In either case, the lack of publicity about extension activities in specific locations is likely to have its reduced potential impact. 14 on a regular basis. A significantly higher proportion of the same farmers (86% of those who ever received information and 93% of regular receivers of information) also indicate that they discuss general agricultural information with other farmers. The flow of information from CFs to NCFs therefore is not working as well as may be expected." Contrary to the projects' objectives, the methods that appear to be more widely in use today are the same as were popular in the pre-project (pre-T&V) period: most farmers, including CFs, rely on barazas and other fora to meet their extension agent. This outcome is at significant odds with the intent of T&V, which sought to bring about a qualitative change in the nature of extension contact with farmers by moving away from barazas and other public meetings, which provide little opportunity for interaction. Public meetings rely largely on exhortation and are useful for broadcasting simple messages, but they are not conducive to effective learning or substantive exchange on technical problems. System performance. As extension is following alternative methods, it is important to consider the change in the overall efficacy and efficiency of the extension under NEP-I and II in reaching out to farmers, irrespective of the method used. The measures of outreach typically used are whether farmers have met an extension agent (which includes any type of contact) or the time of first contact. These measures, however, are poor indicators of either the quality or the effectiveness of the extension service. Nevertheless, they are pursued to allow a comparison of the systemic outreach between the pre- and post-NEP-I periods. Since the start of NEP-I, the proportion of farmers who have ever met extension workers has increased, with the increase being significantly higher for "new" farmers than for veteran farmers.22 However, the increase in outreach is almost directly related to the increase in staff numbers, indicating little or no increase in staff productivity or systemic Figure 3. Staff Allocation and Poverty by District efficiency. 3 0.7 Biases. The previous 0.6 extension system was 2.5 known to be biased 20.5 x in favor of the more 0.4 "progressive" 1.5 farmers and higher productivity areas. . Statistical tests show 0.1 that some of the 0 . 0 earlier biases against Muranga Nzola Kisumu Talta Bungoma Kericho Machakos women, small MM1982 C1990 1M1997 - Poverty farmers, and those Source: OED survey and "Poverty Profiles," Mukui 1994 living far from access roads, have been rectified. These have largely been a result of the expansion of the service to previously neglected areas. At the 21. These results are consistent with recent findings from beneficiary assessments in several African countries, which show that the CFs are likely to be less well connected with the rest of the community than is hoped (Salmen 1999). 22. This analysis is based on farmer recall of their first meeting with extension. Comparisons with three independent data sets for three different time periods since 1992 show that the recall bias is not likely to be significant. 15 same time, new biases have entered in favor of the more educated farmers, higher-potential zones, areas closer to markets and areas closer to Nairobi. The latter result is consistent with the headquarters-centric development of the Kenyan extension system noted earlier. The bias against relatively poorer areas, however, has been maintained (Figure 3). Farmer assessment of extension. While the quantity of extension advice delivered is less than expected, the farmers' approval rating of the meetings and "messages" by the recipients, measured in terms of "usefulness" and "applicability," is very high (86%)?3 Yet, few farmers (about 40%) have actually applied the extension recommendations. More important, a majority of even the CFs (51%) have also not applied the recommendations. This disparity, between the positive assessment by the farmers and their reluctance to apply the recommendations, is discouraging.24 The findings indicate that while the farmers think that the messages are probably good, they are just not meant for them, calls into question the relevance of the advice delivered. There is also a mismatch between the recommendations that farmers find most applicable, interpreted as the effective supply of information, and the farmers' demand for information. The former are generally simple or unsophisticated agronomic practices, while the latter are generally the more sophisticated input-application or intensification issues?5 Suspension ofNEP-H disbursements. Finally, an incident with potential consequences for this evaluation was the suspension of NEP-I disbursements in 1996. Although the survey was conducted nine months after the suspension was lifted, providing sufficient time for the system to get back on course, the farmer responses could reflect the adverse effects of the suspension on extension activities. At the same time, the disruption of service provides a rare opportunity for a "counterfactual" to evaluate the current effectiveness of the system. Figure 4. Impact of NEP-II Suspension, 1996 Since CFs are the most likely to be affected by the disruption of service 50- under the T&V system, the responses 40 - of CFs are of particular significance. 30 - The results show that a majority of the 20- CFs (60%) did not notice any change 10 - in the delivery of extension services in o i 1996, despite the disruption of funds CF NCF flow for almost the entire year (Figure 4).26 As may be expected, the ill% Noting Anyce proportion of NCFs reporting no 0% Reporting reduced visits change is considerably higher (88%). Source: OED survey Whatever the current methodologies being used by extension, the finding that a majority of the farmers did not notice any change raises two possibilities. One is that the service continued as usual by substituting non-project funds, questioning the additionality of NEP-II funds. The other, a more likely explanation, is that 23. Among the farmers who report receiving advice at least once a year, about 41% of the sample. 24. In part, the problem is that the term "useful," which is often used to determine the effectiveness of extension services, is ill-defined and vague. Farmers also are reluctant to volunteer criticisms. 25. "Supply" is defined here as the effective supply of information as revealed by the farmers' responses on the recommendations that they find most applicable. It is assumed that the recommendations that farmers find most applicable are either those they receive advice on, or are the ones that are relevant to their circumstances. 26. Of the CFs noticing a change, about 23% actually reported an increase in extension visits; 37% reported fewer visits and 32% reported no visits at all. The remainder gave unspecified or other responses. Surprisingly, the pattern of responses of CFs is almost the same as that of NCFs. 16 the frequency of contact of the CFs, as for most farmers, is generally low, and hence a disruption in its delivery was not noticed by many. This more likely scenario indicates the lack of effectiveness of the current extension system. Conclusions The key finding is that the CF approach, central to the design of NEP-I and II, is not working as anticipated. There is no apparent improvement in either the qualitative or the quantitative aspects of the extension-farmer interaction relative to what the situation presumably was before the projects. More specifically, there has been no improvement since 1990, when NEP-II started, and even then the approach was performing very poorly. Farmers think that information is less available now than it was 10-15 years ago. The qualitative nature of the extension-farmer meetings is not as expected, and appears to be no better today than it was before the project period. The extension agents have started using alternative methods, and many appear to have reverted to the old methods, particularly barazas, to reach the farmers. For the relatively simple messages, this approach may be more cost-effective. However, for advice on more complex practices and for solving farmer-specific problems, this method is not likely to be very effective. While the systemic outreach has increased, and some biases of the previous system rectified, this has been largely a result of increase in staff numbers, not of improved staff productivity. The advice delivered is apparently of limited relevance to farmers, as a majority of the farmers have not actually tried the recommendations, and is not responsive to farmers' needs. The lack of effectiveness of the current system is suggested by the fact that the disruption of the service for almost a year went largely unnoticed by the CFs and NCFs alike. 5. Outcomes: Awareness and Adoption In the causal chain from investment in extension and the desired impact, farmer awareness and adoption of technological components are important indicators of the extension service. They measure its proximate impact and provide a backdrop for assessing its potential economic impact.' The impact will surely be limited if extension is unable to appreciably increase the level of farmer awareness. Further, despite awareness, potential benefits from increased productivity will be limited if the farmers do not adopt the recommendations.28 The degree of non-adoption reflects in part the quantity and relevance of extension advice, especially given the technical, marketing and resource constraints that farmers face" Information diffusion typically occurs through a number of channels, with extension service being one, but not the only source. The role of extension becomes important when the normal process of diffusion is slower than is desirable. It becomes particularly important for more complex types of information (e.g., the type and quantity of fertilizer to use with a particular crop or a new variety) and solving farmer-specific or localized problems (e.g., pest control or soil 27. The actual impact of extension requires measuring the increase in agricultural productivity that can be associated with extension, and is dealt with in a later section. 28. Increased productivity can take the forms of diversification into higher valued crops, an increase in the efficiency of input use, a change in the use of productive inputs, or a combination of the three. 29. A farmer's decision to adopt a particular technology is influenced by a number of economic and technological factors, including extension advice (Feder, Just, and Zilberman 1985). 17 micro-nutrient deficiency). In the T&V approach, the selective "infection" of the CF with new information is expected to speed up the usual diffusion process?o The details on the analytical methods used, the data and results are presented in Working Paper 2. The main findings are summarized here. The analysis in this section deals with recommendations for cropping activities." Findings Awareness. All farmers have heard maize-related messages; most are aware of the messages on cash and minor food crops; and about two-thirds have heard of the crops being currently promoted by extension. The proportion is higher in the more productive districts or where the crops have a relatively long history. Thus, at a very basic level, messages appear to be reaching the farmers, with some room for improvement for the "new" crops. Awareness of simple agronomic recommendations is high, but falls significantly with the increasing complexity of practices. Surprisingly, the levels and pattern of awareness among CFs are similar to those Figure 5. Timing of Awareness for other farmers. The lack of awareness of complex messages among CFs is disappointing, considering that intensive 90 - face-to-face interaction has an advantage over other methods of extension in 6 delivering such advice?0 During NEP-I and II, awareness appears . e30 - - -,,M. to have increased, but only modestly a. (Figure 5).3 More importantly, the pattern 10 w- by activity has remained virtually the o1 same. Thus, the simpler messages for 1 2 3 4 5 6 7 8 9 10 11 12 13 which current level of awareness is high Activity complexity are also those for which awareness was already high before NEP-I.34 For complex - - - 1982 - '1990 i 1997 practices, the low levels of awareness have persisted. For most activities the change was relatively greater during NEP- Source: OED survey I than during NEP-II. 30. This process is akin to the spread of an epidemic or an infectious disease (Feder and Umali 1993). 31 Livestock extension services were included in NEP-II only after 1994. A preliminary analysis shows that the vast majority of the households who are aware of livestock related recommendations were already aware of them by 1994. The attribution of the awareness or adoption of these recommendations to NEP-II is thus likely to be tenuous. The analysis of the awareness and adoption of livestock recommendations will be conducted separately. 32. Less than a third of the sample is aware of the recommendations on fertilizer and other chemical inputs. 33 The activities corresponding to the numbers in figure 5 are given in Annex G, footnote 68. 34. Several early surveys showed that many of the simpler and maize-related recommendations were already widely in use at the start of NEP-I (see Gerhart 1975, and references therein; Ongaro 1990). 18 Awareness of maize-related messages is relatively high, but it is significantly lower for other crops, especially cash and non-traditional crops." Even for maize, however, the difference between simpler and complex messages persists. Given the high levels of awareness of the simpler messages, the marginal returns to additional efforts in extending these messages are likely to be low. The data from the current survey, as well as the findings frdm the 1990 survey suggest that similar efforts in NEP-II have also likely yielded limited payoffs. For those who are aware, government extension is a sizeable, but not the largest, direct source of information. This is to be expected since extension can only reach a small proportion of its client population. However, even among CFs, who meet extension agents most regularly, less than one- half cite extension as their source information. This finding is consistent with the earlier findings that even most CFs do not meet extension on a regular basis. An inter-temporal analysis of information sources reveals that even before NEP-I, public extension was the main source of information for spacing and the more complex practices. Its share of other, simpler messages was relatively low (and that of friends and family was quite high). During NEP-I and H, the share of extension for simple messages increased significantly, but its share for the complex messages fell (particularly during NEP-II). Equally significant is the increase in the shares of the private sector for complex practices and "other" sources (e.g., specialized services, cooperative societies, and youth clubs) for simpler practices. These trends reflect the increased focus of NEP-I on simpler messages. They also reflect the dynamism of an information system that is undergoing a transition, with non-public sources becoming increasingly important providers of information?" Adoption. The patterns of adoption follow those of awareness. In general, the adoption levels are very low, with the exception of some simpler practices (planting time and weeding activities). Less than a quarter of the sample has ever tried any of the 80 - recommendations for the complex practices. A breakdown by current and . 60 - past adopters shows that the proportion 40 - of farmers currently applying the l recommendations is almost negligible 20 - for the more complex practices (Figure 0 - 6). Not surprisingly, current adoption 1 2 3 4 5 6 7 8 9 10 11 12 13 rates are relatively higher for maize than Complexity ) for other crops. However, a comparison with the ATD survey results, which is BAIl Farmers EContact Farmers feasible only for maize practices, Source: OED survey indicates that the levels of adoption have remained almost the same since 1990. As expected, among the reasons for non-adoption (or discontinuation) of the recommended practices, lack of funds is cited most often. Together with land and labor, resource constraints are 35. Maize has been the focus of government extension since the mid-1960s. The cumulative impact of these efforts on the relative levels of awareness for maize practices is thus to be expected. 36. The results could also be interpreted as reflecting the efficient working of the spread effect of the CF approach. However, if this were the case, there should be a corresponding increase in the share of friends, neighbors, and family as a source of information. The results show a contrary trend: the share of friend, neighbors, and family has steadily declined over time for all activities. 19 the reasons reported for non-adoption by a sizeable proportion of the sample (40%). However, an almost equally substantial proportion of the sample (34%) cites reasons that could be addressed through proper extension advice. The most significant finding, however, is that a very high proportion of those who are aware have adopted the practices (over 80% for even the more complex recommendations). Thus, while credit and resource constraints may be important factors, the primary problem constraining the adoption of recommended practices is the lack of information.' Statistical tests of extension 's impact. The statistical analysis sought to establish the factors that influence the probability of awareness or adoption of each recommended practice." A detailed discussion of the approaches used and the results are given in Working Paper 2. The main finding is that the current supply of extension cannot be associated with an increased probability of awareness or adoption of individual extension messages. However, the supply of extension in 1982 continues to have a strong positive impact on the current levels of awareness and adoption. These results suggest that information diffusion has proceeded at its own pace and the impact of NEP-I and II is not apparent from the current data. The test of the efficacy of the contact farmer approach also yields disappointing results. Being a CF increases the probability of awareness for only spacing and cultural plant protection practices (without instrumental variables). Allowing for the endogeneity of the measured contact variable (i.e., with instrumental variables), CFs have a higher probability of awareness only for 5 of the 13 messages tested." Among other factors, social capital increases the probability of awareness of simpler messages, while farm size has the same effect for complex messages. Education has positive short- and long-term impacts: primary education for simple practices and higher education for complex practices. Better infrastructure (roads) increases the probability of awareness for most practices, while the distance to Nairobi and off-farm work generally have a negative effect. The adoption analysis tried a new specification to directly estimate delayed impacts, but fails to show a positive influence of extension activities since 1982.4 The initial (before NEP-I) stock of knowledge has a strong positive influence for every practice, but the subsequent supply of extension does not show any significant or systematic impact. The only significant impact discernible is the negative effect on the adoption of complex practices for early years of NEP-I. 37. It is reasonable to assume that, with over 80% of those aware having adopted the recommendations, if the remaining two-thirds of the sample who are currently not aware of the recommended practices are provided with appropriate advice, another almost 50% of the sample could be potential adopters. 38. These include the supply of extension, measured as the ratio of extension worker to the farm families in each location. The cumulative effect of extension advice over years is captured by using a weighted, lagged structure of extension supply over a number of years. 39. For spacing, seed rates, and chemical plant protection measures the predicted contact farmer variable is significant at the 5% level; for crop variety and cultural pest control, it is significant at 10%. 40. This new specification, a multiplicative parametric specification of the supply of extension over discrete time intervals between 1982 and 1997 was also tried for the awareness estimation. The results for awareness with respect to extension were similar to the adoption results. 20 Conclusions Both the descriptive and statistical analyses show that the focus of the Kenyan crops extension service has remained on providing simple agronomic practices. The pre-1982 bias in favor of the simpler messages and maize has continued. The data do not reveal any significant correction of this bias in subsequent years. The evolution in the levels of awareness and adoption also suggests that the underlying dynamics of the diffusion process have been little influenced by extension activities. Thus, the less sophisticated messages that are amenable to quick diffusion through informal communication channels or casual contact, and messages that have been known for a long time, have continued to spread. The less well known, and the more sophisticated, have lagged behind, with low levels of awareness and adoption. While the focus on simpler messages and the primary subsistence crop, maize, may have been justified in the early years, the benefits to such efforts are clearly bounded. For continued increases in productivity, a graduation to more sophisticated practices would be the natural path of evolution in an effort to intensify agricultural production. Such a transition, however, is not evident from the current data, either in terms of the awareness or the adoption of the more complex practices. This could be because of either a poor focus of extension efforts or a lack of expertise to deliver the more complex messages, or perhaps both. The continued emphasis on simple practices and maize, despite seemingly high levels of awareness of these practices before the project period, on one hand, and the lack of association of post-1982 extension with adoption rates, on the other, suggest that the potential impact of NEP-I and II on agricultural productivity may be limited. This is apparent at least for NEP-II from a comparison with the results from the ATD survey of the same population, which shows that adoption rates have remained unchanged since 1990. This in turn raises concerns about the cost-effectiveness of the T&V approach as applied in Kenya, particularly given its considerable cost. With the major share of the system's cost stemming from its focus on face-to-face extension, which is best suited to deliver more sophisticated and context-specific advice, the preoccupation under both NEP-I and NEP-II on simple messages clearly indicates that the design features of the projects were not fully exploited in Kenya. This is particularly noteworthy for NEP-II, at the start of which it was clear (from the ATD study) that most farmers were aware of, and a large proportion had even adopted, the simpler messages for maize. The economic justification for the investment in extension under NEP-I and II-whether the returns to extension justify the costs incurred-requires an estimation of the actual impact of extension on agricultural productivity. That exercise is the subject of the next section. However, the findings from this section indicate that that the potential impact of the T&V system as implemented in Kenya since 1982 is not likely to have been significantly greater than that of the extension system it replaced. 6. Results I: Farmer Efficiency and Productivity Change The impact of extension is most directly measured by relating it to farm productivity. Changes in productivity over time can occur as a result of improved efficiency or due to technical change. Productive efficiency is a measure of the farmers' level of skill and knowledge, often termed managerial skills, in producing the most out of a given set of production inputs. Or alternatively, in producing a given level of output with the minimum amount of inputs. Technical change is a result of technological advance, e.g., by improvements in the quality of inputs. Extension has an 21 important role to play in both respects: the former is achieved by imparting knowledge and advice on best practices suitable to the local circumstances to improve farmers skills, the latter by disseminating information on the latest technological advances. This section investigates the level of efficiency in crop production of the sample farmers at two points in time using Data Envelopment Analysis. DEA is a non-parametric method that allows a ranking of households by their level of efficiency. Also measured is the change in farm-level productivity between 1982 and 1997 using the Malmquist index, again using the DEA method. A detailed description of the technical aspects of the analysis, data considerations, the assumptions maintained, and the results are given in Working paper 4. The analysis uses the 1982 RHBS survey data to obtain the pre-project situation, and the 1997 OED survey data for the current situation. The two surveys have 285 households in common, which are used to calculate the Malmquist indices of productivity, efficiency, and technical change. The data for 1982 are limited, but it is possible to obtain some input-based technical efficiency measures. The data for 1997 are more complete, and allow estimating overall cost efficiency and its components, the technical, scale, and allocative measures, to help identify the source of inefficiency." What is measured is relative efficiency, that is, each observation is measured against the best practice, or "production frontier," generated from within the sample. Thus, all observations are ranked against the most efficient farmer in the sample. Findings Relative efficiency. The average farm was operating at a very low level of technical efficiency in 1982 (31%), and while there has been some improvement, the relative efficiency is still quite low in 1997 (45%). The scale of farmers' operations has improved, but a majority still operate at a sub-optimal scale; statistical tests fail to reject the non-decreasing (increasing) returns to scale. A comparison of cost or allocative efficiencies between 1982 and 1997 is not possible for lack of price data for 1982. For 1997, the results show that the average level of economic (cost) efficiency is very low (15%), much lower than technical efficiency. Thus, there appear to be significant potential savings for farmers, even with the current level of technology, by simply changing their input mix to one that is more economical given the current market conditions. The above measures, calculated over all regions, are useful in putting the overall picture in perspective. To allow different regions to vary in their productive potential and agro-ecological endowments, the district-specific measures were also calculated for 1997. As expected, the average level of technical efficiency increases (with an average efficiency level of 69%), but the economic efficiency is still very low (30%). The low level of cost efficiency implies that allocative efficiency of households is quite low, which in turn indicates that the farmers are not using economically optimal levels of inputs. A statistical analysis using the 1997 district-specific efficiency measures, fails to reveal a statistically significant relationship with any of the efficiency measures (cost, technical, or allocative) and the supply of extension services. In these tests, the cluster average technical efficiency for 1982 is used to control for the regional effects for each location, and extension supply is measured as a weighted average of lagged extension staff-farms ratios. Qualitatively, extension has a small positive coefficient in the cost and technical efficiency relationships, but a 41. Technical efficiency is a measures the physical productivity aspects, that is relating physical inputs to output; scale efficiency measures the deviation of each farm from the optimal size of operation, and allocative efficiency measures the deviation from the optimal of the input mix given the current market conditions as reflected by the current prices. For 1982, input price data are not available and hence only the technical and scale measures are calculated. 22 negative coefficient for allocative efficiency. The 1982 efficiency level is positive and significant for cost and technical measures, and is positive but weakly significant for allocative efficiency. Although there is no clear-cut rationale for including district-specific effects since efficiency measures are calculated by district, the consequence is that the effect of the supply of extension on technical efficiency is still low and positive (0.056), but is now significant at the 10% level. Overall, the results do not change much, instead they reinforce hints of mild effects of extension on technical efficiency; cost and allocative efficiency results do not change. To test for the effectiveness of alternative extension methodologies, indicators for the normal meeting place and the frequency of meetings were tested in a separate set of regressions. The frequency of meetings has no influence on efficiency. Those who meet extension agents at cooperative society meetings have a weakly significant but quantitatively large effect on both cost and technical efficiencies.42 It might also be expected that contact farmers would be more efficient. However, the current data do not show this for any measure of efficiency. Among other variables, farm size has a strong negative effect on cost and technical efficiency, i.e., smaller farmers are more efficient. Distance to markets has a significant negative effect on allocative efficiency, no effect on overall cost efficiency and a positive effect on technical efficiency. This suggests that farmers farther from markets may be specializing in specific crops and hence may be more efficient; those close to markets may be diversifying, and while they gain in allocative efficiency, they compromise technical efficiency. As may also be expected, agro- ecological variables affect all measures of efficiency (in varying ways). Larger families and land fragmentation have a negative influence on cost and technical efficiency, and farmer age has a weak negative effect on technical and cost efficiency. Social capital has a weakly positive effect on allocative efficiency, while distance to dirt roads is weakly negatively significant for both cost and allocative efficiency. Households whose heads have attained higher levels of education also have a lower cost efficiency, but the result is only weakly significant. Productivity change. Relative measures for individual years do not indicate how efficiency or productivity has changed over time. These changes are measured using the Malmquist index for productivity change, which is also decomposed into indices measuring technical and efficiency changes. The indices are calculated by district to control for regional effects in the production and economic environment. The results show that, on average, productivity increased by 28% over the period 1982-97. This increase was largely due to positive technical change, which raised productivity by about 56%. The technical improvement, however, was moderated by a decline in efficiency of, on average, 31%, relative to what existed in 1982. By district, the more productive districts have experienced productivity declines on net (Muranga, Kericho, and Trans Nzoia), while the others have experienced substantial gains (with the largest gains in Machakos and Kisumu, followed by Bungoma and Taita Taveta). Most districts show technical progress, but declines in efficiency. The 42. Weak significance refers to significance at the 10% level. 43. Since only 285 observations are used, some tests were done to check for potential selection bias for the retained observations (a result of inability to contact all of the households in the 1997 survey locations). A simple t-test of the 1982 level of relative efficiency shows that the average efficiency of retained observations is slightly lower than that of the rest of the 1982 sample, and significant at the 5% level. However, once agro-ecological factors are controlled for, in a Probit regression, the 1982 efficiency level difference is no longer significant at the 5% level. 23 exceptions are Kericho, with no technical change but a decline in efficiency, and Trans Nzoia, with significant technical regression but a modest gain in efficiency.' The measures reveal some unexpected trends. Figure 7 plots the linear trends in the cluster-level averages of the three Malmquist indices."5 The data are sorted (in ascending order) by the 1982 cluster level average relative efficiency. The trends show that clusters that had high average efficiency levels in 1982, mostly in the more productive areas, have gained less in terms of total productivity, and some may have even have regressed.46 The trend in efficiency is similar, but shows relatively smaller changes. The technical change trend is much flatter, but again negatively correlated with the 1982 level of efficiency. Without any major technological advances, these results show a convergence across regions toward homogeneity in the level of productivity. Figure 7. Productivity Change 1982-1997 Trends in Cluster Averages of District-Specific Scores 6 3 5 2.5 0 4 2 ~1.5 L -f ft ft -f - - -L- - 1 ft_0.5 &9 0 0,R Clusters sorted by average 1982 efficiency scores 1982 Staff/Farm ratio f m 1997 Staff/Farm ratio Malmquist Index M " Efficiency Change Technical Change Source: Results of DEA analysis using OED Survey data How does this relate to extension? Also presented in the graph are the trends of the staff-farm ratios in 1982 and 1997. Other ratios are not shown since the trend for all is similar, i.e., positively correlated with the 1982 level of efficiency. In fact, the 1990 trend is steeper than that of 1982, indicating that the allocation of front-line staff during NEP-I generally favored the more productive regions. During NEP-II, there has been a freeze on recruitment. The decline in the slope of the 1997 trend may thus reflect natural attrition in the front-line workforce. As is clear, productivity change is inversely correlated with the allocation of extension staff. 44. A drawback of DEA is its sensitivity to measurement errors. To minimize these, the analysis used only observations with positive outputs, and for all variables, observations in the top and bottom 1% of distribution of intensities (output or input per unit of area) were eliminated; the exception to this rule was observations with zero non-labor cash inputs. 45. Malmquist indices are calculated in a way that scores below one represent a positive change or gain, while scores above one represent regress. To make the presentation more transparent, the graphs depict the inverse, that is, scores greater than one represent gains in productivity or efficiency. 46. These trends are consistent with farmer complaints in the high-potential districts, such as Trans Nzoia, that they are obtaining lower yields with the same or more inputs than before. 24 Conclusions The conclusions from this analysis, especially as summarized in Figure 7, are striking. It shows that there has been little change in areas that were more productive in 1982, while the other regions have been catching up. This suggests that the more productive areas may have reached an upper bound, and with little new technology forthcoming to substantially raise the production frontier, their productivity has stagnated. However, extension effort has continually been focused on these areas. At the same time, a general lack of improvement in efficiency has meant that even in districts that have experienced technical progress, the overall potential for productivity gains has been compromised. These results, combined with the fact that the overall level of efficiency (as measured by cost- efficiency) is still very low and the high degree of variation in the level of efficiency across farmers, suggest that despite room for improvement, extension resources do not appear to have been efficiently utilized. While extension may have contributed to the growth in the less productive areas, the overall effectiveness of extension appears to have been limited. The minor differences in the effect of extension, albeit statistically insignificant, between the cost, technical, and allocative efficiencies suggests that extension has generally concentrated on disseminating technical messages rather than help farmers optimize on resource use or tailor their messages to the prevailing economic environment. The lesson emerging is that the extension service could have been allocated more efficiently. By properly assessing the potential for technical gains (especially in the main maize areas that have already benefited substantially from past research and extension efforts) a greater deployment of extension staff in less productive areas may have been more cost-effective. Considering the significant resources needed to sustain the current system, and the extremely tight budget constraints on the Ministry of Agriculture (as for other Kenyan ministries), it is apparent that much fewer resources could have been used to achieve the same results. The returns to such investment in extension could conceivably have been much higher. 7. Results II: Production Effects of Extension To determine the returns to the investment in extension, its impact on agricultural production needs to be estimated. The previous section shows that the allocation of extension is biased in favor of more efficient and, hence likely more productive, areas. This bias makes the measurement of the impact of extension difficult in a cross-sectional framework, that is using data from one point in time. This problem has been demonstrated in the context of the 1990 ATD evaluation of the impact of extension in Kenya. A reconsideration of the results from the ATD study, including the technical details of the difficulty in interpreting its results, is presented in Working Paper 3.47 The main finding is that the high returns to T&V extension in Kenya estimated by the ATD study are highly sensitive to regional effects. At the same time, correcting for inadvertent data errors makes the results less robust. The sensitivity of the results precludes any judgment about the returns to extension being positive. To overcome the methodological limitations imposed by a cross-sectional framework, this evaluation uses a more robust method. For this, the 1982 RHBS data and the 1990 ATD data are combined with a fresh survey of the same households to develop a panel data set. The technical details of the analysis and results are presented in Working Paper 5. The primary objective of the 47. The working paper has been recently published as World Bank Policy Research Working Paper, No. 2098. 25 analysis is to identify the impact of extension on crop production by appropriately controlling for as many unobserved factors as possible. The factors of primary concern are the unobserved natural productivity effects and other inherent regional socio-economic or agro-ecological effects. A fixed-effects approach. The problem with data from a single cross-sectional specification is confirmed using the 1997 data. Statistical tests show that while the marginal effects of variable production inputs on farm production are stable to the inclusion of alternative regional and agro- ecological indicators, the coefficient on the extension variable is sensitive. To overcome this problem, the panel nature of the data is exploited.48 A difference model allows controlling for the unobservable regional and agro-ecological factors. Using this specification, a separate model is estimated for each of the three 2-year panels. In addition to the variable production inputs, household characteristics are included in differenced form since these have also changed over time. Varying weather conditions are controlled for by including the farmers' reported performance of crops (normal or poor relative to good). These indicators are available for the 1990 and 1997 data, but not for the 1982 data. The remaining complication is the extension supply variable. The data on staff-farm ratios, used to measure the supply of extension services in each location, go back only to 1982. Thus, for 1997 and 1990 it is possible to model lagged impact of extension, using weighted lags going back seven years each.49 For 1982, only the single-year measure can be used. While it would be desirable to have data on the previous years' supply of extension, it is assumed that past extension efforts are embodied in the 1982 level of production. The post-1982 changes in the supply of extension help identify the impact of the new system of extension. In addition, the 1982 staff- farm ratio provides additional control for the base level of extension supply. What the difference model measures then is the change in productivity that can be attributed to changes in extension after 1982.0 Extension supply can thus be modeled either as a difference of the cumulative extension supply for the later year (1990 or 1997) and the 1982 supply, or by allowing the coefficient to vary over time by including both variables independently. Findings The results from previous section point to the importance of distinguishing between the program effect, or more precisely the efficiency in the allocation of extension resources, and the direct extension effect. Given that the allocation of extension staff is, and has historically been, biased in favor of the more productive areas, it is necessary to control for the initial conditions to be able to identify the impact of extension, as it is likely that the growth in agricultural production has been uneven across different areas. That this has been the case in Kenya is demonstrated by the results in the previous section on productivity change. To control for the effect of initial conditions, a more flexible approach is used than a simple difference model. Extending the analytical model (see Working Paper 5), it is possible derive an empirical model that allows a distinction between program efficiency, or the "program effect", and the impact of extension, or the "extension effect". To proxy for the initial conditions, the 48. Between the three surveys, because of attrition and incomplete data for some variables, the number of observations common to 1982 and 1990 data are 306; 216 are common to the 1990 and 1997 data; and 258 to the 1982 and 1990. 49. Alternative lag structures and lengths were tried, but the qualitative results do not change significantly. 50. To the extent there are any carryover effects from the previous system, the impact of early NEP-I years is likely to be over-estimated. However, this is not considered a major limitation. 26 average cluster level yields for the base year are used." Applying the model to the three panels confirms that the allocation of resources has been inefficient. The base year yields have a significantly negative coefficient. The extension effect, however, is not significant in either the production function or the reduced-form supply function specifications for any of the panels.52 To confirm the hypothesis of the confounding effect of initial conditions on the impact of the supply of extension services, the simple difference model for all specifications yields a result similar to that found in the previous section. That is, in the pure difference specification, productivity change is negatively (and significantly) correlated with extension variable. As noted earlier, however, a naive interpretation of this result is misleading as it suggests that extension has a negative impact on production. Conclusions The main finding of this analysis is that the allocation of extension resources in Kenya has been inefficient. This confirms the findings of the previous section, which also suggest that the resources have been poorly targeted. At the same time, however, once the initial conditions are controlled for, a statistically significant impact of extension cannot be established. It thus appears that the less productive farmers and areas have been catching up as new technology reaches them. And while it is likely that extension has played a role in extending these technologies, this cannot be firmly established with the current data. A more rational allocation of resources would likely have achieved the same results in a more cost-effective manner. The lesson emerging is the need to build a flexible and responsive system. With little new technology forthcoming, e.g., for maize, it is not economical to maintain a high level of extension presence. Instead, reaching new areas or previously uncovered farmers would have a greater marginal impact, both on production and on poverty. What is needed perhaps is a "smart" system that flexibly responds to information disequilibria, targeting existing or emerging gaps between average and best practices. At other times, it would be more efficient to keep a leaner presence to maintain local equilibrium, rather than provide a blanket coverage of all regions at all times. 8. Client Focus: Farmer's Valuation of Extension Benefits The benefit of a public service to the ultimate beneficiaries is a critical issue for policy. If the "beneficiaries" were in fact paying "clients," the value of the service provided would simply be its market price. For most public services, however, there is no market. Traditionally, extension advice has been provided free because of its nature as a public good (low "excludability" and "rivalry"), with substantial positive externalities. With limited resources, however, the issue of the efficient allocation across a number of possible public goods provision remains, and hence the need to measure the benefits of the service provided. A measure of benefits would also allow considering cost recovery measures. Even if only partial, cost recovery has several benefits: it provides appropriate incentives (and hence accountability and client responsiveness); it brings 51. For each observation, the average was taken over all other observations in the cluster to avoid spurious statistical association since the household production level enters the dependent variable calculation. 52. Note that district dummy variables are included, in conjunction with distances to market and roads, to proxy for local prices, which are not available for the 1982 and 1990 data. 27 budgetary respite; and promotes pluralism, by allowing alternative providers, particularly private ones, to enter the "market." In the context of extension and poor farmers, some pertinent issues are their demand for advice, their willingness to pay for such advice, and their ability to afford the payments. Theoretically, the upper limit an individual would be willing to pay would be the maximum private net benefit derived from the service. That benefit can estimated either directly or indirectly. One indirect method is to estimate benefit from the impact of the service on the productivity of farmers, as attempted in the preceding section. This method, however, presumes that the service is being delivered, and delivered in a manner that is efficient and effective. It also does not reveal whether the farmer would be willing to pay for the service. A direct method is the contingent valuation method (CVM), which elicits from the farmers their willing to pay for the service and gives some idea of their perceived benefits from the service. The approach used in this evaluation is briefly discussed in Annex H. A more detailed discussion of the survey design, an important part of CVM, the tests and controls to check of consistency and reliability of farmer responses, and the detailed descriptive and analytical results are presented in Working Paper 6." The key findings are summarized here. Findings Desired frequency of visits. Some farmers (4%) indicated they do not want any extension advice, and some (another 4%) do not want the current service to continue. For the remainder, the median number of desired visits is three per year, with a modal value of two.4 Over two-thirds of the farmers want less than one visit every three months. Even among contact farmers, almost one-half want to meet the extension agent no more than once every three months. These responses suggest that the biweekly or even monthly visit norm under NEP-I and II exceeds what farmers want. Willingness to pay. Overall, a small proportion (9%) of those who would like to receive advice (including 12% of CFs) are not willing to pay for it. Over one-half of the farmers are willing to pay individually, while the rest prefer to pay in a group." The mean willingness to pay (WTP) for individual contributors is Ksh 67 per visit, and for group contributors it is Ksh 51 per visit; the mean WTP is Ksh. 60. At the time of the survey, the daily wage rate for agricultural labor was Ksh 60. The farmers who are not willing to pay (at all or as individuals), most frequently cited lack of funds as the reason. The total annual WTP is on average Ksh. 346, with a median of Ksh 160. The range is quite wide, from 0 to 8,640, but the mean WTP is significantly different from 0 (standard error of 32). The inter-quartile range, however, is tighter between 60 and 360. Factors influencing willingness to pay. A systematic variation in the WTP by socio-economic or agro-ecological characteristics, or with alternative extension methodologies currently in use is important for policy makers to more effectively and efficiently target future services. It is also important in determining the perceived benefit from the services as available to different farmers. 53. The survey followed the professionally accepted guidelines for CVM questioning: interviews were conducted in person; the more conservative willingness to pay (rather than willingness to accept) was elicited; sample and item non- response were quite low; open-ended as well as double-bounded referendum questions were asked; the program was well-understood by farmers; and opportunities were given to farmers to explain their responses (See Arrow, et al.) 54. Eleven respondents (under 2% of the sample) mentioned "on-demand" in response to the desired frequency of visits. These observations have been dropped for the rest of the analysis. 55. Farmers were also asked the mode of payment they preferred, and a majority chose cash. 28 The influence of existing extension services on the WTP is viewed from three aspects. One is the current supply of extension services. This has a small positive effect, but does not attain statistical significance in any specification tried. Next, to test the influence of alternative methodologies, variables indicating the households' normal meeting place with extension were tested. Households meeting extension agents either on their own fields or in a baraza do not have a significantly higher WTP than others (i.e., relative to those who do not normally meet extension or meet very infrequently). Those who normally meet at a cooperative society have a negative and significant effect. Finally, a variable indicating whether the farmer is a contact farmer was tested, but it failed to attain significance. Listening to radio programs has a strong positive and highly significant influence on the WTP. This could be explained by the fact that information dissemination over the radio encourages farmers to seek additional or more detailed information-enough to convince them that it would be worthwhile to pay to get the additional information. It may also reflect that fact that they may not currently have access to such additional information to follow up on what they hear on the radio. The results also show that the WTP is significantly higher for those willing to pay in a group. Both gender and education make a difference: female headed households and households with heads with primary or, especially, higher-level education have a significantly higher WTP. Other variables do not appear to influence the WTP. Social capital variables show weak but mixed effects. Household membership of groups (farmer or spouse) increases the WTP. However, a higher incidence of groups in a location has a generally negative effect. The latter perhaps reflects a greater access to information, while the former probably reflects the benefits of collective actions. Among infrastructural variables, only access to dirt roads had an effect on WTP (households living farther from a dirt road are willing to pay more), although the effect is weak. Household living in lower- potential zones have a lower WTP, as do those living on hills and undulating terrain. Conclusions The findings reveal that a significant proportion of farmers would like to receive extension services and are willing to pay for them. The perceived benefit, as reflected in the total amount that farmers are willing to pay, however, is well below what the government currently spends per farm on extension services. Also, the frequency of visits desired is much lower than was presumed in the NEP-I and II design. An econometric analysis suggests that the WTP is not related in a systematic way to the level or methods of extension currently in use. This suggests that the farmers' WTP reflects an unmet demand for services. This is also indicated by the fact that the willingness to pay for CFs and for those who have never received advice before are statistically indistinguishable. Another indicator is the desired frequency of visits, which is approximately the same across all categories of farmers. Finally, the strong influence of listening to the radio on the WTP probably reflects a lack of ability to follow up on information that farmers get over the radio. The results have important implications for the design of future extension services. The most important is the implication for cost recovery and the possibility of incorporating an endogenous quality control mechanism in the delivery system. Considering that even those who do not regularly receive extension, or meet infrequently and in public meetings, are willing to pay clearly indicates that farmers value agricultural advice and would be willing to share the cost of receiving such advice. It also reflects the lack of an alternative source of information at this time. At the same time, the uniformity of the level of demand (frequency of visits) and WTP, suggests that it would be more efficient to cover a larger number of farmers, but with lower intensity and higher quality of contact. The large positive influence of radio programs on the WTP suggests that complementary extension approaches should be exploited for potentially significant 29 synergistic effects. It is also possible that radio programs could be used to whet the farmers' search for information, which could then help promote the provision of more specialized private extension or extension for a fee. 9. Conclusions The overall success of the T&V extension system as implemented in Kenya has been disappointing. The current system is unsustainable and the extension approach adopted by NEP-I and II has not proven to be effective. Although the geographical coverage of the system and staff skills have improved as a result of the projects, the outreach of the system is low and the qualitative nature of the interaction between extension agents with farmers is well below what was anticipated. The evaluation reveals that there is an unmet demand for extension services, and the farmers value the access to such advice enough to be willing to pay for it. Despite the substantial scope for improvement, however, the data do not provide evidence of any significant impact of the current extension system on crop productivity. On the contrary, all approaches using the available data indicate that the current institutional arrangements have been ineffective in delivering the much needed services to the vast majority of Kenyan farmers. It is likely that the NEP-I had some beneficial impacts early in its implementation period. These benefits, however, appear to have been short-lived. The available evidence does not indicate any significant impacts even by 1990. The results do show that the allocation of extension resources has been inefficient. Based on the various estimates obtained, a positive rate of return to the current expenditures on extension cannot be established. Further, the amount that farmers are willing to pay is also well below what the government is currently spending per farm to deliver the services. The findings suggest that a more rational allocation of extension resources would been more cost-effective. 30 References Actionaid Kenya, 1997. Kwa Maoni Yetu (From Our Perspective). Report prepared by Actionaid Kenya, Nairobi, for the Operations Evaluations Department, World Bank. Arrow, K., R. Solow, P.R. Portney, E.E. Learner, R. Radner, and H. Shuman, 1993. "Report of the NOAA Panel on Contingent Valuation." Federal Register 58, 1993, pages 4601-4614. Benor, D. and M. Baxter, 1984. Training and Visit Extension. The World Bank, Washington, D.C. Benor, D., J.Q. Harrison, and M. Baxter, 1984. Agricultural Extension: The Training and Visit System. The World Bank, Washington, D.C. Birkhaeuser, D., R.E. Evenson and G. Feder, 1991. 'The Economic Impact of Agricultural Extension: A Review". Economic Development and Cultural Change, 607-650. Bindlish, V., and R. Evenson, 1993. Evaluation of the Performance of T& V Extension in Kenya. World Bank Technical Paper No. 208, Africa Technical Department Series. Bindlish, V., and R. E. Evenson, 1997. "The impact of T&V Extension in Africa: The Experience of Kenya and Burkina Faso". The World Bank Research Observer, 12 (2): 183-202. Bindlish, Vishva, Robert Evenson and Mathurin Gbetibouo, 1993. Evaluation of T& V Extension in Burkina Faso. World Bank Technical Paper No. 226. Africa Technical Department Series. Washington, D.C. Cleaver, Kevin, 1993. A strategy to Develop Agriculture in Sub-Saharan Africa and a Focus for the World Bank. World Bank Technical Paper Number 203. Africa Technical Department Series. Washington, D.C. Feder, G., R. Just and D. Zilbernan, 1985. "Adoption of Agricultural Innovations in Developing Countries: A Survey". Economic Development and Cultural Change, 33, 255-298. Feder, G., Lau, L.J. and Slade, R.H. (1987), "Does Agricultural Extension Pay? The Training and Visit System in Northwest India", American Journal ofAgricultural Economics 69(3),677-86. Feder, G. and Slade, R. (1986), "The Impact of Agricultural Extension: The Training and Visit System in India", World Bank Research Observer 1(2), 139-61. Feder, G., Slade, R. and Lau, L. (1985), The Impact ofAgricultural Extension: The Training and Visit System in Haryana, World Bank Staff Working Paper 756, World Bank, Washington, DC. Feder, Gershon and Dina Umali. 1993. "The Adoption of Agricultural Innovations: A Review". Technological Forecasting and Social Change, Vol 43: 215-239. Feder, G., A. Willett, and W. Zijp. 1999. "Agricultural Extension - Generic Challenges and the Ingredients for Solutions". Mimeo, The World Bank, Washington, D.C. 31 Gerhart, J., 1975. The Difusion of Hybrid Maize in Western Kenya. Abridged by CIMMYT. Centro Internacional de Mejoramiento de Maiz y Trigo, Mexico City. Hussain, Syed Sajidin, Derek Byerlee and Paul W. Heisey, 1994. "Impacts of the training and visit extension system on farmers' knowledge and adoption of technology: Evidence from Pakistan". Agricultural Economics, 10: 39-47. Kandie, E.K., 1997. "Proposals to Improve Effectiveness of the Agricultural Extension Services in Kenya", Agricultural Extension Policy Project, Ministry of Agriculture, Livestock and Marketing, Government of Kenya, and GTZ. MALDM (Ministry of Agriculture, Livestock Development and Marketing), 1997a. Proposal on The National Agricultural and Livestock Extension Programme (NALEP), Second Draft, December. MALDM (Ministry of Agriculture, Livestock Development and Marketing), 1997b. Alternative Approaches to Agricultural Extension: ASIP Preparation Study. Nairobi. MALDM (Ministry of Agriculture, Livestock Development and Marketing), 1997c. Staffing Norms Analysis: Revised Draft Final Report, Volumes I-III Agriculture Sector Management Project II, Government of Kenya. Mukui, J. T. 1994. Kenya: Poverty Profiles. Consultant report prepared for the Office of the Vice-President and Ministry of Planning and National Development, Nairobi, Kenya. Narayan, D. and D. Nyamwaya, 1995. A Participatory Poverty Assessment Study - Kenya. A report prepared for the World Bank, sponsored by the British ODA and UNICEF. A revised version of the report was published as Environment Department Participation Series Paper No. 034, Learning from the Poor: A Participatory Poverty Assessment in Kenya, 1996. Ongaro, W., 1990. "Modern Maize Technology, Yield Variations and Efficiency Differentials: A Case of Small Farms in Western Kenya". Eastern Africa Economic Review, 6(1):11-29. Picciotto, R. and Anderson, J.R. (1997), "Reconsidering agricultural extension", World Bank Research Observer 12(2), 249-59. Purcell, D.L. and Anderson, J.R. (1997), Agricultural Extension and Research: Achievements and Problems in National Systems, World Bank Operations Evaluation Study, World Bank, Washington, DC. Salmen, L. 1999. The Voice of the Farmer in Agricultural Extension. A Review ofBeneficiary Assessments ofAgricultural Extension and An Inquiry into their Potential as a Management Tool. Draft report to AKIS Thematic Team, World Bank. Tiffen, M., R. Purcell, F. Gichuki, C. Gachene and J. Gatheru (1996). National Soil and Water Conservation Programme, Kenya. Sida Evaluation 96/25, Department for Natural Resources and the Environment, Swedish International Development Cooperation Agency, Stockholm. Venkatesan, V. and Kampen, J. (1998), Evolution ofAgricultural Services in Sub-Saharan Africa, World Bank Discussion Paper No. 390, Africa Region Series, World Bank, Washington, DC. 32 World Bank, 1983. "Staff Appraisal Report: Kenya: National Extension Project". Washington, D.C. World Bank, 1990. "Staff Appraisal Report: Kenya: Second National Agricultural Extension Project". Washington, D.C. World Bank, 1992. "Project Completion Report: Kenya: National Extension Project (Cr. 1387- KE)". Washington, D.C. World Bank, 1994. Agricultural Extension: Lessons from Completed Projects. Operations Evaluation Department. Washington, D.C. World Bank, 1996. Performance Audit Report: Kenya: National Extension Project (Credit 1387- KE). Report No. 15652. Operations Evaluation Department. Washington, D.C. World Bank (1997). Rural Development: From Vision to Action, World Bank, Washington, DC. World Bank (1998). Kenya: Country Assistance Strategy. Washington, D.C. 33 Annex A Annex A. Impact Model Food Secu uai Fniends, neighbors,ui] Ln ult ---+ Innovative farmers ................................ ... tiCr..edit..Ri.sk Spillovers Adaptive i Research -+ Public Extension Service Weather and Pests Farmer Organizations, NGOs Private Sector: Input suppliers, processors, consultants ---- Household N - - - - I objectives Output Media: Audio, Video, Print idigenous: Systems ---Field Days-' ......................................................I L abor Prices Demonstrations/ Field trials 1 __rstutueEducation II I I Technology Generation Knowledge Delivery Farm Decision-Making Impact Inputs Activities Output Outcomes Results Research- Institulional Euaension Links Development Access Awarene%s Producmvitv ReconimendAlions Sustainabihr- Coniact Adoption ElTiciency Training Efficacy Distribution Feedback Plurahit) 35 Annex B Annex B. Background Among the several goals often cited, one most agricultural extension services aspire towards is agricultural development (Feder, Willett, and Zijp 1999). The objective of extension services is to sustainably increase agricultural productivity by, among other things, improving the knowledge of farmers about new crops, varieties, inputs, and better husbandry and management practices. The importance of science-based technological advances in raising farm productivity gives agricultural extension a key role to play in the development, and has accordingly been consistently supported by the World Bank in many of its borrowing countries. Over the past two decades, the Bank has invested about US$ 4 billion world wide in extension projects. A large number of these projects used the Training and Visit (T&V) system of management (along the principles expounded in Benor, Harrison and Baxter 1984). In Africa, agricultural extension has been central to the Bank's development strategy in the ipast (Cleaver 1993). The strategy for the new millennium, designed to "focus on a few selected national and thereby systemic programs of high impact," also lists extension as a key area for Bank support for Africa (World Bank 1997). This strategy in the past has relied to a large degree on the T&V system, with national program in over 22 countries designed following its guidelines (Venkatesan and Kampen 1998). The effectiveness of the T&V system of extension, in particular its cost-effectiveness, has been a subject of much debate. The debate has largely centered on the institutional design of T&V and its efficacy relative to alternative mechanisms for delivering extension advice (Picciotto and Anderson 1997). Within the Bank, the debate has been passionate and often emotional. The focus has largely been on conceptual issues, but with little supporting evidence. The limited evidence provided usually comes from assessments made on field visits. As expected, such assessments are generally positive by the supporters of T&V, and negative by its critics. While there is probably some truth in all assessments, the reality is largely obscured by the fact that most field visits are unlikely to be representative or unbiased; either in the manner in which the "data" are obtained or in their interpretation. The facts are also often colored by moral hazard on the part of the local extension staff when dealing with so-called random donor visits (see Box B.1). Despite the intensity of the debate, there have been very few attempts at rigorously measuring the impact T&V, or the lack of it. The notable exceptions, which attempt to estimate the returns to T&V investments, are only three.' All three were conducted by the Bank, in India (Feder and Slade 1985), Kenya (Bindlish and Evenson 1993) and Burkina Faso (Bindlish, Evenson and Gbetibouo 1993). These studies found positive, but varying, degrees of impact in their respective settings. All three use survey data, but they are subject to various limitations imposed by the available data. Other studies have considered the effectiveness of the T&V approach in other settings (e.g., Hussain, Byerlee, and Heisey 1994), and the findings have been generally mixed. There are also a number of studies on the operational aspects of T&V, most of which are critical of the approach, but fail to assess the full impact of the extension system. The Kenyan study, of particular interest here, was part of an effort by the Africa Region to assess the impact of the large amount of development resources going to extension in Africa. The Africa Technical Department undertook the study to evaluate the impact of the agricultural extension 56. There have been a large number of studies attempting to measure the impact of extension in general, including several of T&V systems. However, most studies suffer from conceptual or methodological limitations as noted by Birkhaeuser, Feder and Evenson (1991) and Feder and Umali (1993). 36 Annex B Box B.1. The difficulty with drawing inferences from field visits: some experiences from thiq evaluation Most field visits were arranged through the extension service. Each visit typically entailed an entourage of: mission members, resident mission staff, Ministry (headquarters) representative, provincial and/or district staff, often including the officer in charge, several subject matter specialists, divisional staff, and the local frontline staff. The group usually arrived in a motorcade of 3-5 vehicles. The visits are generally to farmers or groups who extension agents normally work with. Almost always the more successful farmers, or those who have benefited from extension are visited. The farmers visited apparently receive a lot of missions, as most keep an impressive diary which visiting "dignitaries" are obliged to sign. Such experiences are unlikely to be insightful or representative. The moral hazard confronting the field extension staff was revealed during a visit organized by a bilateral donor to view an alternative extension approach. Most previous visits ha< been to districts funded only by NEP, and most discussions on the issues and problems with the extension service as presently organized were positive, pointing to the benefits of the system. Th( visit organized by the bi-lateral agency, however, was to a district receiving both NEP and bi- lateral aid funds. At the start of the visit to the district extension office, however, the extension staff seemed uncomfortable. The problem was that the "mission" had representatives of both funding sources, the local program head of the bi-lateral agency and a Bank staff. The district officials confided with the accompanying local staff (some of whom had previously worked for the government service, but now were with the bi-lateral agency) with their dilemma. The problem was which program to discuss, and more importantly which program to praise since the approaches were very different. The accompanying project staff resolved the issue by noting that the Bank staff was actually from OED and the extension staff should feel free to talk as they felt about NEP. Of course, the rest of the discussion revealed that the program funded by the bi- lateral was the program of choice. projects it had supported in Kenya and Burkina Faso in 1990 (henceforth the ATD study)." As noted, the studies estimated very high returns to extension, especially in the case of Kenya. However, the findings have been controversial because of various limitations, some of which the authors themselves noted (Bindlish and Evenson 1993, p. 29). At a time when many borrower countries were becoming concerned with the high costs of the T&V approach, and there was increasing concern within the Bank about the development effectiveness of its extension portfolio, the high estimated returns were greeted with mixed feelings, and even skepticism in some quarters (World Bank 1994, subsequently published in Purcell and Anderson 1997). Nevertheless, with the evidence based on household survey data," using formal statistical methods, the high estimated returns lent credibility to the benefits of T&V claimed by its supporters. The findings vindicated the Bank's stated policy of using extension as a major plank in the overall rural development strategy for Africa (Cleaver 1993), and justified speeding up the already rapid pace of introducing the T&V system in Africa. At the end of 1997, 22 countries had a national extension program using the T&V system of management, with active Bank projects supporting a total investment of over $700 million. 57. Kenya was the first country in Africa to reform its national extension service along the T&V lines. 58. And the data were collected by an independent agency, the Central Bureau of Statistics and not MALDM 37 Annex C Annex C. The Design of NEP-I and II Beginning in 1982, the Bank introduced the T&V system of management as a pilot in two districts.59 Following the brief pilot, the system was expanded to 30 of Kenya's 41 districts over a three year period, covering all of the high and medium potential areas. NEP-I was designed as a first phase of a longer term institutional development plan of first introducing the T&V system and then to improve its operation overtime. It was primarily an institution building project, and as such it was anticipated that external assistance would be required for a 10-15 year period. The project design followed the standard T&V principles (as noted in the SAR, p 19). The project activities would provide farmers, on a regular and systematic basis, with up-to-date advice on farming practices best suited to their specific conditions. The program was initially restricted to the crop extension service. Extension workers would visit farmers regularly and receive systematic training and technical support from research staff. Each FEW was to divide the farm families in his or her jurisdiction (then anticipated to be between 400 to 800 farmers) into 8 groups. Each group was to be visited every fortnight (4 in one week and four the next). Since it was impossible to visit each group member on any given day, 5-10 contact farmers were to be selected from each group. Thus, about 10% of the farmers would thus be designated contact farmers and the extension worker was to work mainly with the CFs, demonstrating to them the practices that would be followed in the next two weeks, and was to involve as many other farmers as possible in the demonstrations and discussions. The selection process was to entail an inventory of all farmers, with identification of particularly poor farmers whose progress was to be monitored through the M&E system. The initial focus was to be on simple messages, concentrating on a few important crops and the most important aspects of crop production activities. The initial focus was also on low-cost improvements, which the majority of farmers could afford. The implicit assumption being that once there was an increase in productivity and revenues, the farmers could graduate to more costly technological components with the additional income generated. Each FEW was to be backed up and supervised by Technical Officers and Subject Matter Specialists. Every two weeks, the FEWs were to receive a full day of intensive technical training on the messages to be delivered the following fortnight. The Subject Matter Specialists were to upgrade their knowledge and skills through the monthly training workshops to be attended by research scientists. By design, at the district level alone, the ratio of non-FEW staff to FEWs was 1:3.' The project also provided funding for transport to increase staff mobility, allowances for field staff, audio-visual equipment, civil works to build office space where none existed, incremental operating costs, to facilitate the AICs to produce, publish and update extension manuals for all staff. NEP-II sought to further strengthen the extension service and support its expansion to hitherto uncovered areas, including the dryer zones; provide funds for improved transportation of staff; improve the use of mass media and communications; rehabilitate and refurnishFTCs; and promote research-extension linkages by providing funds for transport and equipment for greater participation of research staff in extension training sessions, demonstrations and on-farm trials. The project's goal was to effectively deliver technical messages tailored to the needs of smallholder farmers, especially women, and aimed at increasing yield levels for both staple and export crops. 59. The pilot was focussed on one crop, maize, and was conducted with weekly visits, rather than fortnightly as in the standard T&V prescription. 60. Non-few staff included the DAO, SMS/AO, DEO, TO, Sr. Account Clerk, Clerks and Enumerators and Drivers. 38 Annex D Annex D. Institutional Features Management: The feature that is immediately notable is that not only is the monitoring and evaluation (M&E) non-functional, but even the basic management information is missing. Thus, the data on the number of extension staff, or the capacities in which they are operating, or the even annual expenditure on extension are not readily available. While some of the underlying factors go beyond the extension department, and indeed beyond the MALDM, it is apparent that NEP-I and II had no impact on this important aspect of management of the extension service. The overall management of the projects was also weak. This proved to be particularly important in NEP-II, during which the management of the service rested with a Working Group with insufficient authority and which was unable to coordinate the activities of the three departments (agriculture, livestock development and veterinary). As a result, the management during most of NEP-II was ineffective. This is particularly significant as the SAR for NEP-I prominently states that the T&V system it was introducing was first and foremost a management system. Another shortcoming in institutional strengthening is the continuing lack of a strategic vision or a national policy for agricultural extension. As noted by several observers, this reflects a preoccupation of extension's management with the modalities of the delivery mechanisms without regard to either the planning, policy or management of the extension services" The new paradigm introduced by NEP-I helped increase the level of energy at all levels in the early years with a large influx of operational and development funds, the availability of new vehicles, payment of allowances and significant training. These factors helped raised the morale and motivation of the field staff, and the detailed program of implementation, with a clear chain of command and well-defined bureaucratic assignments of staff, dispensed with the need for policy or planning. With deteriorating financial situation and ineffectiveness of the service, efforts have been made more recently towards developing a national policy and a framework for the future development of extension in Kenya. The progress so far, however, has been driven by multi-faceted pressures, including from the Bank and other donors in the context of the preparation of the Kenya ASIP, as well as through support from other Bank projects. The resulting studies, as MALDM documents, reflect a renewed thinking on extension and provide an in-depth and candid assessment of the current state of affairs in Kenya's extension efforts. The actual progress on developing a national policy, however, has been slow and the policy paper is yet to be finalized. Incentives: Another consequence of the mechanistic implementation of the program design has been inappropriate incentives, both institutional and individual. The "rules of the game" are key elements of institutional development, especially for service delivery, as they determine the incentive structure. While the bureaucratization of the service cannot be attributed to the projects, the hierarchical structure of the T&V design has not helped improve the situation relative to the pre-NEP days. The result is that service is both top heavy and headquarter centric (e.g., excessive supervisory staff with a large concentration in Nairobi). At the individual level, the relative early success, and the relatively free flow of funds led to an intolerance of dissent and neglect of the early emerging problems. The perceived management benefits of the projects, in terms of "monitorability" of project outputs and "accountability" of staff (number of visits, number training sessions, whether or not the FEWs were strictly following their assigned "route maps") and the focus on the delivery of specific and well-defined messages, 61. These observations were made during (individual and group) meetings by current and past extension and non- extension staff of the Ministry, academics, researchers, and donors. 39 Annex D in fact put in place adverse incentives. As in any bureaucratic organization, the accountability of staff was to their supervisors, not to the clients, and the monitoring was of number of visits not the quality of the meetings. The lack of accountability to farmers is brought out in both the household and staff surveys. The household survey results show that proportion of farmers needing advice (i.e., the demand for information) on more complex messages is twice those needing information on simple agronomic practices. On the contrary, the proportion that find simple agronomic messages most applicable (reflecting the effective supply of information) is twice that find the more complex messages applicable. Similarly, the mismatch between what the farmers want and what the extension supplies is reflected in the methodology of extension. According to the FEWs themselves, field visits are the least popular method with the farmers (5%), and demonstrations and field days are most popular, a majority of the FEWs prefer to use field visits. Sustainability: The available estimate on total expenditure on extension does not present a complete picture." How the expenditures have evolved overtime is not known? Nevertheless, based on the most reliable estimates available, from the "printed estimates", for 1996/97 at approximately KP 156 to 177 million (or US$ 54 to 61 million),' out of a total ministry budget of KP 340 billion. By these estimates, about 46% of the Ministry's budget goes to extension activities. A review of the public expenditures in agriculture also estimated that about 60% of the agricultural budget was devoted to extension activities, of which 70% is donor funded. With a total of about 3.44 million farm families (Welfare Monitoring Survey, and Staffing norms study), these estimates suggest that on average, Kenya spent about US$15.11 per farm family during 1996/97 for extension services, or an equivalent of KSh. 876.38." Comparable estimates for 1982 are not available. However, estimates for the districts included in the 1990 ATD study are US$3.92 for 1982 and $4.67 for 1990 (in constant 1991 dollars). These figures compare with the current estimate of $13.29 (in 1991 constant Shillings converted to dollars at the 1991 average exchange rate of Ksh 27.5:$1; or $15.11 in 1997 dollars, at Ksh 58:$1). While the optimal level of expenditure on extension is debatable, the problem facing the MALDM is that the current system is too expensive and financially unsustainable. This is clear from the fact that even towards the end of NEP-II, project funds were financing 90% of the systems operating costs (non-salary). Government budget is insufficient to keep the staff mobile and effective. A vast majority of both the FEWs and SMSs indicate funds for transport and allowances as the a "serious or very serious" constraint to effective delivery of extension services. 62. The problem is that although the estimates for wholly extension projects can be identified, the expenditures on extension as a component in a number of other projects, a number of which are donor funded, are not yet accounted for. 63. The data provided by the Ministry to OED on two occasions, one for the NEP-I audit report and a second time for this evaluation, are inaccurate. A cross check with the appropriation accounts for 1994/95 reveals that the data given are for the expenditures (development and recurrent) are for the whole Ministry and not just for extension. 64. Using a conversion rate of USI to KP2.9 (Ksh 58) for 1997. 65. The exact number depends on how much of the KP21.7 million that is spent on "information management" is attributable to extension activities. 66. An alternative estimate can be approximated from the staffing norms study. Assuming that the proposed short-term staffing norms 1998/99 reflect the current levels of staffing, for the study districts the associated personnel costs, for district level staff alone, are about Ksh 374 per farm family. Adjusting for inflation (15%), assuming these costs represent 80% of the recurrent budget, and doubling it to account for the development budget yields an estimate of Ksh. 794 per farm family. This is close to the estimate from the 96/97 budget, considering that it is an underestimate because of omitted staff expenses for all staff above the district level. 40 Annex D Similarly, training sessions and monthly workshops have also been reduced significantly because of funds shortage. A major reason for the inadequacy of funds is the large numbers of extension staff. Available estimates indicate that at the end of 1996, over about 24% of all ministry staff were engaged in extension. At the field level, the agricultural frontline extension workers (FEW) were approximately 6841, more than double the number in 1982 of 3328. Veterinary and livestock production FEWs were 357 and 547, respectively (comparable estimates for 1982 are not available). This gives an average of about 500 farms per agriculture FEW alone. Comparable numbers for 1982 are not available. The number of technical (non-FEW) agriculture staff are estimated at about 1577, or about 4 FEWs per supporting technical staff. The unsustainable growth in staff through the 80s, which currently takes up about 80% of operating costs, led to a recruitment freeze in 1990. While the increase in staff numbers was result of government policy and presumably unrelated to NEP-I (during which most of the increase occurred), it is noteworthy that the ratios of farm families to FEW and of FEW to technical staff are very close to the original prescription of NEP-I of 500:1 and 3:1, respectively. Apparently, the number of farm families was underestimated at the start of NEP-I, when the farm:FEW ratio was much higher than the reported 500:1. These estimates of staff strength, however, are inconsistent with the data provided by the districts included for this evaluation. The ratio of farms to FEWs is on average about 1100:1. The discrepancy is significant and not easy to explain. It may due to the poor identification and classification of staff by their current assigned duties (which do not necessarily correspond to the assigned job codes), and it probably also reflects and a concentration of staff at headquarters, provincial and district offices, where staff are involved in essentially non-field activities. This is reflected in the staff survey, where a majority of the SMS indicate they have responsibilities other than the areas of their subject expertise. In any event, it is apparent that the current system is overstaffed and expensive. Pluralism: Pluralism was clearly not a characteristic of T&V as implemented in Kenya. NEP-I introduced the contact-farmer, later group, approach, with all of the extension staff time devoted to delivering advice to the contact-farmers. This was achieved at the cost of eliminating all other activities, which were perhaps not effective for all types of extension activities, but were nevertheless efficient alternatives deliver certain types of general information. The NEP-I and II approaches, however, were not very successful in reaching the non-contact farmers for various reasons (a brief prepared by extension staff in one of the districts for a focus group meeting provides some insight into the various reasons, and the effectiveness of the NEP-I and II approaches; this is attached as Annex F), but the institutional energies have remained focused on the T&V approach. A vast number of FEWs and supervisors acknowledge working on projects of other donors, with NGOs, and the private sector. This is mostly for additional incentives (allowances), mobility and training. Most such activities are also differently organized and generally considered by the staff to be more effective. Whether or not the latter is true or a reflection of better working conditions is unknown. However, despite the significant level of ad-hoc activities taking place, there have been no efforts at trying to institutionalize such linkages to make the system more effective or rational. A negative impact of NEP-I on an alternative institutional arrangement was the discontinuation of the soil conservation program (Tiffen, et al. 1996). The program was well established and functioning reasonably well since 1974. However, the collective action required for soil 41 Annex D conservation extension, however, was not amenable to the contact-farmer or even a small group approach. It cannot be demonstrated on small plots, nor can it be reduced to simple messages. The program was therefore discontinued, but was eventually re-introduced in 1988 as a separate branch office supervised and supported by SIDA. The new program introduced the catchment area approach. A more recent incidence of unintended impact of the T&V approach is on livestock extension. NEP-I had concentrated on agriculture, and even though Ministry of Livestock was merged with the Ministry of Agriculture briefly in the early 80s, till 1991 the livestock extension department operated separately, using its own approach, largely funded by other donors. Early attempts at including livestock extension in NEP did not succeed as the livestock department resisted the T&V approach. The re-amalgamation of the Ministries, and the more recent unified approach to extension promoted by NEP-II, are creating significant tensions. In general, livestock extension is also not amenable to delivering messages at pre-determined times, nor does it require frequent visits. Livestock advice is based on solving specific problems and is not seasonal. It also requires significant amount of training to convert specialists to generalists and vice versa, which is resented by the already trained and experienced livestock staff. Training: The benefit of NEP-I and II that is most widely agreed upon is the upgrading of staff skills through training." NEP-I provided substantial training to existing, older staff and untrained staff. The regular training schedules were effective and had a positive impact staff quality. After the initial years, however, funding constraints, strained research-extension links, and lack of new technology reduced the effectiveness of training sessions. The quality of extension staff is also attested to by positive assessment of the competence of extension staff by farmers (as noted in sections 3 and 4) and by NGOs and other donor projects who often use extension staff for their projects (albeit with additional training). Despite these significant efforts, however, a majority of the SMSs feel that the FEWs are inadequately qualified to carry out their responsibilities. A majority also feel there are too many subjects to handle effectively; this sentiment was also voiced by a large number of FEWs as well. Finally, limited funds have restricted training sessions, although the primary reason given by a vast majority of SMSs for reduced frequency of training was "nothing new to say". This reduced frequency of training, however, is reflected in the FEWs demand for increased training sessions, as they see their effectiveness declining as a consequence. Research-extension linkages: A critical element in the high intensity T&V approach is a regular flow of "messages" from research. In the early years the linkages were weak, but sufficient to ensure a supply of simple messages. Over time as funding became tight problems started to emerge. Eventually when KARI was separated from the MALDM, the links were totally severed. They remained weak till in 1993, when renewed efforts under NEP-II led to a memorandiun of understanding between KARI and MALDM was signed to reestablish the linkages. The linkages, however, are still very weak, as revealed by the staff survey. A majority of both the FEWs and SMSs note inadequate and infrequent meetings with researchers and limited participation in field trials. One constraint is the lack of adequate adaptive research to generate new messages. Limited feedback from farmers through extension led several of the regional research centers to conduct their own participatory rural appraisals to identify farmer problems and for targeting their adaptive research. More recently, under the Farming Systems Approach to Research, Extension and Training initiative, under the lead of KARI, the links appear to be improving and some new technologies are emerging. 67. Part of the improvement in staff quality can also be attributed to the rapid increase in staff numbers through the 80s which brought in better trained graduates from the agricultural colleges. 42 Annex E Annex E. Change in the access and quality of services over the past 10-15 years. SERVICES CHANGE IN ACCESS CHANGE IN QUALITY Better Worse No Don't Better Worse No Don't Change Know Change Know Village Shopping Center 14.4 0.2 85.2 0.2 69.8 6.9 23.1 0.2 Public Transport 7.6 1.2 91.2 0.0 41.3 17.9 40.8 0.0 Public Health Dispensary 23.7 1.9 74.3 0.2 30.8 48.2 20.2 0.9 Private Health Centers 62.0 0.6 34.8 2.6 54.2 4.9 26.7 14.2 Piped Water 33.7 18.0 59.4 3.8 11.9 20.4 46.7 21.0 Primary School 14.5 0.3 85.2 0.0 50.6 14.1 33.7 1.6 Secondary School 20.5 1.2 78.3 0.0 51.2 8.7 34.7 5.5 Banking Services 18.7 1.3 77.5 2.5 27.9 3.2 43.2 25.8 Other credit sources 10.6 1.4 78.5 9.5 22.5 8.9 40.6 28.0 Telephone Facility 48.6 2.2 48.0 0.3 30.4 8.9 46.4 14.4 Electricity 51.7 1.3 46.3 0.7 20.8 1.1 58.6 19.4 Tarmac Road 14.7 1.9 83.1 0.3 26.1 30.2 40.7 2.8 Dry Season Road Access 6.5 3.3 90.0 0.2 22.9 50.3 26.5 0.4 Seed Dealer 34.6 1.0 63.9 0.5 51.7 6.2 37.5 4.7 Fertilizer/Chemicals Store 31.5 1.0 66.4 1.0 48.7 3.7 40.2 7.4 Output Market 14.5 1.3 84.0 0.2 53.3 10.6 34.3 1.8 Extension Services 16.6 3.8 75.0 4.6 11.4 39.4 32.9 16.4 Public Veterinary Services 17.2 2.4 77.3 3.1 16.0 27.3 42.8 13.9 Private Veterinary Services 51.7 0.7 32.5 15.1 42.1 5.1 30.3 22.6 43 Annex F Annex F. Briefing (Prepared by one of the field extension staff for a focus-groups meeting to discuss the impact of NEP-I and H in one of the districts.) NEP I * Farmer selection was not done well. In most cases farmers were handpicked by extension staff. * Farmers were not willing to go to the same home every now and then hence Fortnightly Programme did not angur well. * Follower farmers did not in most cases take up messages and replicate in their homes due to non-follow up. * Repetitive messages and lack of clear technology packages led to monotony in the information being passed to the farmers. * Feedback mechanism from farmer to research through extension and vice versa were not effective. * Research programmes in most cases rarely addressed what the farmers needs in the field were, leading to poor linkages. * Individual farmers were left out and felt extension service was only aimed at a few well to do farmers. This is the reason for the permanent attack on extension by administration that extension agents are never seen. * Supervision by divisional and district staff was difficult due to lack of transport and route maps that were unrealistic. * Integration with other programmes such as soil conservation, Home Economics was minimal. * Whereas funding would be adequate for various programmed activities. More than 90% of the funds go towards staff activities and less than 10% go directly to farmers. This would be the reason for low adoption rates. Extension packages were deemed to be not practical for simple farmers. NEP II * Coverage was fairer than contact farmer approach however most areas did not have worthwhile groups. Mostly women groups had a merry-go-round agenda with very little agricultural activity. * Frontline Extension workers meet 4 groups a day and with most groups being women groups, they meet once a month therefore the FEW would either meet the owner of the home (chairlady) alone or very few members of the group. 44 Annex F * There was lack of serious technical packages and that again resulted in repetitive and boring messages. * Most of the demonstration sites were based at the Chairlady's home and little benefit went to the members. * There were no clear packages coming from research except for the normal Agronomic, i.e. practices that have been practiced for a long time by farmers. * Farmers problems are still not yet solved by the existing extension approach e.g. (i) Correct seed varieties for beans, maize, sunflower, etc. (ii) Crop pests and diseases are still being researched on with little assurance of success coming soon. (iii) Credit supply through groups was not administered leading to poor or low adoption rates. * Mobility continued being the most problematic thing at the frontline level leading to very poor coverage at the field level. Good vehicles at the district but no sufficient funds for maintaining them. Poor remuneration for field staff. * Administration system of funds always led to failure of demonstration plots compared to those of farmers due to problems with district treasuries liquidity problems. * Research programmes are funded based on the interest of the donor agency which mostly did not address farmers needs. For example, only one cluster was selected for an entire district of 14 divisions with very varied farmers needs. * Top-down approach has resulted into farmers expecting free things and the approaches being implemented seem to be imposed on the farmer. Farmers view on how extension should be conducted need to be taken into account. 45 Annex G Annex G. Awareness and Adoption of Extension Messages This annex provides a brief summary of the data used for the awareness and adoption analysis, the results of which are discussed in Section 5. Complete details on the OED survey, the methodology used for the analysis and the results are given in Working Paper 2. Stock of messages: As a first step, this study sought to establish an inventory of extension messages and technologies that are available from the research system. This effort yielded limited results. There have been few new technologies recommended during the course of NEP-I and II, and these have generally been in the form of updated varieties, without major changes in practices. The recommendations for most practices have remained essentially the same for the past 15 years. Obtaining the specific extension messages proved to be difficult as the district farm management guidelines have not been updated. In itself, the fact that the technology stock and associated "messages" have remained fairly constant is noteworthy. For the purposes of this evaluation, however, established that recommendations for the 13 main activities for the crops most commonly grown in the study districts do exist. Survey design: The OED survey covered four categories of crops: maize as the main crop, a cash crop, a minor food-crop and a new or promotional crop." The questionnaire was designed to obtain an appreciation of the level of "sophistication" of farmers' knowledge and covered messages on 13 cropping activities." The coverage of the different crop categories was motivated by the fact that the main crop, which was maize for all study districts, has been the target of extension activities since the mid-1960s, and hence may not be very informative about the impact of extension services at the time of the survey. Cash and minor (or non-maize) food crops were covered to obtain a more holistic picture, and the new/promotional crop category was included as a test of the effectiveness of the information dissemination system. Some limited comparative results for 1990 are available from the ATD survey. It should be noted, however, that the ATD data have information on only one crop (mainly maize) for the vast majority of the sample (of 420).70 Thus caution needs to be exercised when comparing the results from the 1990 survey and the current survey. 68. The module of the survey on awareness and adoption of practices was conducted in the second round of the OED survey, at which time 34 households from the original sample of 596 could not be contacted again. Hence, the following analysis is based on the remaining sample of 562 households. 69. Recommendations targeted ranged from simple messages (crop varieties, planting time, spacing, seed rate, weeding time and number of weedings), to practices of intermediate complexity (type of basal and topdress fertilizers), to more complex messages (the quantity of basal and top-dress fertilizers, time of top-dressing, and chemical and cultural pest/disease control measures) 70. The results reported in Bindlish and Evenson (1993) are mostly for maize, with beans as the only second crop for about 25% of the 1990 sample. Coverage of crops other than maize intercrops is negligible. 46 Annex H Annex H. The Contingent Valuation Method (CVM) and its Application The contingent valuation is a method used to elicit individuals' use and non-use values for a variety of public and private goods and services. The technique relies on describing a hypothetical situation to a sample of individuals and asking them to state their willingness to pay (WTP) to avoid a particular change in that situation, or willingness to accept (WTA) compensation for a proposed change. The name of the technique derives from the fact that reported WTP or WTA is contingent upon the scenario described to individuals. In its simplest form, CVM directly asks individuals their WTP to maintain the status quo (i.e., to avoid a change in the provision or quality of the good or service). The method has been used widely in developed countries to estimate non-use values, typically for environmental and public goods. Its applications in developing countries are growing and have been used to value, among other things: improved sanitation services; household water services; surface water quality improvements; tsetse control; and forest protection; and wildlife viewing. This is the first known application of CV to elicit WTP for agricultural extension services. This also the first known application for an impact evaluation of a project. As part of the OED survey, the farmers were first asked if they wanted to continue receiving extension advice, or would like to starting receiving advice (for those who do not currently receive advice). Those who answered in the affirmative, were then asked about their desired number of extension visits per year. The following statement was then read to the farmers: " The cost of providing extension advice (including transport costs, salaries, etc.) have been mostly financed by the government. The lack of funds is a major obstacle in providing extension services. This could lead to irregular visits by the extension workers, and a deterioration in the quality of the service. There is also the possibility that the extension program could be eliminated altogether". This was followed by a question on whether the farmer wanted the extension program to continue. For those who wanted the service to continue, the farmers were reminded of their desired number of annual visits, and asked if they would be willing to pay individually for extension services. Those who refused were instead asked if they would contribute as a group member if such group were to be organized. Then the farmers were asked how much they be willing to pay per visit. The format in which the CV questions were asked was varied between the first and second rounds of the survey to test for various types of biases. In the first round, about half of the farmers were asked double-bounded referendum questions (or closed-ended questions) and the other half an open-ended CV question. In the second round, farmers closed-ended questions in the first round were asked the open-ended question, while the rest were asked a cost-less choice question, i.e., whether they preferred to receive extension services or a market good (sugar or kerosene) worth approximately 100 Ksh. The results reported here are from the pooled responses to the open- ended questions. The detailed description of the survey design and results is given in Working paper 6. 47 Annex I REPUBLIC OF KENYA MINISTRY OF AGRICULTURE Taihone 718870. Fax 720568 Ti- PEPj4ANN r N r AR' Wh= rplying plasac quote ILO HOUSE OFTA/9 CArAMRAL ROAD RAZ N . .. ............................... P.O. Box 3ooz = date NOB 28 June, 1999 Gregory K. Ingram. Manager, Sector and Thematic Evaluations Group Operations Evaluation Department The Worid Bank Washington DC 20433 1JSA FAX 202-5223123 Dear ' RE:. COMMENTS ON IMPACT EVALUATION O. AGRICULTURAL EXTENSION PROJECTS Refer to your letter dated 10th June 1999 on the above subject. Please enclosed herewith final our comments on the document for your consideration and incorporation to the final document Yours AMS. JOS I1A K. TERER PERMANENT SECRETARY Encl. 48 Annex I Ministry of Agriculture's comments on the Impact Evaluation of Agricultural Extension Projects in Kenya. 1.0 Introduction: 1.1 Generally the document has captured most of the areas in evaluaring the Agricultural Extension Projects in Kenya. However, we have the (otlowing comments to make:- 2.0 Specific Comments: 2.1 In 1994, the animal health extension was incorporated in NEP I actvitUes after the Mid-Term Review of August, 1994, however, the contribution of T&V towards livestock activities has not been captured in the document. 2.2 After four joint IDA and GOK monitoring and Evaluation Missions. it was concluded that management of NEP II was the main constrainr towards unification and co-ordination of extension services. Firstly, Managerneor of NEP II was through a Project -Working Group and this may have had a far reaching effect on the project implementation. This was a very important conclusion that the evaluators of efficacy of T&V should have considered. Secondly, the implementation process followed the procedures laid down by the Ministry of Agriculture. Procurement was an important management tool in NEP ( and NE? II. Motor vehicles, motor cycles, bicycles and office equipments and iLLrnitures were procured. The extent to, which it was realized should be evaluated. 2.3 Extension Services provision in NEP I were mainly con:6ned to monthly workshops, monthly training sessions, staff seminars and courses, staff/ farmer educational tours, on-farm demonstrations, farmers field days, agriculturaI shows, farmers courses and seminars, group visits and meetings. These were cxTension methods used by extension agents to create awareness and encourage adoption of technologies by farmers. However, on page 6 of the document in tabic 1, it has been observed that T&V had its strengths and weaknesses that the evaluators should have considered to be able to arrive at a balanced evaluation report about the efficacy of T&V Management of Extension Services. 2.4 In he document enumrtors have used a stzuctured questionaie. Unfortunately no sample of questionnaire is appeded hnce it is diEaut to es=bish the quality of data. Thee is also no indiwadon of use of informal survey methods to verify the results from the formal survev. 2.5 Comparison has been made using data and information of 10-15 years ago There is no indication of any survey done before the start of project implementation, hence the rationale mentioned here does not hold. It is also stared in the document that there was no baseline study done to ascertain pre-project situation . 1 49 Annex I 2.6 The document has concentrated basically on information access/dissemination within the household and no other stakeholders are involved. In addition, the indication that only 47% of respondent say that information is less available now than it was before relates to asidng direct questions or through a questionnaire which may not be conclusive. Other forms of verification should have been explored. 2.7 The document also indicates that key findings of the survey are that there has not been apparent improvement in the quality of delivering of extension services. There are no conclusive indication of the same. The kind of data the evaluators have obtained from the questionnaire can not be conclusive. Other verifiable indicators are required. 2.8 Extension Services covers a wide range of agro-ecological zones. In all places it is not possible to apply contact farmer and follow-up frmers principles of T&V. So the extension agent has to apply other methods to deliver information to the farmers. In ASAL areas they can use extension barazas but it should be noted that these are not administrative barazas. Also in administrative barazas, extension agents are invited to address the public but this should not be construed to mean it is an extension barazas . 2.9 One year of suspension is too short to notice any signifcant change in delivery of extension services as reflected in the document. There were other parallel extension related projects, which may have supported extension service in 1996, when NEP II funds were suspended. The Govermment may have redirected the recurrent funds from other sources towards extension services during that period of suspension. 2.10 Regarding the outreach, first contact with farmers has been used to analyze the efectiveness of an Extension Services. This measure does no say much about the efficacy of T&V. It leaves a lot of room for criticism because the baseline data on pre- T&V situation were not available. This is may therefore be treated as a proxy output indicator. 2.11 The document has largely covered crops at the expense of livestock which is often a major enterprise in some farms, therefore it may not represent a holistic picture of the Extension Services. 2 50 Annex I 2.12 In T&V extension approach, it is assumed that farmers learn about innovation/technology by becoming aware of it, become interested in it and adopt it. Learning occurs by adopting ideas that come from outside. The evaluation has not considered the possibility that farmers are active problem solvers on their own and that they are not passive consumers of technology, but part of its development born out of long experiences in farming. The term "message" here could acquire a more pluralistic meaning if the term "infbrmation" was used instead of message. 2.13 The use of statistical analysis to be able to explain process such as awareness and adoption is not plausible. It is because there are other confounding variable like attitudes and past experiences of farmers that will very much influence farmers probability of becoming aware of technology or information and adopting it. This section is an academic exercise and does not explain actual reality. 214 The Ministry of Agriculture through the Division Extension Service also recognized the constraints of naon-adoption due to irrelevant technology. A linkage was established between extension and KARI with the mandate to address the oroblem of inappropriare technology. Indeed some NEP I fumds were allocated to KARl Regional Research Centers(RRC) for linkage activities. The results may not have been captured during the survey. 4.15 The evaluatuators conducted T&V system as stated in theoretical books and did not consider the modification that this approach had undergone, the complex farming system and socio-economic circumstances of smallholders. Therefore the evaluation is academic. 4.16 Despite the increased farmer/extension contact and significant increase in level of technology awareness as observed in the documents, the evaluators have not considered that adoption rate for these new technologies was also limited to a geater extent by low returns fiom farm produce. 3 51 Annex J Annex J. OED Response to Borrower Comments. The report summarizes the main findings of the OED evaluation, the details of which are contained in six supporting working papers. The evaluation is based on evidence obtained through a broad-ranging household survey and on the information and data available from two preceding surveys in 1990 and 1982, as discussed on page 2 of the report. The evaluation also surveyed front-line extension staff, subject matter specialists, and supervisors in the study districts. As noted on page 2 and elsewhere, that these data were combined with secondary sources of information obtained from the various reports and documents, including several reports of the Ministry of Agriculture (MOA) itself (as noted on page 5). The historical evidence is collated from various studies and published reports from as early as the early 1970s (references in the report and in working papers). In addition, the report also uses the Government of Kenya's participatory poverty assessments (as noted in Section 3) and an independent Beneficiary Assessment conducted by an NGO in full cooperation with the MOA. Responses to specific comments: Para. 2.1 The household and staff surveys and the beneficiary assessments covered all extension activities, including livestock production, animal health, soil conservation, etc. Thus, the analysis in Sections 2, 3, and 4 is not restricted to crop production. In the other sections, dealing specifically with the physical or technical impacts, the analysis is restricted to crop activities. The text in sections 5, 6 and 7 has been modified and footnote 31 added on page 17 to clarify this. The primary reason for focusing on crop related activities is that livestock production was included in NEP-II only after 1994. Thus, for most of the period under study, livestock extension activities were financed and managed under a different system. Also, a preliminary analysis of the awareness and adoption of livestock production practices shows that almost all farmers had heard of and adopted the practices by 1994. A deeper analysis of livestock production activities will be undertaken and communicated separately, although the association with NEP-II is likely to be tenuous. Further, as discussed in Annex D, the limited experience of the livestock extension staff with T&V has not been positive. Para. 2.2 The text has been suitably modified on page 6 and annex D page 38. OED considers procurement a function of project management, but not a management tool. Para. 2.3 In OED methodology, efficacy is assessed by comparing project outcomes with its goals. The findings in Sections 3 and 4 show that the project design was not efficacious. As noted in Section 2 and summarized on page 8, the reasons noted in table 1 are the underlying causes. Para. 2.4 The questionnaire was deliberately omitted from the final report for the sake of brevity and it is available on request. A copy of the survey, however, was sent to the MOA at the time of the survey. Para. 2.5 The comparison to the situation 10-15 years ago is a subjective assessment by the users of the service, the farmers, of the change in the access to and the quality of extension services. As beneficiary perceptions, OED is inclined to respect these views at face value. However, these views are not the only basis for the evaluation in the report. In the absence of appropriate baseline data on various aspects of the extension services, recall data are used where feasible. Formal statistical tests in Working Paper 1 comparing recall estimates with estimates from three independent surveys, for different time periods between 1982 and 1993, on the contact with extension show that recall bias is not likely to be significant. Further comparisons in Working Paper 2 on the awareness and adoption of maize practices also suggest that recall bias is 52 Annex J not likely to be significant. More important, comparable findings from a number of studies noted in Working Paper 1, some as early as 1972, provide a significant degree of confidence in the broad results emerging from the OED survey. As for the technical impact (on farmer efficiency and productivity), the 1982 Rural Household Budget Survey provides the appropriate baseline for NEP-I and the 1990 ATD survey for NEP-II. Para. 2.6 As noted above, the evaluation included a survey of extension staff. The decline in the effectiveness of the system since 1982 and 1990 was reported by most staff, as noted on page 7. An alternative assessment by the beneficiaries is summarized in Annex E on page 42, which queried about 19 separate services in a contextually independent way; there is no compelling reason to believe that the responses for extension services would be biased. In this context, it should be noted that the evaluation strategy (page 3) included beneficiary assessments (BAs). It is unfortunate that the BAs were not carried out, as discussed on page 10. It was agreed that the BAs would be conducted by the MOA with the full support and cooperation of OED (including financial contribution) and the Bank's Kenya country department. However, after initial preparations, the ministry unilaterally decided not to implement the BAs. Para. 2.7 The qualitative aspect of the delivery of services refers to the projects' design, and the text has been modified on pages 16 and 29 to clarify this point. The projects' design anticipated a high intensity and qualitatively different meetings between extension and the contact groups. As discussed in Sections 3 and 4, the outcome on this aspect is unsatisfactory. Para. 2.8 OED agrees with the assessment and the report is consistent in noting that a uniform method should not be used in all locations, as advocated in the T&V design. Footnote 10, on page 6, has been modified to reflect the correct interpretation of barazas. Para. 2.9 The impact of the suspension of funds is discussed in further detail in Working Paper 1. Considering the prescribed extension-contact farmer/group meeting schedule envisaged by T&V, the suspension would be noticeable if the system was working as anticipated. While the possibility of using other funds exists, it does not appear to have been the case, as discussed on page 15. None of the officials met during the evaluation indicated that alternative funds were made available. In any event, if other funds were indeed used, it would indicate that Bank funds were not needed, raising the issue of the additionality of NEP-II. Para. 2.10 OED agrees with the comment and Working Paper 1 makes the same point. Footnote 11 on page 7 has been made more explicit to clarify this point. The analysis, however, does go beyond first contact and covers more meaningful aspects of extension-farmer meetings. Para. 2.11 The response is the same as in 2.1. Para. 2.12 OED fully agrees with the premise of the comment, which is the basis for judging the incrementality attributable to NEP-I and II. It should be reiterated that this is an evaluation of the impact of NEP-I and H; both relied on "messages" being delivered in a time- bound fashion. Para. 2.13 The analysis of awareness and adoption in Working Paper 2 includes a number of social and agro-ecological factors in addition to extension. The statistical results are consistent with the descriptive findings and the focus of NEP-I and II activities, as noted by several senior extension staff and other observers. 53 Annex J Para. 2.14 The report notes improved research-extension linkages and staff training as the beneficial aspects of the projects in Section 2. As inputs into the extension system, the results of these activities would naturally affect the flow and content of the extension activities in the field. These would thus be fully reflected in the impact at the farm level in terms of greater awareness and adoption of practices, and improved productivity. Para. 2.15 Farmer characteristics are included in all aspects of the evaluation. The analysis is, for the most part, general and encompasses alternative methods, so long as they were carried out by "government extension agents." However, as an evaluation of NEP-I and II, the focus has to be on the projects' objectives and design as contained in the Staff Appraisal Reports for 1EP-I and II and as presented in the various documents of MOA noted earlier. Hence, particular attention is paid to the specific aspects of the extension methodology, but the farmers' quantitative and qualitative assessments are generally independent of any particular method. Para. 2.16 The results show increases in awareness and adoption only for a few simple agronomic messages. As noted on page 17, these are also known to have been relatively high even at the start of NEP-I and, especially, NEP-H. The levels for the more complex messages are still very low. It is true that low returns, along with other constraints such as the credit or input availability, are important factors. However, a negligible proportion of farmers indicated low returns as the reason for non-adoption (see Working Paper 2). A number of farmers did note financial constraints, and some noted input market problems. Also, as noted on page 18, alrmost 80% of those who are aware of the recommendations (including the more complex ones) have adopted them. This clearly shows that while other concerns may be valid, the lack of information is a serious constraint. Finally, according to the Economic Survey, 1998 (published by the Government of Kenya), the agricultural output price index for 1997 (1982=100) was 598.9; the total input price index was 520 and the fertilizer price index was 314.3. Thus it is not clear whether adverse market conditions are behind the non-adoption of the practices.