POLICY RESEARCH WORKING PAPER 3 022 Sending Farmers Back to School The Impact of Farmer Field Schools in Indonesia Gershon Feder Rinku Murgai Jaime B. Quizon The World Bank Development Research Group Rural Development April 2003 l POLICY RESEARCH WORKING PAPER 3022 Abstract Feder, Murgai, and Quizon evaluate the impact of the period 1991-99 in Indonesia. Their analysis, farmer field schools, an intensive participatory training employing a modified "difference-in-differences" model, program emphasizing integrated pest management. Their indicates that the program did not have significant effects evaluation focuses on whether participation in the on the performance of graduates and their neighbors. program has improved yields and reduced pesticide use The authors discuss several plausible explanations for among graduates and their neighbors who may have this outcome and suggest recommendations for gained knowledge from graduates through informal improvements. communications. The authors use panel data covering This paper-a product of Rural Development, Development Research Group-is part of a larger effort in the group to analyze the impact of extension and education approaches. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Pauline Kokila, room MC3-510, telephone 202-473-3716, fax 202-522-1151, email address pkokila@worldbank.org. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at gfeder@worldbank.org, rmurgai@worldbank.org, or jquizon@worldbank.org. April 2003. (41 pages) The Policy Research Working Paper Senes disseminates the findings of work in progress to encourage the exchange of ideas about development issues An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the vietv of the World Bank, its Executive Directors, or the countries they represent Produced by the Research Advisory Staff SENDING FARMERS BACK TO SCHOOL: THE IMPACT OF FARMER FIELD SCHOOLS IN INDONESIA Gershon Feder, Rinku Murgai, and Jaime B. Quizon* Keywords: Extension; Farmer Field Schools; Impact Evaluation; Indonesia; Participatory Training *Gershon Feder and Rinku Murgai are Research Manager and Economist, respectively, in the Development Research Group of the World Bank. Jaime Quizon is a Senior Evaluation Officer in the Evaluation Group of the World Bank Institute, World Bank. Email addresses for correspondence are: dfeder(aworldbank.org, rmurgaiaworldbank.or , and iquizon(gDworldbank.org. SENDING FARMERS BACK TO SCHOOL: THE IMPACT OF FARMER FIELD SCHOOLS IN INDONESIA I. Background Agricultural extension and farner education programs are key policy instruments for governments seeking to improve the productivity of agriculture while protecting the environment. Correspondingly, there is great interest in the impact of such public investments and in their financial viability. The experience of extension systems over the past few decades has been mixed. Some studies estimate high rates of return to the investment in extension (Birkhaeuser, Evenson, and Feder), or to farmer education (Jamison and Lau; Lockheed, Jamison, and Lau). Yet, many observers document poor performance in the operation of extension and informal education systems, due to bureaucratic inefficiency, deficient program design, and some generic weaknesses inherent in publicly-operated, staff-intensive, information delivery systems (Feder, Willett, and Zijp). One deficiency highlighted by researchers and practitioners is the tendency of many public officers dealing with the transmission of knowledge to conduct their assignmnent in a "top-down" manner. Often, the information that is conveyed is presented as a technological package comprised of recommended practices. This is perceived as a less effective method for improving knowledge as compared to more participatory approaches (Axxin; Braun, Thiele, and Fernandez). In recent years, a number of development agencies, including the World Bank, have promoted farmer field schools (FFS) as a more effective approach to extend science-based knowledge and practices to farmers. Though pioneered and first promoted by the Food and Agriculture Organization (FAO) as a practical way of diffusing knowledge-intensive integrated pest management (IPM) concepts and practices for East Asian rice-based systems (Kenmore 2 1991; van de Fliert), the FFS has since evolved to include a much broader coverage of other farm-relevant topics in its curriculum. The FFS training program utilizes participatory methods "to help farmers develop their analytical skills, critical thinking, and creativity, and help them learn to make better decisions" (Kemnore 1997). Such an approach, in which the trainer is more of a facilitator, rather than an instructor, reflects a paradigm shift in extension work (Roling and van de Fliert). As an extension approach, the FFS concept does not require that all farmers attend FFS training. Rather, only a select number of farmers within a village or local farmers' group are trained in these informal schools, which entail weekly meetings in a season-long training course. However, in order to disseminate new knowledge more rapidly within the community, selected farmers receive additional training to become farmer-trainers, and are expected to organize field- school replications within the community, with some support from public sources. Furthermore, all FFS graduates are encouraged to share their knowledge and experiences with other farmers within their local village and community organizations. These farmer-to-farmer diffusion effects are expected to bring about cost-effective knowledge diffusion and financial sustainability, issues that have hampered many public extension systems in both developed and developing countries (Quizon, Feder, and Murgai; Hanson and Just). However, there are risks inherent in the FFS approach. Given the complexity of the information conveyed in the training program, the ability to convey effectively to other farmers complex decision making skills through informal communications may be limited and may curtail the diffusion process. Moreover, the requirement from trainers to abandon earlier top- down approaches to which they may have been accustomed in favor of a facilitation mode is a challenge that poses risks to the effectiveness of the program. 3 Previous studies evaluating the impact of FFS at the farm-level report significant impacts of program participation on farm-level yields and profits, and a decline in pesticide use. For example, studies by Nanta in Thailand and Ekneligoda in Sri Lanka claim that pesticide applications decreased with more IPM knowledge and FFS training, while rice yields increased by as much as 25%. A similar study by Ramaswamy, Shafiquddin, and Latif for Bangladesh notes that FFS-schooled farmers had 8-13% higher rice yields than their non-FFS counterparts. Similarly high impacts on farm profits are also reported by studies conducted in Vietnam, Ghana, Cote d'Ivoire and Burkina Faso (cited in Kenmore 1997). Increases of profits of 40% in Sri Lanka, 30% in Thailand, and 10-25% in China are cited in FAO (2000, p. 18). However, most previous studies have not accounted for econometric problems that arise in estimating program impact when the placement of the program across villages and the selection of farmers for participation in the program are not done at random. These, and other econometric issues discussed later in the paper, are likely to bias estimates of program impact. To our knowledge, the actual farm-level impact of this diffusion concept in an area where FFS operated on a large scale over a reasonable period of time has not been studied rigorously. This paper evaluates the impact of the FFS effort on farm-level outcomes (yields and pesticide use), focusing on Indonesia as a case study.' We employ a modified version of a "difference-in-differences" approach to evaluate program impacts, utilizing a panel household survey that includes information on both field school graduates and other farmers. The empirical strategy allows us to separately identify the direct impact on farmers who participated in the program, as well as the secondary impacts through farmer-to-farmer diffusion from graduates to other community members. It also enables us to control for concurrent exogenous events and interventions that are likely to affect farm performance over time, and to deal with selection 4 biases that arise due to the potential endogeneity of participation by farmers and biases inherent in non-random placement of training programs across communities. The Indonesian experience holds lessons for development agencies and governments in developing countries, who are being encouraged to expand and promote the FFS approach on a wider scale. Presently, there are large national FFS programs in several countries, and pilot or smaller scale programs in many countries of Asia, Africa, and Latin America. Of the various countries, Indonesia has had amongst the longest experiences with the FFS approach and therefore, provides the opportunity to study long-term secondary diffusion impacts that may not yet be manifest in other countries where FFS is relatively recent. The Indonesian FFS program is typical of large scale applications of the FFS concept, and has served as a model for programs elsewhere (Kenmore 1991). Thus insights of the performance of the program in Indonesia are of relevance in other countries considering national programs. Farmer field schools are a significant public expenditure undertaking if implemented on a large scale, as the cost per trained farmer is significant (Fleischer, Waibel, and Walter-Echols; Quizon, Feder, and Murgai) and a rigorous investigation of their impact is important. Furthermore, the effectiveness of reliance on farmer- to-farmer diffusion of complex information is of general interest in contemplating extension approaches. Our empirical results do not indicate that farner field schools in Indonesia have induced significant improvements in yields or reduction in pesticide use by graduates relative to other farmers. Not surprisingly then, secondary diffusion effects on those exposed to graduates are also not significant. While we are unable to pinpoint a specific reason for these results, we advance some plausible arguments that might explain the absence of significant program impacts. 5 The rest of the paper is organized as follows. Section II develops a conceptual framework that underlies the empirical work. Section III describes the FFS program in Indonesia and the data set utilized in the study. In Section IV, we formulate the empirical specification and testable hypotheses, and in Section V present the key findings of the empirical analysis. The last section highlights the conclusions. II. Conceptual Framework FFS training aims to affect farmers' knowledge, which can be interpreted broadly to include the possession of analytical skills, critical thinking, and ability to make better decisions, as well as familiarity with agricultural practices and understanding of interactions within the agricultural ecosystem. Improved knowledge is, in turn, reflected in fanners' cultivation procedures, input decisions, and crop yields. Because knowledge and farmer performance can improve over time even in the absence of FFS training (through communication with other farmers, contact with other sources of information etc.), we model performance as a growth process. As improved knowledge is reflected in cultivation decisions, farm performance indicators such as yields and input use (denoted by Y) may also change over time, at some rate, say a..233 A field school can be expected to improve this performance growth rate for farmers by increasing farmers' knowledge. Improvements in performance are expected for two groups of farmers. First, FFS training directly affects outcomes of field school "graduates" (i.e., those who have participated in the training course), as it improves their ability to acquire and process information (through experimentation, interpretation of experience, etc). Second, the training may also indirectly affect farmers who are "exposed" to FFS graduates. These are farmers who live in villages that have field school graduates, and therefore may have gained knowledge indirectly through 6 farmer-to-farrner diffusion. "Control" farmers, who live in villages where no farmer has received training, are unaffected by the program, and therefore continue to grow at the pre-program growth rate of a. In order to capture these features, consider a simple model in which an FFS program is introduced in some villages in cropping season T*, while other villages are not exposed to the program. Define a period between two points in time To and T1, where To < T* ,B > a. The impact of FFS on performance of graduates and exposed farmers can be measured by (a - a) and (,/ - a), respectively. While in the case of yields the program is expected to induce a higher growth rate, the reverse holds in the case of pesticides where the desired outcome is a reduction in use.4 With the conceptual framework formulated, we turn to a description of the Farmer Field School program and its implementation in Indonesia. The econometric specification and testable hypotheses based on this conceptual framework are developed in Section IV. III. Institntional Setting and Data FFS in Indonesia The typical FFS educates farmer participants on agro-ecosystems analysis, or what can be more generally described as integrated pest and crop management (IPCM), as it includes practical aspects of "... plant health, water management, weather, weed density, disease surveillance, plus observation and collection of insect pests and beneficials" (Indonesia National IPM Program Secretariat). Studies suggest that the information contained in the training program could, if property applied, lead to improved farm performance: As demonstrated in Settle et al., pest damage in rice could be contained using an integrated approach relying on biological and physical control while reducing costly pesticide applications (see also the review by Way and Heong). Pingali, Moya, and Velasco concluded that yield increases were possible if improved 8 "second generation" cultivation practices were followed, and emphasized that training programs are particularly important for promoting such practices. Extensive supervised trials in Vietnam confirmed that the technology has the potential to reduce pesticide costs and improve yields, (Pincus). The FFS approach relies on participatory training methods to convey knowledge to participants so as to make them into "confident pest experts, self-teaching experimenters, and effective trainers of other farmers" (Wiebers). An archetypal FFS entails some 8-12 weeks of hands-on, farmer experimentation and non-formal training during a single crop-growing season. A facilitator (typically government employee, but, in some cases, NGO or specially-trained fanner) leads this village-level program, focusing initially on problem-solving approaches in pest management, but also conveying knowledge pertaining to overall good crop management procedures and practices. Through group interactions, FFS participants sharpen their decision- making abilities, and are empowered by learning leadership, communication, and management skills (van de Fliert). Farmer field schools were introduced in Indonesia in 1989 for disseminating IPM technology among rice growers. Villages and farmer participants were purposively selected for inclusion in the program. Village accessibility and the presence of active farmer groups served as criteria for FFS locations. The more affluent and better informed farmers in these villages were invited to participate in the field schools (van de Fliert, p.157). In 1994, the National IPM Training Project took over these pilot FFS activities and launched a nationwide FFS effort (with funding from the World Bank) for promoting integrated pest management and improved crop cultivation, mainly for rice, but also including some non-rice crops. 9 During the expansion phase, villages in which field schools were introduced were still non-randomly selected'by Ministry of Agriculture officials, based on certain criteria (e.g., rice- growing, presence of a farmer group, cooperation of village leaders, accessibility of the village's field school to potential participants). Certain criteria were used to select the field school participants as well (e.g., rice farmer, ability to read and write, ability to participate in FFS activities and discussions). This selection was done with assistance from village-level officials and farmer group leaders. The program's strategy was not to train all farmers in the community, but rather to rely on the spread of knowledge through farmer-to-farmer diffusion. This included the formal use of farmer-facilitators, or select farmers identified during FFS who were invited to attend a "training-of-trainers" program towards their becoming FFS facilitators themselves. Furthermore, informal diffusion through farmers' communications among themselves was expected to improve the knowledge within the wider village community which did not participate in the field schools. Since their inception in Indonesia, farmer field schools have remained separate activities, implemented in addition to the regular agricultural extension activities undertaken nationwide by the Ministiy of Agriculture. There are some private sector-led extension efforts, including FFS activities run by NGOs and training activities sponsored by input supply companies, but these are very few and of limited coverage. By 1999, more than 500,000 farmers in tens of thousands of villages had been trained in field schools. Over 20,000 farmers had attended "training-of- trainers". Data The data come from a panel survey of Javanese households conducted by the Indonesian Center for Agro-Socioeconomic Research (CASER) in April/May 1991 and again in June 1999. 10 The baseline sample included randomly selected rice-growing villages that had already been covered by the program, as well as villages that were not yet covered by the program but were in areas where the program was planned to be implemented. All villages were visited in the repeat survey, but our analysis focuses only on those villages that had not yet been exposed to an FFS at the time of the baseline survey in 1991. Five of these villages had not yet been served with an FFS program even by the time of the 1999 survey, and the 52 households from these villages are thus a control group. Of the 268 households from villages where a field school had been implemented between 1991 and 1999, only 112 had actually participated in the training (whether administered by officials or by farmer-trainers) while the remaining 156 households had not attended a program, but had been potentially exposed to some of its effects through informal communications with graduates of the program.5 Therefore, our data allow us to separately identify the effects of FFS on graduates and exposed farmers. Another source of variation in program participation comes from the fact that field schools were first introduced in the sample villages at different times between the two surveys. Of the 21 program villages in the sample, 13 were first exposed to field schools within three years after the baseline survey (i.e., by April/May 1993). 5 villages were exposed the following year, and the remaining 3 were exposed after 1995/96. Therefore, our data also allow us to explicitly account for the length of pre- and post-program exposure for graduates and exposed farmers. The 1991 survey collected information on households' farm operations and characteristics for the 1990/91 wet season and on their household attributes, activities and assets. The 1999 resurvey covered the wet season 1998/99 and included most of the questions of the baseline effort. To the extent possible, interviewers avoided changing the language and format 11 of the original 1991 questions in order to preclude any bias that may arise when later comparing responses from both panels. The 1999 survey, however, collected additional data, such as when the household participated in FFS training, more information about the community, and some retrospective details, such as information on when the community was first exposed to FFS, details pertaining to the training attended by FFS participants and follow-up activities. Table I reports selected descriptive statistics for the three household types. Household size and composition is similar across the three categories. However, consistent with non-random farmer selection, there are significant differences in asset ownership across the groups, with graduates owning more land and farm assets (e.g., sprayers), and being more educated than exposed households. In addition, graduate households are significantly more likely to participate in community activities and work as village officers. IV. Empirical Specification and Testable Hypotheses Any empirical analysis assessing the impact of FFS must take account of the special aspects of FFS program implementation. A commonly used approach to measure impact would be to use ordinary least squares (OLS), regressing farm-level outcomes on a variable indicating a farmer's participation in the program and on other variables that could affect farm outcomes. This type of regression is effectively a single difference that compares mean outcomes of program participants to mean outcomes of non-participants in order to measure program impact. However, in the Indonesian context (as in other places where FFS programs have been established), two problems arise with such single difference comparisons based on cross- sectional data. First, the establishment of FFS programs in communities is determined by several factors, some of which are not observed by researchers and may be correlated with farm-level outcomes. For example, if villages where yields were the highest were selected into the program, 12 cross-sectional comparisons of participants and non-participants may show that field schools are associated with higher yields. This is not really the effect of the program but instead comes from the rules that guide the placement of the program. Second, farmers who are selected to participate in FFS are likely to be different from other farmers, in ways that are unobserved to the researcher. For examnple, if more motivated farmers were more likely to be selected, comparing graduates to other farmers in an OLS regression would over-estimate the program's impact on farm-level outcomes. Because of these features - non-random program placement and participant selection - it is well recognized that single difference comparisons of outcome measures (e.g., between participants and non-participants) can give severely biased estimates of impact. A popular approach for addressing this problem is the difference-in-differences or double difference (DD) estimator, obtained by comparing the change in performance before and after the program for a treatment group (i.e., graduates or exposed households) to the change in performance over the same time period for a control group that is unaffected by the program. Since the DD estimator relies on comparing changes in outcomes between participants and non-participants, it is not affected by selection biases that arise from time-invariant household or village unobservables (Glewwe and Jacoby).6 The empirical specification for measuring FFS program impacts implied by the conceptual framework laid out in Section II is a straightforward modification of the DD estimator. Using the same notation as earlier, the equation to be estimated is obtained by taking the natural log of equation (1) and rearranging: (2) A(ln Y) = a(T* -To )+fl DN(Tl-T )+ ±/IDG (Tl-T )+ YAX +65AZ 13 where A(ln Y) = (In Y, - In YO ) and (T* = T,) for control farmers. Equation (2) can be estimated by regressing the change in log of performance indicator Y between 1990/91 and 1998/99 on a set of variables that measure the length of pre and post-program exposure and a set of other household, village, and district controls.7 To see how equation (2) is related to the standard DD estimator, consider a case where all villages are exposed to FFS programs in the same season (i.e., T* is the same across all graduates and exposed farmers in the sample). In such a case, the length of exposure variables are the same across all exposed and graduate farmers, and therefore equation (2) collapses to a regression of the change in ln(Y) on the exposed and graduate dummies (scaled up by the constant (T1-T*)), and the set of other household, village, and district controls. In such a regression, u and ,B measure the change in Y over time for graduates and exposed households relative to the change in Y for control households - these are the typical DD estimators of program impact. Equation (2) modifies the DD estimator in order to account for the fact that FFS programs were introduced at different times across the sample villages. The underlying logic is that at the community level, diffusion of knowledge among individuals takes place over time. The longer the time interval between the time when some community members undertook training and the 1999 survey, the greater the spillover effects on the performance of non- graduates of FFS within the community. Similarly, for field school graduates, the longer the time elapsed since their training, the greater the opportunities to experiment with the new concepts and modes of decision-making, and thus the improvement in their performance is likely to be larger.8 For these reasons, the length of time elapsed since the FFS was implemented is 14 explicitly incorporated by multiplying both the graduate and exposed participation dummies with the length of post-program exposure. Estimation of equation (2) in essence compares the performance growth rates of graduates and exposed farmers in the post-program period to the growth rate experienced by control farmers. It can identify parameters a (the pre-program growth rate in Y for all farmers), ,/ (the growth rate of Y for exposed farmers after FFS was implemented in their community), and ,u (the growth rate in Y for graduates after they undertook training). Given these parameter estimates, the impact of FFS on graduates and exposed farmers is simply (,u-a) and (ft-a), respectively. Note that because program impact is estimated in first differences, any time-invariant unobservable farmer or village characteristics that may have affected farmer selection or program placement are differenced out of the regression, and therefore do not bias estimates of the program impact parameters. Another feature of equation (2) is that program impact on graduates and exposed households is estimated by comparing changes in their performance relative to changes in the performance of control households. Therefore, any variation in performance due to factors that affect all farmers, such as systemic climate changes, concurrent policy and price changes, nationwide non-FFS extension efforts or other secular trends, is also eliminated. In addition, in order to control for differences in extension efforts across regions at the sub-national level, we include district dummies in the estimation. This implies a 'within-district' estimation (i.e., graduates and exposed households are effectively compared to control households within the same district), and therefore the program impact estimates are not confounded with differences in access to extension efforts or other public services across districts. 15 In the empirical work that follows, we focus on two performance indicators: yields and pesticide use. As pointed out earlier, some observers have claimed that improvements in these indicators were in fact obtained by the FFS program in Indonesia, although those assertions were not backed by rigorous studies (Kenmore 1997, FAO 2000). Other inputs, such as labor and fertilizers, are not included as performance indicators, as it is not clear a priori in which direction their use should change as a result of the program. In the case of yields, both graduates and neighbors are expected to accelerate their rate of improvement in yields, although the improvement for neighbors is not expected to be as high as graduates, who receive intensive training. Pesticides are a special area of focus of the FFS, and IPM principles are highlighted in the curriculum. It is thus expected that pesticide use will be reduced after attending the program. In order to test these hypotheses, in the case of the yield regressions, we perform the one- sided tests of (,u>za) and (,&>a) against the null hypothesis of no difference in pre- and post- prograni growth rates. In the case of the pesticide regressions, the signs of the tests are reversed, since the training aims to induce lower pesticide use. V. Empirical Results Table 2 reports the means of the yields and pesticide expenditures in the two survey years, for the three groups of farmers. Several observations are pertinent: First, yields have decreased between 1991 and 1999 for the whole sample (and for each group). This is compatible with an overall drop in Java yields of 5% over the same time period, as reflected in national statistics also presented in Table 2.9 The second observation is that the sample, which was drawn from FFS program areas, has consistently higher yields compared to the Java average, apparently because the FFS program focuses on the relatively more productive, better irrigated rice producing areas. Third, none of the differences in changes over time between the sub-groups in 16 either of the performance indicators are statistically significant.'0 And finally, note that expenditures on pesticides have significantly increased for the sample and within each group. This is consistent with aggregate production figures for the rice pesticide market, which show an increasing trend in production (in termns of both active ingredient weight and value) during the early to mid-nineties (Oudejans). We turn now to multivariate analyses. Table 3 reports estimates of the pre-program and post-program growth rates of yields and pesticides using equation (2)."1 The estimates of pre- program growth rates reflect the trends reported in the descriptive statistics: on average, during the sample period, yields were declining and pesticide use increasing before the introduction of farmer field schools. Parameter estimates of post-program growth rates in yields and pesticide use suggest that these trends continued even after the introduction of FFS, both for graduates and exposed farmers. The question of interest is whether there was a significant acceleration (deceleration) in growth rates of yields (pesticides) for graduates and exposed farmers, relative to control farmers, after the introduction of field schools. This is tested in hypothesis tests A and B. Both tests find no evidence of impact, as there does not appear to be a significant difference in the trend growth rates in the post-program period for either graduates or exposed farmers. Test C which examines differences between graduates and exposed farmers in the post-program period similarly does not find any significant differences. Table 4 checks the robustness of these results by estimating a less restrictive specification than equation (2) that allows for the possibility that graduates have a different pre-program growth rate in outcomes compared to exposed and control farmers. Differences in pre-program growth rates might be expected if those selected to participate in field school training were superior performers not only due to fixed traits (e.g., superior managerial ability), but also due to 17 dynamic traits (e.g., faster learners). Therefore, in Table 4 separate pre-program growth rate parameters are estimated for graduates (a2) versus other farmers (a,). The results in Table 4 indicate clearly that the graduates are not randomly selected to participate in FFS training. Test A, which compares the pre-program growth rates in outcomes for graduates with other farmers indicates that graduates had significantly higher yield growth and lower pesticide use growth than other farmers even before any FFS training had taken place. This implies that analyses not accounting for this selection bias (such as cross-section comparisons, or even the difference-in-differences. formulation underlying Table 3) may confound the inherent advantage of these select individuals with the program effect. The main tests of program impact, comparing changes in the growth rates for exposed farmers (Test B) and graduates (Test C) between the pre-program and post-program periods also do not provide evidence that yields of these farmers have increased or pesticide use declined after exposure to the program. These results do not lend support to the contention that the program has been effective. Table 5 further checks robustness of the basic findings reported in Table 3 by specifying an alternative specification that also accounts for the intensity of FFS activities within a village. The intensity of activity (represented by the number of FFS conducted) within a village is claimed to be an important factor in the extent to which individual farmers apply FFS'principles (FAO 1999, p.21). In order to allow for variation in the degree of impact with the intensity of FFS activity, in Table 5, the length of post-program exposure for exposed households is interacted with the number of FFS conducted in the village.'2 The results support the conclusions from Table 3. The tests of program impact (Tests A and B) do not provide evidence of improvements in yield growth and reductions in pesticide use for either graduates or exposed 18 households. No differences in performance seem to have emerged between graduates and exposed farmers after the program (Test C). The absence of significant impact that can be attributed to the FFS programs gives rise to an obvious question: How can an education effort focusing on matters of direct relevance to fanners' cultivation activities not influence positively their performance? This study cannot provide a direct explanation, but some plausible arguments can be advanced: (i) Even if graduates gain knowledge in their training that could over time be reflected in improved performance, the change is rather small and cannot be detected in the econometric study which necessarily imposes a specific structural form that is only assumed to reflect reality. The rather impressive gains cited by promoters of the program likely exaggerate program impact because of improper attributions, confusion of selection biases with true program effects, and extrapolation of observations from small non-representative pilot situations and samples to the wider population. (ii) It is difficult to achieve significant yield gains when there are systemic factors causing yield declines such as decline in soil fertility, increased plant diseases, and climate trends. Thus, the gains that could be achieved through training may have been small to begin with. (iii) The knowledge gained in the course of FFS training is complex, as "farmers do not master a specific set of contents or 'messages', rather, they master a process of learning that can be applied continuously" (Dilts). The curriculum of an FFS includes agro-ecosystem concepts and decision making principles, that, if conveyed in casual and informal conversations, are not likely to be effectively transmitted. Indeed, a 19 recent study of knowledge diffusion related to FFS activities in the Philippines (Rola, Jamias, and Quizon) found no significant differences in knowledge scores between farmers exposed to FFS graduates and other farmers. If neighbors of graduates do not adopt IPM measures, then the pests from their fields can re-infest the fields of graduates even if the latter apply IPM principles which reduce initially their infestation. This will induce pesticide use even by graduates, and yet the yield may decline due to increased pest resistance. (iv) During the implementation of the World Bank-financed expansion of the FFS program in Indonesia, there were periods when training activities were afflicted by untimely transfers of funds to the field training organizers resulting in training not being fully synchronized with the rice-growing season calendar, irregular supply of training materials, and irregular availability of meals for participants. There was a relatively large rate of farmer absenteeism in school sessions during part of the period. These issues may indicate that the quality of knowledge achieved by graduates was adversely affected. These implementation problems are not uncommon when pilot projects get scaled-up to the national level. (v) As the training program got scaled-up to tens of thousands of villages, a large number of trainers had to be inducted into the program. The average quality of trainers and their commitment to bottom-up approaches rnay have been negatively affected in the move to mass volume. (vi) Pest management, on which the FFS training in Indonesia placed a major emphasis, is but a small component of the rice farmers' cost structure in normal years. Thus the cost of pesticides is less than 10% of the total production costs. Farmers are not 20 placing as much importance on what is to them minor production impact as they should if the full environmental and health costs of pesticides were to be considered. VII. Concluding Observations The empirical results of the preceding section do not provide evidence of significant improvement in economic performnance. And, although the analysis has focused on economic benefits, there is no evidence that the expected environmental and health benefits of the program are significant, because the empirical results do not indicate a program effect on pesticide use. The study shows that it is risky to extrapolate the results of small and early pilots, and the non- rigorous analyses that typically accompanied them. The impact of the FFS training can be much smaller than envisaged, when the program is applied on a large scale, rendering the economic, environmental and health benefits much less attractive than what decision makers were expecting. The discussion of possible reasons for the insignificant program impacts suggests the need to exercise more caution in the design of FFS programs and in deciding on whether to scale them up to the national level. If the impacts on yields and pesticide use are not large, then the viability of the approach depends on maintaining low costs per farmer trained. This is particularly true in the case of rice in Asia, where yields are stagnant or falling due to systemic factors. The current design is expensive, once all overhead costs are accounted for.'3 Furthermore, the economic viability is undermined by the possibility that diffusion through informal communications among farmers is hindered due to the complex nature of the information. There thus seem to be several implications for improving and reorienting the Farmer Field School initiative so as to improve the likelihood of economic viability: 21 (i) There is merit in reviewing the curriculum, and focusing the training on the highest priority topics, while simplifying the presentation of the information. There is a balance between presenting the operational implications of scientific research and the articulation of the science base on which these implications are based.14 The FFS approach may have tilted more than is beneficial into the latter, thus diluting the operational effectiveness of the information. (ii) The simplification of the program's content will make it not only more effective with respect to improving the performance of graduates, but also increase the likelihood and speed of diffusion of new knowledge among other farmers. Diffusion can also be enhanced (and made more cost-effective) by employing mass-media and other dissemination approaches for key aspects of the knowledge (e.g., safety rules regarding the use of chemicals). (iii) The narrowing and prioritizing of the curriculum will also shorten the length of the training, and will thus cut the cost of the program - a key source of concerns regarding fiscal sustainability. With a significant reduction of the per-farmer training cost, a much larger number of farmers can be trained directly. When larger groups of farmers within a village are trained, there will be better prospects of collective action in pursuing coordinated pest management, so that cross-farm infestations do not occur. This will make the training more effective in yielding results. (iv) Simplification of the curriculum, shortening of the training, and increasing the extent of simple decision rules in the training will make the program less dependent on trainer quality and thus more amenable for scaling up. 22 With changes of the types outlined above, there is less risk and higher prospects of success for large scale FFS programs. 23 References Amien, I., P. Rejekiningrum, A. Pramudia, and E. Susanti. "Effects of Inter-annual Climate Variability and Climate Change Variability on Rice Yields in Java, Indonesia." Climate Change Variability and Adaptation in Asia and the Pacific East. L. Erda, W. Bolhofer, S. Huq, S. Lehnart, S. Mukherjee, J. Smith, and J. Wisniewski, eds., pp. 29-39, Dorbecht, the Netherlands: Kluwer Academic Publishers, 1996. Axxin, G. Guide on Alternative Extension Approaches. Rome: Food and Agriculture Organization, 1988. Birkhaeuser, D., R. Evenson, and G. Feder. "The Impact of Agricultural Extension: a Review." Economic Development and Cultural Change 39, no. 3 (April 1991):607-50. Braun, A., G. Thiele, and M. Fernandez. "Forever Field Schools and Local Agricultural Research Committee: Complementary Platforms for Integrated Decision-Making in Sustainable Agriculture." ODI, Agricultural Research and Extension, Network Paper No. 105, July 2002. Cassman, K., D. Olk, and A. Dobermann. "Scientific Evidence of Yield and Productivity Declines in Irrigated Rice Systems of Tropical Asia." International Rice Commission Newsletter 46(1997):7-18. Dilts, R. "Facilitating the Emergence of Local Institutions: Reflections from the Experience of the Community IPM Program in Indonesia." Paper presented at the Asian Productivity Organization Meeting on the "Role of Institutions in Rural Community," Colombo, 21-29 September 1998. Ekneligoda, I.A. "Integrated Pest Management in Asia and the Pacific: Report of APO Study Meeting 17-22 July 1995." Country Paper #10: Sri Lanka, Asian Productivity Organization, Tokyo, 1996. Food and Agriculture Organization (FAO). Technical Assistance to the Integrated Pest Management Training Project: Indonesia. Report No. AG: UTF/INS/072/INS, Rome, 1999. . FAQ Inter-Country Programme for Community IPM in Asia: Phase IV Mid Term Review. Rome, December 2000. Feder, G., R. Just, and D. Zilberman. "Adoption of Agricultural Innovations: A Survey." Economic Development and Cultural Change 33, no.2(January 1985):255-98. Feder, G. and D. Umali. "The Adoption of Agricultural Innovations: A Review." Technological Forecasting and Social Change 43(1993):215-39. Feder, G., A. Willett, and W. Zijp. "Agricultural Extension: Generic Challenges and the Ingredients for Solutions." Knowledge Generation and Technical Change: Institutional Innovation in Agriculture. S. Wolf and D. Zilberman, eds., pp. 313-56, Boston: Kluwer Academic Publishers, 2001. Fleischer, G., H. Waibel, and G. Walter-Echols. "The Costs of Transforming Public Extension Services Towards Participatory Approaches." Paper presented at the CIMMYT Impact Assessment Conference, Costa Rica, February 2002. Glewwe, P., and H. Jacoby. "Recommendations for Collecting Panel Data," Designing Household Survey Questionnaires for Developing Countries: Lessons from 15 Years of the Living Standards Measurement Study. M. Grosh and P. Glewwe, eds., pp. 275-314. Washington DC: World Bank, 2000. Greene, W.H. Econometric Analysis. Prentice Hall, Third Edition, 1997. 25 Hanson, J. and R. Just. "The Potential for Transition to Paid Extension: Some Guiding Economic Principles." American Journal ofAgricultural Economics 83, no.3(August 2001):777-84. Heckman, J.J. and R. Robb, Jr. "Alternative Methods for Evaluating the Impact of Interventions: An Overview." Journal of Econometrics 30, no.1-2(October-November 1985):239-67. Indonesia National IPM Program Secretariat. Farmers as Experts. The Indonesian National IPM Program. Indonesian National IPM Program, Jakarta, Indonesia, 1991. Jamison, D., and L. Lau. Farmer Education and Farm Efficiency. Baltimore, Johns Hopkins University Press, 1982. Kenrore, P.E. Indonesia 's Integrated Pest Management: A Modelfor Asia. Manila, Philippines: FAO Intercountry IPC Rice Program, 1991. _ . "A Perspective on IPM." Center for Information on Low External-Input and Sustainable Agriculture Newsletter No. 13, 1997. Lockheed, M., D. Jamison, and L. Lau. "Farmer Education and Farm Efficiency: A Survey" Economic Development and Cultural Change 29 no. I (October 1980):37-76. Moffitt, R. "Program Evaluation with Nonexperimental Data" Evaluation Review 15(1991):291 - 314. Nanta, P. "Integrated Pest Management in Asia and the Pacific: Report of APO Study Meeting 17-22 July 1995." Country Paper #11: Thailand, Asian Productivity Organization, Tokyo, 1996. Oudejans, J.H.M. "Studies on IPM Policy in South-East Asia: Two Centuries of Plant Protection in Indonesia, Malaysia, and Thailand." Wageningen Agricultural University Papers 99.1, 1999. 26 Pincus, J. "The Impact of Farmer Field Schools on Farmers' Cultivation Practices in their own Fields," Report submitted to the FAO Inter-country Programme for Community IPM in Asia, 2000. Pingali, P., M. Hossain, and R. Gerpacio. Asian Rice Bowls: The Returning Crisis? CAB International in association with the International Rice Research Institute, 1997. Pingali, P., F. Moya, and L. Velasco. "Prospects for Rice Yield Improvement in the Post-Green Revolution Philippines." Philippine Review of Economics and Business 27, no.1 (June 1990):85-106 Quizon, J., G. Feder, and R. Murgai. "Fiscal Sustainability of Agricultural Extension: The Case of the Farmer Field School Approach." Journal on International Agricultural and Extension Education 8(Spring 2001):13-24. Ranaswamy, S., K. Shafiquddin, and M.A. Latif. "A Review of IPM Activities and their Impacts during 1992 Boro Rice Season in Bangladesh." FAO Intercountry Program for IPM in Rice in South and Southeast Asia, GCP/RAS/101/NET, 1992. Rogers, E. Diffusion of Innovations. New York: Free Press, 1983. Rola, A., S. Jamias, and J. Quizon. "Do Farmer Field School Graduates Retain and Share What They Learn?: An Investigation in Iloilo, Philippines." Journal of International Agricultural and Extension Education, in press. Roling, N., and E. van de Fliert. "Transforming Extension for Sustainable Agriculture: The Case of Integrated Pest Management in Rice in Indonesia." Agriculture and Human Values 11 (Spring/Summer 1994):96-108. Settle, W. et. al. "Managing Tropical Rice Pests through Conservation of Generalist Natural Enemies and Alternative Prey." Ecology 77, no.7(October 1996):1975-1988. 27 van de Fliert, E. "Integrated Pest Management: Farmer Field Schools Generate Sustainable Practices. A Case Study in Central Java Evaluating IPM Training." Wageningen Agricultural University, Netherlands, Papers No. 93-3, 1993. van de Fliert, E., J. Pontius, and N. Roling. "Searching for Strategies to Replicate a successful Extension Approach", Journal ofAgricultural Education and Extension 1(1 995):41-63. Way, M. and K. Heong. "The Role of Ecosystem Biodiversity in the Dynamics and Management of Inset Pests of Tropical Irrigated Rice." Bulletin of Entomological Research 84(1994):567-87. Wiebers, Uwe-Carsten. "Integrated Pest Management and Pesticide Regulation in Developing Asia." World Bank Technical Paper Number 211. Asia Technical Department Series, World Bank, Washington DC, 1993. 28 Econometrics Appendix A) Basic difference-in-difference (DID) estimation A commonly used approach to measure the impact of FFS on a farm-level outcome is to regress the outcome on a variable indicating a farmer's participation or exposure to the program and on other variables that could affect farm outcomes. For example, using the notation from section II, log of performance indicator Y for farmer i in villagej and time period t is given by (1') ln(Y,,, )= a + 8DNL,, +,uDGYI + 7XSJt,,5Z1l ±2, J + ±7e 'J where X and Z are vectors of household and village characteristics that may change over time, and DG and DN are graduate and exposed household dummy variables. 2, and 77, are unobserved determinants of Y that are fixed over time within a household and village respectively, and c,, is a residual that represents all time-varying components of the error. In this framework, u and ,B measure the direct impact of FFS on graduates and the secondary effects on exposed households, respectively, under the assumption that the graduate and exposed household dummies are orthogonal to the residual. However, nonrandom program placement across villages and farmer selection into the program are likely to violate the orthogonality assumptions. Nonrandom program placement across villages leads to correlation between DN and i,, while nonrandom farmer selection into the program leads to correlation between DG and A,. These correlations imply that p()IS and ,0Is are biased estimates of the true causal effect of FFS on graduates and exposed households. One solution to the problems of participant selection and program placement bias is to estimate probit models that explain which villages and farmers are selected for participation in FFS, and then use the Heckman lambda approach (Greene, p. 981) to correct for selection bias in 29 equation (1'). However, this model would be only weakly identified in the Indonesia case, as there are no evident instrumental variables that would explain farmer or village participation but would have no direct effect on performance. For example, the criteria typically used for selecting farmers for training in Indonesia - rice farmer with irrigation, owner-operator, educated, belongs to an active farmer group - are also likely to influence farm yields and pesticide use. In the absence of valid instruments, the probit selection correction would be identified only by relying on a strong distributional assumption of joint normality of the error terms in the selection and perforrnance equations in order to identify program impacts; indeed, in the absence of valid instruments, it is impossible to identify program impacts with cross-sectional data without invoking strong distributional assumptions (Heckman and Robb). Weaker identifying assumptions are required when panel data with observations before and after the program, for treatment and control groups, are available. With two periods of data, household and village level unobservables (A, and qj) can be eliminated by estimating the regression in first difference form. Subtracting Y in the first period from the second provides the change in Y, conditional on change in program participation: (2') A(n Y,) =a +f,ADNu, + pDGIJ, + rAX., + SAZJ + AeC, = -a + /DNIJ + JDGuI + YrAXu, +AZ,1 ±ACy where ADG and ADN are dummy variables for graduates and exposed households, respectively (1 if household graduated from or was exposed to FFS between the two survey years, 0 otherwise), while control households are the omitted category.'5 u (/1) measures the change in Y over time for graduates (exposed households) relative to the change in Y for control households. This difference-in-difference (DD) estimator provides an unbiased estimate of the effect of FFS 30 on graduates and exposed households under the identifying assumption that change in Y for graduates, exposed, and control households would have been the same in the absence of the program, even though levels of Y in any given season may differ (Moffitt).16 Notice that the cross-sectional estimator in equation (1') requires that the levels of Y be the same for graduates, exposed, and control households in the absence of the program. The identifying assumption in the first difference estimator is a much weaker condition and will more plausibly hold in the Indonesia context given the methods of program placement and farmer selection outlined in section III. B) DlD) estimation, accounting for variation in length of program exposure The specification in equation (2') can be readily adapted to incorporate the dynamic growth process outlined in the conceptual framework. As in the basic DD estimation, the identifying assumption is that in the absence of the program, farm performance would have changed at the same per period growth rate for controls, exposed and graduate households, and program impact on graduates (exposed) can be measured as the change in the per period growth rate for graduates (exposed) relative to the per period growth rate for controls. The main difference of the text equation (2) with the specification in equation (2') is that now the graduate and exposed household dummies are interacted with the length of programn exposure in order to account for the fact that the change in outcomes for households that have been exposed for longer periods may be greater. Incorporating these considerations (and suppressing subscripts to simplify the exposition) leads to: (3') A(lnY) = a(T* - To)+ 3DN (T, - T*)+,uDG (Tl - T*)+ yAX +AZ + Ae 31 where To and T, refer to the two survey years, and T* is the year in which graduates underwent training within the village, such that To t) Prob(T < t) A: 3> a ,B a. P< p<3 0.810 0.445 Notes Dependent variables are the differences in logs of household-level yields and pesticide use on the main rice plot between the 1998/99 and 1990/91 rice growing seasons. Both regressions control for household and village characteristics and district dummies. These sets of variables are jointly significant at 5%. Absolute value of robust t-stats, corrected for clustering within villages, reported in parentheses * significant at 5%. Sample size in both regressions is 320. 35 Table 4. Robustness check: allowing for differences in pre-program growth rates . Parameters Yield Pesticides Pre-Droeram growth rates Exposed and Control Farmers (a,) -0.006 0.141 (0.58) (2.59)* Graduates (a2) 0.005 0.084 (0.52) (1.22) Post-program erowth rates Exposed farmers () -0.012 0.260 (1.16) (2.88)* Graduates, after own training (ji) -0:017 0.266 (1.45) (2.79)* R2 0 35 0.37 Hvpotheses Tests (p-values): Yield Pesticides Prob(T > t) Prob(T < t) A: a2 > al A: a2 >a, B: ,B a2 C: p < a2 0.999 0.998 Notes: Dependent variables are the differences in logs of household-level yields and pesticide use on the main rice plot between the 1998/99 and 1990/91 rice growing seasons. Both regressions control for household and village characteristics and district dummies These sets of variables are jointly significant at 5%. Absolute value of robust t-stats, corrected for clustering within villages, reported in parentheses * significant at 5% Sample size in both regressions is 320 36 Table 5. Robustness check: allowing for differences in number of FIFS Parameters Yield Pesticides Pre-program growth rates All Farmers (a) 0.006 -0.120 (1.06) (2.07)* Post-program growth rates Exposed farmers XNo. of FFS (3 -0.001 0.007 (0.78) (0.71) Graduates, after own training (±) -0.005 0.011 (1.08) (0.42) R2 0.34 0.36 Hypotheses Tests (p-values): Yield Pesticides Prob(T > Prob(T < t) A: ,>a c a cc B p. < p 0.853 0.568 Notes Dependent variables are the differences in logs of household-level yields and pesticide use on the main rice plot between the 1998/99 and 1990/91 rice growing seasons. Both regressions control for household and village characteristics and district dummies. These sets of variables are jointly significant at 5%. Absolute value of robust t-stats, corrected for clustering within villages, reported in parentheses. * significant at 5%. Sample size in both regressions is 320 37 Endnotes It is important to note that in this study, the analysis is limited to measurable economic outcomes - i.e., yields and pesticide use which are two performance indicators frequently highlighted by promoters of the program. The analysis does not deal with other effects that field school training is claimed to bring about such as improved health, enhanced political activism, a sense of pride and empowerment, and build-up of social capital (Dilts; Roling and van de Fliert). 2 Modeling performance as a dynamic process is compatible with the way sociologists and economists perceive the process of innovation uptake. For example, Rogers describes the process of innovation adoption as "the mental process an individual passes from first hearing about an innovation to final adoption." Similarly, the leaming-by- doing literature suggests an evolution of performance over time (Feder, Just, and Zilberman, p. 259). The dynamic performance model is also compatible with Bayesian information-updating processes, describing the way perceptions regarding technology parameters change over time with improved information (Feder and Umali, p. 216). 3 While the conceptual framework is applicable more broadly to a range of farm-level outcomes, in the empirical work we focus specifically on rice yields and pesticide use on the main rice plot, as the improvements in rice yields and reduction in pesticide use have been claimed to be important economic achievements of the Indonesian FFS program (Kenmore 1997; FAO 2000). 4 Note that even if the "without program" trend in yields is negative (due to external factors affecting the whole farm sector such as systemic climate changes or declining soil fertility), the program's impact is expected to be positive in that it enables participants (and their neighbors) to slow the rate of decline compared to control farmers. Similarly, even if the overall "without program" trend is one of increased pesticide use (due to factors such as increased pest resistance), the program is expected to slow the growth of pesticide use by graduates and exposed farmers compared to control farmers. 5In the empirical analysis, we do not differentiate between graduates trained by officials versus farmer-trainers because previous studies have shown that the quality of training provided by the two sources was similar (van de Fliert, Pontius, and Roling). 6 For a more detailed discussion of the econometric issues in our context, please see the Appendix. In brief, the econometric problem caused by non-random program placement and participant selection is that if selection is based 38 on unobservable household and regional traits that also affect the outcomes of interest, then the participation variables in a cross-sectional regression of outcomes on participation are correlated with the error term. Therefore the estimates of parameters associated with these variables are biased. The DD or first-difference estimator eliminates this source of endogeneity as long as the unobservables are time-invariant. 7Household controls include plot size and irrigation, education of household head, and household size and composition. Village-level variables control for availability of different types of infrastructure and services, including cooperatives, schools, observers, production input kiosks, irrigation, road quality, and distance from district center. District-level differences are controlled for by including district dummies. 8 While it may be argued that the passage of time could cause "erosion" of knowledge, this seems to be less likely in the case of knowledge gained through FFS, as the program is said to have engendered communal activities and continuous focus on FFS themes through vibrant alumni associations (FAO 1999, pp. 19-2 1). 9 A secular decline in Asian irrigated rice yields over time has been observed by researchers in recent years (e.g., Cassman, Olk, and Dobermann), and is attributed to declines in soil fertility, increased resistance to pesticides, and increased vulnerability to diseases such as sheath blight and blast. In addition, global warming has been associated with a decline of Java rice yields (Amien et. al.). Finally, the 1998/99 rice season in Indonesia was mildly affected by La Nina. While these factors negatively impact on all farmers, it is expected that farmers trained in FFS (and those who learn from them) will have experienced smaller declines due to their better knowledge of pest and crop management. ° The only exception is the difference in change in yields, between exposed and control group households. Yields for the control group declined by significantly more than the decline observed for the exposed households. " Dependent variables in the regressions are the differences in log(yield) and log(pesticide use per hectare) on the main rice plot between the main 1998/99 and 1990/91 rice growing seasons. 12 Post-program exposure for graduates is not interacted with number of FFS since it is most likely that the main effect on their performance is related to the intensive training they have received and the diffusion through informal conversation is of second order magnitude. 13 A conservative estimate of the cost per farner trained is approximately $49 per farmer (Quizon, Feder, and Murgai). 39 14 Pingali, Hossain, and Gerpacio (pp. 267-271) discuss the trade-off between costly and intensive IPM training and cheaper dissemination of simple science-based messages. X5 Note that the change in participation variables are the same as the dummy variables defuned m equation (1') since none of the households in the sample had graduated or been exposed to FFS in the first survey year. That is, ADG = DG and ADN = DN. 16 This first difference estimator is often termed a diference-in-differences estimator because it computes the program impact by comparing the change in outcomes for participants to the change in outcomes for non- participants. 17 Interacting the participation dummies with the length of exposure variables implies that the three variables attached to parameters a, fa, and u are specified as follows for the three groups: (a) for all control households, the variables take the values (Tr-To), 0 and 0; (b) for exposed households, the variables are (T*-To), (T1-TP), and 0, where 7* varies across exposed farmers depending on the season when the first FFS was held in the village; (c) for graduates, the variables are (78-To), 0, and (T-T7*) where P varies across graduates depending on when the graduate attended a FFS, respectively. 40 l I .qyG Graduates , | ~ ~ ~~~~I - I I '~~~' ~Exposed X ~~~~I / 2...... -K yo I l II TT Time (T) 1Fhguire 1. Tlhe EvoRuiton of 1Performance over Tihme Policy Research Working Paper Series Contact Title Author Date for paper WPS2996 Bank Competition, Financing Thorsten Beck March 2003 K. Labrie Obstacles, and Access to Credit Asl1 Demirguic-Kunt 31001 Vojislav Maksimovic WPS2997 Financial and Legal Institutions Thorsten Beck March 2003 K Labrie and Firm Size Asl1 Demirgu,-Kunt 31001 Vojislav Maksimovic WPS2998 Does Micro-Credit Empower Women9 Mark M Pitt March 2003 P Kokila Evidence from Bangladesh Shahidur R Khandker 31001 Jennifer Cartwright WPS2999 Explaining Liberalization Philipp Harms March 2003 P Flewitt Commitments in Financial Aaditya Mattoo 32724 Services Trade Ludger Schuknecht WPS3000 International Climate Regime beyond Franck Lecocq March 2003 V Soukhanov 2012 Are Quota Allocations Rules Renaud Crassous 35721 Robust to Uncertainty? WPS3001 An Introduction to Financial and Antonio Estache March 2003 G Chenet-Smith Economic Modeling for Utility Martin Rodrfguez Pardina 36370 Regulators Jose Maria Rodriguez German Sember WPS3002 Information-Based Instruments for Uwe Deichmann March 2003 Y. D'Souza Improved Urban Management Somik V Lall 31449 Ajay Suri Pragya Rajoria WPS3003 The Investment Climate and the Mary Hallward-Driemeier March 2003 P Sintim-Aboagye Firm. Firm-Level Evidence from China Scott Wallsten 37644 Lixin Colin Xu WPS3004 Institutions, Trade, and Growth David Dollar March 2003 A. Bonfield Revisiting the Evidence Aart Kraay 31248 WPS3005 Vouchers for Basic Education in Varun Gauri March 2003 H Sladovich Developing Countries. A Principal- Ayesha Vawda 37698 Agent Perspective WPS3006 Social Rights and Economics: Claims Varun Gauri March 2003 H Sladovich lo Health Care and Education in 37698 Developing Countries WPS3007 The Impact of Urban Spatial Structure Antonio M Bento March 2003 V Soukhanov on Travel Demand in the United Maureen L. Cropper 35721 States Ahmed Mushfiq Mobarak Katja Vinha WPS3008 Technology, Computers, and Wages Chris N Sakellariou March 2003 N. Vergara Evidence from a Developing Economy Harry A Patrinos 30432 WPS3009 Agriculture in the Doha Agenda Patrick Messerlin April 2003 P Flewitt 32724 WPS3010 Vulnerability in Consumption, Edmundo Murrugarra April 2003 E Murrugarra Education, and health: Evidence from Jose Signoret 34452 Moldova during the Russian Crisis Policy Research Working Paper Series Contact Title Author Date for paper WPS3011 Renegotiation of Concession J. Luis Guasch April 2003 J. Troncoso Contracts in Latin America Jean-Jacques Laffont 37826 Stephane Straub WPS3012 Just-in-Case Inventories: A Cross- J. Luis Guasch April 2003 J. Troncoso Country Analysis Joseph Kogan 37826 WPS3013 Land Sales and Rental Markets in Klaus Deininger April 2003 M. Fernandez Transition. Evidence from Rural Songqing Jin 33766 Vietnam WPS3014 Evaluation of Financial Liberalization. Xavier Gine April 2003 K. Labrie A General Equilibrium Model with Robert M Townsend 31001 Constrained Occupation Choice WPS3015 Off and Running9 Technology, Trade, Carolina Sanchez-Paramo April 2003 H. Sladovich and the Rising Demand for Skilled Norbert Schady 37698 Workers in Latin America WPS3016 Partisan Politics and Stuti Khemani April 2003 H. Sladovich Intergovernmental Transfers in India 37698 WPS3017 Why is Unemployment so High Jan Rutkowski April 2003 J. Rutkowski in Bulgaria? 84569 WPS3018 Integrating the Least Developed Paul Brenton April 2003 L. Tabada Countries into the World Trading 36896 System The Current Impact of EU Preferences under Everything but Arms WPS3019 Foreign Aid, Conditionality, and Ghost Thilak Ranaweera April 2003 P. Rathan Raj of the Financing Gap: A Forgotten 33705 Aspect of the Aid Debate WPS3020 Consumption, Health, Gender, Anne Case April 2003 D. Ballantyne and Poverty Angus Deaton 87198 WPS3021 Institutional Arrangements for Public Elizabeth Currie April 2003 M. Rosenquist Debt Management Jean-Jacques Dethier 82602 Eriko Togo