23017 WORLD BANK OPERATIONS EVALUATION DEPARTMENT SPRING 2001 NUMBER 205 Progress in Java's Villages OED's 1999 evaluation of the Bank's assistance to Indonesia found that projects were not sufficiently geared to poverty re- duction and that agriculture projects had not been targeted at the poorest. To probe development outcomes at the grassroots, OED revisited Indonesia in 2000 to evaluate the performance of three proj- ects designed to improve diverse aspects of villagers' livelihoods; the focus was on the main island, Java. Based on a survey of 197 people in 33 poorest with impact on the less poor, or villages, OED concluded that interventions (given the wide geographical reach of the in health, agriculture, and infrastructure projects) to compare villages inside and have had a positive impact. While it was outside each project. not possible to quantify the degree of OED used three separate questionnaires: poverty reduction realized, project objec- one to village chiefs (N = 33), one to village tives were broadly consistent with poverty midwives (N = 15), and one to mothers of reduction and these objectives were met children under five years of age (N = 148). even though they did not always reflect The relative weight given to this last group villagers' top priorities. In two of the three responded to OED's finding that projects projects, the positive results are likely to in the 1990s had not adequately targeted be sustained. Thus, for Java at least, the rural women, who account for a dispro- projects do not seem to have been materi- portionately large share of the poor. The ally affected by the 1997 financial crisis evaluation also applied a quality checklist and its political aftershock. to 32 roads built by 2 of the projects. The following sections present, first, the Approach nature and overall outcome of the projects OED visited randomly selected villages in and, second, the survey results. Central Java and Yogyakarta. In these two provinces, respectively, 14 percent and 8 The Projects percent of the rural population had One of the projects trained midwives to live incomes beneath the poverty line, compared and work in villages, in an effort to raise the with a nationwide average of 12 percent. health standards of mothers and children. When housing quality, schooling, and The other two had in common an effort to other poverty indicators are taken into involve villagers in building and maintaining account, the level of poverty in these small-scale infrastructure. One of them also provinces is greater than the income data sought to make agriculture more sustainable suggest. The evaluation examined project in order to raise the incomes of farm house- outcomes in an area that is generally poor. holds. The objectives and outcomes of each It was not able to compare impact on the project are outlined below. 2 World Bank Operations Evaluation Department Fifth Population Project (Part B only, USS25nillion) through the creation of 150 village demonstration plots Part B of this project (the rest of the project did not have (target, 140) and 111 hamlet nurseries (target, 80). a rural focus and was therefore not evaluated) helped the Community development initiatives included distribution Health Ministry develop a workforce of village-based of seedlings and livestock, construction and rehabilita- midwives who would, in collaboration with family plan- tion of roads and bridges, improvements to water supply ning workers, health center staff, traditional birth atten- and sanitation, and creation of 1,586 hamlet revolving dants, and village groups, work to reduce the death rate funds benefiting 175,000 farm families. of mothers and newborns. The project met or exceeded OED rated project outcome as satisfactory. The only its relevant objectives efficiently. shortfall of the project was the weak demonstration First, the regulatory framework was strengthened. effect of the agriculture technology transfer component: A ministerial decree regulating the registration and farmers were not always willing to provide the addi- practice of midwives was issued in 1996. This increases tional labor input that the new technology called for. the autonomy of midwives, particularly in handling abnormal deliveries. Village Infrastructure Project ) / 2 mion! The second objective-strengthening training The project was designed as a pilot, targeted at the capacity-was partially achieved. The project developed poorest 20 percent of villages on Java. It sought to an accreditation system for training schools, and 10 involve villagers in decisionmaking about the use of percent of the schools were formally accredited by grant funds, which were employed to build small-scale project close. In 1994 a more systematic and process- public works. It created jobs paid in cash for oriented curriculum replaced that of 1991, reducing the underemployed villagers to construct the infrastructure, training period for midwives from two-to-three years to and it mobilized villages to contribute to the cost of the one year. This change was made in response to the works. The project covered 1,230 villages throughout government's push to accelerate the deployment of Java (compared with an appraisal estimate of 1,200), midwives to the villages. assisting the building of 3,680 kilometers of rural roads, The third objective was fully achieved: the project set 7,790 meters of bridges, 2,427 water systems, 1,230 out to support the training of 16,000 village midwives communal sanitation units, and two piers. In addition, and it helped train 16,085, or 30 percent of the total there was a substantial short-term employment impact: number of midwives the government sought to deploy in unskilled laborers received an "incentive payment" in 1996. Training was spread over 97 nursing schools in exchange for 13.4 million person-days of labor and, over the 13 project provinces. OED rated the outcome of Part a two-year period, 246 field engineers were employed. B as highly satisfactory. OLD rated outcome as highly satisfactory. YoCmakarta Upland Amrca Development Project The Survey Results (USS 1s55 e nilgln) The OED survey probed whether the projects were This project aimed to make a sustainable improvement relevant to villagers, whether their objectives were in the incomes and living standards of upland residents, achieved, and whether results were sustainable. mainly farmers, through better resource management. The project included components to improve upland area Dieid the projects resPond to vilaters'perceived nends? soil conditions, introduce alternative upland conservation OD asked village chiefs, midwives, and mothers to technology options, foster upland community develop- imagine that their village had none of the following: access ment, and improve rural accessibility through rural road roads, drinking water supply, latrines, primary school, or and bridge construction. The project covered 230 villages health post. They were asked to assume that there were in Yogyakarta, compared with an appraisal estimate of funds for building only one of these items: which would 140. With hindsight, greater focus on off-farm employ- they vote for? Each group named drinking water their top ment would have enhanced the relevance of the project. priority: overall, 53 percent placed this first, with little Yet, physical achievements were substantial. Two between-group variation. Chiefs and mothers gave second hundred and thirty-two kilometers of roads were built, place to roads (respectively, 33 percent and 1S percent and 39 bridges were improved, compared with appraisal backed this option). For midwives, health posts ranked estimates of 173 kilometers and 22 bridges. The project second (capturing 27 percent of their vote). A primary completed conservation measures in 518 micro- school came last, with only 6 percent of all respondents watersheds, against an appraisal target of 500. These making it their top priority. measures included construction of bench and contour A key constituency from the perspective of assessing terraces, improved waterways and drop structures, and the needs of the poor is the mothers. Almost 60 percent infiltration pits and terrace risers. The research and made drinking water their first priority-an option that extension objective, which sought to promote alternative was not central to any of the three projects. Although land use strategies in the watersheds, was supported drinking water supply is listed as one of the eligible Pr6cis 205 3 village projects in Yogyakarta Upland Area Development, the eight that were targeted by the audit. At the same time, there is no data in the completion report to indicate how diversification is proceeding apace. Fifty-two percent of the many systems were built; the main infrastructure built was mothers surveyed say that the head of their household's roads (voted first priority by one-third of village chiefs in main source of income comes from off the farm. The the audit survey). Many villages in the limestone regions of project's support for backyard livestock (goats) possibly Gunung Kidul are seriously short of drinking water-as made sense given that this is not a land-intensive activity several villagers pointed out to the OED mission-and it is (and, moreover, because it targets women). But the case for therefore surprising that this was not reflected in the investing heavily in soil stabilization is more difficult to allocation of project funds. The mission estimated that in demonstrate. A majority of village chiefs in Yogyakarta the subdistrict of Rongkop a household would have to (59 percent) reported that soil erosion was not a major spend the equivalent of 5 percent of the minimum wage on problem. In many cases this was probably not a reflection purchasing-from tankers-the 25 liters of water needed of the project's success: many villages had developed to cover daily needs. This calls into question whether the impressive control measures (for example, stone facing of project was really demand-driven. terrace risers) well before the project. In the Village Infrastructure Project, the menu of infrastructure options was completely open, yet only 13 Did the prolects achieve their objectives? percent of the village grant was used to finance drinking In the Fifth Population Project there was a question about water and sanitation, compared with 76 percent for roads. whether government attempts to accelerate the deployment The implementation completion report notes that "women of village midwives had compromised both the quality of are the main beneficiaries from water supply installa- the training and the focus on locating midwives close to tions," which may explain why so many of the 149 their area of origin. The survey found that the midwives mothers interviewed by the audit mission listed drinking were better trained than expected and in a good position water as their top priority. The emphasis given to roads to be integrated into the life of the village where they may reflect the preferences of village chiefs and field had been deployed. Of the 15 midwives interviewed, engineers retained by the project. The mission learned that, 70 percent were under the age of 30; all had graduated in their consultation with villagers, project and govern- from senior secondary school, as well as nursing school; ment staff did not promote the openness of the community 53 percent had received two or more post-assignment infrastructure menu as fully as they might have done. training courses; 53 percent had been born in the district In the case of the Fifth Population Project, the audit where they were now working; 93 percent spoke the local compared reported use of the midwives with use of language; and 60 percent had been offered housing by the traditional providers to see if villagers are expressing a village on arrival. demand for the new service. There were three findings. For the evaluation of the Yogyakarta Upland Area First, women are more likely to use a combination of Development Project, on-farm inspections and interviews services from traditional and new sources, rather than were carried out In 17 Yogyakarta villages. Terraces making sole use of one or the other. (This seems to be appear to be well maintained, and there is ample evidence because midwives are treated as complements, not of grasses and perennial crops-which may enhance soil substitutes, for traditional providers: the latter offer stability-although some of these measures predated the support for a month or so after the birth-baby care, help project. The roads that were built are of adequate quality. with chores, massages). Second, poor women are more There has been no significant increase in the goat herd as a likely to use traditional attendants than trained midwives. result of the project; and revolving funds-whether or not Third, comparing mothers' plans for future births with they were created by the project-have continued to grow. what they did in the past, support for making sole use of With respect to works built by the Village Infrastructure trained midwives appears to be growing. Thus, the project Project, the mission inspected 20 roads, finding that design passes the relevance test because it appears to have quality was average to good. Based on the audit team's correctly anticipated a latent demand for trained midwives. roadside evaluation, road quality was comparable to that Moreover, previous attempts to upgrade the services in the Yogyakarta project. However, village chiefs in provided by traditional providers-for example, through Yogyakarta rated road quality more highly than their hygiene training-had not been successful. counterparts in Central Java. Of the three projects, Yogyakarta Upland Area Develop- In Gunung Kidul, Yogyakarta, the survey examined ment, while of adequate quality, is the least relevant. The eight villages that had been served by both the Upland major emphasis given to boosting farming productivity Area Development and the Village Infrastructure Project. begs questions. The natural resource base is unpromising- OED asked villagers which of the two projects had made large "farming" areas of Gunung Kidul are almost bare the larger positive impact on their lives. There was no clear rock. In the villages visited by the audit mission, chiefs preference between the projects: of the 14 persons repeatedly talked of the need to diversify out of agriculture. interviewed, 7 preferred the Village Infrastructure Project, Census data show that this district is the least diversified of 5 preferred the Yogyakarta Project, and 2 said there was 4 World Bank Operations Evaluation Department no significant difference in impact between the projects. chiefs indicated the fund had grown in size during the Even in the same village there was not always a consensus: previous five years. Not all villagers have the same demand in three of the eight villages, preferences were evenly for, or access to, these funds. Of the mothers interviewed, divided between the two projects. Those who favored the 38 percent indicated that they had taken a loan in the last Village Infrastructure Project tended to single out the large five years, but only 18 percent had borrowed from the volume of paid, short-term employment created by road revolving fund. Village midwives prefer to borrow from building. Those who preferred the Yogyakarta Project were banks: almost half of those interviewed had taken loans, more likely to emphasize the longer-term benefits and the but none had borrowed from revolving funds. They have direct impact on farming productivity. used loans to buy midwifery equipment and supplies, renovate clinic space, and purchase the bicycles and Are the positilue -csilts of these proweets likciy to endure? motorbikes that are indispensable for their work. Eight of the 15 midwives interviewed had been in the village more than 3 years-that is, they had remained when Conclusions their first contract expired. Ten midwives indicated that Project results do not appear to have been materially dis- they would stay in the village at the end of their current rupted by the 1997 East Asian financial crisis. Evidently contract. There is also clearly a demand for the service the crisis was felt more in the cities than in the countryside. from fee-paying clients. In the survey villages, 9 of the 15 A majority of village chiefs and women reported to OED that midwives indicated that their income from private fees was most people were better-off than they had been five years at least double the amount of their government stipend. before. It was striking that all of the 33 villages visited had Thus, the project results seem likely to be sustained. robust financial institutions, including well-capitalized The improvement to rural roads made by both the revolving funds that are accessible to people of diverse Yogyakarta Upland Area Development Project and the income groups. This provides a sound context for further Village Infrastructure Project seems to have been sustained. development and continued work to reduce poverty. Another In the areas visited by OED, maintenance is carried out Bank survey of rural Central Java reached the same conclusion, regularly, mainly by unpaid villagers drafted by the village finding agreement among men, women, and young people that chief. Maintenance activities rarely seem to extend beyond well-being had increased in the ten years up to 1999, with a the filling of potholes with sand and small stones; without 10-15 percent fall in the proportion of poor households. mechanized compaction-no evidence of this was found- this is barely adequate for steeply sloping roads. Neverthe- Based on interviews with 72 people in rural Genengsari, Central Java; see "Indonesia: Consulrations with rhe Poor," report presenred to the Global Synthesis less, almost three-quarters of village chiefs interviewed Workshop, September 22-23, 1999, Poverty Group, World Bank. claimed that roads are better maintained now than they were 5 years ago. In some places, villagers have used their own funds to asphalt the steepest inclines: this was true of roads built under both projects. Also, both projects score roughly the same on erosion resistance. In both cases, C . C E D therefore, sustainability is rated as likely. OED PARTNERSHIPS AND KNOWLEDGE GROUP The revolving funds set up by the Yogyakarta Upland Area Development Project are still being maintained. In Editor-in-Chief: Elizabeth Cinpbell-Pag6 Kepek village, a goat farmer told the mission that the local Series Editor/Writer: Caroline McEuen revolving fund had received Rp 700,000 from the project Contributing Editor/Writer: Pat McNees and now had a paid-up capital of Rp 3 million, a growth Dissemination: Juicy Qureishi-fluq rate of 16 percent in real terms. Ten of the 24 households in Pr6cis are available at no charge by contacting the OED Help Desk: the fund had borrowed sums of Rp 40,000-60,000, paying tel: 1-202/458-4497, or fax the form below to: 1-202/S22-3125, back a minimum of Rp 1,000 per month plus an interest You can also e-mail your request to: elne(xvotldbank.org charge of 0.5 percent. Two of the ten borrowers had This and other OED publications are on the Internet, at temporary repayment problems, but these had been http://w.-orldbaiik.otglhtiniloed resolved using a combination of group support and DISCLAIMER: OED Pr9is are produced by the World Bank Operations sanctions. A similar pattern was found in other villages, and Evaluation Deparment, Partnerships and Knowledge Group (OEDPK), in no case did a fund collapse. Therefore, the sustainability Outreach and Dissemination Unit. The views in this paper are those of the ofOperations Evaluation staff and editors and should not be attributed to of e thi s c ompon en to the n p rojeti raedoliely, s nd i the World Bank, its affiliated organizations, or its Executive Directors. The ease of access to loans from revolving funds and, in some cases, banks has probably enhanced the sustain- ISSN 1564-6297 ability of all three projects. Each of the 33 villages visited Prkis 205 Progress in Java's Villages had an active revolving fund and, in every case, village SThis i3 is based on the 2000 evaluation work of John Heath. pc tk a