BOLIVIA Rural Alliances Project Report No. 132905 DECEMBER 20, 2018 © 2019 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2018. Bolivia—Rural Alliances accuracy of the data included in this work. The Any queries on rights and licenses, including Project. Independent Evaluation Group, Project boundaries, colors, denominations, and other subsidiary rights, should be addressed to Performance Assessment Report 132905. information shown on any map in this work do World Bank Publications, The World Bank Washington, DC: World Bank. not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 132905 PROJECT PERFORMANCE ASSESSMENT REPORT BOLIVIA RURAL ALLIANCES PROJECT (CREDIT NO. 40680, 45580) December 20, 2018 Financial, Private Sector, and Sustainable Development Independent Evaluation Group Currency Equivalents (annual averages) Currency Unit = Boliviano (Bs) 2005 $1.00 Bs 8.36 2011 $1.00 Bs 6.96 2018 $1.00 Bs 6.96 Abbreviations ICR Implementation Completion and Results Report IEG Independent Evaluation Group MACA Ministry of Peasant and Agricultural Affairs PPAR Project Performance Assessment Report UCN National Coordination Unit All dollar amounts are U.S. dollars unless otherwise indicated. Fiscal Year Government: January 1–December 31 Director-General, Independent Evaluation Ms. Sophie Sirtaine (acting) Director, Financial, Private Sector and Sustainable Development Mr. José Carbajo Martinez Manager, Sustainable Development Ms. Midori Makino Task Manager Mr. Christopher Nelson ii Contents Preface ................................................................................................................................................................vii Summary .......................................................................................................................................................... viii 1. Context.............................................................................................................................................................1 2. Relevance of the Objectives and Design.............................................................................................3 Objectives ....................................................................................................................................................... 3 Relevance of the Objectives .....................................................................................................................4 Design ..............................................................................................................................................................4 Components ................................................................................................................................................................5 Project Process ............................................................................................................................................................6 Implementation Arrangements ............................................................................................................................. 7 Relevance of the Design ............................................................................................................................ 8 3. Implementation ......................................................................................................................................... 10 Planned versus Actual Expenditure by Component ....................................................................... 10 Implementation Experience .................................................................................................................... 10 Safeguards Compliance ..........................................................................................................................................11 Financial Management and Procurement ........................................................................................................11 4. Achievement of the Objectives ............................................................................................................ 11 Study Findings............................................................................................................................................. 12 Findings from IEG Workshops ............................................................................................................................. 14 Caveats ........................................................................................................................................................................ 16 5. Efficiency ...................................................................................................................................................... 17 6. Ratings .......................................................................................................................................................... 18 Outcome ....................................................................................................................................................... 18 Risk to Development Outcome ............................................................................................................. 18 Bank Performance .................................................................................................................................... 22 Quality at Entry ........................................................................................................................................................ 22 Quality of Supervision ........................................................................................................................................... 23 Borrower Performance ............................................................................................................................ 23 Government Performance.................................................................................................................................... 23 iii Implementing Agency Performance................................................................................................................. 23 Monitoring and Evaluation .................................................................................................................... 24 Design ......................................................................................................................................................................... 24 Implementation ....................................................................................................................................................... 25 Use................................................................................................................................................................................ 25 7. Lessons.......................................................................................................................................................... 25 Bibliography..................................................................................................................................................... 28 Box Box 1.1. Generic Features of the Rural Alliance Model in Latin America.......................................... 3 Tables Table 3.1. Planned versus Actual Expenditure, by Component ........................................................ 10 Table 4.1. Producer Perceptions of Project Impact............................................................................... 14 Appendixes Appendix A. Basic Data Sheet ................................................................................................................... 30 Appendix B. Analysis of Project Process ............................................................................................... 33 Appendix C. Workshop Method .............................................................................................................. 35 Appendix D. Workshop Results ................................................................................................................ 36 Appendix E. List of Persons Met and Alliances Invited to Workshops ...................................... 41 This report was prepared by John R. Heath (consultant), who visited Bolivia to assess the project in May–June 2018. Christopher D. Nelson (task team leader) supervised the assessment. The report was peer reviewed by Willem Janssen and panel reviewed by Jack van Holst Pellekaan. Vibhuti Kharang provided administrative support. iv Principal Ratings Indicator ICR ICR Review PPAR Outcome Satisfactory Satisfactory Highly satisfactory Risk to development Negligible to low Negligible to low Negligible to low outcome Bank performance Satisfactory Moderately satisfactory Highly satisfactory Borrower performance Satisfactory Satisfactory Highly satisfactory Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of the ICR. PPAR = Project Performance Assessment Report. Key Staff Responsible Management Appraisal Completion Project Team Leader Ethel Sennhauser David Tuchschneider Sector Manager or Practice Manager Laurent Msellati Sector Director or Senior Global John Redwood Juergen Voegele Practice Director Country Director Marcelo Giugale Alberto Rodriguez v IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About This Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, country assistance strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, highly unsatisfactory. Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes). The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vi Preface This is the Project Performance Assessment Report (PPAR) for the Bolivia Rural Alliances Project (P083051). A follow-up project, the Bolivia Rural Alliances Project 2 (P127743), which is still active, lies outside the scope of this assessment. The project was selected for a performance assessment to advance learning about the scope for smallholder groups to benefit from integration with market value chains. The World Bank’s Board of Executive Directors approved the Bolivia Rural Alliances Project on May 26, 2005. The total project cost at appraisal was $34.88 million, which consisted of $28.40 million from an International Development Association (IDA) credit and $6.48 million from beneficiary contributions. Additional financing of $30 million in IDA credit and $7.07 million in beneficiary contributions was approved on April 7, 2009. The actual project cost at closing was $79.74 million. The IDA credit closed on March 31, 2014, 30 months later than originally anticipated. This PPAR is based on a review of project documents, special studies sponsored by the project, and interviews with World Bank staff and national counterparts. It is also informed by the findings from nine regional workshops organized by the Independent Evaluation Group (IEG). The workshops included a questionnaire survey and interviews with 64 representatives of producer groups and approximately 20 of the buyers with which they were allied (see appendixes C and E for details). IEG thanks everyone who contributed to the assessment. The support given by project counterparts in Bolivia is also gratefully acknowledged. As per standard IEG procedures, the borrower was invited to comment on the draft report. No comments were received from the Borrower. vii Summary Around the turn of the millennium, based on lessons learned from projects in Bolivia and elsewhere, the World Bank began tinkering with the model of decentralized, community-driven development, trying to make it a more effective vehicle for boosting incomes generated by private sector productive activities in poor rural areas. The conviction was growing that past efforts to raise production incomes had underperformed because they had not, at the project design phase, paid enough attention to the potential of existing—and, more importantly, new—markets, nor had they developed ways to better link small-scale producers to those markets. The rural alliances model has now been applied to 18 operations in 10 countries throughout the Latin America and Caribbean Region. It seeks to promote links between buyers and organized groups of poor rural producers. No individual alliance model is appropriate for all countries. In Bolivia, the approach was developed in a difficult context. In poor rural areas, producers operated at levels of productivity well below regional standards, marketed surpluses were small, and there were few ties to export markets. The trust between farmers and buyers was traditionally limited; before the project, for example, many indigenous producers sold surpluses only to their kin. Between 2005 and 2014, the Bolivia Rural Alliances Project tested, then scaled up, an approach that included the following steps: • A broad-based public information campaign to raise awareness of the opportunities presented by the project, and of the eligibility criteria. • The joint preparation of a business plan by producer organizations and buyers. • Product specifications and purchasing conditions agreed on by buyers (buyers could also provide technical assistance to producers). • Financing that producer organizations could request from the project. If approved, the producers were required to contribute 30 percent of the total investment cost in cash. • Funding and technical support for each producer organization that became a member of the alliance, so that they could meet the terms and product specifications set by the buyers. The objective of the project, as stated in the development credit agreement, was “to test a model to improve accessibility to markets for poor rural producers in pilot areas.” The viii agreement stated that the objective would be achieved “by: (a) promoting strategic productive alliances between different economic players at the local level; (b) empowering rural producers through the development of self-managed grass-root organizations; (c) increasing access to productive assets and technology; and (d) promoting more effective, responsive and accountable service organizations at the local level.” IEG rates the project’s outcome as highly satisfactory. The project’s objective and design were both highly relevant. The objective responded to lessons learned from past operations in Bolivia and elsewhere and was consistent with both the thrust of the Bolivian government’s policy and the World Bank’s corporate strategy to boost the production-based incomes of poor rural producers by identifying and seizing market opportunities. The project design embodied a set of components and activities that were necessary and sufficient to test the alliance model. The project achieved its objective to a high extent, an achievement supported through two aspects. First, the project process was thorough enough to allow for a full testing of the model’s potential and its likelihood of delivering sustainable results. Process highlights included (i) rigorous application of the cash counterpart payment required of producers, which helped to ensure that only subprojects proposed by the most committed and capable producers were approved for financing; (ii) the involvement of brokers to improve accessibility to markets from the very start of subproject formulation; and (iii) the application of procedures for organizing producer groups that led to sound administration, transparent accounting, and the delivery of a full suite of services, including technical assistance, input procurement, and marketing support. All of these process steps were successfully negotiated, and each was critical for achieving the project’s objective of boosting market access. Second, an important indicator of the project’s achievement was that the increase in net income of producers exceeded the “without project” counterfactual by 160 percent. Although the project was not designed to target the poorest of the poor, it reduced the incidence of poverty among project beneficiaries. The share of moderately poor project beneficiaries after project intervention (50 percent) was 12 percentage points lower than for nonbeneficiaries; 33 percent of project beneficiaries were extremely poor after the intervention, 10 percentage points lower than nonbeneficiaries. The poverty gap (the extent to which individuals fall below the poverty line as a percentage of the poverty line) also narrowed more for beneficiaries than it did for nonbeneficiaries. The substantial increase in net income was driven by increases in the volume sold, improved product quality, and improved trading terms, although the precise weight attributable to each of these factors was not quantified by the impact analysis. ix The project also used resources very efficiently. Its estimated economic rate of return (based on results from the 70 percent of subprojects for which individual completion reports were available) averaged 25 percent and, after price shocks are allowed for, remained above the opportunity cost of capital. Project management absorbed only 11 percent of total project costs, which is a low figure for projects involving a complex structure of national and regional offices. The risk to development outcome is rated as low. Most of the producer groups are still operating 7 to 10 years after they received project financing. According to two survey estimates, between one-half and two-thirds of producer groups continue to sell to the buyer with which they formally allied when the subproject was approved. Survival rates vary by region and product line, and it is likely that producer groups will continue to thrive: First, several groups have developed their own arrangements for providing finance to their members by setting up revolving funds and providing guarantees for the loans that commercial banks make to individual producers. Second, even producer groups faced with downward price trends—notably, for milk and quinoa—have continued to operate, in some cases by diversifying products and in others by pursuing defined-origin labeling. The performance of the World Bank and of the borrower are therefore both rated highly satisfactory, based on close collaboration in project design, adaptive project implementation, and leadership continuity. IEG draws six lessons from the assessment: • In a country such as Bolivia, where the productivity of small-scale producers is low and there is substantial scope for increasing sales to the domestic market, the first step for a productive alliance is to boost the quantity and quality of the marketed surplus. The impact of rural alliances will vary depending on the country context. In Bolivia, productive alliances successfully boosted the quantity and quality of marketed surpluses of small-scale farmers. Other alliance projects in Brazil, Colombia, and Central America show how the same model can be adapted to higher levels of market development (World Bank 2016). • Once producer groups are well organized, alliances can help producers obtain sustainable, postproject finance, enhancing the sustainability of the alliance arrangement. The project design promoted fiduciary responsibility by empowering producer groups to manage project funds directly through systematic record keeping and public presentation of accounts. Producer groups then went on to set up revolving funds on their own initiative and also provided x guarantees for commercial bank loans. As well, many alliance buyers advanced cash and made loans to producers they had come to trust. • Project management can be greatly enhanced when strict quality controls are applied by independent parties, without political interference. In this project, staff selection procedures and performance evaluations were competitive and transparent. An independent firm was used to advertise positions and recruit staff, with every effort made to attract as broad a range of applicants as possible. Another independent firm evaluated staff performance each year based on mutually agreed-on individual work plans and beneficiary feedback. Subproject proposals were also independently vetted, a process that ensured rigorous assessment of feasibility, allowing for variations in context and producer organization capability. The rules were clearly defined at the outset, thereby reducing the scope for management to be compromised by political favoritism. • Technical assistance works best when it is based on a flexible menu that accommodates the varied capacity building needs of different subprojects. The menu of technical assistance options in this project included accounting, procurement, input purchase, negotiation with buyers, storage, processing, and packaging—all valid areas for skills development, although the precise mix of options varies from one subproject to the next and should be specified when the subproject proposal is first elaborated. • Agile disbursement of project funds enhances beneficiary commitment and increases the efficiency of subproject implementation. When payments are made directly from the government budget to the participants’ bank accounts, producers feel trusted and enabled and have the means to respond flexibly to implementation challenges without having to cope with the problems caused by disbursement delays. In this project, transfers of public funds to producer organizations was formalized in the government’s budget law and channeled through the public sector’s financial management system, a model that can be replicated elsewhere. • Having a knowledgeable national coordinator who helps design the project and provides long-term leadership greatly enhances the achievement of project objectives. A distinguishing feature of this project was that the initiative came from a national counterpart familiar with rural Bolivia who brought his knowledge and skills to project preparation, working closely with World Bank staff to develop a pragmatic and flexible project design. The same person led the counterpart team for the duration of the first project and remained in charge for xi the follow-up operation, winning the trust and respect of the government while diligently supervising and remaining in close contact with staff in six regional offices. José Carbajo Martinez Director, Financial, Private Sector and Sustainable Development Independent Evaluation Group xii 1. Context 1.1 The World Bank Agriculture Global Practice has identified “linking farmers to markets” as a major development challenge in recognition of the obstacles faced by smallholder producers as they seek to compete in rapidly expanding modern agricultural value chains (World Bank 2016). The aim is to improve the competitiveness and entrepreneurship of smallholder producers in developing countries. One approach—pioneered in Bolivia—seeks to promote productive alliances that strengthen the links between producer groups and buyers. Worldwide, producer organizations have spread rapidly. It is estimated that 250 million farmers in developing countries belong to a producer organization (World Bank 2007b, 72). To succeed, productive alliances need to be context specific. Not all countries are ready for an approach centered on high-value export markets. Most farmers in Bolivia are oriented toward the domestic market, which has substantial room for expansion given rapid urbanization. Small-scale farmers have a precarious foothold in markets, however, primarily because low productivity and high production costs limit their capacity to produce a marketable surplus and reduce their competitive edge. Factors contributing to low productivity include the extreme fragmentation of farm holdings, restrictions on the sale and lease of farmland, limited access to credit, soil erosion, and the low application of advanced technologies (Baldivia Urdininea 2008). Weak producer organization aggravates the problem: Acting alone, small-scale farmers have little power to negotiate input and output prices, particularly when faced with an oligopsony. As well, where producers are not organized, it is more difficult, and more expensive, to provide them with technical assistance and market intelligence. The Rural Alliances Project was partly an attempt to more fully realize the growth potential implicit in recent moves to decentralize government. Following passage of the Popular Participation Law (1994) and the Administrative Decentralization Law (1995), Bolivia began removing institutional constraints to investment in the countryside by creating rural municipalities, devolving decision making to these local governments, and assigning them an increasing share of public investment resources. The government also gave local communities and indigenous groups the legal right to identify and formulate public investments and to participate in and control their implementation (World Bank 2005b). Although decentralization increased government responsiveness to local needs and improved education and health services (Faguet 2001), by the turn of the 1 millennium, it was evident that decentralization had not raised rural incomes. This was partly because transfers from the central government to municipalities were earmarked in a way that continued to privilege social programs over productive investments. As well, investments in rural production (including those funded by World Bank interventions) had addressed producer needs rather than market opportunities. In particular, the FY01 Indigenous People’s Development Project showed that attempts to expand production will be ineffective if producers do not begin by identifying markets, carefully assessing the opportunities and threats they pose (World Bank 2006). The attention to openings in domestic and export markets was consistent with the government’s 2005 rural development strategy, which identified the need to integrate rural producers with region-specific value chains linking them to the final consumer (MACA 2005, 69). At the same time, World Bank staff were acknowledging the limited contribution, worldwide, that community-driven development initiatives had made to boosting private enterprise. The principles and tactics of decentralized rural development were still endorsed, but a search began to link these approaches to new interventions designed to boost incomes, by improving the terms on which producers connected to value chains. In Latin America, this developed into the rural alliance approach (box 1.1). 2 Box 1.1. Generic Features of the Rural Alliance Model in Latin America “A rural productive alliance is an agreement between formally organized producers and at least one commercial buyer. The agreement specifies product characteristics, such as size and varieties to be produced; quantity to be produced or bought; how a product will be delivered, by whom, and when; grading and packing requirements; means of payment; price determination criteria; and the buyer’s contribution, such as technical assistance, specific inputs, and arrangements for input reimbursement (for example, at the time of sale). The alliances aim to increase incomes and employment for rural producers through their participation in modern supply chains, sometimes with an emphasis on lagging regions or indigenous populations. Producers overcome market barriers and gain stability through consistent, higher prices and buyers receive a consistent, reliable supply of goods meeting their quality standards. Alliances are initially funded through grants for technical assistance (in production, management, and marketing) for the producer organization, along with infrastructure and equipment. Grant recipients in some organizations repay a share of the grant to the organization to create revolving funds that will provide credit to members when external funding ends. Producer organizations need to build marketing skills and may benefit from a third-party agent or broker to enter high-value markets. Buyers can improve the alliance through sensitization to the benefits and transactions costs of working with small-scale producers and through support to optimize the marketability of niche products. Projects require a handover strategy so that domestic actors can fund, implement, and scale up activities when project support ends.” Source: World Bank 2012, 95. 2. Relevance of the Objectives and Design Objectives 2.1 The objective of the project as stated in the development credit agreement was “to test a model to improve accessibility to markets for poor rural producers in pilot areas.” The agreement stated that the objective would be achieved “by: (a) promoting strategic productive alliances between different economic players at the local level; (b) empowering rural producers through the development of self-managed grass-root organizations; (c) increasing access to productive assets and technology; and (d) promoting more effective, responsive and accountable service organizations at the local level” (World Bank 2005a, 22). The objective is identically worded in the project appraisal document (World Bank 2005b, 4). The project development objective was not revised during implementation. 3 Relevance of the Objectives The project pioneered a new approach to linking small producers to markets, building on lessons learned from three preceding World Bank rural development operations in Bolivia: the FY96 Rural Communities Development Project (P006202), the FY98 Participatory Rural Investment Project (P040085), and the FY01 Indigenous Peoples Learning and Innovation Credit (P057416). The first lesson learned from these projects was that decentralization is a necessary but insufficient way to boost the incomes of the rural poor. Second, private productive and public infrastructure investments need to complement each other. Third, productive investments will have limited impact if they are not preceded by a detailed assessment of where the markets are and what the buyers want. This assessment is more likely to work if it is a joint exercise between producers and buyers, culminating in a partnership agreement (that is, an alliance). Finally, recognizing that no tried and tested model existed for developing these rural alliances, the project took an experimental approach, exploring three different models. The project objective reflected the assimilation of these lessons. Before the project was approved, the World Bank had worked closely with the government to prepare a rural development strategy (MACA 2005). The project objective aligned with that strategy’s emphasis on linking rural producers to market value chains. Throughout approval and implementation, the project objective was consistent with the aims of the various World Bank assistance programs for Bolivia. The FY12–15 Country Partnership Strategy sought to “reduce extreme poverty in rural areas by increasing agricultural productivity.” The strategy acknowledged that “income generation capacity for rural producers is constrained by their difficult and unpredictable access to consumption markets” (World Bank 2011a, 32). The project objective was also in line with the World Bank’s corporate agriculture and rural development strategy, one of whose aims was to “strengthen farmer-to-market linkages” (World Bank 2003, 50). The relevance of the project objective is rated high. Design The original project covered three regions: (i) the northern expansion zone of Santa Cruz, (ii) the Cochabamba valleys, and (iii) the area around the Uyuni Salt Lake in Oruro and Potosi. With additional financing approved in 2009, the geographic scope was extended to include the Lake Titicaca region and the La Paz-Beni subtropical area (World Bank 2009, 6). 4 Components Component 1: Institutional support. This component financed the technical assistance and training needed to create productive alliances, helping small-scale producers become partners in new marketing arrangements with companies in the private sector. The component supported a project-information campaign, actions to empower groups of poor rural producers, the call for proposals, and the preparation of prefeasibility and feasibility studies and their evaluation and approval; it also helped formalize the alliances. Subcomponents comprised (i) communication and dissemination, (ii) institutional facilitation, (iii) capacity building for service providers and local governments, and (iv) appraisal of alliances. Component 2: Implementation of rural productive alliances. This component supported implementation of the rural alliances prepared under component 1. It also supported the formation of partnerships between producers and buyers, helped upgrade product standards to meet the requirements of new markets, gave special assistance to small producers, and ensured that service providers and local governments participated in the preparation of investment plans. This work entailed cofinancing producer’s subprojects up to the storage stage, increasing producer’s access to the credit they needed for product marketing and processing, working with local governments to build public infrastructure, and providing incentives to buyers to offset the risks associated with entering into financing and marketing arrangements with small rural producers. Subcomponents comprised (i) farmer organization subprojects, (ii) municipal subprojects, and (iii) finance enhancement incentives. Component 3: Project management. This component covered the setup and operation of a project coordinating unit in the Ministry of Peasant and Agricultural Affairs (Ministerio de Asuntos Campesinos y Agropecuarios; MACA), as well as the development of a monitoring and evaluation system to measure market access by poor producers and growth in their incomes. Subcomponents comprised (i) studies, (ii) technical services, (iii) financial management, (iv) monitoring and evaluation, (v) equipment, and (vi) operating costs (World Bank 2005b, 5–6). 2.9 Additional financing of $30 million was approved by the World Bank’s Board on April 7, 2009. There was no borrower contribution to the additional financing. Beneficiaries made a substantial contribution, exceeding initial projections. It was expected at appraisal that they would provide $6.48 million, to be augmented by $7.07 million as part of the additional financing; their actual eventual contribution was $19.80 million (25 percent of the final cost). Most of the additional financing was used to 5 expand project activities into the two new subregions (World Bank 2009, 6). The original project covered 54 municipalities; additional financing added 29 more municipalities. With the approval of additional financing, the components were modified. The finance enhancement incentives subcomponent was eliminated because demand for complementary credit from outside institutions could be satisfied through technical assistance provision to the alliances themselves, rather than through incentives to financial institutions. The municipal subprojects subcomponent was reduced in scope because fewer than the expected number of alliances required complementary public works to be viable, and there was less need for additional project funding for public infrastructure because general funding transfers for both local and regional governments increased substantially following changes to the hydrocarbon laws in 2005 (World Bank 2009, 5). Project Process The design of the project was based on the following assumptions and steps (appendix B provides further details): • Poor rural producers have limited access to markets, but some will have the resources needed to become market-oriented producers and allow the project to test ways of linking poor producers with commercial potential to buyers. • In selected, poor rural areas, advertise widely the terms of the experiment and invite producers and buyers to sign up. • Make it clear from the start that participants must make a large, upfront cash contribution to subproject costs, and give preference to producers already organized in groups, thereby ensuring that only producers with potential self- select into the test. • With the help of brokers familiar with market opportunities, bring producer groups and buyers together to formulate a proposal for the former to sell to the latter on agreed terms, thereby creating an alliance. • Invite participants to frame their proposals in terms of one of three alliance models. Model 1 (the simplest—the one that was expected from the beginning to garner the most support) was designed for small-scale producers and financed goods and technical assistance up to the postproduction storage phase. Models 2 and 3 would finance buyers as well as producers and covered processing and marketing but was not tested because of lack of interest by participants. 6 • Based on (i) assessments of the financial, social, and environmental feasibility of the subproject proposals; (ii) the (cofinancing) commitment shown by producer groups; and (iii) the availability of complementary funding from local governments and financial institutions, select the most promising proposals for financing. • Require producer groups and buyers of the winning proposals to sign an alliance agreement, specifying product quantity and quality and timing of delivery. • Implement the alliance subproject, ensuring that participants receive full technical support—not only during but also after the implementation period; participants choose technical assistance providers, paying for their services with subproject funds. • Evaluate the performance of the subprojects, in terms of growth in the volume of sales, incomes, and jobs. • Modify the alliance model in the light of lessons learned from the test. Implementation Arrangements The project was initially implemented by MACA, which set up a National Coordination Unit (UCN) and Regional Operational Units in the three project areas. Reflecting successive reforms of the executive branch, MACA was succeeded by the Ministry of Rural Development, Agriculture and Environment and finally by the Ministry of Rural Development and Land. MACA and its successors represented the project’s strategic management, ensuring that government policies were implemented in coordination with other sector programs and projects. The UCN was responsible for the management information system, monitoring and evaluation, and financial management and procurement, including the appraisal and financing of alliance proposals. The UCN comprised a project coordinator and individual consultants appointed by MACA and its successors. Staff performance was evaluated annually by an external agency. A project council made up of members of relevant ministries; national representatives of small-scale producers, traders, business people, prefectures; and the national project coordinator was set up to approve management guidelines, review annual operating programs, assure coordination with other programs, and recommend strategies and policies to facilitate project implementation. It was originally proposed that this council would meet twice a year. When additional financing was approved, the council was abolished because the Evo Morales government considered the new social participation mechanisms to be sufficient for project purposes. During project implementation, the project became the lead entity of a deconcentrated unit, 7 EMPODERAR (Emprendimientos Productivos para el Desarrollo Rural, or Productive Initiatives for Rural Development Program), within the Ministry of Rural Development and Land that coordinates other programs financed by external donors and the Treasury (World Bank 2014, 4–5). Relevance of the Design When this project was prepared around 2000, poverty was widespread in rural Bolivia, and its incidence had risen over the previous decade. Eighty-four percent of the rural population fell under the poverty line, with 67 percent defined as extremely poor. In Potosi and Cochabamba, two departments covered by the project, approximately 95 percent of the rural population was poor. According to a contemporaneous poverty assessment, on average, the incomes of the rural poor covered less than half of the basic consumption basket (World Bank 2005d, 2). In these circumstances, area-based targeting, rather than household-based targeting, made the most sense. The area-based targeting identified 54 municipalities (which expanded to 110 after additional financing) “with a high incidence of poverty combined with economic potential” (World Bank 2005b, 77). Seventy percent of the population in these municipalities self-identified as indigenous, a group that on average had a higher rate of poverty than the rural population as a whole. Project records show that 90 percent of project beneficiaries belonged to an indigenous group. Although such groups are known to be overwhelmingly poor, the project’s design served the objective of reaching poor producers who had access to financial resources and who had the potential to participate in rural alliances. The area-targeting scheme drew on a municipal index of basic needs satisfaction devised by the Ministry of Planning and Development (Salguero Lowenthal 2012). Inevitably, within each municipality, income levels differed among households; the project did not target the poorest of the poor. Household-based targeting would not have been consistent with the self-selection that was central to project design. Self-selection filtered out producers willing and able to make a cash contribution to the subproject cost. This method of selection was an appropriate way to test a model of increased market access that was aimed at the poor with productive potential. The choice of project components and activities was consistent with, and sufficient for, the declared objective of testing a model for promoting alliances between producer groups and buyers. The provision for information campaigns was enough to ensure that interest would be aroused, serving to attract a large number of alliance proposals. The project design allowed for multiple, independent evaluations of 8 proposals before subproject financing was approved, ensuring that the testing of the models would be fair and rigorous; the scope for political manipulation of subproject selection was limited. Testing required a fully funded provision for monitoring and evaluation, which the design allowed for. The appraisal document noted that there would be “an external and independent evaluation conducted at the time of midterm review” (World Bank 2005b, 3) but did not provide for the baseline survey and selection of treatment and control groups that a rigorous impact evaluation would entail. However, once the project was under way, the design was adjusted to allow for a partial evaluation of impact, the limitations of which are discussed in the Monitoring and Evaluation section. 2.16 The project objective refers to “improving accessibility to markets” (World Bank, 2005b). This can be construed as increasing the volume of goods sold in existing markets, as well as supplying new markets. Neither of these outcomes would guarantee an increase in producer incomes or employment, but it is clear from the project’s original choice of key performance indicators that income and job growth were explicit, expected project outcomes if the project objectives were achieved. 2.17 The appraisal document clearly states that the project was not intended to serve the poorest, but to serve poor rural producers with potential (World Bank 2005b). Neither the appraisal document nor the operating manual defines “poor rural producers.” Farm size is an obvious criterion, but this is not explicit in the project guidelines. World Bank staff associated with the project told the Independent Evaluation Group (IEG) that, in practice, this size criterion was used in eligibility screening, with the ceiling adjusted between regions to reflect agroclimatic variations. The project targeted ostensibly poor regions, not poor communities or households. The project design also included a social evaluation to ensure that subproject proposals selected for financing addressed the needs of women and indigenous groups (World Bank 2005b). 2.18 Two government funds set up independently of the project were used to enhance subproject viability by providing complementary sources of finance. An incentive fund (Fondo de Desarrollo del Sistema Financiero y de Apoyo al Sector Productivo) provided grants to financial institutions willing to assume the risks associated with lending to alliance members. A Productive and Social Investment Fund was designed to elicit support from local government by providing cofinancing for public infrastructure works that were an explicit part of the alliance plan. 2.19 Design relevance is rated high. 9 3. Implementation Planned versus Actual Expenditure by Component The increase in total project costs reflected the expansion of the area covered by the project under additional financing, rather than increased unit costs (table 3.1). Final expenditures on components 1 and 3 were less than projected at additional financing approval. Table 3.1. Planned versus Actual Expenditure, by Component Estimate at Actual as Appraisal Additional Actual or Percentage Estimate Financing Latest Estimate of Appraisal Component ($, millions) ($, millions) ($, millions) Institutional support 3.05 5.29 5.14 168 Alliance 23.88 57.50 65.67 193 implementation Project administration, 4.43 9.16 8.93 276 monitoring, and evaluation Price contingencies 3.57 0.00 0.00 0 Total cost 33.93 71.95 79.74 229 Source: World Bank 2014. Implementation Experience The project was approved on May 26, 2005, and the credit became effective 12 months later. The original closing date was September 30, 2011, and was extended, with additional financing, to March 31, 2014. The project’s effectiveness was delayed during President Rodriguez’s caretaker administration (2005–06) by objections from the Ministry of Finance, partly because of fiscal constraints but also due to hesitation about financial transfers to communities and poor producer groups. Following effectiveness, project startup was again delayed by 18 months, which slowed the hiring of staff for the UCN and the Regional Operational Units. The parent ministry at that time (the Ministry of Rural Development, Agriculture and Environment) was reluctant to sign agreements with regional governments led by opposition parties. Consequently, the project’s development objective was given a moderately unsatisfactory rating in the two implementation supervision reports for 2006 (World Bank 2014, ix). Subsequently, the government became a strong supporter of the project, introducing rural development policies that aligned with the project’s objectives. The price trend of agricultural 10 commodities was favorable, tripling for quinoa, an Andean staple produced by 13 percent of the rural alliances sponsored by the project. Safeguards Compliance The safeguards that applied to the project were Environmental Assessment (Operational Policy [OP]/Bank Procedure [BP]/Good Practice 4.01), Pest Management (OP/BP 4.09), Cultural Property (OP 4.11), Forests (OP/BP 4.36), and Indigenous People (Operational Directive 4.20). In all supervision reports, safeguard compliance was rated either satisfactory or moderately satisfactory. Implementation of the Indigenous Peoples safeguard (OP/BP 4.10) was of paramount importance: 90 percent of direct beneficiaries identified themselves as belonging to an indigenous group, primarily Aymara, Quechua, and Guarani (World Bank 2014, 16). The project’s social assessment emphasized the needs of indigenous groups, and workshops on this theme were organized during implementation. The only significant failing of the safeguard enforcement involved the misapplication of integrated pest management techniques. This failing was corrected following the midterm review. Financial Management and Procurement All audits were unqualified and, overall, financial management was satisfactory. Only one instance of fund mismanagement was reported—this involved the leader of a producer organization who was subsequently put on trial. Procurement was also satisfactory. Procurement was mainly handled by the small producer organizations, under the control of their members and the guidance of the regional operating units. Procurement specialists in the national and regional offices trained the staff of each producer organization. 4. Achievement of the Objectives The project process lived up to design expectations by attracting poor smallholders with significant productive potential. The project was also socially inclusive: 90 percent of beneficiaries self-identified as indigenous, a group that accounts for most of the poor in Bolivia. Relative to the norm for Bolivian indigenous society, women were well represented among project beneficiaries. The project team staged 17 regional workshops intended to promote women’s participation in alliances. Women accounted for 32 percent of the membership of producer organizations and 27 percent of their leadership (World Bank 2014, 15). The self-selection targeting mechanism, with its emphasis on counterpart payments in cash, helped ensure that the project attracted the most committed and 11 enterprising participants. To ensure that the counterpart obligation was not too high a hurdle for participants, the payment was spread out over three to four tranches. Informal arrangements also boosted participation. Through its interviews with producers, IEG found that it was a common practice for producers to advance money to colleagues who were temporarily short of funds. In Salar region (the poorest of the six covered by the project), for example, quinoa buyers would pay producers in advance so that they could cover the counterpart, subtracting the advance later on when the product was delivered; milk wholesalers offered the same service to producers in Lago and Valles. As well, although the counterpart was specified in cash, there seems to have been some scope for producers to substitute labor and materials. Study Findings The project significantly boosted the net incomes of participating producers. One Food and Agriculture Organization study surveyed 140 alliances in three of the six regions, covering 18 percent of all the alliances that were financed (Salguero Lowenthal 2012). One of the project’s achievements was to ensure that detailed baseline and completion reports were prepared for each subproject, using a standard methodology. Given that project beneficiaries were not randomly assigned, and that the choice of subprojects was limited to early starters who had a completion report already in hand, this method could not be used to estimate the difference made by the project. However, it provided a wealth of data about the actual performance of the many subprojects and the variance between them. The data show that the 140 alliances experienced an average increase of 160 percent in net income; only 11 percent of these alliances experienced a net loss. The data do not show substantial variation in net income gains by product line— groups producing the same product showed a wide range in net income gains. This suggests that it was the attributes of the producer group, not the product, that explained the difference. The same report found that, although all the areas covered by the project had high levels of poverty, given the project’s intention to favor districts with growth potential, the poorest municipalities may have been left out. One survey, based on a purposely selected sample of 140 alliances in three of the six regions covered by the project (Uyuni, Valles, and Tropico), found that 43 percent of the municipalities that scored relatively high on the basic needs satisfaction index attracted 56 percent of the total funds invested in project-sponsored alliances and 51 percent of all beneficiaries (Salguero Lowenthal 2012). A separate impact evaluation—the methodology of which is explained in the Monitoring and Evaluation section—was conducted in four of the six project regions 12 (Chaco, Norte, Valles, and Tropico). The study found that that the increase in net income attributable to the project was 63 percent higher for project beneficiaries than for a control group of matched nonbeneficiaries. Moreover, the project’s impact on income was progressive, with producers in the lowest income quintile gaining four times more than those in the next highest quintile. Income per producer from the alliance-supported product increased on average by 73 percent. The increase in net income was highest in Norte, followed by Tropico, Valles, and Chaco. The increase was highest for beef (136 percent), followed by milk (102 percent), honey (76 percent), coffee (57 percent), and cocoa (51 percent; Monterrey 2016). The report by Monterrey (2016) relates increases in income to project-driven improvements in the organizational and technical capacity of producer groups. Sixty- one percent of the groups collectively procured inputs for their members, 59 percent used subproject funds to hire technical assistance, and 53 percent collectively negotiated prices with traders and delivered their product as a group rather than individually. The study shows that the use of technical assistance and improvements to product collection and storage significantly boosted income gains. Together, these factors improved the terms of market entry, in line with the project objective. The project contributed less than expected to the creation of new wage-earning jobs. According to the Implementation Completion and Results Report, the increment in person-days of wage employment (not including family labor or labor in marketing value chains) was only 38 percent of the target (World Bank 2014, vii). The impact evaluation by Monterrey (2016) found that beneficiaries generated 0.15 more person- days of wage employment than did nonbeneficiaries—a difference that was small but statistically significant nonetheless. However, this limited impact was more than compensated for by the increase in the level of on-farm family employment, resulting from, among other things, an expansion in the areas planted and grazed. The growth in the cultivated area was 3.1 hectares more for project beneficiaries than it was for nonbeneficiaries (Monterrey 2016). Most strikingly, the project significantly reduced the incidence of poverty (Monterrey 2016). The share of project beneficiaries that were moderately poor following project intervention (50 percent) was 12 percentage points lower than for nonbeneficiaries; 33 percent of project beneficiaries were extremely poor after the intervention, 10 percentage points lower than for nonbeneficiaries. The poverty gap (the extent to which individuals fall below the poverty line, as a percentage of the line) also narrowed more for beneficiaries than it did for nonbeneficiaries. Although the project team was unable to conduct a full double-difference analysis, it partially compensated for this flaw by testing four strategies for matching 13 beneficiaries and nonbeneficiaries. Depending on the matching procedure, the project increased agricultural sales and total agricultural income between 28 percent and 39 percent over the average agricultural sales and total agricultural income of the control group. These improvements were significant at the 5 percent level. Household labor income increased by about 28 percent, an improvement that was also significant at the 5 percent level. At project completion, a beneficiary survey of 86 randomly selected producer groups provided additional evidence of the project’s positive results. Most of the producers surveyed reported that the main positive impacts were improved quality of products, increased productivity, and the development of productive capacities (table 4.1). There was a shared perception that this had led to increases in household income over the medium term. Table 4.1. Producer Perceptions of Project Impact Percent of Producers Mentioning Type of Impact an Impact Improvement in produce quality or quantity 75 Increase in labor productivity 71 Organizational strengthening 68 Net income increase (sales less costs) 57 Market access 36 Increased productive capacity to generate income 32 Source: World Bank 2014. Findings from IEG Workshops The rate of participation in the nine IEG workshops was a good measure of the robustness of the project’s effects: 59 percent of the producer groups randomly sampled by IEG sent a representative to one of the workshops. To maximize workshop attendance and enhance participation by those living far away, or those with limited means, IEG reimbursed the travel expenses of participants and provided modest refreshments at the workshops. Appendixes C and D outline the workshop method and findings in detail. All represented groups were still fully operational 7 to 11 years after receiving project financing. There is some suggestion of age attrition, however: Of the alliances funded by the project in 2010, the participation rate was 64 percent, while for those funded in 2007, it was 53 percent (appendix D, table D.3). However, even if all the “no- shows” at workshops (41 percent of those invited) corresponded to groups that are no 14 longer operating (although they may still be operational; this is unclear), the participation rate is still a significant indicator of the viability of the funded producer groups and reflects positively on the project process. The producer groups provided substantial services to their members. All the producer representatives who attended the workshops noted that they had received technical assistance and attended training courses, and most participants vigorously endorsed the quality of the orientation provided. More than 90 percent of producer groups set up collective arrangements for procuring inputs and provided marketing support to their members, including visits to trade fairs. More than half of the groups helped their members secure individual loans from commercial banks. There was a tendency, however, for producer groups to underinvest in technical assistance, spending much more on goods, which were deemed to be of greater practical value. As well, once the subproject investment was fully disbursed, delivery of technical assistance tended to languish, because producers were not willing to fund this from their own pockets. The follow-up project has addressed this issue by offering top-up funding for technical assistance to alliances created under the first project; this is a stopgap measure, however. Comments by IEG workshop participants suggest that the method used by the project to organize producers was rigorous and comprehensive. In addition to having a committee administer project funds, producer groups established committees to monitor the quality of their administration, to evaluate bids from competing input suppliers, and to ensure that the suppliers who won the bid delivered the agreed-on input quantity and quality. Administrative costs were covered by a levy on sales revenues. Fines were collected from producers who failed to attend committee meetings. Producer groups needed to have a legal identity if they wanted to receive project funds. Some of the producers interviewed by IEG reported that legalization could take up to one year and often required them to travel to provincial capitals, a considerable distance. Although the project made provision for formalizing producer groups, there could have been a bias toward self-selection by long-established groups—those that were likely to have a legal identity already. In the event, the bias was not pronounced: 52 percent of the producer groups funded by the project had been running for less than a year; presumably they were project creations. Only 14 percent of the groups had been running for 10 years or more (although, for reasons that are unclear, in the Valles region, this share was as high as 39 percent, and for producers from milk and quinoa-producing groups, the share was over 20 percent). In other words, the project added value by investing substantially in producer organizations: It did not merely cater to well- established groups (which, arguably, did not need assistance). 15 Given the constraints of the Bolivian context, the immediate challenge for the project was to improve the terms on which small-scale producers entered existing markets, rather than adding new markets. As well, access to the domestic market was more important than access to export markets: Only 28 percent of alliances produced for export (World Bank 2014, 32). The first step to improving access was producing larger marketable surpluses of higher quality. Indeed, beneficiary surveys sponsored by the project found that 75 percent of producers cited increased product quality and quantity as a positive impact of the alliance (World Bank 2014, 42). Participants in the IEG workshops said that the idea of involving brokers from the very start of subproject design was sound, because it would help identify buyers and strengthen the basis for price negotiation. For the most part, brokers helped improve the terms of access to existing markets, rather than helping producers link to new markets. In Lago region, a buyer of dry llama meat noted that she was interested in exporting more of her product but that the technical assistance provided by the project did not adequately cover marketing costs. In the same region, one of the technical assistance providers told IEG that markets for milk were already well established; brokers were not needed to help producer groups identify markets, but they did help improve the terms on which they negotiated with existing buyers. Perhaps the most striking finding from the workshops was the mix of resilience and flexibility that producers demonstrated in their interaction with markets. Producers with long-established ties to highly capitalized buyers (which was the case for milk) had responded to falling prices by diversifying into dairy by-products. Producers of annual crops that were faced with adverse price trends had substituted more profitable crops instead. These attributes increased the chances that producer groups would survive, a topic that is examined in depth in the Risk to Development Outcome section. The overwhelming evidence of producer group sustainability is a measure of the project’s achievement of its objective—it is likely that access to markets will continue improving over the long term as producer groups build on the expertise they acquired from the subprojects. Caveats Two points need to be clarified. First, the project development objective was to test a model (not models) to improve accessibility to markets. Although at the design stage three potential models were developed, only one was taken up during implementation: Since this choice was made by the project beneficiaries themselves (and not imposed by project management), the rejection of the other two models was itself a kind of test. The rejected alternatives were more sophisticated and buyer centered: It was noted that buyers simply were not interested in the limited funds that the project 16 offered to support product processing and other aspects of their operation. As well, little use was made of the financial incentive fund (Fondo de Desarrollo del Sistema Financiero y de Apoyo al Sector Productivo) that was intended to encourage lending to producers by banks. Once again, this does not count as a project failing; rather, it underscores the robustness of the demand-driven test allowed for by the project design. Second, the municipal investment subproject option built into the project’s design proved difficult to promote. Only two percent of the alliances funded involved municipal cofinancing of infrastructure (mainly roads and bridges). The original target was for the Productive and Social Investment Fund to fund 86 alliance subprojects; the revised target was 42. By project end, only 19 subprojects had been financed (World Bank 2014, 30–32). Before the project’s additional financing phase, local government budgets were buoyed by central government transfers funded from natural gas exports. In the last decade, however, natural gas prices have fallen sharply, reducing transfers and squeezing municipal budgets. In Lago, a municipal government officer told IEG that the municipality could not afford the 20 percent cofinance share that would be required for it to participate in the project. As well, in any given region, the producer groups receiving project finance would tend not to be co-located, so it was difficulty for a single infrastructure project to serve several groups at a time. Finally, because of the exigencies of the local government budget cycle, it proved difficult to synchronize the release of the municipal counterpart payment with the project timetable. Although this was a lesson learned, it does not detract from the project’s achievement of its objective to improve accessibility to markets for poor rural producers. Achievement of the project objective is high. 5. Efficiency A thorough economic and financial analysis was conducted before project completion, based on a sample of 140 producer groups (Salguero Lowenthal 2012). The analysis found that, over a 10-year horizon, 74 percent of the groups achieved an economic rate of return above the 12 percent discount threshold. At project completion, the analysis was repeated for the 535 producer groups for which individual subproject completion reports were available—that is, 70 percent of the subprojects financed. This follow-up survey estimated the economic rate of return as 25 percent, falling to 19 percent in the event of a 20 percent reduction in incremental benefits (World Bank 2014). Ninety percent of all the groups surveyed experienced incremental net income gains. The product lines that performed best were quinoa (an incremental revenue of $120,850 per alliance and $3,128 per family) and coffee ($139,642 per alliance and $2,230 17 per family), followed by potato seed production ($81,397 per alliance and $1,536 per family) and peaches ($61,401 per alliance and $1,616 per family). At the level of the alliances, annual incremental revenues are higher for coffee than for quinoa, but the opposite is true at the family level. This is due to a higher number of participating families in coffee alliances (63 members per alliance average) compared with quinoa (39 members per alliance average). The same occurs for potato seeds, with an average of 53 members per alliance compared with 38 members per alliance in peach production. Using the spreadsheets generated by the 2012 study, IEG tested the sensitivity of the rate of return to poststudy price trends. The two most frequently produced products were milk and quinoa, accounting for 21 percent and 6 percent of the study sample, respectively. In 2018, the price of milk was Bs 3.00 per liter, 70 percent of the level when the study was conducted (2011–2012). The price for organic quinoa (Bs 475 per quintal) was 88 percent of the 2011–2012 level. After adjusting the net income stream accordingly, IEG found that, except for 6 of the 30 milk-producing groups and 2 of the 9 quinoa-producing groups (which had negative net present value in 2012), the other 31 groups had an internal rate of return well above the 12 percent threshold. Project management (component 3) accounted for only 11 percent of total costs, which is low by the standard of comparable projects in Bolivia and elsewhere and is particularly noteworthy given the need for five regional offices to support decentralized implementation. Efficiency is rated high. 6. Ratings Outcome Because relevance, efficacy, and efficiency are each rated high, outcome is rated highly satisfactory. Risk to Development Outcome There were a number of primary risks to alliance survival. Some producers did not have enough cash on hand when the counterpart payment fell due. In Salar region, drought and soil erosion obliged some producers to temporarily seek work elsewhere, limiting the time they could devote to the alliance subproject. Some of the alliances with a predominantly female or indigenous makeup had been approved even though the members lived far from towns and had few links to markets, contrary to the project’s 18 principle. Other reasons for failure included weaknesses in the producer organization, demoralization following longer-than-usual lags between alliance approval and the release of project funds, lack of experience with the alliance product, and misunderstandings about the copayment obligation. All of these factors should, in principle, have been picked up in the various rounds of preapproval screening, and the alliances in question should not have been approved (Balderrama Mariscal 2011). Seven years after project funds were released, 47 percent of the producer groups were still working with the original buyer and, in 53 percent of cases, the original alliance had collapsed—meaning either that the producer group had partnered with a new buyer (30 percent) or had not formed a new partnership, reverting to the free market instead (23 percent). This finding was based on a random, statistically representative sample of 86 alliances (Morales 2015). In cases in which the initial alliance had ended, producers cited the low price offered by the buyer as the main reason, followed by failure to meet the volume target, either because the buyer could not afford to buy all that the producer wanted to sell, or because the producer could not deliver the agreed-on quantity to the buyer. Of the alliances that fell apart, 72 percent did so within a year of receiving project financing. There was a marked difference between the six project regions in the frequency with which producer groups continued to work with the original buyer. The rate was highest in Norte (75 percent) and Lago (69 percent), where, respectively, cocoa and milk are the main products. These products are both linked to well-organized, highly capitalized, and long-established marketing chains. Alliance survival rates were lower in the other four regions: 13 percent in Chaco, 24 percent in Tropico, 33 percent in Salar, and 50 percent in Valles. In these regions, a large share of producer groups was producing annual crops, where the scope for year-to-year substitution of product and/or buyer is higher. In preparation for the 2018 mission, IEG randomly chose producer groups from the database for four regions that it intended to visit (Lago, Valles, Salar, and Tropico). Regional project staff were asked to contact each producer group to find out if it was still operating and might be willing to participate in an IEG workshop (appendix C). Data on group survival were forwarded by two regional offices. In Tropico, 75 percent of the 32 sampled groups were still active. In Lago, 59 percent of the 29 selected alliances were fully operational; among milk-producing alliances, the proportion was 65 percent. Membership stability is another test of the resilience of producer groups. Fifty- four percent of the producer groups interviewed by IEG have either increased their membership since alliance setup or have the same number of members as at the start. 19 But membership had fallen overall in Tropico region, and quinoa-producing groups were more likely to have suffered a decline in membership than were milk-producing groups (appendix D, table D.8). Whereas the 2015 (six-region) survey found that 53 percent of producer groups had dropped the original buyer, in IEG’s more limited (four-region) survey the alliance had collapsed in only one-third of cases. The longer ago that producer groups received project funding, the more likely they were to have dropped the buyer with whom they originally formed an alliance. None of the 16 groups to receive the most recent round of funding (2011) had changed their buyer, however. In the municipality of San Julián (Tropico), all eight of the participating producer groups had changed their buyer; seven of these did so because they produced annual crops and had decided to switch crops and buyers in response to price trends. Other products had more stable alliances. In the case of milk, 22 percent of producer groups had changed their buyer, while all of the quinoa-producing groups stuck to their original buyer. The main reason for dropping the original buyer was dissatisfaction with the price offered (57 percent of cases)—in other words, at the time of sale, the price in the spot market was higher than the price negotiated earlier with the alliance buyer (appendix D, tables D.6 and D.7). Alliance survival depends substantially on the level of trust that grows between producers and buyers, a factor that was heartily acknowledged by workshop participants: “It’s like a marriage, after all,” said one participant. Trust was more important than the written agreement, which was co-signed by the producers and buyers forming an alliance—a contract that was not, in any event, legally binding and would have been too much trouble to enforce had either of the parties wanted to do so. The absence of postproject funding is always a threat to development outcome. The project design did not provide for revolving funds (which have a poor reputation in Bolivia) but some of the producer groups that IEG interviewed had made similar arrangements on their own initiative. Approximately one-fifth of producer groups had their own internal credit arrangements (including revolving funds), while one-half of producer groups helped their members secure loans from commercial banks (appendix D, table D.5). In Lago, producers told IEG that the banks reach out to them— such is their trust in the producer groups’ creditworthiness and capacity to supervise repayment. Little credit seems to be provided by buyers; the 2015 survey found that only 16 percent of alliance buyers offered this service to producers (Morales 2015), a finding that is consistent with IEG’s 2018 observations. Price shocks pose the most severe risk to alliances, and dissatisfaction with prices is the main reason producer groups break with the buyers with which they had 20 originally allied. In the IEG workshops, the most common products were milk (52 percent of the producer groups attending) and quinoa (18 percent). The producer price of both products has recently fallen sharply, and yet milk and quinoa producer groups interviewed by IEG have remained intact: All of those who attended workshops were still fully operational and the no-shows (groups that may have ceased operating) accounted for 44 percent of milk invitees and 41 percent of quinoa invitees. Since 2011, milk producers have been hit by a collapse in the world price, following a surge in exports from the European Union. Government price controls provided limited buffering to producers. Wholesalers in Bolivia substituted part of their domestic supply of fresh milk with cheaper imports of milk powder; the purchase quotas negotiated with Bolivian producer groups were cut. The government- administered farmgate price is now Bs 3.00 per liter, 17 percent lower than it was five years ago. Because of milk marketing oligopsony, there are few buyers of fresh milk from which producers can choose, which is the main reason why 78 percent of the milk- producing groups at IEG workshops had remained with the original buyer. However, as proof of their resilience, many producers have diversified into cheese and yogurt. Quinoa producers face greater price uncertainty than do milk producers. Based on demand in overseas markets (almost all quinoa is exported), the price per quintal (45 kg) surged to over Bs 1,500 in 2013 before dropping back to Bs 400 in 2018. Counterintuitively, the threat to the integrity of producer groups was arguably higher when prices peaked in 2013 than it is today. Quinoa is grown on communally owned land and when the price surged, many comuneros who had migrated in search of better livelihoods returned to demand that their claim to land be reinstated, threatening to break up producer groups strengthened by the project. Most of these producer groups appear to have survived, however, partly because there are few alternative farm products that can thrive in the resource-poor Salar region. Some of the producer groups interviewed by IEG are now taking steps to develop niche markets for the Salar’s distinctive quinoa variety (which is richer in amino acids than are competing Peruvian varieties), seeking to obtain controlled-origin status with the European Union. In the Tropico and Valles regions, the picture is altogether different. Soils in these regions are more fertile, producers are better off, there are more product alternatives, markets are more competitive, and the supply of annual crops is price elastic. In Tropico, 76 percent of the producer groups interviewed by IEG were no longer in the same alliance that the project had helped promote, and 38 percent had changed product line— switching, for example, from sesame to soybean and sunflower (appendix D, tables D.6 and D.7). Producers in this frontier region are entrepreneurial and the groups interviewed by IEG are mechanized and sophisticated—in these workshops, the need to 21 invest in water-conserving irrigation and to reduce pesticide contamination was frequently voiced. The alliance was never intended as an end in itself—what is important is whether the skills, knowledge, and experience generated by the alliance experience lead to a sustained improvement in producer incomes resulting from improved market access, an outcome that will survive the failure of the original alliance (Morales 2015). Of the 86 alliances surveyed, 64 percent of the producer groups perceived that alliances had a positive impact on household income; even among producer groups that had reverted to selling on the spot market (that were no longer in alliances), 55 percent reported that alliances resulted in higher household incomes. Seventy percent or more of producer groups also cited improved product quality, increased productivity, and strengthening of the producer organization as positive impacts of the alliance model. Interestingly, the impact to receive the lowest rating (51 percent of producer groups) was increased market access, which is central to the project’s statement of objective. However, overall, the 2015 data amount to a vote of confidence by producers in the alliance approach, and IEG’s interviews with producers and buyers in 2018 suggest that there was substantial enthusiasm about, and commitment to, the alliance model, with producers demonstrating solidarity with each other. Risk to development outcome is rated low. Bank Performance Quality at Entry 6.16 The project was designed jointly by World Bank team leaders and the Bolivian counterparts subsequently charged with national coordination; the closeness of their collaboration, and the quality of the lessons they derived from past operations, were vital to the development of a workable project design. From the beginning, the designers placed a premium on flexibility, and this fed through to the implementation phase, with producer groups being offered a broad menu of options, together with the technical guidance needed for them to choose the organizational forms and product procedures that worked best for them. Several judicious choices raised quality at entry. For example, the decision to place the financial analysis of all alliance proposals in the hands of an independent firm ensured uniformity of appraisal and protection from political interference. Project preparation included close consultation with leaders of producer and indigenous organizations, resulting in the development of an innovative and influential social analysis methodology that was vital for screening would-be project beneficiaries. Quality at entry is rated highly satisfactory. 22 Quality of Supervision 6.17 Supervision was characterized by continuity and thoroughness. There were only two task team leaders, and they worked closely during the handover period. There were never fewer than two missions per year; all missions included technical, fiduciary, and safeguards staff; and the stationing of a World Bank staff member in the La Paz office helped ensure follow-up between missions and a quick response to queries from national counterparts. Although there were no significant midcourse corrections to project design, operating procedures were continuously refined, benefiting from the well-designed management information system. The restructuring was handled expeditiously and contributed to the highly satisfactory outcome. Supervision is rated highly satisfactory. Bank performance is rated highly satisfactory. Borrower Performance Government Performance 6.19 Once the loan was made effective, there were delays in hiring staff for the national and regional offices because the project’s parent ministry was slow to sign agreements with regional governments that were in the hands of the political opposition. A ruling by the Bolivian Treasury that formal producer organizations must comply with tax rules reduced the incentive for groups to formalize, slowing the pace of implementation (because only formal producer groups were eligible to receive project financing). After 2007, there was a substantial increase in government support for the project, including the introduction of complementary policies and a new rural development strategy, that were conducive to the attainment of project objectives. Government performance is rated satisfactory. Implementing Agency Performance 6.20 IEG concurs with the completion report finding that the project’s substantial achievements were mainly the result of solid performance by the national and regional offices that were set up to implement it. The UCN was results focused and transparent. This unit was established at the right distance from the Ministry of Peasant and Agricultural Affairs (MACA): sufficiently insulated from interference in day-to-day operations meddling but close enough to respond to the minister when needed. One of the reasons for the project’s sound performance was the depoliticized process of recruiting project staff: An independent agency was placed in charge of finding and hiring staff, and only persons of the highest technical caliber were appointed. The same 23 agency was responsible for an annual, peer review-based evaluation of staff performance. Although there was significant rotation of staff in the regional offices— some of it disruptive—the same person remained in the post of national coordinator and was intimately involved with the project from the design phase onward, ensuring sound coordination of staff and close control over the project’s performance. This continuity was a major factor behind the project’s success. Implementing agency performance is rated highly satisfactory. 6.21 Overall, borrower performance is rated highly satisfactory. Monitoring and Evaluation Design A centralized management information system, with georeferenced data on each subproject, was generated by the six regional offices. Overall, the selection of performance indicators was judicious, although two were ambiguous: The increase in the wage rate and in the product price indicators were dropped during the implementation phase because neither could be directly attributed to the project. An impact evaluation was conducted in 2014, with a sample of project beneficiaries and matched controls that totaled 3,946 producers, drawn from four of the six project regions. Because beneficiaries self-selected into the project, they were possibly more enterprising and likely to succeed than nonbeneficiaries, even without the project intervention. It was therefore necessary to correct for this selection bias. As well, although there were ex ante data on beneficiaries (based on the feasibility studies prepared before alliance approval), in the absence of an adequate baseline study, there were no ex ante data on nonbeneficiaries. The designers of the study addressed this limitation by constructing a control group from participants in the follow-up project: producers with an alliance plan that had been approved for project financing but for which funds had not yet been released. The producers in the control group were selected based on observable characteristics similar to those of producers in the treatment group, using two different matching techniques. This was a robust approach that avoided the problem of imperfect nonbeneficiary recall of the situation ex ante. However, data were insufficient for any estimate of the factors responsible for the (substantial) observed increase in net income: It is not clear how much could be attributed to an increase in the volume sold, improved product quality, or more favorable trading terms. 24 Implementation The aide-mémoire and supervision reports show that the monitoring arrangements were fully implemented, with regular updating of the data on performance indicators and progress toward targets. Many supervision missions were preceded by special studies of particular regions or product lines, and the findings from these studies influenced the focus of the mission and the recommendations made. The management information system allowed for a regular flow of information between the central and regional offices, with an integrated database— “no separate, ‘orphan’ spreadsheets, no ‘phantom’ subprojects”—as the national coordinator told IEG. Particularly noteworthy are the detailed completion reports that were prepared for each subproject. Use Findings from the monitoring system and the evaluation studies did not lead to major design changes—either midcourse corrections to the project, or adjustments to the design of the follow-up project. However, the quantity and quality of data generated greatly enhanced subproject supervision by the national and regional project teams, helping to account for the success of the project. The findings of an evaluation report produced for the midterm review were used to make the case for scaling back the municipal investment component and focusing efforts on just one of the three alliance models proposed. Most importantly, the quality of project monitoring and evaluation was high enough to allow for a test of the alliance model—the objective of the project. A shortcoming of the test was that the extent to which the increase in net income of producers was attributable to the increase in the volume sold, improved product quality, or more favorable trading terms, could not be ascertained. Monitoring and evaluation quality is rated substantial. 7. Lessons In a country like Bolivia, where the productivity of small-scale producers is low and there is substantial scope for increasing sales to the domestic market, the first step for a productive alliance is to boost the quantity and quality of the marketed surplus. The impact of rural alliances will vary with the country context. In Bolivia, productive alliances successfully boosted the quantity and quality of marketed surpluses of small- scale farmers. Other alliance projects in Brazil, Colombia, and Central America show how the same model can be adapted to higher levels of market development (World Bank 2016). 25 Once producer groups are well organized, alliances can help producers obtain sustainable, postproject financing, enhancing the sustainability of the alliance arrangement. The project design promoted fiduciary responsibility by empowering producer groups to manage project funds directly, through systematic record keeping and public presentation of accounts. Producer groups then went on to set up revolving funds on their own initiative and also provided guarantees for commercial bank loans. Many alliance buyers also proved willing to advance cash and make loans to producers they had come to trust. Project management can be greatly enhanced when strict quality controls are applied by independent parties without political interference. In this project, staff selection procedures and performance evaluations were competitive and transparent. An independent firm was used to advertise positions and recruit staff, with every effort made to attract as broad a range of applicants as possible. Another independent firm evaluated staff performance each year based on mutually agreed-on individual work plans and beneficiary feedback. Subproject proposals were also independently vetted, a process that ensured rigorous assessment of feasibility, allowing for variations in context and producer organization capability. The rules were clearly defined at the outset, thereby reducing the scope for management to be compromised by political favoritism. Technical assistance works best when it is based on a flexible menu that accommodates the varied capacity building needs of different subprojects. The menu of technical assistance options in this project ranged from accounting, procurement, input purchase, negotiation with buyers, storage, processing, and packaging—all valid areas for skills development. However, the precise mix of technical assistance varies from one subproject to the next and is best specified when the subproject proposal is first elaborated. Agile disbursement of project funds enhances beneficiary commitment and increases the efficiency of subproject implementation. When payments are made directly from the government budget to the participants’ bank accounts, producers feel trusted and enabled, and have the means to respond flexibly to implementation challenges without having to cope with the problems caused by disbursement delays. In this project, transfers of public funds to producer organizations was formalized in the government’s budget law and channeled through the public sector’s financial management system, a model that can be replicated elsewhere. Having a knowledgeable national coordinator who helps design the project and provides long-term leadership greatly enhances the achievement of project objectives. A distinguishing feature of this project was that the initiative came from a national 26 counterpart familiar with rural Bolivia who brought his knowledge and skills to project preparation, working closely with World Bank staff to develop a project design that was pragmatic and flexible. The same person led the counterpart team for the duration of the first project and remained in charge for the follow-up operation, winning the trust and respect of the government while diligently supervising and remaining in close contact with the staff in six regional offices. 27 Bibliography Balderrama Mariscal, Carlos. 2011. “Evaluación del Proyecto de Alianzas Rurales (PAR): Evaluación de las Alianzas Abandonadas.” Informe Final, La Paz, Bolivia, December. Baldivia Urdininea, José. 2008. “Bolivia: Pequeños Productores e Inserción en Mercados.” Food and Agriculture Organization of the United Nations and World Bank, Rome, November. Faguet, Jean-Paul. 2001. “Does Decentralization Increase Responsiveness to Local Needs? Evidence from Bolivia.” Policy Research Working Paper 2516, World Bank, Washington, DC. MACA (Ministerio de Asuntos Campesinos y Agropecuarios). 2005. Estrategia Nacional de Desarrollo Agropecuario y Rural. La Paz, Bolivia: MACA. Monterrey Arce, Javier. 2016. Evaluación de Impacto del Proyecto Alianzas Rurales (PAR 1). La Paz, Bolivia. Morales, Miguel Angel. 2015. “Estudio sobre Condiciones de Sostenibilidad en las Alianzas Rurales.” La Paz, Bolivia. Salguero Lowenthal, José Alejandro. 2012. “Evaluación de Resultados Incrementales del Programa de Alianzas Rurales (PAR).” Rome, Italy: Food and Agriculture Organization of the United Nations. World Bank. 2000. “Bolivia—Rural Communities Development Project.” Implementation Completion and Results Report 20549, World Bank, Washington, DC. ———. 2003. Reaching the Rural Poor: A Renewed Strategy for Rural Development. Washington, DC: World Bank. ———. 2005a. “Bolivia—Rural Alliances Project.” Development Credit Agreement. Washington, DC: World Bank. ———. 2005b. “Bolivia—Rural Alliances Project.” Project Appraisal Document Report 32080, World Bank, Washington, DC. ———. 2005c. “Bolivia Rural Alliances Project: Social Analysis for Inclusion, Cohesion and Accountability (English).” Social Analysis Good Practice Dissemination Note Report 34059, World Bank, Washington, DC. ———. 2005d. Bolivia—Poverty Assessment: Establishing the Basis for Pro-Poor Growth. Washington, DC: World Bank. ———. 2006. “Bolivia—Indigenous Peoples Development Project.” Implementation Completion and Results Report 35911-BO, World Bank, Washington, DC. 28 ———. 2007a. “Bolivia—Participatory Rural Investment Project.” Independent Evaluation Group, Implementation Completion and Results Report ICRR0000125, World Bank, Washington, DC. ———. 2007b. World Development Report 2008: Agriculture for Development. Washington, DC: World Bank. ———. 2009. “Bolivia—Rural Alliances Project: Additional Financing.” Project Paper Report 47478-BO, World Bank, Washington, DC: World Bank. ———. 2011a. Bolivia—Country Partnership Strategy for FY2012–15. Washington, DC: World Bank. ———. 2011b. “Bolivia—Community Investment in Rural Areas Project.” Project Appraisal Document Report 59877, World Bank, Washington, DC. ———. 2012. Agricultural Innovation Systems: An Investment Sourcebook. Washington, DC: World Bank. ———. 2014. “Bolivia—Rural Alliances Project.” Implementation Completion and Results Report ICR3388, World Bank, Washington, DC. ———. 2016. Linking Farmers to Markets Through Productive Alliances: An Assessment of the World Bank Experience in Latin America. Washington, DC: World Bank. 29 Appendix A. Basic Data Sheet Bolivia Rural Alliances Project (P083051) Table A.1. Key Project Data Actual or Current Appraisal Estimate Estimate Actual as Percent of Financing ($, millions) ($, millions) Appraisal Estimate Total project costs 34.88 79.74 228 Loan amount 28.40 59.94 211 Table A.2 Cumulative Estimated and Actual Disbursements Disbursements FY06 FY07 FY08 FY09 FY10 FY11 Appraisal estimate ($, millions) 1.41 4.77 9.47 15.73 21.93 26.33 Actual ($, millions) 500 1.05 4.59 12.62 20.16 34.36 Actual as percent of appraisal 35 22.01 48.46 80.22 91.92 130.49 Table A.3. Project Dates Event Original Actual Concept review May 22, 2006 May 22, 2006 Board approval May 22, 2006 May 26, 2005 Signing Sept. 7, 2005 Closing date Sept. 30, 2011 Mar. 31, 2014 Table A.4. Staff Time and Cost World Bank Budget Only Staff time Costa Stage of Project Cycle (no. weeks) ($, thousands) Lending FY05 66.72 254,391.27 Total 66.72 254,391.27 Supervision or ICR FY06 21.06 32,439.75 FY07 23.27 38,868.21 FY08 21.93 83,982.87 FY09 23.74 79,258.92 FY10 27.40 88,012.71 30 FY11 10.38 28,913.79 FY12 9.29 52,983.91 FY13 37.55 136,804.04 Total 174.62 541,264.20 Note: ICR = Implementation Completion and Results Report. a. Including travel and consultant costs. Table A.5. Task Team Members 31 32 Appendix B. Analysis of Project Process Table B.1. Steps and Actors in the Project Process Actors Brokers Project Producer and Local Financial Evaluation Steps Staff Groups Buyers Facilitators Government Institutions Agencies 1. Disseminate X ? project guidelines 2. Formalize ? ? ? producer groups 3. Contract and X supervise alliance facilitators and brokers 4. Prepare alliance X X X X proposals 5. Mobilize ? ?a ?b complementary funds 6. Screen proposals X for eligibility 7. Evaluate alliance ? ? X X proposals 8. Choose proposals X ?a ?b to finance 9. Formalize X X X alliances 10. Implement X X X X Xa Xb subprojects 11. Monitor and X X evaluate 12. Adjust project ? design Source: Adapted from project appraisal document (World Bank 2005b). Note: Italics and ‘?’ indicate steps that were not spelled out in the project guidelines. a. The project included a fund (Productive and Social Investment Fund) that provided counterpart finance to local government infrastructure projects that complemented alliance subprojects. b. The project provided incentives to financial institutions to lend to unbanked producers and buyers using a special fund (Fondo de Desarrollo del Sistema Financiero y de Apoyo al Sector Productivo). 33 Questions Posed by the Independent Evaluation Group Assessment Step 1. Did the Regional project staff engage local government as partners in the dissemination process? Did they promote all three alliance models to the same degree? Step 2. Did Regional project staff devote resources to organizing and formalizing producer groups, or did they work mainly with existing organized/formal groups? How easy was it to formalize producer groups (for example, did government agencies facilitate)? Step 5. Did the Regional project staff encourage local governments and financial institutions to provide complementary funding (through the Productive and Social Investment Fund and Fondo de Desarrollo del Sistema Financiero y de Apoyo al Sector Productivo)? Step 6. Did the eligibility criteria, and the manner in which they were applied, target poor rural producers? Step 8. Was the existence of complementary funding (from local governments and financial institutions) taken into account by project staff when deciding which proposals to finance? Step 9. How binding were the alliance agreements that producer groups and buyers were required to sign? Step 10. How systematic and sound was the technical support to subproject implementation provided by project staff, brokers, and facilitators—including postimplementation follow-up? Step 12. How much was the project design adjusted to reflect findings from monitoring and evaluation? 34 Appendix C. Workshop Method In four of the six regions covered by the project, the Independent Evaluation Group (IEG) organized nine workshops for producers and buyers in May–June 2018. Of the 108 alliances randomly selected by IEG and invited to a workshop (listed in appendix E), 64 (59 percent) sent at least one producer representative. Adding in producers who supplemented or substituted for the invitees, 78 producers attended (table C.1). Each workshop was also attended by two to three buyers connected with the invited alliances. The dominant products were milk (accounting for 52 percent of the producers who attended the workshops) and quinoa (18 percent). Producers of beef, pork, chicken and llama meat, eggs, fruits, and vegetables were also represented. At the start of the workshop, each producer filled out a questionnaire. After this, the project staff and the IEG consultant facilitated a discussion among the group. A translator was on hand to assist participants from ethnic minorities whose first language was not Spanish. The IEG consultant separately interviewed the buyers who attended the workshop. Outside the workshops, the consultant also interviewed local government representatives, the project staff, and project facilitators who had provided technical assistance to producers that participated in the project. The producers who participated in the workshops received a modest stipend to cover transport costs and refreshments. Table C.1. Workshop Attendance (Producers) Workshops Total Region (no.) Municipalities Invited Attended Added Attendance Lago 2 Pucarani 26 16 4 20 Salar 2 Uyuni, Tomave 21 12 6 18 Valles 2 Sipe Sipe, Punata 29 15 2 17 Tropico 3 El Torno, San Julián, 32 21 2 23 Porongo, La Guardia Total 9 108 64 14 78 35 Appendix D. Workshop Results The number of members per producer group averaged 39, one-third of whom were women. The mean subproject size was the equivalent of $82,696, or approximately $2,000 per producer—a per-capita investment that is well above the norm for projects of this nature (table D.1). Although the producers’ share of subproject costs averaged 30 percent (consistent with project guidelines), there was significant variation between the four regions, ranging from 25 percent in Salar (the poorest region) to 38 percent in Lago (where closeness to the capital, La Paz, pushes up incomes, possibly increasing producers’ capacity to pay). There was a good response rate to the workshop invitations, with 59 percent of the producer groups choosing to send a representative. (All of the groups that attended were still fully operational—a survival rate of almost 60 percent.) There is some suggestion that older producer groups were less likely to accept IEG’s invitation to participate in a workshop, possibly because they were more likely to have ceased operating. Of the alliances funded by the project in 2010, the participation rate was 64 percent, while for those funded in 2007 (more than 10 years before IEG’s visit) the rate was 53 percent (table D.3). Approximately half of the producer groups appear to have been created explicitly to take advantage of project funding, with the others set up over a year before the prospect of funding was announced. A sizeable minority of producer groups (14 percent) had existed for over 10 years. There were sharp regional differences, with almost 40 percent of the producer groups in Valles having operated for more than 10 years, while there were no such groups in Salar. Quinoa-producing groups tend to be of more recent vintage than dairy product groups (table D.2). Two-thirds of producer groups had warehouse space where the alliance product could be received and stored. Nearly half had machinery and a dedicated space for product processing. Few of the groups had their own vehicles (table D.4). Almost all of the producer groups that participated in workshops provided technical assistance and training to their members, purchased inputs on behalf of the group as a whole, and supplied help with marketing and product promotion. Approximately one-fifth of producer groups had their own internal credit arrangements (including revolving funds set up at the initiative of the members), while one-half intervened with banks to help secure loans for their producers (table D.5). Overall, one-third of the groups had dropped the original buyer. The longer ago that producer groups received project funding, the more likely they were to have dropped 36 the buyer with whom they originally formed an alliance; none of the 16 groups to receive the most recent round of funding (2011) had changed their buyer. In the municipality of San Julián (Tropico), all eight of the participating producer groups had changed their buyer; seven of these did so because they produced annual crops and had decided to switch crops in response to price trends. In the case of milk, 22 percent of producer groups had changed buyer, while all of the quinoa-producing groups stuck to their original buyer (table D.6). The main reason for dropping the original buyer was dissatisfaction with the price offered (57 percent of cases): In other words, at the time of sale, the price in the spot market was higher than the price negotiated earlier with the alliance buyer (table D.7). Fifty-four percent of the producer groups interviewed by IEG have either increased their membership since alliance setup or have the same number of members as at the start. However, membership had fallen overall in Tropico region, and quinoa-producing groups were more likely to have suffered a decline in membership than milk-producing groups (table D.8). Table D.1. Basic Data on Alliances IEG Sample Total for Sample Whole Lago Salar Valles Tropico Total Project Alliances (no.) 26 21 29 32 108 768 Total investment 2,028,139 1,476,835 3,054,819 2,371,411 8,931,204 c. 44.2 million in alliances ($) Share of total 38 24 27 28 30 30 investment paid (mandated) by producers (%) Producers (no.) 782 893 1420 1130 4,225* 28,896 Mean investment 78,005 70,325 105,339 74,107 82,696 57,539 per alliance ($) Mean investment 2,594 1,654 2,151 2,098 2,114 1,529 per producer ($) Mean producers 30 43 49 35 39 38 per alliance (no.) Sources: Implementation Completion and Results Report (World Bank 2014). Note: The Independent Evaluation Group sample is derived from the project database. The total for the whole is derived from the Implementation Completion and Results Report. a. Thirty-four percent of all the producers in the sample were women. 37 Table D.2. Years before Alliance Startup that Producer Group Operated (no. [percent]) Region Product Lago Salar Valles Tropico All Milk Quinoa Less than 1 8 (50) 7 (58) 3 (23) 14 (67) 32 (52) 12 (44) 6 (60) year 1–10 years 6 (37) 3 (25) 5 (38.5) 7 (33) 21 (34) 9 (33) 2 (20) More than 10 2 (13) 2 (17) 5 (38.5) — 9 (14) 6 (23) 2 (20) years Total 16 (100) 12 (100) 13 (100) 21 (100) 62 (100) 27 (100) 10 (100) Sources: Project database; workshop questionnaires. Note: Refers to alliances randomly selected by IEG that participated in workshops and for which producer group origin date was available (n=62). Table D.3. Year that Alliance Received Funds from Rural Alliances Project Region Workshops (no. alliances) Attending Invited Difference Year Lago Salar Valles Tropico (no.) (no.) (percent) 2007 — 2 4 4 10 19 53 2008 — 1 2 8 11 20 55 2009 — 1 6 6 13 21 62 2010 9 2 1 2 14 22 64 2011 7 6 2 1 16 26 62 Total 16 12 15 21 64 108 59 Table D.4. Producer Group Facilities (no. [percent]) Product Processing Vehicles and Warehouse Machinery or Transport Facility Status Storage Space Equipment Have 42 (67) 31 (48) 4 (6) Don’t Have 22 (33) 33 (52) 60 (94) Total 64 (100) 64 (100) 64 (100) Source: Workshop questionnaires. Note: Tropico is an outlier: Only 24 percent of producer groups have a warehouse, compared with 67 percent of all producer groups. 38 Table D.5. Services Provided to Producers by Producer Group (percent) Producer Groups Offering this Service Service (N=64) Product processing 42 Product transport 11 Technical assistance 100 Training courses 98 Input purchase 95 Loans/rotating funds internal to producer group 22 Help in securing bank loans 53 Representation with government agencies 80 Support for product marketing 97 Source: Workshop questionnaires. Table D.6. Alliances with and without Change of Buyer Since Startup (no. [percent]) 2011 Start Lago Salar Valles Tropico Milk Quinoa up All No 15 12 9 5 21 10 — 43 (67) Change (94) (100) (69) (24) (78) (100) Change 1 — 4 16 6 — 16 21 (6) (31) (76) (22) (100) (33) Total 16 12 13 21 27 10 16 64 (100) (100) (100) (100) (100) (100) (100) (100) Source: Workshop questionnaires. Table D.7. Reasons for Alliance Buyer Dropout Producer Groups Citing this as Reason (N=21) Reason Stated (percent) Producer group could not meet quantity required by buyer 10 Price agreed with buyer lower than spot price at time of sale 57 Producer group failed to deliver on time 38 Product did not meet quality standards required by buyer 10 Problem in transporting product to buyer 24 Producer changed producta 38 Source: Independent Evaluation Group workshop questionnaires. Note: a. This reason applies only to producers of annual crops in one municipality (San Julián). 39 Table D.8. Membership Stability of Producer Groups Membership All Change since Lago Salar Valles Tropico (no. Milk Quinoa Alliance Setup (no.) (no.) (no.) (no.) [percent]) (no.) (no.) Increased 7 4 7 1 19 (30) 10 3 Unchanged 7 — 3 5 15 (24) 7 — Decreased 2 8 5 14 29 (46) 10 7 Total 16 12 15 20 63 (100) 27 10 Source: Independent Evaluation Group workshop questionnaires. 40 Appendix E. List of Persons Met and Alliances Invited to Workshops At the World Bank, Washington, DC David Tuchschneider, Task Team Leader, Rural Alliances Project Luz Berania Diaz Rios, Task Team Leader, Second Rural Alliances Project In Bolivia* Jhonny Delgadillo, National Coordinator, Rural Alliances Project, La Paz Marco Monasterios, Planning and Monitoring Specialist, National Coordinating Office, La Paz Ana Colgue Marrani, Buyer, Lago Region Jorge A. Guillen Wilde, Buyer, Lago Region Omar Carrasco, Municipal Secretary, Pucarani, Lago Region Juan Carlos Lopez Cantuta, Agricultural Officer, Pucarani, Lago Region Yesenia Apaza H., Buyer, Lago Region Mauro Marrani Alanoca, Buyer, Lago Region Rodolfo Apaya, Broker, Lago Region Susana Vargas, Broker, Lago Region Gladys Ticona G., Broker, Lago Region David Pari Flores, Alliances Project Officer, Lago Region Oscar Cazorla Villarpando, Alliances Project Officer, Lago Region Adalid Monje Arauco del Villar, Project Officer, Lago Region Juan Carlos Aroni M., Agronomist, Uyuni, Salar Region Remberto Morales Cruz, Buyer, Challavinto, Salar Region Nelson Perez Paco, President, Quinoa Growers Association, Salar Region Emilio Huaygua Ali, Alliances Project Officer, Salar Region Javier Tejada Inza, Alliances Project Officer, Cochabamba, Valles Region Adela Quiroz Mairana, Buyer, Valles Region Eliseo Camacho O., Buyer, Valles Region Rene Sanchez, Broker, Tropico Region Abilio Moreira Cruz, Broker, Tropico Region Willy Calderon Coca, Buyer, Tropico Region Gabriel Marrani Yucra, Buyer, Tropico Region Felicidad Terrazas Ponce, Buyer, Tropico Region Limberg Duran Paniagua, Buyer, Tropico Region Henry Garcia Gutierrez, Fisheries Specialist, Tropico Region 41 Nue Moron Carrasco, Buyer, Tropico Region Jorge Augusto Lopez, Buyer, Tropico Region Rodolfo Ayala Saavedra, Alliances Project Officer, Tropico Region *This list does include the names of producers who participated in the IEG workshops. Alliance ID Alliances Invited to Workshops LGO-1261-003-09 Asociación Piscícola Multiactiva “Tuni Ichucota” LGO-1261-026-09 Desarrollo Integral Lechero Iquiaca CENTRO LGO-1261-047-09 Asociación de Ganaderos Lecheros de la Comunidad Chuñavi (AGLECCH) LGO-1261-060-09 APALEP PRODUCE LGO-1261-061-09 ASOCIACIÓN DE PRODUCTORES LECHEROS AGROPECUARIOS LGO-1261-077-09 Producción y Comercialización de Huevos de Gallina Criolla LGO-1261-083-09 LLAMASUMITA ASOCIACIÓN DE PRODUCTORES ECOLOGICOS CHOJASIVI “APEC- LGO-1261-087-09 CHOASIVI” LGO-1261-105-09 Alianza de producción lechera de la comunidad Ancocagua LGO-1261-108-09 Asociación Productores Lecheros Agropecuarios Huarialtaya LGO-1261-111-09 Asociación Productores Lecheros de Seguenca II LGO-1261-113-09 APLECOTA – CAL DELIZIA LGO-1261-125-09 Fortalecimiento a la producción LECHERÍA–MUCUÑA LGO-1261-126-09 Desarrollo Lechero de Iquiaca LGO-1261-155-09 Asociación de productores de leche “San Carlos” LGO-1261-158-09 APLEMA LGO-1261-159-09 Asociación de Productores de Leche Módulo Cúcuta “ALECU” LGO-1261-160-09 APLEMCAU Asociación de Productores Lecheros Módulo Calería Uno LGO-1261-161-09 APLEK –ILPAZ LGO-1261-166-09 Asociación de Productores de Leche Unión Pucarani (APLUP) – PIL Andina S.A. LGO-1261-206-09 Asociación de Trabajadores Lecheros Agropecuarios Seguenca ATLA-S LGO-1261-212-09 Asociación de Productores Lecheros Asunción Catavi “APLEAC” LGO-1261-214-09 ASOCIACIÓN PRODUCTORES VILAQUE PAMPAJASI LGO-1261-215-09 APLEV PRODUCE LGO-1261-247-09 Módulo de Productores de Leche “CHOJÑACOLLO I” LGO-1261-298-09 APALIG UYU-1530-004-08 APROACAV—REAL ANDINA UYU-1530-011-09 ASOCIACIÓN DE PRODUCTORES DE QUINUA NORTE QUIJARRO UYU-1530-012-09 PRODUCCIÓN INTEGRAL AGROPECUARIO “AGUA DULCE” UYU-1530-013-09 APAINDIC—REAL ANDINA UYU-1530-014-09 APRA CHACALA—REAL ANDINA UYU-1530-017-07 EL PORVENIR Asociación de Productores de quinua Machacuyo (APROQUIMAC)—Real UYU-1530-022-09 Andina UYU-1530-026-07 APROA Q – ANAPQUI 42 UYU-1530-033-07 APRACOVES-QUINUA BOL UYU-1530-037-09 AIAQ-REAL ANDINA ASOCIACIÓN DE PRODUCTORES AGROPECUARIOS DE NUEVA UYU-1530-040-09 ESPERANZA UYU-1530-042-07 APRAVI–QUINOA BOL SRL. UYU-1530-051-09 AIPPAAL—REAL ANDINA UYU-1530-052-08 APQUICARM – LINO VELIZ UYU-1530-053-07 APROQUIRGAC-PROANBOL SRL. Asociación Regional Indígena de Productores Integral Agropecuario Coroma UYU-1530-060-09 ARIPIAC UYU-1530-101-07 Sociedad Indígena de Productores Agropecuarios Vinto (Coroma) UYU-1531-000-06 BOLIVAR–ASCEX UYU-1531-031-07 ALIANZA AGROPECUARIA TOMAVE UYU-1531-035-07 ALIANZA DE PRODUCTORES FORRAJEROS Y GANADEROS. UYU-1531-041-09 AIPAKC-ANAPQUI CBB-1303-138-08 Asociación de módulos lecheros Cochabamba “AMLECO” CBB-1303-143-08 Asociación agropecuaria y agroindustrial Santa Rosa "ASAROSA" CBB-1303-152-08 Asociación de lecheros de Mallcochapi CBB-1303-154-08 Asociación de módulos lecheros Señor de Santiago “AMLESA” CBB-1303-165-08 Asociación agrícola lechera y sus derivados de Viloma—Cochabamba (ALVICO) CBB-1303-168-08 Asociación de productores agropecuarios Quiroz Rancho (APAQ) CBB-1303-174-08 Asociación módulo lechero Huañakawa (AMLH) CBB-1303-182-08 Asociación de productores pecuarios Montenegro zona 1—Sipe Sipe (APPMSS) CBB-1303-185-08 Asociación Porcinocultores Valle Bajo (APOVAB) CBB-1303-226-08 Asociación módulos lecheros del valle bajo “AMLEVB” CBB-1303-244-09 Asociación de Productores de Cebolla del Valle Central y Bajo (ASOPROC) CBB-1303-283-09 Asociación Integral de Productores Agropecuarios Coachaca (AINPAC) CBB-1303-291-09 Asociación de productores de leche Sorata (ASOPROLES) CBB-1303-340-09 Asociación de Porcinocultores Suticollo (APS) CBB-1315-002-06 Asociación de productores lecheros del Valle Alto “APLVA” CBB-1315-040-07 Asociación de productores de leche Villa Rosario (ADPPLEVR) CBB-1315-048-07 Asociación de productores de leche Punata “APLP” CBB-1315-054-07 Asociación de pequeños productores de leche APPL “Tambillo Grande” CBB-1315-055-07 Asociación de pequeños productores de leche APPL “Valle de Punata” CBB-1315-056-07 Asociación de pequeños productores de leche APPL “Señor de Milagros” CBB-1315-063-07 Asociación de pequeños productores Khuska Purina CBB-1315-087-07 Asociación de pequeños productores de ganado porcino El Rosal CBB-1315-122-07 Asociación de lecheros agropecuarios Punata CBB-1315-196-08 Asociación de productores de leche Wiñay Causay (APLWC) CBB-1315-199-08 Asociación de productores de cebolla de Punata “APROCEP” CBB-1315-225-08 Asociación de Productores de leche Nueva Esperanza CBB-1315-231-09 Asociación de Productores Lecheros Sivingani CBB-1315-271-09 Asociación Agropecuaria Productiva Punata (AAPP) CBB-1315-351-09 Asociación de Productores de Durazno Distrito II Punata (APRODD) SCZ-1705-068-07 Nuevo Amanecer de los Apicultores de Villa Florida SCZ-1705-070-07 Las Mujeres Exitosas Productoras de Carne de Cerdo en Jorochito 43 SCZ-1705-074-07 Asociación Solidaria La Purita SCZ-1705-075-07 Leche La Forestal SCZ-1705-087-07 Junta Pirai SCZ-1705-088-07 Asociación de pequeños Productores de ganado “Los Hornos” SCZ-1705-108-07 Asociación de Pollos Parrilleros 12 de abril SCZ-1705-117-07 Asociación de Productores de Ganado de Leche SCZ-1705-120-07 Asociación de Pequeños Productores de Ganado “Villa Paraíso” SCZ-1705-152-07 Alianza de Cítricos León SCZ-1739-007-08 Asociación de Productores Agropecuarios “Los Cóndores” SCZ-1739-008-08 Asociación Integral de Productores Agropecuarios de San Julián (AIPAS) SCZ-1739-009-08 Producción y Comercialización de Frejol Negro SCZ-1739-017-07 Asociación De pequeños Productores Agropecuarios del Oriente APPAO SCZ-1739-027-07 Producción y Venta de Cooperativistas SCZ-1739-132-07 Abriendo Sendas SCZ-1739-141-07 Asociación de Productores Agropecuarios 15 de mayo SCZ-1739-142-07 Asociación de Agricultores “21 de agosto” Cultivo de Maíz San Julián SCZ-1739-143-07 Productores de Sésamo 12 de mayo (San Julián) SCZ-1739-144-07 Asociación de Pequeños Productores “SINCHIHUAYRA” de Maíz SCZ-1703-007-09 Granja de Chanchos Porongo SCZ-1703-054-07 Agropecuaria “Patriota” SCZ-1703-061-07 ASAPAI—APIBSA SCZ-1703-075-08 Producción y Venta de Leche Porongo SCZ-1703-082-07 Sindicato Agrario El Chorito 2 de agosto Agua Dulce SCZ-1703-097-09 Asociación de Pequeños Productores Agropecuarios Sombrerito SCZ-1704-053-09 Asociación 25 de octubre La Guardia SCZ-1704-058-07 ASAPIGUARDIA-APIBSA SCZ-1704-110-07 Productores de Mani—Totorales SCZ-1704-111-07 Asociación de Avicultores 23 de mayo SCZ-1704-112-07 Producción Lechera “Los Colonos” SCZ-1704-114-07 Lechería “Naranjillos” Note: Alliances in bold are those that participated in Independent Evaluation Group workshops; the other organizations were invited but did not participate. 44