IEG Report Number: ICRR14802 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 09/11/2015 Country: Bolivia Project ID: P083051 Appraisal Actual Project Name: Bo Rural Alliances Project Costs (US$M): 34.88 79.74 L/C Number: Loan/Credit (US$M): 28.40 59.94 Sector Board: Agriculture and Rural Cofinancing (US$M): Development Cofinanciers: Board Approval Date : 05/26/2005 Closing Date: 09/30/2011 03/31/2014 Sector(s): General agriculture; fishing and forestry sector (70%); Agricultural extension and research (20%); Agro-industry; marketing; and trade (10%) Theme(s): Rural markets (40%); Rural services and infrastructure (40%); Rural policies and institutions (20%) Prepared by: Reviewed by: ICR Review Group: Coordinator: Ebru Karamete J. W. Van Holst Christopher David IEGPS1 Pellekaan Nelson 2. Project Objectives and Components: a. Objectives: The project development objective (PDO) stated in the Development Credit Agreement (p.22 ) was "to test a model to improve accessibility to markets for poor rural producers in the Pilot Areas by: (a) promoting strategic productive alliances between different economic players at the local level; (b) empowering rural producers through the development of self-managed grass-root organizations; (c) increasing access to productive assets and technology; and (d) promoting more effective, responsive and accountable service organizations at the local level". The PAD statement of the project development objective was "to test a model to improve accessibility to markets for poor rural producers in selected sub-regions of the country" (p. 4). This review uses the Development Credit Agreement version of the objective to assess this project’s achievements. The PAD defined a rural alliance as an “economic agreement between a group of small rural producers and other economic actors, in which all contribute and gain, thus assuring the continuity of the agreement in the medium and long term" (p. 33). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project had three components: 1. Institutional Support (Appraisal Estimate : US$3.33 million, Estimate at Additional Financing : US$5.29 million, Actual: US$5.14 million ). The component aimed at establishing institutional capacity development of small producer organizations to become partners in the new marketing agreements with the commercial partners. The main outputs were the formation of rural productive alliances and the preparation of viable alliance plans. The component financed technical assistance and training for the following specific activities: project dissemination campaign, calling for proposals, pre-feasibility and feasibility studies and their evaluation and approval; and the formalization of the alliances. 2. Implementation of Rural Alliances (Appraisal Estimate : US$26.63 million, Estimate at Additional Financing : US$57.50 million, Actual: US$65.67 million). This component supported the implementation of the rural alliances prepared under the first component. Project financing was provided for works, goods and services for the implementation of producer’s sub-projects up to the storage stage (maximum of approximately USD 1,500 per participating household and the producer organizations to contribute at least 20% of the alliance investment costs in kind or cash). In addition, support was provided for agencies to facilitate access to credit for both producers and purchasers when required; and for local governments which decide to support alliances through the design and construction of public infrastructure (in collaboration with the Productive and Social Investment Fund). The Finance Enhancement Initiatives sub-component was dropped during the additional financing, as it became clear that no specific incentive was needed for promoting access to credit. In addition the scope of Municipal Infrastructure sub-component was reduced as fewer alliances needed complementary public works investments. 3. Project Management (Appraisal Estimate : US$4.91 million, Estimate at Additional Financing : US$9.16 million, Actual: US$8.93 million). The component financed technical assistance, goods, equipment and incremental operating costs for the establishment and operation of a project coordination team in Ministry of Peasant Affairs and Agriculture (MACA) and its successors--the Ministry of Rural Development, Agriculture and Environment (MDRAMA) and the Ministry of Rural Development and Land (MDRT). In addition funds were provided for setting up and operation of a management information system, the implementation of monitoring, evaluation and learning arrangements, and the completion of technical studies. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs: Total costs increased from the appraisal estimate of US$34.88 million to US$ 79.74 million because an additional financing was added to the project in order to scale up operations into two new areas. Financing: The original IDA grant for US$28.40 million was fully disbursed. Additional financing of US$30.00 million was added in April 2009 in order to scale up project activities into two new sub-regions of the country. By project closing, US$59.94 million had been disbursed, exceeding the original and additional financing estimates. Borrower Contribution : There was no Borrower contribution. It was expected at appraisal that beneficiaries would provide US$6.48 million. The actual contribution from the beneficiaries was considerably higher at US$19.80 million. Dates: On April 25, 2009, with the additional financing, the original closing date of September 30, 2011 was extended by 2.5 years to March 31, 2014 in order to complete the activities in the two new activity areas that were funded by the additional financing. Restructuring: The additional financing introduced revisions to the following project outcome indicators without changing the PDO, and IEG’s assessment is that these changes do not necessitate a split evaluation: (i) Outcome indicators “increase in income per day of labor” and the “increment in the sale price of product” were eliminated, since both indicators depended mostly on factors outside of project control. Nevertheless these indicators were not closely linked to measuring the outcome of the project. (ii) The expected values of the “growth in income of rural productive units” and the “Increase in the volume marketed per rural productive unit” were modified from an accumulated end-of-project increase of 54% to an accumulated three-year increase of 45% per alliance, and this was a different way of measuring change in the outcomes. (iii) The indicator “number of financed alliances with an internal rate of return greater than 12.5%” was added, in order to provide another measure of achievement. This indicator was not relevant before the Mid Term Review as most alliances had not been established then, thus a split rating was not needed. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial The project development objectives were substantially relevant to the country priorities and strategies. Increasing small scale farmers' access to markets would mean economic opportunities and increased incomes for small farmers, which was fundamentally needed in rural areas. The Government reforms in the 1990s and early 2000 did not result in the expected rural growth and increases in rural incomes. Laws enacted in the mid to late 1990s were intended to encourage peasant and indigenous rural production through the transfer of resources to local governments for productive investments. The National Dialogue Law of 2000 strengthened incentives to local governments to support rural production through: (a) the creation of the Municipal Productive Councils at the municipal level; (b) eligibility for support to productive initiatives from the Highly Indebted Poor Countries (HIPC 11) resources; and (c) support for productive investments and public productive infrastructure from the Productive and Social Investment Fund (FPS) as part of the National Compensation Policy which defined policies and priority actions for poverty reduction (PAD, p. 3). In the early 2000s, the contribution of agriculture to GNP remained at about 16%, and the contribution of small producers who focused mostly on the domestic markets, stagnated (ICR p. 1). The incidence of rural poverty in 2002 increased to a startling 83% (from 78% in 1997) and extreme poverty to 67% (from 58 % in 1997), and it was also substantially higher than in urban areas (CAS FY 04, p. 2). The project development objectives were aligned with the Bolivian national development strategy . The project was designed by a different Administration that also prepared the National Rural Development Strategy (ENDAR). ENDAR's main objective was to increase rural incomes and employment through the provision of tools and knowledge to increase market accessibility in a sustainable way. The new administration, which was elected in 2005 and took office in 2006, was supported by a broad coalition of indigenous and peasant movement, therefore, was very much committed to increasing income opportunities for the peasant and indigenous groups, through productive investments. The ICR (p. 6) noted that the new Administration introduced rural development policies fully aligned with the Project’s objectives; particularly EMPODERAR (Productive Initiatives for Rural Development Program) was created by the Ministry of Rural Development and Land as part of its Rural and Agrarian Revolution Agricultural Sector Plan whose main line of action was to improve access to markets, production and competitiveness on the part of poor rural producers (ICR, p.9). The project objectives were relevant to the Country Assistance Strategy (CAS for FY 04-08) at appraisal and to the Country Partnership Strategy (CPS for FY 12-15) at closing. Although the project was not specifically mentioned in the CAS 04, the objectives were aligned with the CAS aim of addressing rural poverty issues and particularly the thematic area of productive development of micro small and medium enterprises. This project was included in the recent CPS (FY 12-15) and the project objectives were aligned with the CAS objective of reducing extreme poverty in rural areas by increasing agricultural productivity. b. Relevance of Design: Substantial The design components were in general consistent with the stated development objectives. The Project’s design was to test three alliance models as well as to test rural infrastructure enhancements for market access. Project design allowed for tailored support for these initiatives, however, they were demand driven thus would only be implemented if there was a demand. The alliance models to be tested were (a) co-finance small producer groups (up to storage) to improve market accessibility; (b) support small producer groups (up to processing) to improve market accessibility; and (c) support small producer groups and their market agents (PAD, p. 9). The models would establish formal linkages between producer groups and buyers and the alliance agreements would specify the rules by which both parties would act. Component 1 supported establishment of productive alliances and development of alliance plans. Component 2 provided funds to the members of producer organizations for implementation of sub-projects, as well as for public infrastructure projects to local governments that tried to enhance access to markets. Through the sub-projects for the producer organizations the project was designed to test the three models using bottom-up processes. During implementation there was demand only for first model. The design also aimed to strengthen producer organizations' capacity to organize and negotiate by financing a network of marketing service providers. According to the PAD (p. 72), the poverty targeting was achieved mainly through selecting areas that had high density of poverty, focusing on local markets, empowering organizations of poor producers, supporting market and service agents to increase partnering with the poor, and establishing rules for the selection of beneficiaries that favor the poor, the indigenous and women. The ICR provided no information on how well these criteria worked to ensure that "poor rural producers" were prominent among the project's beneficiaries. The PAD’s original results matrix provided a partially adequate analysis of the results expected from the project, as some outcome indicators were not very relevant (i.e. income and jobs generated). Nevertheless the rest of the results matrix was more relevant and measured adequately the results from the project’s implementation. 4. Achievement of Objectives (Efficacy): The project objective was to test a model to improve accessibility to markets for poor rural producers in the Pilot Areas. The three elements in this objective (testing, improved access to markets, and a focus on poor rural producers) will be addressed separately below. The project was designed to test three alliance models for improving the marketing of agricultural production and to provide support for complementary municipal infrastructure. On the basis of the demand from producer groups, the first alliance model (sub-projects up to storage level) was preferred. Consequently models 2 and 3 were dropped during implementation. The demand for the municipal infrastructure investment was also lower than anticipated. However, the ICR noted that there is now some emerging demand for model 2 in the follow-up project, as producer organizations build internal management capacity and increase post-harvest activities (p. 9). The following intermediate outcomes were defined in the PAD (p. 4): (a) promoting strategic productive alliances between different economic players at the local level; (b) empowering rural producers through the development of self managed grass root organizations; (c) increasing access to productive assets and technology; and (d) promoting more effective, responsive and accountable service organizations at the local level. They represented core activities in rural alliances and therefore provided an important framework for testing the feasibility and effectiveness of the preferred alliance model. (a) Promoting strategic productive alliances between different economic players at the local level .  The project launched 4 information campaigns (invitations to submit proposals), achieving below the revised target of 8 campaigns (ICR p. 32).  The project carried out 17 promotion workshops aimed at women groups, exceeding the target of 12 workshops (ICR p. 32).  Feasibility studies were performed for 1,403 alliances (above the revised target of 806) (ICR p. 32).  1,137 alliance plans were selected to be supported (above the target of 375) (ICR p. 33).  The project carried out 39 feasibility studies for the complementary infrastructure investments, failing to meet the revised target of 42 feasibility studies (ICR p. 33). (b) Empowering rural producers through the development of self -managed grass-root organizations;  The project supported 752 producer groups in their organization (above the revised target of 752). (ICR p. 32)  The project trained 12,072 alliance members on the operation of alliances, i.e. organization, management and negotiation skills (exceeding the revised target of 4,050 members) (ICR p. 33).  602 producer groups (out of 768 that were financed) maintained their agreement with the marketing agent (ICR p. 33).  19 alliances that required public infrastructure obtained it (but this was below the revised target of 42 alliances) (ICR p. 33). (c) Increasing access to productive assets and technology .  The project financed 768 alliances (above the revised target of 675) and 28,896 households benefited, amount per family given was US$1,529. 48% of alliances were for agricultural crop production and 45 percent were for livestock production (ICR p. 35).  The ICR reported that average crop yields increased by some 45% during implementation (ICR p. 12), but no information on years or control group comparisons were provided.  58% of producers were applying Integrated Pest Management methodology (less than the target of 80%) (ICR p. 33). It was not clear if this was due to the project.  187 were farms under organic production (no target was set on this) (ICR p. 33). It was not clear from the ICR if this was due to the project.  The project also included complementary infrastructure investments by municipalities in order to increase access to markets. The ICR (p.11) reported that, the municipal infrastructure implementation was slow, due to the limited direct interest of municipality managers in such investments, as well as slowness in Social and Productive Investment fund (FPS) to channel the government funding to the municipalities. The scope was reduced with the approval of the Additional Financing, from a target of 18,000 beneficiaries to 12,000 and the target number of sub-projects was 42 and only 19 were financed by closing. The project supported improvement of eight rural roads, the construction of eight bridges and three irrigation schemes. (d) Promoting more effective , responsive and accountable service organizations at the local level .  The project developed a network of service providers. It hired 608 brokers to provide support with feasibility studies and implementation of sub-projects (ICR, p. 32).  1,402 facilitators assisted producers in aspects of production, marketing and management of producer organization. These service providers are also working in the 2nd Phase project, selling their services to the producers (ICR, p.12).  The project recruited a consulting firm as an independent evaluator of the alliance business plans (ICR, p. 12).  However, no indicator measured the effectiveness, responsiveness and accountability of these service organizations. On the basis of the above, this Review concluded that testing of the model (the first element of the project's objective) was substantially achieved. Outcomes: There were two expected outcomes from this project, namely: (a) improved access to markets; and (b) targeting of project benefits at poor rural producers. (a) Improving access to markets - Substantial.  Based on the project's M&E system, the average increase in marketed produce reached 60% (in two years) compared to the target of 45% (over three years). However there was no control group comparison for this improved outcome (ICR, page 11) .  According to the impact evaluation agricultural sales of alliance producers were between 29 and 39% higher than those for a control group in 2014 (ICR, p. 10).  The project also supported municipal infrastructure investments to improve market access, though the ICR noted that there was no indication that the municipal infrastructure investments had a clear positive effect on overall market access (p. 10). (b) Targeting of project benefits at poor rural producers - Modest  The ICR did not explicitly report on how much of the project benefits were targeted to poor rural producers.  The ICR (p. 17) reported that: “The project has most likely also had a positive impact on the depth of poverty in the rural areas by providing additional income to the poorest quintiles of rural society”. It is also stated on page 18 that "90% of direct beneficiaries identify themselves as belonging to an indigenous group: 35% Quechua, 41% Aymara, 9% Guarani and 5% other indigenous groups. Bolivia’s rural indigenous and small farmers are over-represented amongst the very poor".  The targeting at the regional or community level does not guarantee that the project benefits actually went to the poor and are not captured by the elite in those areas or communities. This matter is also closely linked to the effectiveness of the eligibility criteria and selection method used for the project beneficiaries and it is not clear from the ICR, how it was ensured that poor rural producers actually benefitted from the project.  The project team subsequently shared with IEG, the highlights from the Impact Evaluation, which concluded that “Producers who have larger areas of land used for agricultural crops are on average 8.2 percentage points more likely to be beneficiaries of the project”. This is indeed one indication that PCDP II was less likely to benefit poor rural farmers who typically have small areas of land used for agriculture. The Impact Evaluation therefore made an important contribution to the testing of the model. In addition , the ICR provided achievements in terms of increased incomes. Accordingly:  Average agricultural income of project beneficiaries (of the farmer groups) at the project's close was 28-37% higher than for a control group. In addition the operations of 96% of alliances generated internal rates of return that were higher than the discount rate of 12.5% (Economic Analysis)  Increased income, as well as number of alliances with IRRs greater than 12.5%, both of which exceeded the targets (ICR, Data Sheet, p. vi).  New wage earning jobs created were (308,506 man-days) in the project area which was only 38% of the target 809,590 man-days (Impact Evaluation) 5. Efficiency: Substantial. Ex-ante economic analysis estimated results for 3 different project models as well as municipality investments. The analysis included indicative models of potential alliances and estimations were based on secondary information from ongoing development projects and financial institutions. Sub-project benefits included increased market sales, increased production for family consumption, cost savings and reduction of post-harvest losses. Sub-project costs included purchased goods and services, contracted labor and family labor. Flows of net benefits were calculated for 15 years, considering the expected life of main investment items. Model 1 analysis included results for organic quinoa as well as several annual and permanent crops. Economic NPV and IRR for model 1 alliances, ranged between US$9,000-16,500 and 15-17% respectively; and the financial NPV and IRR for model 1 alliances, was on average US$33,500 and 23% respectively. Including other two model estimates and municipal investments the average Financial Rate of Return and Economic Rate of Return was 17 % and 16% respectively (PAD p. 14). The ex-post financial and economic evaluation of the project was carried out based on the data registered in the Geo-referenced Management Information System of the project. As there was no implementation of the two models, the estimates were based on Model 1 only. The analysis included nine sub-sectors (milk, quinoa, coffee, cattle, pig breeding, cocoa, potato seed, peach and sesame) that reportedly dominated the investment. These represented 72.45% of the investment in alliances, involving 535 small producers’ sub-projects (70% of the total number of sub-projects) and 70% of the direct participants. No analysis was conducted on the municipality infrastructure investments. The ICR reported that (p. 37) the results and impacts of the investments were analyzed at the sub-sector level and at project level as a whole. Within each sub-sector, economic parameters and indicators were estimated for each rural alliance as well as for the average alliance and beneficiary family. In order to estimate the aggregate impacts at the sub-sector level, the incremental net income flows for the total number of alliances in the analyzed sub-sector were added, and relevant aggregated indicators calculated. To estimate aggregated impacts at project level, the aggregated net income flows for each sub-sector analyzed were added and aggregated feasibility indicators were estimated. In terms of financial analysis, considering all project costs against only the incremental net benefit derived from the 535 analyzed producers’ subprojects, the NPV and FIRR were reported as USD 40.4 million and 26% respectively. A 20% reduction of income level to account for risk factors (technical or social/organizational aspects) and climate or market related variability would result in a NPV of USD 18.57 million and FIRR of 21%. For the 535 analyzed alliances, assuming all project costs against only the incremental net benefit derived from the analyzed producers’ sub-projects, the NPV and EIRR using "economic" prices" were USD 31.29 million and 25% respectively; and a 20% reduction of income level would result in a NPV of USD 13.59 and IRR of 19% (ICR page 13). Also, the ICR reported that 96 % of all alliances had an IRR higher than the discount rate of 12.5 % (p. iii). Nevertheless, there were some shortcomings with the economic analysis methodology: (i) it was not clear from the ICR if the alliances/sectors included in the analysis were randomly selected. Furthermore, although the economic analysis used data from the M&E database, which included 100 % of the sub-projects, it was not clear why 30 % of sub-projects were left out from the analysis; (ii) the details of the benefits and costs streams were not provided and (iii) the costs did not account for incremental family labor, as it had not been recorded in the M&E system, however no estimation was included for that in the economic analysis. No administrative/operational inefficiencies were reported in the ICR. Also, project management costs were moderate compared to similar projects, with only 12% of total project costs. Even though there was lack of clarity regarding the sample selection and methodology of the economic analysis, due to high economic rates of return, even with 20 percent reduction in income levels, and no administrative and operational inefficiencies, this Review rates the overall efficiency of the project is rated as substantial. a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 16% 100% ICR estimate Yes 19% 75% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of the objective is rated substantial, due to its alignment with country priorities and World Bank assistance strategies. The relevance of design is also rated substantial as design components are in general consistent with the stated development objectives with the caveat that project design was not clear cut about identification and monitoring of poor producers. The project substantially achieved its objective of testing a model to improve accessibility to markets, and focus on poor rural producers. However, while the outcome for improved accessibility of markets is rated substantial, the outcome in terms of the focus on poor rural producers in this project is rated modest. Although there was some lack of clarity on the sample selection and the methodology for the economic analysis, the overall efficiency of the project is rated as substantial due to significant rates of return and no administrative/operational inefficiencies. Overall there were minor shortcomings in the project's relevance, achievements and efficiency. The project's outcome is therefore rated satisfactory. a. Outcome Rating: Satisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR considered that the risks to development outcomes are low because of the high level of alliance sustainability (75 % of Producer Organizations still partnered with their original commercial partner, and 100 % of Producer Organizations continued their operations) as well as the economic advantages for producers, as evidenced by the high marketed products and higher incomes for producers. According to the beneficiary survey with randomly selected 86 alliances, the commercial buyers also reported benefits from their perspective such as guaranteed product supply and product quality (ICR p. 46). The ICR also reported that there is no direct evidence to assess if sustainability of access to markets is lost for producers who are no longer allied with buyers, but information from the beneficiary survey indicate that market access is less of a risk than market risk (ICR, para 3.6.2) Finally, the initial success of the follow-up project is an additional motivation for existing alliances to continue and also facilitates upgrading the alliances’ operations through credit and technical assistance. The project also established an institutional network for service delivery through agents, this should also contribute to the sustainability of the project model. Although, for the existing alliances no access to finance mechanisms were established under the project, the follow up project included support for access to credit as well as stronger capacity building component that would support sustainability of the alliances. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry: The ICR noted (p. 19) that the project preparation efforts included a social assessment that included consultations with the leaders of peasant and indigenous organizations. The Project was based on a sector analysis and lessons learnt from previous operations in Bolivia (particularly the Rural Communities Development (1995-1999) and the follow up Participatory Rural Investment (1998-2006) projects) that focused on empowerment of rural communities, municipal strengthening and investments in rural infrastructure as well as similar projects in Colombia and Brazil. Some of the important lessons that were incorporated into the project design were (PAD p. 7): (i) The recognition that the main achievements by municipal government programs in the productive front have been in providing support for public infrastructure to support production, e.g. roads. (ii) Producer organizations for small producers in Bolivia have limited capacity in management, which led to the inclusion of project activities to build the capacity of these grass-root organizations in marketing and business. (iii) The recognition that to promote rural growth, in addition to supporting agriculture production, measures to address land degradation, over exploitation of the natural resource base and reduction in soil fertility was also needed. The wider approach of this project on rural production as a whole, was aimed at addressing this. Also, the sub-project selection favored environmentally friendly projects and integrated pest management was promoted. (iv) Bolivia had poor service delivery and technical assistance in the rural sector, the project tried to address this by allowing the producers to procure the TA themselves as part of the alliance plan, and by providing support in the identification of service providers in Bolivia and abroad, as needed. The project development objectives were relevant but too output oriented. The design components were consistent with the stated development objectives, however lacking in design was the identification and monitoring of poor producers as project beneficiaries. In terms of M&E there were a couple of shortcomings: no indicator measured poverty targeting and impact even though 'poor rural producers' was to be targeted, by the PDO. Another issue was that an institutional capacity development of producer organizations indicator in order to measure the Project’s institutional impact, was lacking . The Project was initially implemented by the Ministry of Peasant and Agricultural Affairs (MACA) which set up a National Coordination Unit (UCN) and Regional Operational Units (UOR) in the three project areas. At the Ministerial level, the Project has formed the basis for its rural program executing unit, in charge of implementing donor- and government-funded programs focused on economic opportunities of rural peasant and indigenous producers. The moderately satisfactory rating is due to issue with inadequacy of targeting of poor beneficiaries within the project design and the shortcomings in the M&E framework regarding some of the outcome indicators. Quality-at-Entry Rating: Moderately Satisfactory b. Quality of supervision: The ICR reported that (p. 19) over the project implementation period there were two supervision missions annually, supported by technical, fiduciary and safeguards staff as needed. The task team leader was stationed in La Paz, which allowed for constant communication by the Bank and engagement in all aspects of project implementation. The ICR claimed that supervision missions identified several issues that helped overcome operational challenges, but no explanation was provided vis-a-vis what these challenges were and how they were resolved. The project team subsequently explained that: "The technical and fiduciary discussions were based on extensive field visits. The Government and Bank teams constantly asked what could be improved and how could be simplified. Some examples were: the progression of instruments for buyer assessment; the decision to move to a strictly cash-based counterpart system; the adoption of the PO report, countersigned by the PO members as a key fiduciary support document; the refinement of ex ante social and environmental evaluations; the evolution of the alliance support functions; the design of the MIS". The ICR also noted that the supervision team instituted a permanent process of improvement and self learning on administrative, financial, procurement, business, and environmental, social, planning and monitoring aspects, and the task team subsequently explained that this was endogenous and not prompted by outside agents or processes. A shortcoming that the ICR did not mention was that an adequate system for the identification, monitoring and tracking for poor rural producers (the intended project beneficiaries) was not established during supervision, in collaboration with the relevant government ministries, and surprisingly not even at the time of the additional financing. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The ICR noted that (p. 5), President Mesa’s government led the formulation of the rural development strategy, which supported the preparation of the project. Although the effectiveness was delayed during President Rodriguez’ caretaker administration (2005-2006) by objections from the Ministry of Finance due to hesitations about financial transfers to communities and poor-producer groups; this characteristic later became one of the pillars of GOB’s rural development policy. The ICR also reported that after an initial period of hesitation and some resistance to the project staff recruitment procedures, the Administration became fully engaged in the project and provided full support during implementation (p. 19). Government Performance Rating Satisfactory b. Implementing Agency Performance: The ICR (p. 21) noted that, overall the project implementation units (UCN, UORs) were efficient, stimulated innovation and incorporated lessons learnt during implementation, therefore rated as Satisfactory. The details are as follows: (i) The National Coordination Unit (UCN) provided operational guidance, quality assurance on technical, economic, social and environmental aspects, as well as fiduciary control. It provided effective project management function by implementing operational, administrative and monitoring aspects. It also focused on results and transparency. (ii) The Regional Coordination Units (UORs) implemented the Project activities in the field including calls for proposals, data collection, as well as supporting producer associations throughout the alliance cycle. The ICR reported that their was generally solid, but uneven at times, due to rotation of staff as a result of low salary levels and distant office locations. (iii) Social and Productive Investment Fund -FPS, which managed the municipal infrastructure part of the Project performed slower implementation because FPS increased its own portfolio five times and therefore had less control over the Project investments. Also, FPS required financial commitments from the Municipalities, who sometimes lacked the funds, which caused further delays. Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The monitoring and evaluation system included the establishment and operation of an Information System to monitor project implementation, procurement and financial audits. It also comprised impact evaluation and beneficiary survey. However, the design of the M&E Framework had some weaknesses: although the PDO was mainly about testing the project model to improve accessibility of markets for poor rural producers, some of the outcome indicators were related to higher level objectives that the project could contribute to, such as: income increase and generation of wage earning jobs. In addition, no indicators, monitoring or tracking system were developed to measure the impact on 'poor rural producers' - the target group for this project. Lastly, the ICR noted (p. 7) that, an indicator to measure institutional capacity development of producer organizations could have been useful to measure the Project’s institutional impact. b. M&E Implementation: The ICR reported that (p. 7), the Management Information System became a fully-fledged Geo-referenced system, which would help the project Implementation Unit to generate a wealth of data on process and results. For the impact evaluation a baseline survey was carried out, however, the sample was too small to generate a control group to measure impact. To work around this problem the baseline survey for the follow-up project (PAR II) was used to estimate the counterfactual outcomes for the project. There was no assessment in the ICR of the reliability of this approach. Also, the ICR reported that there were some initial problems in data collection because the technical staff did not regularly update the M&E system and there were inconsistencies in data collection. The initial monitoring by the producer organizations themselves was also weak as the monitoring committee members were insufficiently trained and did not appreciate the importance of project monitoring. However, over the years, the the M&E system became a more reliable information base. c. M&E Utilization: The ICR did not report on utilization of M&E data. Nevertheless, the M&E data were used to monitor project progress. Also, the economic analysis in the ICR was based on M&E data. M&E Quality Rating: Modest 11. Other Issues a. Safeguards: According to the PAD (p. 18), the Safeguard policies applicable to the Project were Environmental Assessment (OP/BP/GP 4.01), Natural Habitats (OP/BP 4.04), Pest Management (OP/BP 4.09), Cultural Property (OP 4.11), Forests (OP/BP 4.36) and Indigenous People (OD 4.10). The ICR omitted the Natural Habitats safeguards policy. In addition the ICR reported very limited information on the implementation of the safeguards. The ICR reported that (p. 8) no cultural property issues were identified, and the supervision missions rated safeguards as Satisfactory or Moderately Satisfactory in all Implementation Supervision Reports. The Mid-Term Review identified some environmental issues with the application of Integrated Pesticide Management techniques, which were properly addressed in the remainder of the Project. The ICR also noted that (p. 18) the project success in the implementation of the Indigenous Peoples safeguard (OP/BP 4.10) is demonstrated by the fact that 90% of direct beneficiaries identify themselves as belonging to an indigenous group: 35% Quechua, 41% Aymara, 9% Guarani and 5% other indigenous groups. No additional information is provided by the ICR regarding on the details of these or on the compliance with all the other policies. b. Fiduciary Compliance: The ICR reported that (p. 8) financial management was rated Satisfactory and the audits were unqualified. There was an issue regarding mismanagement of funds by a leader of a producer organization and the project team took immediate action and a court trial was ongoing by the time the project was closed. The ICR on page 22 noted that: " The Project established a model in which transfers of public funds to producer organizations became explicitly provided in the Budget law and) were processed through the country’s financial management system. This allowed payments/disbursements to be made directly from the Treasury Single Account to beneficiaries’ bank accounts, opened and maintained in formal financial sector institutions. Such arrangement allowed for a more transparent, efficient and agile flow of funds". In terms of procurement, the ICR mentioned that the ex-post procurement audits did not identify any important procurement issues and rated the performance Satisfactory (p. 8). The producer organizations had a substantial role in procurement, under the guidance of Regional Operating Units. The procurement specialists of the national and regional offices trained the fiduciary staff of each producer organization responsible. The ICR on page 21 noted that the project linked social accountability and formal controls, i.e. approval by organization members of tranche execution reports (executed activities, procurement and expenditures) in open assemblies was the basis for Project fiduciary control. This was a sound method that provided not only fiduciary control but also social accountability. c. Unintended Impacts (positive or negative): d. Other: Gender Issues: There were no specific indicators used to measure gender aspects. Monitoring gender specific indicators together with poverty impact would provide rich analysis on the linkages between poverty and gender. However, the ICR reported that, according to the Borrower’s Completion Report, 32% of the members of producer organizations were women and 27% of the representatives in the alliance management and monitoring committees were women (p. 17). The project also organized 17 sub-regional workshops specifically promoting the participation of women in alliances. 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome: Bank Performance: Satisfactory Moderately Based on lack of identification, Satisfactory monitoring and tracking of poor rural producers as beneficiaries in the project design, and failing to address this shortcoming during supervision. Consequently both Quality at Entry and Quality of Supervision are rated as moderately satisfactory in this Review . Borrower Performance : Satisfactory Satisfactory Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR provided useful lessons, the main lessons with some reformulation of language are as follows:  Use of self-selection instruments transparently helps to (i) ensure that the producers who join an initiative have the necessary resource endowment , and the risk profile for engaging in the innovation processes; (ii) avoid over-dimensioning of the investments . The project had the requirement that 30% of the alliance investment cost to be covered by the participants in cash and upfront in every tranche. This provided a self selection mechanism for the eligible beneficiaries. In addition, the participants’ contributions ensured that the alliances remains profitable from their point of view. Hower, it is important that counterpart requirement is publicized clearly at the outset.  Formal accountability mechanisms and documentation requirements can be significantly strengthened and simplified by the use of and reliance on social accountability mechanisms within the producer organizations . The project experience showed that approval by the producer organization members for the executed activities, procurement and expenditures, in open assemblies provides a sound basis for fiduciary control. Also, social control is enhanced if the producers provide cash contributions themselves.  In order to encourage the further growth of the producer organizations after alliance financing ends , a new line of financing should be available to producers so as to graduate them from donor support and to facilitate their access to credit . This requires developing capacities of producer organizations to provide specific services to its members after the end of the Project, including, as in the case of the Colombia Rural Productive Partnerships Project (P104567), the possibility of implementing revolving funds for member credit.  Women’s participation requires differentiated support . Thirty two percent of the producers in alliances were women but their representation at the producer organization directorate levels remains low. To improve access to project opportunities by women it is necessary to deploy a wider range of targeted activities, including radio messages in indigenous languages at adequate times, separate workshops for women and women’s groups, more intensive technical assistance during the whole alliance cycle, and targeting women in leadership training activities.  Producer organization capacity building needs to take into account business needs . Capacity building of organizations should not only focus on general managerial requirements (accounting, procurement) but also be tailored to the specific functions each organization assumes in the alliance. For example training needs on operating collective infrastructure (e.g. storage, processing and packaging), as well as provision of technical assistance services to the producer organization members can also be needed. This requires specific knowledge and support and should be included in the alliance plan.  Use of country public financial management systems increases transparency and efficiency . The Project established a model in which transfers of public funds to producer organizations became explicitly provided in the Budget law and) were processed through the country’s financial management system. This allowed payments/disbursements to be made directly from the Treasury Single Account to beneficiaries’ bank accounts, opened and maintained in formal financial sector institutions. Such arrangement allowed for a more transparent, efficient and agile flow of funds. This ICR Review finds the following lessons:  When the PDO includes targeting poor rural producers , the project design should include identifying , monitoring and tracking the poor beneficiaries . However, if the alliance model is more suitable for better off producers, who have the capacity or risk profile required for entering into market-driven partnerships, as indicated by the ICR in the 'Lessons Section", then PDO needs to be adapted accordingly.  Commercial buyers need to participate more in the design of alliances , and receive more financial support to ensure continuity of partnerships . Some producer organizations found that their increased productive capacity (including that to innovate) surpassed their current buyer’s capacity. Private sector participation in the design of the alliances (commodity type, location) could have increased their relevance and thus led to more sustainable partnerships. Also, commercial partners can be supported via financial resources to increase their capacities stemming from additional supplies provided by the small farmers. In addition, brokerage services for organizations that outgrow their current alliance arrangements; and increased outreach to potential buyers outside the local spheres of most organizations, could be other measures. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR had a good formulation and rich discussion on lessons but there were several shortcomings:(i)There was a lack of analysis on the poverty targeting and poverty impact of the project. (ii) There was a lack of information on the economic analysis methodology, particularly regarding sample selection, (iii) Bank and Borrower performance sections lacked details and examples. (iv) There was lack of information in the ICR on safeguard compliance. a.Quality of ICR Rating : Satisfactory