Document of The World Bank Report No: ICR00002843 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-73270 IBRD-77320) ON A LOAN IN THE AMOUNT OF US$ 54.35 MILLION AND AN ADDITIONAL FINANCING IN THE AMOUNT OF US$30.0 MILLION TO THE STATE OF BAHIA FOR A BAHIA STATE INTEGRATED PROJECT: RURAL POVERTY July 28, 2014 Agriculture Global Practice Brazil Country Management Unit Latin America and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective January 31, 2014) Currency Unit = R$ 1.00 = US$ 0.414 US$ 1.00 = R$ 2.413 FISCAL YEAR ABBREVIATIONS AND ACRONYMS AF : Additional Financing ARM : Arrangements for Results Monitoring BFAF : Bahia Family Agriculture Forum BNB : Bank of Northeast Brazil BNDES : National Development Bank CA : Community Association CAR : Regional Development and Action Company (State Technical Unit) CDD : Community Driven Development CEDRS : State Sustainable Rural Development Council CEPI : State Indigenous Peoples’ Council CET : State Territorial Council CMDS : Municipal Sustainable Development Councils CMDRS : Municipal Councils for Sustainable Rural Development CODEVASP : Company for the Development of the Sao Francisco Valley CONTAG : National Confederation of Agricultural Workers COPIBA : Bahia State Council for the Rights of Indigenous Peoples CPS : World Bank Country Partnership Strategy CTGA : Technical Committee for Environmental Assurance EA : Environmental Assessment EBDA : Brazilian Agricultural Development Enterprise FECAMP : Economic Foundation of University of Campinas, Sao Paulo FIRR : Financial Internal Rate of Return FUNAI : National Foundation for Indian Affairs HDI : Human Development Index HDI-M : Municipal Human Development Index IFAD : International for Food and Agriculture Development IICA : Inter-American Institute for Cooperation in Agriculture INCRA : National Institute for Colonization and Agrarian Reform IPPF : Indigenous Peoples’ Participation Framework IRPAA : Regional Institutional for Small-scale Agro-livestock Properties ISR : Implementation Status Report JSDF : Japanese Social Development Fund MC : Municipal Council MDS : Federal Ministry of Social Development M&E : Monitoring and Evaluation MIS : Management Information System O&M : Operation and Maintenance OP : Original Project PAA : Federal Food Acquisition Program PAC : Project Community Schemes PGRH : Water Resources Management Project PNAE : National School Lunch Program POA : Project Annual Operating Plan PPA : State Multi-Year Investment Program PROAGUA : Federal Water Resources Management Program RF : Results Framework RPAP : Rural Poverty Alleviation Program/Project RPRP : Rural Poverty Reduction Program/Project SACC : Monitoring and Administrative Management System, State of Bahia SEAGRI : Secretariat of Agriculture, Livestock, Irrigation SEBRAE : National Service for Micro and Small Enterprise SEDIR : State Secretariat of Development and Regional Integration SEMARH : State Secretariat for Environment and Water Resources SENAR : National Rural Apprenticeship Service SEPLAN : State Secretariat of Planning SETRE : Services and Training Enterprise SUAF : Superintendent for Family Agriculture TC : Territorial College/Council TF : Trust Fund UNICAMP : University of Campinas Vice President: Jorge Familiar Country Director: Deborah Wetzel Senior Global Practice Director: Juergen Voegele Practice Manager: Laurent Msellati Project Team Leader: Maria de Fatima Amazonas ICR Team Leader: Maria de Fatima Amazonas BRAZIL Bahia State Integrated Project: Rural Poverty (P093787) CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes .......................................................................................... 14 4. Assessment of Risk to Development Outcome......................................................... 22 5. Assessment of Bank and Borrower Performance ..................................................... 23 6. Lessons Learned ....................................................................................................... 24 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 25 Annex 1. Project Costs and Financing .......................................................................... 26 Annex 2. Outputs by Component ................................................................................. 28 Annex 3. Economic and Financial Analysis ................................................................. 45 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 52 Annex 5. Beneficiary Survey Results ........................................................................... 54 Annex 6. Stakeholder Workshop Report and Results................................................... 62 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 64 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 71 Annex 9. List of Supporting Documents ...................................................................... 72 MAP: IBRD 41028 A. Basic Information Bahia State Integrated Country: Brazil Project Name: Project: Rural Poverty IBRD-73270, IBRD- Project ID: P093787 L/C/TF Number(s): 77320,TF-10751/TF- 58071, TF P118988 ICR Date: July 15, 2014 ICR Type: Core ICR Lending Instrument: SIL Borrower: STATE OF BAHIA Original Total USD 54.35M Disbursed Amount: USD 84.35M Commitment: Revised Amount: USD 84.35M Environmental Category: B Implementing Agencies: SEPLAN/CAR, then SEDIR/CAR Co-financiers and Other External Partners: N/A B. Key Dates Revised / Actual Process Date Process Original Date Date(s) 02/09/2006 (OP) 02/09/2006 Concept Review: 12/20/2004 Effectiveness: N/A (AF) 01/24/2011 05/29/2007 02/05/2010 Appraisal: 02/15/2005 Restructuring(s): 07/27/2012 07/31/2013 09/01/2005 (OP) 03/17/2008 (OP) 11/13/2007 (OP) Approval: Mid-term Review: 06/23/2009 (AF) 03/26/2012 (AF) 07/19/2012 (AF) OP: 07/31/2010 Closing: 07/31/2010 AF: 01/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: MS Government: MS Implementing Quality of Supervision: MS MS Agency/Agencies: Overall Bank Overall Borrower MS MS Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) General agriculture, fishing and forestry sector 36 30 Other social services 18 5 Sub-national government administration 18 10 Transmission and Distribution of Electricity 16 - Water supply 12 55 Theme Code (as % of total Bank financing) Indigenous peoples 17 3 Municipal governance and institution building 17 2 Participation and civic engagement 33 10 Rural services and infrastructure 33 85 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Deborah Wetzel Letitia A. Obeng (Acting) Sector Manager: Laurent Msellati John Redwood Project Team Leader: Maria de Fatima Amazonas Luis O. Coirolo ICR Team Leader: Maria de Fatima Amazonas ICR Primary Author: Anna Roumani F. Results Framework Analysis Project Development Objectives (PDO, from Project Appraisal Document) The objective of the original project (OP), as expressed in the PAD of 2005, was “to increase social and economic opportunities for the rural poor in Bahia by improving their access to basic, socio-economic infrastructure, thus contributing to the Borrower’s objective of increasing the HDI”. This was to be achieved by: “(a) integrating with and/or complementing activities of other Bank loans for the State (under the Bahia State Program), particularly the PGRH and the proposed urban project, Viver Melhor Urbano II; (b) strengthening the cross-sector integration of and providing significantly more emphasis on education, health, culture, natural resources management and environmental sustainability; and (c) scaling up the impact on poverty by using the skills, social capital and experience of the Municipal Councils and community associations to improve the relevance, efficiency, sustainability, targeting and outcomes of non-project State and Federal investments in rural Bahia.” Note: The PDO as expressed in the OP Loan Agreement was: “To increase social and economic opportunities for the Municipalities’ rural poor by improving their access to basic social and economic infrastructure and thus contributing to the Borrower’s objective of increasing the HDI.” Revised Project Development Objectives (as approved by original approving authority) An Additional Financing (AF) loan of US$30.0 million approved by the Board on June 23, 2009 stated the PDO as expressed in the OP Loan Agreement, but revised the Results Framework. The ICR uses the statement of PDO in the Loan Agreements throughout. For clarity, the Results Indicators and Intermediate Outcome Indicators below are listed separately for the OP and AF. Further, the AF Results Framework which revised the PAD original is used to measure achievements of both the AF and retroactively, the OP. The precedent for this approach is established in supervision Aide Memoires under the AF where key indicators are shown – using the 2009 revisions - for both stages. The original RF is considered inadequate to measure the OP PDO, although it is included below (to comply with ICR Guidelines) and measured. The Main Text Section 2.3 explains the design aspects of the original Results Framework and relationship to the Arrangements for Results Monitoring. All listed indicators below are from the Results Frameworks presented in the PAD and Project Paper respectively. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Original Project (OP): Indicator 1 : 2,500 subprojects (SP) implemented and maintained by community associations Value Revised to 1,658 SPs quantitative or Zero (at appraisal) 2,500 SP 1,950 (see benefiting 151,978 Qualitative) below) families Date achieved 02/15/2005 07/31/2010 02/05/2010 07/31/2010 Substantially achieved: OP achieved 1,658 SP, 85% of formally reduced target (1,950) due to increased SP costs (inflation and exchange rate movements). The Comments PAD Arrangements for Results Monitoring (ARM) substituted: “% of rural poor (incl. % self-selecting into Produzir, identifying and implementing subprojects” as the achievement) Results Indicator. Output for the substituted RI: 24.5% of total rural poor families in Bahia self-selected into the OP, identifying and implementing SPs. Additional Financing (AF) revised PDO Indicators: Indicator 2 : Evidence of improved living conditions at the beneficiary household level Disease reduced; Value manual labor No target quantitative or Zero reduced; drought established Qualitative) resistance improved. Date achieved 06/23/2009 07/31/2013 01/31/2014 Exceeded: Aggregate OP/AF shows WSS and cistern investments reduced Comments waterborne disease and improved wellbeing for 54% of beneficiaries: better (incl. % water quality (95%); improved drought resistance (96%); decreased hepatitis (9% achievement) to 1%), schistosomiasis (13% to 5%), parasites (42% to 23%) and diarrhea (46% to 35%). Manioc mills reduced food insecurity (see Unicamp 2014). Proportion of productive investments generating full-time employment for Indicator 3 : beneficiaries Value Manioc mills and quantitative or Zero No target agricultural Qualitative) mechanization. Date achieved 06/23/2009 07/31/2013 01/31/2014 Substantially achieved: 94% 785 productive SPs were financed of which manioc mills (MM) and tractors/equipment (TE) were 67%. Productive Comments investments were intended as a source of supplementary income, and jobs were (incl. % not typically full-time. TE and MM (524 SP) created total 1,050 jobs (operators/ achievement) maintenance work). Also, water/sanitation SPs, about 6% of total, also created about 150 jobs (one O&M/SP). Better indicator would have been: “Family and contracted labor, temporary and/or permanent, resulting from productive SPs”. Indicator 4 : IRRs for productive subprojects exceed 12% 12% or above for 26% of agricultural Value mechanization SPs; quantitative or Zero IRRs exceed 12% manioc mills and Qualitative) honey IRRs negative. Date achieved 06/23/2009 07/31/2013 01/31/2014 Partially achieved: IRRs reached or exceeded 12% for 26% of agricultural mechanization investments but were negative for manioc mills and honey due Comments mainly to severe drought. See Annex 3. Activity models prepared for the AF (incl. % (sampling OP productive SPs) suggested reasonable to high IRRs and BC ratios: achievement) rural electrification (IRR 14%, B/C 1.2); water supply (IRR 21%); manioc mills (IRR 23%, B/C 1.4); milk processing (IRR 35%, B/C 2.4); tractors (IRR 39%, B/C 2.0). Min. 60% of MCs deliberating resource allocation from programs outside Indicator 5 : the project MCs, through participation in Value MCs played no Territorial Councils quantitative or significant role in Min 60% (TC), leveraged Qualitative) resource allocation non-project resources. Date achieved 06/23/2009 07/31/2013 01/31/2014 Partially achieved: Unicamp (2014), CAR (2014), Aide Memoires/other sources indicate that political support for the MCs in their traditional form, Comments declined in favor of territorial development policies/forums. Resource allocation (incl. % from non-project sources was successful, the dominant mechanisms being CAR’s achievement) partnerships with State/Federal programs, but MCs within the TCs, the TCs themselves and diverse social movements also leveraged resources. See Main Text 2.2, 3.2 and Annex 2). Ratio of funds leveraged to project funds increases over the life of the Indicator 6 : project Produzir I leveraged Value about US$5 of non- OP: 1:1.1 quantitative or OP/AF target 1:5 project funds for every AF: 1:5 Qualitative) US$1.00 of project funds. Date achieved 06/23/2009 07/31/2013 01/31/2014 Comments Achieved: Ratio of funds leveraged to Loan funds increased over life of the (incl. % project from R$120.0 m. under OP (about US$57.0 m.) to R$293.5 m under AF achievement) (about US$147.0 m.). Ratio of 1:1.1 (OP) and 1:5 (AF). Participatory decision-making being used to target and allocate non-project Indicator 7 : resources Territorial Participatory forums were Colleges/Councils Value used but not the with MC quantitative or Municipal Councils, No target representatives Qualitative) whose role declined from participating, 2006 onwards. leveraged non- project resources. Date achieved 06/23/2009 07/31/2013 01/31/2014 Achieved: The primary leveraging mechanism was CAR’s institutional partnerships with other Federal and State programs. Within this framework, the Comments participatory Territorial Colleges and Bahia Family Agriculture Forum for (incl. % example, supported CAR in leveraging financing from INCRA (land reform), achievement) Ministry of Agrarian Development (food security), as well as CODEVASF and the Ministry of Social Development (reservoirs and cisterns) for the Project’s target population. See Main Text 2.2 and 3.2, and Annex 5. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Original Project (OP): Component 1 Indicator 1 : MIS registers subproject proposals and tracks their implementation MIS registered SP Value MIS established under proposals and (quantitative Produzir I, registered and No target tracked their or Qualitative) tracked about 3,000 SPs. implementation Date achieved 02/15/2005 07/31/2010 07/31/2010 Comments (incl. % Achieved: Dropped by AF. achievement) Indicator 2 : Periodic evaluation, Bank and CAR supervision missions Value Evaluation and (quantitative Zero No target supervision or Qualitative) conducted Date achieved 02/15/2005 07/31/2010 07/31/2010 Achieved: This indicator was achieved but measured only the supervision and Comments M&E process without capturing Component 1 or PDO. Dropped by AF. Final (incl. % Evaluation (UNICAMP 2014) and Borrower Completion Report (2014) were achievement) prepared. Bank/Borrower supervision missions were conducted. Indicator 3 : CAR semi-annual reports Value Semi-annual reports (quantitative Zero 6-monthly prepared by CAR or Qualitative) throughout Date achieved 02/15/2005 07/31/2010 07/31/2010 Comments (incl. % Achieved: Dropped by the AF. achievement) Component 2: Indicator 4 : MIS tracks number and type of training events Value MIS was already No target MIS tracked # and (quantitative established under type training events or Qualitative) Produzir I, and tracking project activities Date achieved 02/15/2005 07/31/2010 07/31/2010 Achieved: Indicator was a process and was dropped by AF. Actual training: (i) Comments 468 events for 13,173 beneficiaries (OP), and 411 events for 14,112 beneficiaries (incl. % (AF); (ii) 88 events for 1,491 CAR technicians (OP), and113 events for 2,144 achievement) CAR technicians (AF). Types: management, tech/operational; institution- building; water management; socio-productive inclusion. Indicator 5 : Periodic evaluation: Bank and CAR supervision missions Bank and CAR Value supervision (quantitative Zero No target missions evaluated or Qualitative) project performance and progress Date achieved 02/15/2005 07/31/2010 07/31/2010 Comments (incl. % Achieved: Dropped by AF. achievement) Indicator 6 : CAR semi-annual reports Value Semi-annual reports (quantitative Zero No target prepared by CAR or Qualitative) Date achieved 02/15/2005 07/31/2010 07/31/2010 Comments (incl. % Achieved: Dropped by AF. achievement) Component 3: Indicator 7 : MIS monitors subproject cost, type, geographic distribution and relation to HDI-M MIS monitored SP MIS established under Value cost, type, previous project but not (quantitative No target geographic monitoring relation to or Qualitative) distribution and HDI relation to HDI Date achieved 02/15/2005 07/31/2010 07/31/2010 Achieved: This indicator was achieved but is process-related. PDO wording is Comments “HDI” not “HDI-M” (the latter feeds into HDI). Relationship to HDI implies (incl. % categorizing subprojects in terms of whether they contributed to income achievement) generation, health and/or education, HDI’s major components. This was done. Indicator 8 : Periodic evaluation: Bank and CAR supervision missions Value Bank and CAR (quantitative Zero No target supervision or Qualitative) missions conducted Date achieved 02/15/2005 07/31/2010 07/31/2010 Comments (incl. % Achieved: This indicator was achieved but is process-related. Dropped by AF. achievement) Indicator 9 : Evaluation framework Value Previous project had Final Evaluation, (quantitative established MIS and No target BCR completed or Qualitative) conducted evaluation Date achieved 02/15/2005 07/31/2010 07/31/2010 Comments Partially achieved: This indicator is process-related, and meaning is unclear. (incl. % Dropped by AF. achievement) Additional Financing: Component 1: Indicator 10 : Min. 800 community subprojects implemented AF: Reduced Value 1,658 under OP (OP to 500 SPs OP: 1,658 (85%) (quantitative closing date used for 800 SPs (July 2012 AF: 1,037 (207%) or Qualitative) baseline) Restructuring) Date achieved 06/23/2009 07/31/2013 07/27/2012 01/31/2014 Substantially achieved under OP (85%) AF target exceeded (207%): Comments Aggregate 2,695 SPs financed vs. 2,450 targeted (110%). Original OP target of (incl. % 2,500 formally-reduced to 1,950, and 85% achieved. AF target reduced to 500 achievement) but achieved 207% due to leveraged funds. OP shortfalls due to cost inflation (construction materials and labor) and fluctuating USD/R$ exchange rate. Indicator 11 : Min. 320 communities benefited in Area A and 480 in Area B Value Total 1,658 SPs under Target min. 320 AF: 1,037 total of (quantitative OP, of which 389 (Area Area A and 480 which 261 (Area A) or Qualitative) A) and 1,269 (Area B) Area B and 776 (Area B) Date achieved 06/23/2009 07/31/2013 01/31/2014 Substantially achieved: Under AF, Area A achieved 82% of target and Area B Comments 162%. Under OP, Area A target groups received 26% of SPs and Area B 74% (incl. % (no explicit target was set for # of SPs). Applying AF targets to OP, Area A achievement) achieved 122% of target and Area B 264% Productive subprojects are min. 35% of total, and generate incremental Indicator 12 : employment and income Value Productive 30.5% (quantitative 28.3% (OP) 35% productive (AF) or Qualitative) Date achieved 06/23/2009 07/31/2013 01/31/2014 Substantially achieved (AF): 30.5% No target was set for OP but achieved Comments 28.3%. Aggregate 785 productive subprojects of which tractors and manioc (incl. % mills generated incremental employment for 1,050 operators/maintenance achievement) workers (est. 2 jobs per SP). Unicamp (2014) reported increased incomes for 54% of manioc mill beneficiaries and 45% of tractor beneficiaries. Indicator 13 : Women receive minimum 30% of all productive subprojects Women Value Women averaged 30% met/exceeded target (quantitative under OP (various types 30% depending on type or Qualitative) of investments) of SP Date achieved 06/23/2009 07/31/2013 01/31/2014 Comments Met/Exceeded: Under AF, manioc mills (women 70%); tractors/equipment (incl. % (women 30%); other SPs (women 35-40%) achievement) Min. 50% reduction in incidence of water-borne disease in communities Indicator 14 : with water supply subprojects Value Results reported for end- Reductions in major Min. 50% (quantitative project only, via diseases and other reduction or Qualitative) evaluation surveys benefits Date achieved 06/23/2009 07/31/2013 01/31/2014 Met/Exceeded: Final Evaluation UNICAMP (2014): Water quality improved Comments (96%) with following results: Hepatitis (9% to 1%); Schistosomiasis (13% to (incl. % 5%); Parasites (41% to 23%); Diarrhea (46% to 35%). Resistance to drought: achievement) 96%. Physically taxing time spent collecting water: 7.2 hours to 0.8 hours/week. Component 2: Indicator 15 : 80% of participating Community Associations (CA) have benefited from at least one training/TA course Value 80% with min. 1 (quantitative 100% under OP 100% AF training/TA course or Qualitative) Date achieved 06/23/2009 07/31/2013 01/31/2014 Comments (incl. % Exceeded: 125% of target, both stages achievement) Indicator 16 : 80% of MCs have participated in at least one training/TA course Value (quantitative 100% of existing MCs 80% of MCs 65% or Qualitative) Date achieved 06/23/2009 07/31/2013 01/31/2014 Substantial achievement under both OP and AF: 267 mobilization and training events (mainly on the new Op. Man., community organization, investment priority-setting and HDI) benefited 100% of Municipal Councils Comments (MC) and 7,154 members soon after OP effectiveness. 270 MCs were (incl. % restructured out of 416 municipalities (65%) and all participated in several achievement) training events. Also, about 350 MCs participated in Territorial Development events on the PPA, Economic and Environmental Zoning/other in the initial stages of the Project. Indicator 17 : STU personnel participate in min, two training courses/person Value Min. 2 (quantitative 100% 100% courses/person or Qualitative) Date achieved 06/23/2009 07/31/2013 01/31/2014 Comments Achieved: 100% All CAR staff associated with the Project participated in at (incl. % least two training courses. achievement) Component 3: Indicator 18 : EOP: Two independent project audits completed and a third contracted Value 2 audits 2 audits done and (quantitative OP audited annually completed (AF) 3rd contracted or Qualitative) and 3rd contracted Date achieved 06/23/2009 07/31/2013 01/31/2014 Comments Achieved: 100% OP and AF were audited annually by independent auditors (incl. % with all audit opinions Unqualified. Final audit (2013) is being contracted and achievement) will cover grace period (audit is due end-2014). Indicator 19 : Full physical performance study completed and reviewed for MTR Value (quantitative OP: Zero Study AF: Zero or Qualitative) Date achieved 07/31/2010 07/31/2013 01/31/2014 Partially achieved: Two MTRs (OP and AF) were conducted by the Bank. Neither completed a Physical Performance Study (PPS) but a Technical Report Comments on physical performance of Produzir was conducted ex-post (June 2014). The (incl. % AF added this indicator because the required study was not done by the OP. Late achievement) effectiveness of the AF left no time for a PPS, hence its ex-post preparation. It should be noted, MTRs are not customary for AFs. EOP: STU has finalized one full-scale impact evaluation study and Bank has Indicator 20 : reviewed AF: Final Value One impact evaluation received (quantitative OP: Zero evaluation study and reviewed by the or Qualitative) Bank Date achieved 07/31/2010 07/31/2013 01/31/2014 Comments Achieved: 100% Unicamp (2014) Final Evaluation completed, delivered to (incl. % Bank and reviewed. Response sent to CAR. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 06/04/2006 Satisfactory Highly Satisfactory 11.73 2 12/22/2006 Satisfactory Highly Satisfactory 38.33 3 06/25/2007 Satisfactory Highly Satisfactory 45.23 4 12/03/2007 Satisfactory Highly Satisfactory 47.23 5 06/17/2008 Satisfactory Highly Satisfactory 48.33 6 11/26/2008 Satisfactory Highly Satisfactory 49.53 7 05/16/2009 Satisfactory Highly Satisfactory 50.63 8 12/24/2009 Satisfactory Highly Satisfactory 53.63 9 04/20/2010 Satisfactory Satisfactory 54.35 10 02/06/2011 Satisfactory Satisfactory 54.35 11 08/10/2011 Satisfactory Satisfactory 57.35 12 01/15/2012 Satisfactory Satisfactory 69.34 13 10/05/2012 Satisfactory Satisfactory 78.27 14 04/24/2013 Satisfactory Satisfactory 81.27 15 08/04/2013 Satisfactory Satisfactory 84.27 16 01/31/2014 Satisfactory Satisfactory 84.35 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions CAR transferred from SEPLAN to Secretariat of Regional Development and Integration 05/29/2007 S HS 43.73 (SEDIR). Loan Agreement amended to show SEDIR as Borrower’s representative. US$3.1 m. reallocated due to reduction (waiver) of Front End Fee; and target for subprojects 02/05/2010 S S 53.63 financed reduced from 2500 to 1950 due increased average cost. US$675,000 reallocated; targeted subprojects reduced from 800 to 500 (under 07/27/2012 Additional Financing) due to increased average cost; and, Cat. 2 revised to include “goods”. Closing Date extended 6 months due to delayed 07/31/2013 Effectiveness and need for additional time to complete subprojects. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.1.1 The Bahia State Integrated Project: Rural Poverty (Loan 7327-BR), known locally as Produzir II, was the second of a two-phase program, the first stage of which was approved for external financing in 2000 and concluded in 2005. Produzir II was designed to both continue the work of the first phase in delivering socio-economic infrastructure and services to Bahia’s poor rural families and to seek a wider impact by strengthening integration both across sectors and between the Project and other State and Federal programs operating in rural areas of the State. The goal was to help the State to increase the Human Development Index (HDI) by investing in activities to improve income generation, health and education, the core components of HDI. 1.1.2 Produzir II was Bahia’s fourth in a series of community-driven development (CDD) projects dating from 1993 when the large-scale Northeast Rural Development Program (NRDP) in 10 Northeast states was restructured by converting a traditional, centralized, top-down, integrated rural development approach into state-financed and managed CDD operations, demand-driven by community associations participating in representative, participatory Municipal Councils. By 2005, Produzir had become a primary instrument of the State Government for improving economic and social inclusion, and strengthening governance and spatial integration in rural areas. The State had performed well under the first phase (2001-2005), achieving project objectives, reaching vulnerable populations, financing a large number of productive investments and leveraging additional resources through cross-sector integration mechanisms. 1 1.1.3 State of Bahia: At appraisal in 2005, the State of Bahia – the largest state in Northeast Brazil - had the highest number of poor households in the region and a Human Development Index (HDI) still ranked 22nd among all states despite improving from 0.386 in 1991 to 0.512 by 2000. 2 Some 47% of all rural families remained in poverty despite improvements in income and non-income measures of poverty over the previous decade. Data from 2003 (the most recent available at appraisal) showed 68% of rural households still lacked piped water, 43% had no sanitation services, and 36% lacked electricity, compared to 12%, 5% and 1% for urban Bahia. The State Government’s long-term goal of increasing HDI from 0.688 to 0.766 (the Brazil median) by 2020 was backed by a strategic development program - Plano Estrategico da Bahia (2003-2020) – seeking social justice and cohesion, environmental sustainability, spatial integration and, economic diversification and competitiveness. The Bank’s response was the Bahia State Program supporting key elements of government’s plan. 1.1.4 Rationale for Bank assistance: Throughout this period, the Bank had maintained a long- term partnership with the State and Federal governments to deliver targeted rural poverty projects in Northeast Brazil. The CDD projects had benefited an estimated 11 million people since 1993, contributed collectively to a significant improvement in basic rural services region-wide, and promoted social capital formation. The Bank had also financed sector investment projects and AAA work relevant to the same issues/region. The FY04-07 Country Assistance Strategy called 1 See ICR, Rural Poverty Reduction Project – State of Bahia, Report #32773, December 2005. 2 Source: UNDP 2013. As per the Loan Agreement: (i) HDI is the Borrower’s Human Development Index measured by the Guarantor’s Institute for Applied Economics and Research (IPEA) on the basis of indicators for health, education and income; (ii) HDI-M means the Human Development Index of a Municipality, as measured by the Fundacao Joao Pinheiro on the basis of the same types of indicators. HDI is an aggregate of HDI-M results. 1 for successive projects under the Northeast program to focus increasingly on integrating these CDD efforts more closely with other government activities in participating municipalities. New projects were to align with state development strategies and to build synergies/links with other Bank-supported operations, and those of other financiers. Project design was intended to reflect this guidance. Produzir I had piloted such integration and intended to expand/mainstream it under the second phase, integrating investments with a wider set of State and Federal poverty projects in health and sanitation, education and environment. Bank support was expected to “leverage” non- project resources at a ratio of 1:5. 3 1.1.5 Produzir II was to contribute to key strategic areas of the State’s Multi-year Development Plan (PPA, 2004-2007) by: (i) integrating directly with the Programa de Gerenciamento dos Recursos Hidricos and PROAGUA – using CDD techniques to deliver small-scale investments in rural water supply while encouraging community participation and consultation for larger public works; (ii) complementing the then-proposed Viver Melhor Urbano II project, with Viver Melhor Rural in the same municipalities as the urban operation; and (iii) emphasizing education, health, culture, natural resource management and environmental sustainability, the first two – along with income generation – linked directly to the HDI. The Project was one of six Bank-supported projects viewed by the State as having strategic importance by directly addressing 10 and indirectly another nine of the 37 programmatic areas in the State’s PPA. The six Bank operations represented about seven percent of the State’s 2004 budget. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 1.2.1 The PDO in the Loan Agreement for the Original Project (OP) was: “to increase social and economic opportunities for the Municipalities’ rural poor by improving their access to basic socioeconomic infrastructure, thus contributing to the Borrower's objective of increasing the Human Development Index (HDI)”. 4 1.2.2 The PAD Results Framework shows one Project Outcome Indicator (POI): "2,500 subproject investments implemented and maintained by community associations”. The PAD Arrangements for Results Monitoring table (ARM) substituted this POI with the following: “percentage of rural poor self-selecting into Produzir, identifying and implementing subprojects”. This substitution remains unexplained but it was monitored throughout the Project. See analysis of Results Framework design in Section 2.3. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 1.3.1 The Additional Financing (AF) of 2009 did not revise the PDO but the Results Framework was revised, as follows: • Evidence of improved living conditions at beneficiary household level • Proportion of productive investments generating full-time employment for beneficiaries • ERRs for productive subprojects exceed 12% • Min. 60% of Municipal Councils deliberating resource allocation from programs outside the Project 3 The PAD text mentions a ratio of 1:10 but this is an acknowledged error. The AF Project Paper confirms 1:5. 4 The Project was appraised in 2005 and the PDO as stated in the PAD comprised a dominant statement and three dependent clauses joined by “by”. The ICR utilizes the statement of PDO as presented (in identical form) in the OP and AF Loan Agreements, throughout. 2 • Ratio of funds leveraged to project funds increases over the life of the Project (ratio 1:5) • Participatory decision-making being used to target and allocate non-project resources 5 1.4 Main Beneficiaries 1.4.1 The PAD quantified subproject investments but not beneficiaries. The latter were the poorest communities and traditionally marginalized groups, mostly family farmers, tenants, sharecroppers and landless laborers with income primarily from farming, wage labor, pensions and remittances. Secondary beneficiaries (of technical assistance/training) were to be the participatory Municipal Councils, community associations and the State Technical Unit (Regional Development and Action Company - CAR). The PAD and Project Paper discuss the inclusion of indigenous peoples under Safeguards provisions, and quilombolas (Afro-descendants of former slave communities) and women under “traditionally marginalized groups”. 1.5 Original Components (as approved) 1.5.1 Project components were as follows: Component 1: Community Subprojects (estimated total cost US$67.5 m., with Bank contribution US$48.9 m., 90%), supported matching grants for 2,500 small-scale socio-economic infrastructure, education, health, cultural, environmental investments designed to support the State’s HDI goals. The Project would cover 407 municipalities (of the total 417), with 40% of Component 1 resources dedicated to the 100 poorest municipalities (by HDI-M). Community associations (CA) would – as under Produzir I - identify their priority investments consistent with a State-run information campaign to raise awareness of HDI and its constituent indices. Associations would execute, operate and maintain their subprojects (with some exceptions), using approved resources transferred directly to their association bank account. Component 2: Institutional Development (estimated total cost of US$4.5 m. with Bank contribution 100%) executed by the CAR, would finance technical assistance (TA) and training for the CAs to carry out their assigned roles/responsibilities in regard to subproject investments; capacity-building for Municipal Councils (MC) including their assessment and supervision of CAs, participatory planning, and financial and environmental management of subprojects; training/TA for CAR’s Regional Offices (RO); and, workshops and seminars for the MCs and CAs to promote the integration of Federal and State poverty reduction programs with Produzir II. Component 3: Project Administration, Supervision, Monitoring and Evaluation (estimated total cost of US$3.0 m. with Bank contribution of US$0.95 m., 32%) would finance costs (excluding salaries) of project administration, monitoring and impact evaluation. 1.6 Revised Components 1.6.1 Components remained as originally designed throughout except for the Original Project reducing the number of subprojects from 2,500 to 1,950 under the February 2010 Restructuring, and the Additional Financing scaling up the number of subprojects under Component 1 by 800, later reduced to 500 under the July 2012 restructuring. 5 See Annex 2 for matrix showing status of the Results Framework over time and what was actually monitored. 3 1.7 Other significant changes • The Borrower’s representative changed. CAR was transferred to the Secretariat of Regional Development and Integration (SEDIR), from SEPLAN. • Loan funds were reallocated twice: (i) In February, 2010 an aggregate US$3,100,000 were reallocated to Cat. 1A and 1B (Area 1 and 2 Grants) and, the target for total subprojects was reduced from 2500 to 1959 due to increased average cost/subproject; and, (ii) July 27, 2012, US$675,000 of AF Loan funds were reallocated to Cat. 2 Consultants’ services/training; subproject target under the AF was reduced from 800 to 500; and, Cat. 2 was revised to include “goods”. • Japanese Social Development Fund (JSDF) Grant of US$1,555,200 (TF058071) under its Sustainability Window to the National Confederation of Agricultural Workers (CONTAG) financed a “Linking Grass Roots Producer Organizations to Global Markets” pilot Project. The Grant Agreement was amended in December 2011 providing CONTAG another US$100,000 (TF10751). See 3.5.9. • JSDF Grant of US$877,614 (TF P118988) of US$877,614 financed a set of activities entitled “Leveling the Playing Field for Quilombola Communities in Northeastern Brazil”. See 3.5 and Annex 2, Appendix 3. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 2.1.1 Background analysis and lessons learned: The Project took its lead, technically and methodologically, from 15 years of Bank-supported CDD projects in the Northeast and from its immediate predecessor in Bahia which had financed over 3,000 investments in basic socio- economic infrastructure and agricultural production, benefiting over one million rural inhabitants organized in 2,500 community associations in 396 municipalities. Some 100,000 poor rural families acquired better sanitation and access to safe water supply from over 1,300 investments, while another 110,000 families improved their incomes from 800 productive investments in agro- processing, mechanization and grain marketing. Lessons of previous experience were taken into account in specific design features, e.g., the use of CDD mechanisms (Municipal Councils) to leverage additional resources from other programs and improve their allocation, and the alignment of community investments with HDI. 2.1.2 Assessment of project design: Produzir II was prepared, negotiated and presented to the Board in about five months, largely due to the relationship between the Bank and Borrower teams and familiarity with the target population and methodology. Studies evinced strong community support for the CDD method of service delivery. 6 Project design reflected evaluation and case studies of similar projects/subprojects in Bahia itself and the Northeast States of Ceara, Pernambuco and Piauí by the Federal University of Campinas, State of Sao Paulo (Fecamp, 2004 and 2005). Project design was similar to its predecessor in most respects except for its emphasis on cross-sector integration to maximize project impact for which the primary instrument was to be the project Municipal Councils; integration with/complementing six key state strategies for rural areas; and conscious effort to align subproject investments with HDI. The intense focus on integration resulted from successful experiences under Produzir I via the “Special Projects” which used the Project’s institutional model and, at the time of appraisal, the Bank’s increasingly strong focus on and belief in the poverty reduction potential of cross-sector integration. 6 See case studies by Fundacao Economia de Campinas, Universidade Estadual de Campinas (Fecamp, 2004) and the Borrower Completion Report (CAR 2005). 4 2.1.4 Quality at entry showed certain shortcomings. The PAD indicates a pro-forma approach, rapid preparation and the sense that design/activities were not in step with the PDO. This might be explained by the project being the second phase of a larger operation presented by Bahia to the Federal Government in 2000 and approved by the External Financing Committee (COFIEX) of the National Treasury as two phases of US$54.35 m each based on tight fiscal rules for state borrowing. The inconsistency is that the more ambitious objectives of Produzir II were paired with a repeater strategy on the ground. A serious issue was the weak Results Framework and flawed Arrangements for Results Monitoring (ARM) which it seems the internal clearance process did not detect (see 2.4). The PAD also lacked certain standard content, e.g., number and description of beneficiaries, discussion of gender, description of Results Indicators in the main text and discussion of participatory processes/consultation (perhaps because the project methodology and institutions were built on participation). 2.1.5 There was also no discussion/roadmap in the PAD for more complex productive activities despite the expectation of links to national and international markets, and the PDO’s seeking to “contribute to the Borrower’s objective of increasing HDI” of which income generation is one index. The Operational Manual shows a brief description of expected types of investments as well as examples of likely education, health, environment and culture investments, but not the PAD. The fact that CAR had about 1,000 subproject proposals registered, analyzed and awaiting financing along with several thousand community subproject demands overall held over from the previous operation, indicates that the Project was designed to absorb that demand with minimum discontinuity with its predecessor. 2.1.6 Targeting design: Some 40% of project resources were to be applied to the 100 poorest municipalities (Area A) based on their rank in the Municipal Human Development Index (HDI- M), while the other 307 (Area B) shared the remaining 60%. Experience in other participating states using this method of weighted allocation of resources showed that the relatively lower level of organization and absorptive capacity of Area A municipalities tended to skew financing towards the somewhat better-equipped and organized Area B, although it should be noted that both cohorts fell well within the income and wellbeing indices for classification as poor. Fecamp (2005) assessed the ex-ante socio-economic profile of Produzir I beneficiaries in Bahia and in three other Northeast states and showed that overall, 40% were illiterate, 50% had insecure access to food, and 75% of families were below the poverty line with incomes equivalent to/less than US$1.00/day. The Final Evaluation (Unicamp, 2014) is consistent with this profile. 2.1.7 Assessment of risks: The Risk Assessment was reasonable but incomplete. The Municipal Councils were well-established and their deterioration could not have been predicted at appraisal (see section 2.2). Even so, in considering risk factors with potential to affect the Councils, calculation would have shown that the Area A/B targeting approach, taking into account a four-year project, implied that about 75% of all eligible municipalities, i.e., the Area B cohort, would receive an average (indicative) annual budget barely sufficient for three to four subprojects over the life of the Project, reducing their credibility and raison d’etre. 7 The PAD risk minimization measures show that the MCs were seen as pivotal institutional mechanisms for avoiding political influence on the Project, maintaining investment quality and ensuring overall support for CDD. Similarly, inflation and exchange rate pressures on subproject costs – requiring the reduction of subproject targets under both stages of the project - could not have been 7 This suggests inconsistency in arguments that the Councils were already actively leveraging resources from other programs (and thus capable of influencing their methodologies, performance and outcomes). 5 predicted at the time of appraisal but emerged rapidly thereafter (Annex 2). Drought risk was not mentioned, perhaps because the Northeast region is traditionally semi-arid and drought-prone. Persistent and severe drought had major repercussions for the Project’s productive subprojects under the Additional Financing. Mitigation measures could have included improved (climate smart) technologies/approaches already introduced by many Bank projects. Such techniques will be stressed in the new Bahia Produtiva operation. 2.2 Implementation 2.2.1 The OP was implemented rapidly. CAR saw the Project as a straightforward continuation of Produzir I and, with its extensive experience, established network of 18 Regional Offices and a stock of 1,000 pre-approved community subprojects held over from the previous operation, was ready to finance/implement immediately after Effectiveness. This is also confirmed through CAR’s drawdown of 100% of its available retroactive financing. By end-2007, at the time of the Mid-term Review (MTR) the Loan was almost fully-committed and the State had already sought COFIEX approval for a US$30.0 million Additional Financing (see 2.2.6, 2.2.7). 2.2.2 Despite the PAD’s emphasis on the MCs as key instruments for governance, transparency, targeting and especially the integration of other Federal and State programs and, apparently unbeknown to the Bank task team until 2011, the MCs were increasingly moribund and with some exceptions, no longer part of the Project’s institutional framework. 8 This was not perceived by Bank supervision missions (nor divulged by CAR) and hence not reported in Aide Memoires, or factored into the Bank/CAR dialogue or the Additional Financing. CAR did not inform the Bank until 2011 (when questioned on the Councils) and with the Project’s new Bank project manager urging solutions, sought to engage social movements and representative groups to assist the community associations in accessing benefits in an organized, participatory manner under the Additional Financing. Forums/entities identified and formally recruited included: the State Quilombola Council, Bahia Family Agriculture Forum, Territorial Colleges, State Territorial Council, State Sustainable Rural Development Council and the State Indigenous Peoples’ Council. 2.2.3 Several factors caused the Municipal Councils’ demobilization: (i) Party politics played a role. The new State Government saw the Councils as an inherited, outdated mechanism inconsistent with its new territorial development framework, although Government supported participatory governance mechanisms per se and the subsequent reconstitution of the MCs, albeit in different form (see 2.2.4); (ii) Periods of reduced resources combined with the Project’s differentiated allocation of project resources to municipalities based on relative HDI-M (Areas A and B), saw many MCs lose interest as the amount of resources they were receiving or likely to receive over the course of the Project was insignificant in relation to demand. This situation was aggravated by competition from the Municipal Sustainable Rural Development Councils (CMDRS) established for the National Family Agriculture Program (PRONAF) which by mid- decade was increasing its coverage and financing; and (iii) according to Unicamp (2014), authorities also favored accelerating project execution via direct interaction between CAR and the associations. Meanwhile the more fragile and unstable MCs were already being substituted by Territorial Colleges and other participatory entities under new Federal development policies (see 2.2.4), although some stronger MCs continued to operate and participate. 2.2.4 The evolving institutional framework: Federally-mandated territorial planning and development policies created new participatory mechanisms - Territorial Colleges - which 8 See Final Evaluation (UNICAMP 2014), Borrower Completion Report (CAR, 2014), Results Evaluation of Coffee Processing Subprojects (AGIR/CAR, 2009), and supervision Aide Memoires (2011 and 2012). 6 include representation of all municipalities within a defined “territory”, along with public authorities and civil society. The colleges prepare participatory Territorial Sustainable Development Plans and under the Additional Financing their working groups positively influenced the selection and transmittal of community demands for approval/financing. Social movements have ample space within the colleges, reducing the influence of municipal authorities on civil society decision-making. Since it was observed that Municipal Councils can strengthen these colleges, from 2012, CAR and SUAF (Superintendence for Family Agriculture) have been working jointly to re-constitute them, unifying where there were several and creating where they did not exist. Each restructured council is formalized via municipal law as a Municipal Council for Sustainable Development (CMDS). As at ICR finalization, 270 MCs had been restructured and were operational. 9 The CMDSs are integral to the new, Bank-supported Bahia Produtiva. 2.2.5 Demand pressures: Demand for the Project was massive from the start, adding to the large number of subproject proposals already registered at effectiveness, held over from Produzir I. All of these proposals pre-dated the Government turnover and had gone through Municipal Councils. Of these, 1,000 were approved for immediate implementation under the original project (OP). The total demand pool by the time of the Mid-term Review in late 2007 greatly exceeded available financing both under the OP (and the subsequent AF), and a significant portion were Council-processed. The AF also pulled from that existing pool, in addition to financing demand generated through the Territorial Colleges and other representative forums (see 2.2.7). 2.2.6 Mid-term Review (MTR): The MTR in late 2007 was well-staffed with experienced specialists and conducted early. The Project had already disbursed US$45.0 m., 83% of the Loan in just 37% of the planned project period and CAR initiated discussion of a possible Additional Financing. Notably, in addition to subprojects already approved and under execution and, due to publicity/information campaigns, CAR had registered some 13,000 additional demands for subprojects. Water supply, sanitation and productive facilities represented 78% of subprojects financed at that time. The ICR questions the rationale for promoting such a high level of demand given the potential for disillusionment since available funding covered barely 15% of it, but possibly the declining role of the MCs removed a key filter which had governed more rational management/flow of demand up till then. Also, other Federal and State programs (e.g., Agua para Todos) would have been expected to absorb some of this excess demand. The envisaged mid-term study with beneficiary surveys was not prepared. The MTR found all major project elements including Fiduciary and Safeguards fully satisfactory but did not comment on/analyze M&E or the status of the Municipal Councils, and missed a key opportunity to restructure the Results Framework. 2.2.7 Additional Financing: The Board approved an Additional Financing loan of US$30.0 million in July 2009. Project design, methodology and implementation arrangements were described in the Project Paper as the same, although the Borrower had diverged from the institutional format. The rationale could have been improved given massive, unmet demand for water supply and sanitation investments, as well as casting the AF as an opportunity to conduct market studies and selectively pilot more complex productive initiatives. Further, Produzir II was highly cost-effective compared to other programs - a feature observed since the early days of 9 Unicamp (2014), states that the erosion of the MCs permitted some political intermediation in the subproject cycle. However, Unicamp also notes that such evidence is fragmentary and is not overtly critical of this development, referring to the benefits and legitimacy of elected officials facilitating the subproject process when needed. CAR’s Completion Report (2014) rejects any inference that the involvement of elected officials/local authorities had negative consequences. 7 the Northeast CDD projects - and this could have been stressed. For example, for a 16,000 liter household water cistern of similar quality, project cost was R$2,600 compared to R$5,400 for the Federal Agua para Todos program, and used local labor for installation. The number of community subprojects to be financed under Component 1 was scaled up by 800 to an aggregate 3,300 (including the OP), designed to benefit an incremental 72,000 families. 10 2.2.8 The Project schedule changed, extended by the Additional Financing’s standard three- year period to July 31, 2013. An 18-month delay in effectiveness of the AF (to January 24, 2011) resulting from conditions at the Federal level beyond the control of the Borrower (standard reviews by the Federal Government and Senate approval), meant that the AF’s effective implementation time was reduced to 2.5 years, insufficient to complete the investment cycle for many subprojects. To compensate, the Bank approved the State’s request for a six-month extension of the AF closing date to January 31, 2014. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 Design: The project benefited from a well-established Management Information System (MIS) already used by previous CDD projects in this state and updated for the Project. It comprised subproject information, financial management and project management modules and included a database of community profiles and quantitative variables on control groups to improve impact evaluation capacity. The online MIS also permitted real-time data entry and monitoring directly from the field through CAR’s Regional Offices. Evaluation studies were planned, continuing the program started under Produzir I and including: annual physical performance reviews; mid-term evaluation with beneficiary consultations; and, final impact evaluation using repeat surveys of beneficiaries and non-beneficiaries. No baseline study was included; the MIS beneficiary entry profiles were designed for this purpose. 2.3.2 The OP Results Framework (RF) was poorly-designed and presented inconsistently both within the PAD (including the RF) and between the PAD and other project documents. The most marked disconnect was between the PDO and RF indicators. There was only one Results Indicator (an output):“2,500 subproject investments implemented and maintained by community associations”. Indicators by component in the RF were divorced from the internal themes/elements of the PDO and merely described processes. In the PAD Arrangements for Results Monitoring (ARM) table, the Results Indicator was demoted to an Intermediate Outcome Indicator and substituted with an intermediate result: “percentage of rural poor self-selecting into Produzir, identifying and implementing subprojects”. Further, the ARM contained component indicators quite different to those in the RF. The Additional Financing in 2009 revised the RF. 2.3.3 Two additional points are relevant. First, experience shows the error of linking project achievements to a major index which the project itself cannot measure, e.g., HDI. However, the ICR interprets the PDO as signaling that planned investments would be aligned closely to (thereby assisting the Borrower to improve) the component indices comprising HDI – income generation, health and education – not that the Project was intending to measure HDI per se, by end-project. This would have been impossible given that HDI is an independent instrument measured every 10 years on the decade; the project team was aware of this. Alignment with the 10 The original target of 2,500 subprojects was reduced in 2010 to 1,950 due to increased average SP cost, itself the result of larger average numbers of beneficiaries/SP in many cases, cost inflation for SP inputs, and appreciation of the Brazilian Real to the US Dollar. In 2012, the AF target of 800 subprojects was reduced to 500 for similar reasons. Thus a planned aggregate (OP and AF) of 3,300 investments was reduced to 2,450. While the PAD did not quantify expected beneficiaries of the OP they were estimated at around 173,000. 8 State’s rural poverty strategy as embodied in its HDI policies was reasonable and a lesson of the previous Bahia CDD project. Second, in retrospect, project design markedly over-estimated the capacity of the MCs and community associations to manage/promote “integration”, the meaning of which was not well-defined. 11 2.3.4 M&E Implementation: Implementation of M&E showed some shortcomings. The Bank team did not realize that what was being monitored (the ARM) was different to the actual RF, and that the RF merited early restructuring. There were two Mid-term Reviews, the first conducted early and the second a round-up of project status during the AF (Additional Financing projects do not usually conduct an MTR). The survey-based study envisaged for the OP MTR was not conducted but an ex-post, good quality Physical Performance Review was prepared (Pereira/CAR, 2014). Completion Reports were prepared for both stages by CAR. The University of Campinas prepared a Final Evaluation including case studies (Unicamp, 2014) and AGIR/CAR conducted interesting studies in 2009 associated with women’s coffee-growing subprojects as well as experiences with household water cisterns and agricultural mechanization investments (Annex 5). 2.3.5 Utilization and dissemination: First, due to the weak original Results Framework and monitoring arrangements and superiority of the AF’s revised version, the ICR follows the lead of the later Aide Memoires in applying the revised framework of indicators to both the AF and retroactively, to the OP, for monitoring project achievements. Second, the Unicamp Final Evaluation was a primary source for preparation of the new operation in Bahia (see Section 4) and the ICR, as were CAR’s two Completion Reports. Finally, CAR organized a results dissemination seminar/workshop, summarized in Annex 6. 2.4 Safeguard and Fiduciary Compliance 2.4.1 Safeguards: CAR’s management of project Safeguards provisions was satisfactory throughout and no problems arose, as summarized below: (a) Environment: Implementation of the Environmental Management Plan was strong institutionally, legally and operationally under both stages of the Project. Compliance with Bank Safeguards and with all applicable State and Federal laws was satisfactory. All subprojects were screened using a long-established and updated system prior to approval for financing to prevent, minimize or mitigate adverse environmental effects. The MTR recommended greater emphasis on environmental training, and that the Project explore the possible financing of environmental subprojects (recuperation, sustainable wood production, and management of residues/recycling), already contemplated in the PAD and Operational Manual. This was attempted with little success in part because communities did not understand the rationale – an issue of awareness-building – were unwilling to devote land to improved practices given subsistence pressures, and/or had other urgent priority needs. The new Bahia operation will address environmental subprojects for the poorest farmers on degraded lands, as well as opportunities for productive activities in organized recycling (an especially successful community activity in the State of Paraiba). The issue of potential cumulative impacts, e.g., water supply systems using wells, was given special attention 11 For purposes of results measurement, the ICR uses the PDO as presented in the Loan Agreements for both the original Project and Additional Financing but comments on the broader PDO as presented in the PAD as a design issue. It was unreasonable to expect the Project to measure the impact of the MCs’ own participatory methodology on the “relevance, efficiency, sustainability, targeting and outcomes of non-project State and Federal investments in rural Bahia”. No indicators were included to measure these factors and the evaluative difficulties are readily apparent. 9 and it was recommended that the MIS include indicators/parameters to measure aggregate environmental impacts, negative and positive. (b) Indigenous Peoples: Compliance with the Project’s indigenous peoples’ Safeguards was satisfactory. The Project intensified activities supporting indigenous communities by integrating project actions with other State policies/programs. The total estimated indigenous population of Bahia at appraisal was around 10,600, about 9% of the national total. Their activities were mainly subsistence-based and extractive. The experience under Produzir I was quite successful and these groups had demonstrated their capacity to mobilize, access and participate in that Project. Produzir II prepared an Indigenous Peoples’ Participation Framework (IPPF) to include mobilization, communication, training and capacity-building, and community investments. CAR would be supported by the National Indian Foundation (FUNAI). The project developed an Action Plan using collective, participatory and culturally-appropriate approaches. In the absence of the MCs (described earlier), indigenous groups accessed project benefits with support from the State Indigenous Peoples’ Council (CEPI) and other social movements/NGOs using the Economia Solidaria strategy via agreements for inter-institutional cooperation with relevant agencies – in this case CAR – to invest in poor rural communities. Some 76 subprojects valued at R$3.2 million benefited 7,288 indigenous families in 15 municipalities. 2.4.2 Financial management and audit: Financial management rated project performance Satisfactory throughout with the exception of a downgrade to Moderately Satisfactory in 2011 due to minor shortcomings: delayed submission of IFRs, delayed updates to the Operational manual, and inconsistent SOEs. CAR complied with agreed corrective actions. The same FM system was used under each stage with key improvements introduced under the OP and updated by the AF. The MTR simplified documentation required to be submitted to the Bank and auditors by associations in regard to subproject conclusion and the appropriate use of the funds by the associations. The overall risk rating remained Low through to end-project. 2.4.3 Audit: Audit performance was generally satisfactory. Auditors’ opinions were consistently Unqualified for Financial Statements and Special Opinions. Management Letters (Carta Gerencial) found the Project’s internal control systems to be satisfactory. The Bank team followed up rapidly with the Borrower on auditors’ recommendations and there were no audit issues pending at close of either the OP or AF. It was agreed with the Bank that the final audit would cover 2013, plus January 2014 and the grace period and be delivered by end-November 2015. CAR delivered the final audit early (June 18) with an Unqualified opinion. 2.4.4 Project costs: Total aggregate project cost was US$216.1 m, 188% of the original estimate; the OP cost US$119.3 m, 159% of appraisal and the AF US$96.9 m, 242% of appraisal. The costs of individual components varied considerably from appraisal. This is explained by the following: (i) Regardless of pressures which subsequently arose and could not have been predicted – at least by the OP - base costs were under-estimated at appraisal of both stages. The estimated average cost of an OP subproject was around R$60-65,000 or roughly US$30,000 and this estimate carried through into the AF. In practice, the average cost was about R$133,000 or US$65,000; (ii) Inflation in the costs of construction materials, labor and equipment and effects of the fluctuating USD/Real exchange rate (see Annex 2) increased average cost. This is perhaps most visible in the case of water supply systems which increased 107% from R$132,074 (OP) to R$253,109 (AF), although the average number of beneficiary families per system rose only marginally from 71 to 77. Manioc mills increased 12% from R$98,278 (OP) to R$110,296 (AF); small-scale dams rose 35%, from R$59,641 (OP) to R$80,521 (AF); and, a concrete bridge increased 19% from R$480,623 (OP) to R$495,804 (AF); and, (iii) Government’s much higher 10 than expected counterpart contribution – overall, more than six-fold the PAD estimate - was increased inter alia, by CAR’s intensive resource leveraging involving many agencies/programs. Administrative costs for such activities were not accounted for at appraisal. See Annex 2, paras 2.15, 2.30 and 2.33 for discussion on costs of individual components. 2.4.5 Procurement: Procurement Post Review (PPR) missions generally rated procurement performance as satisfactory under the OP and AF and procurement risk as average. CAR and the communities were found to be conducting procurement and contracting in a sound manner within a reasonable time and in accordance with Bank Guidelines. CAR’s extensive experience with the Bank facilitated its adherence to proper procedures. Bank recommendations included CAR contracting a procurement specialist well-versed in Bank procedures to speed up processing and to train CAR’s procurement team, along with greater use of the MIS as a procurement aid. Further, a small pilot tested a simplified approach to compliance with Bank procurement procedures. Ten associations were selected and took all procurement decisions in a community assembly, registering decisions/proceedings in “atas” (minutes). The goal was to increase efficiency, economy, transparency and competitiveness in community procurement. 2.5 Post-completion Operation/Next Phase 2.5.1 Operation and maintenance: All subproject investments are formally under the control of the beneficiary community associations, which are responsible for their operation and maintenance (O&M). The status of O&M is discussed below for the four most commonly- financed investments. 2.5.2 Household cisterns and sanitation facilities: Individual household water cisterns and sanitation facilities represented 35% and 25% respectively of total subprojects financed (OP and AF). Cisterns are a simple, inexpensive, highly beneficial technology for water supply in Northeast Brazil and in Bahia, CAR is the principal executing agency for several Federal and State programs in addition to the Bank Project. CAR utilized a standard design of 16,000 liters at a cost averaging around R$2,600 (<50% of the cost of cisterns of the same capacity and quality installed by other programs such as the Federal Agua para Todos); 857 cistern SPs were implemented. Beneficiaries were trained by local Health Agents to manage their cisterns including water quality/treatment, periodic sanitization and ensuring against the entry of pollutants. Studies (Unicamp 2014, AGIR 2009) show that cistern management and maintenance require attention, reflecting the variable capacity of associations and individual households to establish management routines, handle technical issues and/or in some cases understand the rationale for maintenance. 12 Some issues with the quality of training and awareness-building, construction, and associations’ supervision during subproject implementation contributed. An issue reported by Unicamp (2014) was the irregular supply of water, primarily the result of severe drought and inadequate reservoirs/wells to supply households. Some beneficiaries reported that during the height of the drought, only those able to pay for trucked water could fill their cistern (Annex 5). An aggregate 640 sanitation subprojects were also financed. Routine maintenance at the household level is reported as satisfactory and families received guidance on usage and hygiene. Immediate health and wellbeing benefits were reported by beneficiaries (see Annex 5). 12 See Final Evaluation, Unicamp (2014); and Avaliacao do Resultados do Subprojeto de Cisternas Domiciliares, AGIR/CAR, December 2009. The latter is a qualitative analysis by CAR’s Monitoring and Evaluation Unit (AGIR) of a category of project investments, Cisternas Individuais. 11 2.5.3 Water supply: The Project financed 151 water supply systems (WSS), 6% of aggregate investments (OP and AF). The main issue detected by Unicamp (2014) was incomplete/poor construction of some systems, drought-related lack of water, and deterioration of systems from the poor quality water supplying them. About 95% of water supply investments involved direct piping of water to the house, with hydrometer. The larger issue is finding management models that guarantee the economic and financial sustainability of rural water supply systems in dispersed areas. Water rationing devices were installed in some cases and need wider distribution. WSS will be an important focus of the Bank-supported Bahia Produtiva. 2.5.4 Agricultural mechanization: The project financed a total 503 tractors (mecanização agricola), commonly a standard CAR “kit” including a garage and equipment. As noted in Unicamp (2014) and CAR (2014), tractor access among community associations was high (in the range of 70% to 100%), but effective utilization was severely limited by drought from 2011-2013. Tractor use is governed by fees which vary based on the type of activity – plowing, grading, harvesting, transporting material/other – and by whether the user is a member of the association/not; rates for non-members are higher. From July 2012 to June 2013, average household expenditures for usage were R$108 (plowing) vs. R$44 (grading). Fees for tractor usage are commonly saved in a fund and applied to O&M costs: payment to trained community operators (courses provided by SENAR/other entity) and for repairs. Unicamp (2014) found that tractors have a strong impact in terms of: reducing the work/time burden on families, releasing them for other agricultural and non-agricultural activities; reducing labor and tractor rental expenditures; and, fostering permanence on-farm, all factors likely to promote good O&M. The main issue is replacement of the investment beyond its useful life and the funds to adequately cover that, requiring higher rates of tractor utilization (see 3.3 and Annex 3). 2.5.5 Manioc mill: The Project financed 94 manioc mills, using standard designs far superior to the precarious structures common in the Northeast. Beneficiary associations charge a fee for usage, generally lower than what community members were previously paying to private mills. Key processing steps are handled by trained operators who receive about R$1.50 per sack of manioc flour. Fees are usually collected in flour: 5 kg/50 kg sack for members and about 7 kg/50 kg for non-members. The association sells the flour to community members or businesses in the region. Proceeds are held in a fund for maintenance of equipment and payment of the operators. Under-utilization due to drought-driven shortages of raw material was common in the period 2012-2013 and usage will need to increase to cover replacement of the investment. Institutional capacity: 2.5.6 State Technical Unit: CAR is an established, experienced and autonomous public company within the State Secretariat of Development and Regional Integration (SEDIR) and has an extensive network of 18 Regional Offices with adequate technical, social and administrative staffing. It will manage and implement the follow-on Bank-supported operation and its capacity is not in question. 2.5.7 Municipal Councils: As discussed in Section 2.2 and Annex 2, the Councils were not restructured/ renovated by the incoming State Government in 2006 and their capacity declined to the point where only the strongest continued to play a role in the Project by end-OP, including within the participatory Territorial Colleges/Councils. Other institutional arrangements were introduced, strengthened and mainstreamed, both as a result of Federal and State policies already in existence at the time of OP effectiveness and the efforts of CAR. The MCs’ reconstitution as CMDSs is ongoing and they have an important facilitation role under the new project in Bahia. 12 They will receive proposals for social and environmental subprojects from community associations, vet their eligibility and pass qualified proposals to CAR. 2.5.8 Community Associations: The history and capacity of community associations varies state-wide. CAR organized extensive training throughout both stages to strengthen associations’ capacity and social capital (Annex 2). Unicamp (2014) reports that community mobilization/participation was strong in the initial phase of deciding on and executing the subproject but tended to decline thereafter. However, many associations pre-existed the Project and have a long history of cooperative activity. Among sampled beneficiaries, 39% said they had been to all meetings conducted by their association in 2013 while 31% of associations had not met. Some 51% of beneficiaries stated that their association had delivered additional benefits beyond the Project. Associations’ capacity to manage O&M of water supply systems needs focus while ongoing TA and training of associations can equip them to seek and obtain further benefits. 2.5.9 Next phase: A new Bank-supported operation, Bahia Sustainable Rural Development Project (Bahia Produtiva, with loan of US$150.0 million), approved by the Bank’s Board on June 27, 2014, would “increase market integration, net revenues, and food security for organized beneficiaries; and, improve access to water supply and sanitation services of households in project areas”. While demand-oriented, it adopts a differentiated treatment (not just differentiated financing) for two core groups of beneficiaries identified on the basis of national poverty measures. It shifts the focus from local, needs-based productive subprojects to market-driven investments by organized producers with the skills and capacity to integrate with markets, exploiting opportunities identified in value chain analysis and territorial development strategies. The Project will also address in parallel, the needs of the poorest segment struggling with food security and lacking the level of organization or technical/managerial skills to take advantage of market opportunities. 13 Investments for this cohort will focus on improving basic social and environmental conditions and creating capacity for future integration into productive activity via State Territorial Development Plans. Project activities have been carefully aligned with Federal and State policies and programs in poverty reduction, territorial development and universal access to water and sanitation. 2.5.10 Future M&E: While all monitoring data from the Project is preserved in the MIS and easily accessed online, no additional performance indicators were formulated for future M&E of the closed operation. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 3.1.1 Objectives: Continuing relevance of the PDO is rated Satisfactory based on the Bank’s current Country Partnership Strategy (CPS), 2012-2015. The PDO, by stressing socio-economic inclusion through basic infrastructure investments in a very poor Northeast state, remains consistent with the CPS’ first objective: “boosting growth capacity with job and income generation, especially in Brazil’s poorer regions, with special emphasis on the Northeast”. Further, the PDO’s support for improved HDI remains aligned with the second objective of the CPS: “to improve the quality of public services for low income households and expand their provision through public and private channels”. The PDO’s social goals and alignment with HDI 13 These would include combining complementary initiatives at sector level with those addressing cross-cutting issues: gender, territorial development and promotion of socio-economic wellbeing of Afro-Descendants, Indigenous Peoples and quilombolas. 13 by improving public services in water and sanitation remain highly relevant to the CPS, especially in view of continued low access and vulnerability in rural areas. 3.1.2 Design and implementation: Relevance of design and implementation is rated Satisfactory. The Project’s ability to respond rapidly and cost effectively – including during an exceptionally severe and protracted drought – to poor communities’ need for water and sanitation, supports this rating. WSS investments represented about two-thirds of all SPs and benefited around 100,000 families. The Project, through its decentralized, participatory methodology, reached remote and/or dispersed areas where the coverage of larger public programs can be problematic and subject to restrictions on service provision based on minimum numbers of families. Project design and implementation also promoted economic inclusion by reaching and engaging some 106,000 families in small-scale income-generating activities. The Project’s resource leveraging features generated an additional R$413.0 m and reached an estimated 127,000 poor rural families (with repetition). While the impact on local economic development in any organized sense was modest – which may signal the need for design adaptation - the benefits generated were important for those families and the facilities financed have significant potential in a without-drought scenario. 3.2 Achievement of Project Development Objectives 3.2.1 Analysis of PDO achievement uses several sources of evidence of which the most important is the final evaluation by the University of Campinas (Unicamp, 2014). The evaluation methodology is explained in Annex 5 and Section 3.6, and is considered satisfactory but not optimal, the main issue being the absence of a baseline and the use of recall for certain data, although samples are randomly selected and a control group is used for most elements of the study. For example, while broad impacts on health compared treatment and control groups, micro effects at the level of individual diseases did not. But, it is assumed that beneficiaries’ and non- beneficiaries’ perceptions about their general health can be reasonably extrapolated to the micro level, i.e., the types of diseases families commonly suffered and their awareness that their incidence had declined once water and sanitation conditions improved. PDO: Increase social and economic opportunities for the Municipalities’ rural poor by improving their access to basic socio-economic infrastructure thus contributing to the Borrower’s objective of increasing the Human Development Index (HDI). Rated: Moderately Satisfactory 3.2.2 Support to improved State HDI: Bahia’s HDI increased significantly from 2000 to 2010, from 0.512 to 0.660 although its national ranking stayed at 22nd. The link between the Project and HDI outcome was: (i) through an aggregate 1,708 water supply and sanitation investments (household cisterns, water supply systems, sanitation facilities and dams/other) under the OP and AF – about 63% of the total 2,695 SPs financed - benefiting around 100,000 families or 450,000 people, and (ii) 785 productive investments (29% of aggregate SPs, mainly agricultural mechanization, manioc mills and diverse agricultural processing units) benefiting 106,000 families. 14 The “longevity” index of HDI-M (proxy for health) shows that Bahia improved from 0.680 in 2000 to 0.783 in 2010. Beneficiary incomes did improve and the HDI income and longevity indices moved up from 0.594 to 0.663 in the period. Positive correlations with the HDI income index can be inferred although the ICR does not claim direct, measurable 14 The health impacts of an additional 813 diverse water supply investments under Produzir I between 2000 and 2005 should also be noted. These were discussed in Fecamp I, II and III (2004 and 2005), and the ICR, Report No. 32773. 14 attribution. The primary drivers of improved HDI were income and longevity; education lost ground state-wide, which decreased the overall HDI-M for 93% of municipalities. (See table). Table: IDH-M by composite Indices (Income, Longevity, Education), 2000 vs. 2010 Bahia IDH-M IDH-M IDH-M IDH-M Income Longevity Education # Municipalities with decrease since 2000 13 1 415 388 # Municipalities with increase since 2000 404 416 2 29 TOTAL: 417 417 417 417 Source: UNDP/team calculations 3.2.3 Social opportunity: The Project financed 1,708 investments in WSS – household cisterns, water supply systems with household connection, small-scale dams and sanitation facilities. Some 54% of beneficiaries reported improved family health compared to 37% of the control group. Water and sanitation infrastructure led to immediate health improvements in terms of water-borne disease for municipalities with no or low ex-ante sanitation conditions: Hepatitis (9% to 1%); schistosomiasis (13% to 5%); parasitic infections (41% to 23%) and diarrhea (45% to 35%). Some 79% of WSS beneficiaries stated that the Project had improved their health. Further, 96% of beneficiary households reported better drought resilience, 96% said water quality was better and 58% believed their water subprojects improved child and elder health. Some 96% also said that sanitation installation had benefited children and the elderly. Close to three-quarters of all beneficiary households perceived improved quality of life, linked to their subproject, whereas only 53% of controls observed such improvement. Despite the perception of food security being relatively equal between treatment and controls (52.2% and 48% respectively), the worsening of food security was much higher for controls (14%) than the treatment group (3%). Unicamp (2014) notes however, that income transfer programs may also have played a role here. 3.2.4 Supporting HDI longevity outcomes, data from the Institute for Applied Economic Research (IPEA, 2012) shows a sustained upward trend in rural water supply from 2001 to 2009, rising from 25% to 55% compared to a 10% increase in urban water supply in the same period (see graph below and Annex 2). The upward trend is more pronounced from 2006 onwards, coinciding with OP effectiveness and the intensive implementation mentioned. Importantly, CAR’s leveraging of complementary resources from other programs totaling R$413.0 m benefited 127,000 families, of which about 64,000 families received water supply. See Annex 2. 15 3.2.5 Economic opportunity: The Project financed 785 productive investments, generating economic opportunity, interpreted as investments producing income to complement poor rural families’ diversified survival strategy, but not “full-time employment” as expressed by PDO Indicator 3. 15 Even taking the impact of drought into account and the modest scale of most investments, the Project’s productive activities demonstrated capacity to increase income. For Unicamp’s treatment group, 49% beneficiaries of productive investments said they were better off and 14.2% were worse off; among the controls, 42.3% were better off and 30.4% worse off. Some 45% and 54% of beneficiaries of tractors and manioc mills respectively, said income had improved as a result of the Project. IPEA data shows average per capita household income increased in rural areas from 2001 to 2009 with a stronger upward trend from 2006 (see IPEA Graph 8, Annex 2). 3.2.6 An estimated 94% of all productive investments generated some employment. Agricultural mechanization (431 tractors and equipment) and manioc mills (94 modern, electric facilities) represented 67% of all productive SPs. They typically generated two operator/maintenance positions per SP and in both cases “full-time” means seasonal. The Project created an estimated 1,050 jobs from these two types of investments. A Japanese Trust Fund Grant to the National Confederation of Agricultural Workers (CONTAG), supervised by CAR, successfully explored the potential for more sophisticated, market-oriented production, employment and income generation by family farmers. See 3.5.9. 3.2.7 Agricultural mechanization: While past analyses have reported that well-managed tractors can have positive impacts on family income, Unicamp (2014) found that the income effects were suppressed by drought. Among the positive impacts: (i) 45% of beneficiaries reported that their monetary income and subsistence improved and attributed this to the tractor; (ii) 81% said that time spent on family field labor was reduced and 70% were using this saved time on additional agricultural activities (and 44% on non-agricultural activities); (iii) labor costs were reduced for 82% of beneficiary households but there was no indication that tractors caused unemployment. The labor released was absorbed into other agricultural activities. 3.2.8 Manioc mill beneficiaries were also drought-affected with shortages of raw material despite the manioc plant’s drought-resistance. Even so, 54% of beneficiary households reported increased income post-investment, more time to market what was often their only income generating product (flour), better quality flour, more accessible/better-located mills and reduced costs of production, the latter because fees charged to association members for usage were lower than outside mills. Despite technological limitations and the fact that such subprojects address only the transformative stage of an agricultural product, some 60% of households using manioc mills said their production of flour had increased and 70% said its quality had improved. Other achievements: 3.2.9 Resource leveraging: The Project was a catalyst in scaling up socio-economic infrastructure investment in rural areas. Financial integration was substantial. CAR’s successful partnerships with diverse public programs benefited the Project’s defined target population, yielding an additional US$60.0 m and US$147.0 m under the OP and AF respectively (see Tables, 15 CAR insists that the Project’s productive investments were never intended to create full-time employment but were an income complement, i.e., the indicator was inappropriate. The ICR suggests that the Project, using a small sample or focus groups, could have measured temporary and/or permanent family and contracted labor opportunities created. 16 Annex 2), increasing the ratio of funds leveraged to project funds to 1:5 by end-AF and benefiting an additional 127,000 families. As discussed elsewhere, CAR was the primary agent for leveraging resources through institutional partnerships. Within this framework, the participatory Territorial Colleges and Bahia Family Agriculture Forum supported CAR in leveraging from INCRA (land reform), the Ministry of Agrarian Development (food security), as well as CODEVASF and the Ministry of Social Development (reservoirs and cisterns). 3.2.10 Cross-sector integration: As intended at appraisal, the Project was inserted into several rural policy areas specified in the State Government’s Multi-Year Development Plan, mainly to complement six strategic themes (see PAD Table 1). In practice, the Project’s social capital formation through shared responsibilities and decentralization aligned it with Gestao Solidaria e Governo Competente. Investments in productive activities aligned the Project with Bahia que Faz (generating employment and income) and the WSS investments with Bahia de Toda Gente (social action, quality of life and health, citizen protection). This strategy also incorporated the Bank-supported Water Resources Management Project (PGRH) and Federal Water Resources Management Project (PROAGUA), with the Project complementing these large-scale programs by operating closer to the ground in specific, poor communities. 3.2.11 Targeting: Planned Area A and B targeting via the differentiated allocation of Component 1 resources tended - as in all Northeast CDD states with similar targeting design - to skew towards Area B. This resulted from: (i) local, spontaneous, drought-driven demands especially for water supply, exerting heavy pressure on CAR to approve subprojects and causing difficulty in maintaining the required differentiation; (ii) the poorest Area A communities tended to be less-organized/informed despite CAR’s intensive efforts to train their associations and promote demand through information campaigns; and, (iii) decline of the Municipal Councils withdrew the one-stop-shop facilitation function which had supported the associations for almost two decades. Obtaining a subproject became a less predictable process, until the Additional Financing made alternative, formal arrangements for supporting association decision-making and priority-setting. CAR maintains that Bank loan funds adhered strictly to the 40/60 ratio but that State counterpart funds – which were considerable - did not. Unicamp (2014) mentions that while project beneficiaries were “archetypical” representatives of the rural poor in Bahia, project targeting did not adhere strictly to areas of densest poverty concentration. See table and Annex 2. Table: Targeting by Distribution of Resources and Subprojects Areas # # Value # Subproject % Families R$ m. % Municipalities Original Project 1,658 100 151,978 206,152 100.00 391 Area A 389 29 33,759 51.438 24.95 100 Area B 1,269 71 118,219 151.448 75.05 291 Additional Financing 1,037 100 112,702 152.232 100.00 294 Area A 261 25 25,253 37.878 24.88 76 Area B 776 75 87,449 114.354 75.12 218 Source: CAR/MIS 2014 Table: Results based on proportional application of AF targets to OP Area # SPs (Target) # SPs (Actual) Actual (% of Target) Original Project (OP) 1,950 1,658 85% Area A: 780 389 50% Area B: 1,170 1,269 108% Add. Financing (AF) 500 1.037 207% Area A: 200 261 131% Area B: 300 776 259% 17 OP+AF 2,450 2,695 110% Area A: 980 650 66% Area B: 1,470 2,045 139% Source: CAR/MIS 3.3 Efficiency Rated: Moderately Satisfactory 3.3.1 Seven types of investments accounted for 90% of all subprojects and 87% of all families benefited: water supply including piped systems, household cisterns and small-scale dams; household sanitation; agricultural mechanization (tractor and equipment); honey collection and processing; small bridges; and manioc mills. The efficiency analysis assessed the cost- effectiveness of water supply SPs, conducted a financial analysis of 27 randomly-sampled agricultural mechanization, honey production and manioc mill SPs – all of which affected by drought - then for balance compared financial results of similar investments in Bahia and other Northeast states under without-drought conditions. Results are as follows. 3.3.2 Water supply is a core public service which generates known, sufficient economic benefits to justify its universal provision where ex-ante access is low, and was assessed within a cost-effectiveness framework. Key elements of project design helped to ensure that these subprojects were the least-cost, best alternative for delivering small-scale community infrastructure of this type: (i) accurate targeting, confirmed by Unicamp (2014), FECAMP studies (2004 and 2005) and the larger NE portfolio of Bank-supported CDD operations; (ii) use of standard designs and technical parameters, ensuring least cost solutions deployed at scale; (iii) validity of these investments in drought-prone areas; (iv) significant cost savings relative to similar works executed by public agencies, due to delegation of execution to the community associations - savings for cisterns and household sanitation were respectively, 50% and 18% cheaper than market reference prices; (v) direct, competitive contracting on all SPs, with resulting procurement economy frequently enabling benefits to be extended to additional beneficiaries; and,(vi) generally satisfactory O&M. Cost-effectiveness is rated as Satisfactory. 3.3.3 Using primary data analysis, operational budgets were found to be positive for 16 of the 19 tractors sampled but that drought-suppressed utilization rates did not cover depreciation, and FIRRs were negative for 11 of the 19. Results were better where usage was more intense and even at one-half of technical capacity, was sufficient to generate satisfactory returns indicating that a majority could be viable under better climatic conditions. Similarly in the case of the three honey production SPs, drought induced a severe contraction precisely when these investments were expected to contribute to an expansion of honey production. Production was well below installed capacity. Operational budgets were positive but with low values insufficient to guarantee recovery of the investment. NPV and FIRRs were negative for all three. However, prospects for recovery are judged good but require active interventions in TA and financing to re-populate hives and re-build confidence. For the five manioc flour mills, despite low usage due to drought effects on the manioc crop, they all generated positive net operational revenues, but insufficient to compensate for investments and NPVs were negative. One SP presented an FIRR of 0% and the other two -8% and -11%. 3.3.4 Using secondary data to compare the above results with efficiency outcomes presented in the ICR for Produzir I, benefit-cost ratios were greater than 2.0 for the main productive types analyzed and the investments were financially sustainable. Tractors and manioc mills implemented under non-drought conditions showed FIRRs of 37.6% and 15.8% respectively, and in both cases, less than six years were required to recover the investment. It should be noted that these results took into account increased productivity and production. Data from four other 18 Northeast States using the same project methodology (Pernambuco, Piaui, Minas Gerais and Sergipe) showed FIRRs ranging from 25% to 39% for tractors and 17% to 63% for manioc mills. In without-drought conditions, these investments are viable. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 3.4.1 Overall project outcome rating: This rating takes into account the following: • Relevance: Assessed against the current Bank Brazil CPS, relevance is rated Satisfactory. The PDO itself remains relevant in relation to current Bank/country policies, and the project design and implementation methodology were able – with modest shortcomings - to deliver measurable benefits to the target population. • Efficacy: Due to moderate shortcomings, efficacy is rated Moderately Satisfactory. Evaluation shows modest but positive results in generating economic opportunity and robust social outcomes in health, wellbeing and drought resilience. Unicamp (2014), the HDI results for the relevant decade, and IPEA data (Annex 2) show a clear improving trend in the period in both income and longevity (see para. 3.2.2). Financial leveraging by CAR was impressive and focused on similar types of investments. Total subprojects financed were 110% of the formally-revised aggregate. O&M is well-established for a majority of productive subprojects, tempered by management issues affecting about one- third of the water supply investments. This represents a generalized issue for water supply systems in dispersed rural areas and requires an over-riding solution. Household cisterns and sanitation were generally well-managed, with modest shortcomings. • Efficiency: Due to moderate shortcomings, efficiency is rated Moderately Satisfactory. The cost-effectiveness of WSS, 63% of total investments, is evident. Productive investments (30% of total) were hampered by persistent, severe drought but their viability is judged likely under non-drought conditions. This is borne out by consistent, historical evidence of efficiency in similar Bank-supported investments in Bahia and other Northeast States (see Annex 3). 3.5 Overarching Themes, Other Outcomes and Impacts. (a) Poverty Impacts, Gender Aspects, and Social Development 3.5.1 Gender: CAR has a good track record of ensuring women’s participation and leadership under its CDD operations. The fact that of the 2,758 associations financed, 844 or 31% were led by women suggests that conditions for their participation were favorable. Several of these associations were awarded prizes under the Bank-sponsored “Voz Mulher” (women’s voice) competition. Women also met or exceeded targets for access to productive SPs. Unicamp (2014) reported however, that CAR lacked a strategy for reaching and prioritizing female-headed households, as verified in the PAD and Operational Manual. About 74% of all participating households were led by men, whereas 39% of households in Bahia are female-led and poorer. 3.5.2 Youth: Involvement of young people became a focus in 2011 through a partnership with the National Rural Apprenticeship Service (SENAR) to train and motivate them to participate in associative activities. Youth represented 47% of participants of training for Management Support which stressed young people as “multipliers”, able to influence other youth to become involved in community activities. Even with a modest planning effort, the Project was able to involve and engage youth which is expected to have advantages for the Bahia Produtiva operation. 3.5.3 Indigenous peoples: The Project developed an Action Plan using collective, participatory and culturally-appropriate approaches. Indigenous groups accessed project benefits with support from the State Indigenous Peoples’ Council (CEPI) and other social movements/NGOs. Using the 19 Economia Solidaria strategy via agreements for inter-institutional cooperation with relevant agencies, CAR coordinated investments in poor rural communities. Some 25 subprojects valued at R$3.1 million benefited 3,109 indigenous families in 18 municipalities. Agricultural mechanization was 50% and the rest were animal husbandry. 3.5.4 Quilombolas: 16 The Project established a partnership with Projeto Quilombolas to support culturally and legally appropriate activities based on their constitutional rights to self- determination, traditionally occupied lands and natural resources. Some 142 agreements were signed in 15 “territories of identity”, benefiting 12,486 families with investments totaling R$10.8 million. The pattern of investments followed the overall project trend, comprising mostly agricultural mechanization, WSS, cisterns, manioc flour mills and dairy. See also 3.5.11 and Annex 2 regarding the JSDF Grant activities benefiting quilombolas. (b) Institutional Change/Strengthening 3.5.5 CAR is a strong, well-staffed and experienced institution, supported by a network of Regional Offices. Coordinating the Bank-supported projects is just one aspect of its complex role and responsibilities. Continuing its long association with the Bank, CAR will coordinate the proposed new Bahia Sustainable Rural Development Project (Bahia Produtiva) but will need to make a significant technical and operational transition as the objectives, design and methodology of the new project are a departure from the classic, Northeast CDD approach. Institutionally, this framework has evolved beyond the control of individual projects, and the projects themselves are changing. Councils’ role in the new Bank project will be different, i.e., not directly concerned with priority-setting, governance and resource allocation. As discussed elsewhere, the demise of the MCs saw CAR construct an alternative institutional framework to intermediate associations’ demands. Not unexpectedly, intermediation gained traction to fill this vacuum but Unicamp (2014) describes as “fragmentary” evidence that intermediation became interference. Generally, associations benefited from having the advocacy support of social movements/representatives and local officials when needed. As noted, the Councils are under formal reconstitution. 3.5.6 Community associations: The project worked with 2,758 associations many of which pre-dated the Project and 844 of which were female-led. As has been the case for two decades of CDD operations, their dynamism, capacity and leadership are variable and rectifying this involves training, mutual struggle, collective action and a steady accumulation of social capital. Many associations were established to gain access to the Project while others have acquired the autonomy and capacity to demand and obtain resources/services from diverse programs. Unicamp (2014) found that 76% of beneficiaries and 66% of controls had participated in an association, union, community working groups and/or political party and 77% of beneficiaries had participated in an association, before the Project. While 100% of associations held meetings to select a subproject, this attendance tended to fall off afterwards. (c) Other Unintended Outcomes and Impacts (positive or negative) 3.5.7 Drought, while common in the Northeast region, was especially severe and had negative effects on productive investments during the AF period. Drought also intensified community demand for project water supply investments far beyond what could be financed and to a degree skewed the targeting results. While a portion of that overhanging demand will be passed directly to other programs, many communities will likely miss out. This massive demand for water supply 16 Communities of descendants of escaped African slaves and an especially vulnerable, isolated group. 20 and sanitation revealed the extent of ongoing deprivation in rural Bahia and the need for a major push, especially given the positive results reported by Unicamp (2014) on strengthening communities’ drought resilience. See Annexes 2 and 5. 3.5.8 Japanese Social Development Fund (JSDF) Grant (TF058071): 17 The objectives of the JSDF Grant of R$1.56 m to the National Confederation of Agricultural Workers (CONTAG) were: “to strengthen, support and expand market opportunities for poor Rural Producer Organizations (RPO) in Northeast Brazil organized under the umbrella of rural productive alliances working in the fields of agro-processing and trade, handicrafts or other light industry”. An additional R$100,000 in Grant Funds was approved for CONTAG (TF010751) in August 2011. Achievement of planned activities was rated Satisfactory (GRM, 2012). 3.5.10 Activities and results: Participating RPOs, working with CONTAG technicians: (i) identified challenges to gaining market access; (ii) generated options for responding; and (iii) prepared and implemented 30 investments totaling R$1.7 m (US$0.84m) distributed across three basic types: agro-processing (36%); value-added packaging and distribution (35%); and, improved management of small animal husbandry (29%). Product lines included honey, cashew and other fruits (umbu, mango, banana and papaya), animals (chickens, goats, sheep), vegetables (manioc, beans) and dairy. Financial analysis showed significant improvements in margins paid to producers and overall income from the investments. For small ruminants, improved management practices led to healthier and heavier animals, and into higher prices at sale. Fruit pulp processing generated a four-fold increase in value relative to the sale of the fruit alone. The technologies adopted by the RPOs not only resulted in greater productivity but were also fully compatible with the environmental conditions of the semi-arid Northeast. CONTAG conducted extensive follow-up dissemination of results and lessons to the public and private sectors. 3.5.11 Japanese Social Development Fund Grant (TF P118988: A second JSDF Grant valued at US$877,614 and administered by CAR sought to build institutional capacity of Quilombola Councils and execute small subprojects: basic infrastructure, productive activities, literacy training, food basket distribution and monitoring, computer training and housing support. All planned activities were completed. See activities and outputs, Annex 2, Appendix 3. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 3.6.1 The Unicamp (2014) Final Evaluation used a quasi-experimental design to analyze/present the main outcomes of project interventions, defining for survey purposes two groups: the universe of households which benefited from the Project (Treatment Group -TG) and a group of households which sought access but did not succeed (Control Group - CG). There was no baseline study and for some variables, recall and perceptions were used. The Treatment Group comprised 675 households in 42 municipalities and 132 subprojects; the Control Group, 411 households in 39 municipalities, covering 77 subprojects (demanded but not approved). Main results were as follows (details, Annex 5): 17 Three Northeast states participated: Bahia, Pernambuco and Piaui. CONTAG took the lead with the Bahia portion under the oversight of CAR within the AF. The same states participated in the quilombola JSDF. 21 • Drought had a major impact on the Project’s investments in income-generating SPs. • 45% of tractor beneficiaries said income had improved and attributed it to the Project. • 54% of manioc mill beneficiaries said income had improved due to the Project. • 72% of the TG reported improved quality of life vs. 53% of the CG. • 54% of TG said their health had improved vs. 37% of the CG • 92% of beneficiaries said the Project resolved water availability problems; 96% said the water had improved drought resistance; and 96% said water quality had improved. • 96% reported improved health of children and the elderly from sanitation investments. • Water and sanitation infrastructure causes immediate health impacts on water- borne/related disease for municipalities with no or low ex-ante sanitation conditions. • Regions with higher concentrations of poverty did not necessarily receive more funds but project beneficiary families were described as “archetypical” of the rural poor in Bahia. 4. Assessment of Risk to Development Outcome Rating: Moderate 4.1 The development outcome rating is based on the following: • O&M of productive SPs is generally strong, and not questioned in the Unicamp analysis. Associations are paying into funds or making other acceptable arrangements. • Beneficiary associations were trained to manage their water supply systems but management issues affect about one-third, requiring broader, more systematic approaches which are under discussion by the State based on successful models in other states. • Water cisterns and sanitation facilities are managed by individual households, all of which were trained in usage and hygiene practices which – along with their immediate health and wellbeing impacts – are likely to promote good sustainability, with some moderate shortcomings (see Section 2.5). • Quality issues associated with technical and feasibility analysis as well as execution - factors almost entirely avoidable – may also affect the sustainability of some investments. • Sustainability is built from maturation, social capital formation, consolidation and the nature of local conditions/history affecting the beneficiary group, which is not affected by a single subproject. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 5.1.1 Quality at entry is rated Moderately Unsatisfactory for the OP and Satisfactory for the AF, weighing the following factors: Positive: for the OP, (i) attention to scaling up activities successful under Produzir I including deepening cross-sector integration and resource leveraging to achieve greater impact; (ii) attention to indigenous and ethnic groups; (iii) differentiated targeting of resources to municipalities based on relative HDI-M although this may have hastened the decline of the MCs; (iv) effort to ensure alignment of specific types of investments with the State’s HDI goals; and, (v) for quality at entry of the AF, sound design and rational revision/overhaul of the Results Framework. The only issue is the outdated presentation on the MCs but this was recognized after AF effectiveness and corrected rapidly and comprehensively, i.e. it did not affect the subsequent implementation of the AF. This is offset by: (vi) poorly- designed original Results Framework and ARM, not detected by internal review; (vii) superficial PAD with omissions suggesting pro-forma, follow-on approach and technical issues; (viii) lack of analytical guidance for income-generating SPs despite the Project’s intended support for the 22 State’s HDI goals; (ix) similarly, lack of a strategy for education investments (also HDI-relevant) and planned environmental subprojects. 18 (b) Quality of Supervision Rating: Moderately Satisfactory 5.1.2 The quality of supervision varied between the OP and AF, and the latter’s superiority accounts in large part for the above rating (see 5.1.5). The ICR’s assessment of supervision follows in two parts, OP and AF. 5.1.3 Original project: The rating of Moderately Unsatisfactory is based on the following: (i) brief, unevenly spaced missions, although Fiduciary/Safeguards supervision by Bank specialists was consistently good quality; (ii) brief MTR especially for a large state with over 400 eligible municipalities and no beneficiary survey. This may have stemmed from a degree of complacency and deference towards a long-established client with a fast-disbursing project and an early request for an AF, prompting the MTR team to forego more thorough analysis to focus on the proposed new operation; (iii) MTR failure to recognize the weakness of and restructure the Results Framework; (iv) supervision monitored progress using the ARM indicators which had little relationship to either the RF or the PDO; and (v) failure to perceive MCs’ decline and preparation of the Additional Financing assuming the original institutional structure was still in place. 5.1.4 Additional Financing: Supervision quality was Satisfactory, based on the following: (i) turnover in Bank task management, benefiting the Project including by putting the relationship with CAR onto a more rigorous footing; (ii) adoption of a more analytical approach to supervision, prompting a dialogue with CAR leading to an alternative, formalized framework for subproject decision-making and allocation of resources; (iii) putting CAR on notice that solutions needed to be consistent with the methodology reflected both in the Loan Agreement and Operational Manual; (iv) galvanizing/supporting CAR in launching a state-wide effort to re- constitute and unify the MCs; and (v) taking the correct but unusual step of requesting a 6-month extension of the Additional Financing given the 18-month delay between Board approval and effectiveness. This allowed inter alia, CAR to exceed the original (not only reduced) SP target under the AF and intensify its resource leveraging activities with strong outcomes. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 5.1.5 Quality at entry showed significant shortcomings under the OP and modest shortcomings under the AF and in aggregate is rated Moderately Satisfactory. Supervision of the OP showed significant shortcomings causing opportunities for course corrections to be missed. Supervision of the AF was far more rigorous with positive results and is rated Satisfactory. On balance, Bank performance is rated Moderately Satisfactory, taking into account the overall Outcome rating of Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 5.2.1 Government strongly supported the Project through its counterpart funding contribution which far exceeded estimates under both phases, but did not support its institutional structure, at least under the OP. This changed, partly due to the Bank’s firm position and greater attention 18 While the PAD foreshadowed investment in SPs reflecting all three indices, the Project’s actual obligation was to “contribute” to the Borrower’s objective of improving HDI. 23 from CAR management. The state-wide effort ongoing since 2012 to restructure/unify the MCs within a territorial development framework tacitly assumes Government’s support. Government saw the Project as an integral part of its rural poverty strategy, and as a key tool in expanding water supply to the many thousands of small, dispersed communities still unserved. Government supported the AF - its delayed effectiveness was due mainly to slow Federal approval processes – and is firmly behind the new Bahia Produtiva. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 5.2.2 CAR implemented the Project rapidly and effectively, with both stages fully-committed well before scheduled closing. However, CAR is faulted for not engaging the Bank in a dialogue on the status of the MCs until 2011, although evidence also faults Bank complacency in the face of rapid project execution. CAR worked proactively with the Bank team to find interim solutions to support the associations, and launched a state-wide process to re-constitute the MCs in an updated, unified form. Evidence also suggests CAR tried to compensate from early on by putting a strong effort into training the associations under both stages to strengthen their social capital and “reduce the methodological distance” (CAR, 2014) created by the decline of the MCs. Finally, CAR’s resource leveraging was especially successful, as was its proactive use of the AF’s six-month extension which saw AF (and aggregate, revised OP/AF) targets exceeded. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 5.2.3 The rating takes into account Government’s exceptional/sustained financial support for the Project tempered by its lack of political support for the Councils despite their centrality to the Project methodology and status in project legal documents. CAR is faulted for not advising the Bank but, taking into account its strategic approach to implementation – positioning the project for rapid take-off and disbursement once effective (under both stages) – efforts to find solutions, and support for the new Bank project which marks a shift away from the classic Northeast CDD model, the rating is Moderately Satisfactory overall. 6. Lessons Learned 6.1 CDD projects now have two decades of experience in Bahia with demonstrated strengths in improving family wellbeing, social capital formation and productive activities which complement family income. The project demonstrated that the methodology remains a highly cost-effective avenue to basic social infrastructure acquisition at scale. Significant, more systematic employment and income generation requires adaptation of the methodology and differentiation of targeted groups. Elements such as “growth pole” approaches focusing on selected areas more intensively should be tested before expanding horizontally. Such choices within CDD projects have trade-offs but merit the effort. New projects in Bahia (approved) and the Northeast State of Paraiba (proposed) have adopted this concept. Mobilization, training/TA and information need modernization, and linkages to mainstream sources of financing are essential. 6.2 Appraisal of CDD projects of this type needs to weigh the costs and benefits relative to other programs already available and with similar objectives. While a strong argument can be made for the OP appraised in early 2005 when many large-scale public programs in Brazil were still expanding slowly/unevenly, the justification for the Additional Financing appraised in 2009 could have been improved. Appraisal should have looked realistically at the extent to which the proposed AF was complementary or even necessary, especially as CAR’s resource leveraging 24 capacity was proven. A justification on the grounds of water supply and sanitation deficits alone would have been stronger, especially given their proven cost-effectiveness and capacity to rapidly reach otherwise un-serviced and remote rural populations. 6.3 Natural resources management at the community and municipal level in Northeast rural projects needs to move beyond the traditional focus on Safeguards compliance. Much can be done at the community level utilizing the models and lessons learned globally, adapted to the soil and climatic conditions of the Northeast. Basic principles including simple, climate-smart technologies should be introduced, coupled with awareness-building and incentives to convince poor producers of the longer-term payoff of adoption. Extreme drought conditions understandably focused attention on water supply, but awareness-building and selected pilot investments could have been conducted in parallel, demonstrating that drought resistance goes beyond water availability per se to include land and water conservation, and appropriate farming practices. Monitoring and evaluation require intensive focus from the project preparation stage to closing and beyond. Missions need to engage the counterpart in a specific M&E dialogue and adopt a problem-solving approach. Early restructuring of an unsatisfactory RF is essential and project teams need to periodically assess continued relevance/measurability. Results Frameworks need timely baselines, controls and a results chain linked to the Project’s expected change in beneficiary behavior. The evaluation of household water supply and productive investments may need differentiated evaluation streams/methodologies. In the case of productive SPs, if the economic and financial analysis finds that financial flows are positive, the investment is probably sustainable. With household water supply, the test is social and environmental. Above all, the institutionalization of M&E is essential. Successive projects should contribute in an organized manner to the overall body of data/knowledge on rural poverty available for policy-making and planning. 6.5 Rapid preparation of Bank projects needs to be balanced by the need to step back from “follow-on” mode. Teams need to assess both the external environment of the project including political and institutional shifts – although in this case these could not have been known in 2005 - and changing needs of the client and the poor. The market linkage aspiration was not consistent with the existing implementation arrangements, nor was the resource leveraging concept consistent with the real capacity and influence of the MCs at appraisal, implying in this case that a follow-on operation may not have been the most appropriate path. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies: 7.1 CAR’s letter of July 14, 2014 commenting on the Bank’s draft ICR summarizes the Project’s trajectory and key features/successes but is critical of the original design, especially its ambitious role for the MCs, and certain indicators including the expectation that productive investments would create full-time employment. Importantly, CAR notes the drought-related negative financial results for productive SPs and undertakes to provide – via its Regional Offices - technical monitoring/assistance with a follow-up efficiency analysis in two years. CAR disagrees with the Moderately Satisfactory rating for overall outcome, and requests that this rating be revised, pointing to major efforts to achieve project objectives. The Bank team stands by its rating. (b) Co-financiers N/A (c) Other partners and stakeholders N/A 25 Annex 1. Project Costs and Financing A. Original Project (Loan 7327-BR) (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Community Subprojects 66.06 105.72 160.0 Area A: 26.42 27.07 102.5 Area B: 33.58 79.34 236.3 Institutional Development 3.90 5.10 130.8 Project Administration, Supervision, Monitoring and 3.30 7.80 236.4 Evaluation Total Baseline Cost 73.26 119.31 162.9 Physical Contingencies 0.80 0.00 0.00 Price Contingencies 0.40 0.00 0.00 Total Project Costs 74.46 119.31 160.2 Front-end fee PPF 0.54 0.00 .00 Total Financing Required 75.00 119.31 159.1 Note: Values allocated to Physical and Price Contingencies were reallocated to other components (b) Financing: Original Project Appraisal Actual/Latest Type of Co- Estimate Estimate Percentage of Source of Funds financing (USD (USD Appraisal millions) millions) Matching Borrower 20.65 64.96 314.6 Grants International Bank for Reconstruction Matching 54.35 54.35 100.0 and Development Grants B. Additional Financing (Loan 7732-BR) (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Community Subprojects 34.67 76.14 219.6 Area A: 13.86 18.94 136.7 Area B: 20.81 57.20 274.9 26 Institutional Development 2.70 5.10 188.9 Project Administration, Supervision, Monitoring and 1.76 15.60 886.4 Evaluation Total Baseline Cost 39.13 96.80 247.4 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 39.13 96.80 247.4 Front-end fee PPF 0.07 0.00 0.0 Unallocated 0.80 0.00 0.0 Total Financing Required 40.00 96.80 242.0 Note: Values allocated to Physical and Price Contingencies, and Unallocated, were reallocated to other components. (b) Financing: Additional Financing Appraisal Actual/Latest Type of Co- Estimate Estimate Percentage of Source of Funds financing (USD (USD Appraisal millions) millions) Matching Borrower 10.00 66.84 668.4 Grants International Bank for Reconstruction Matching 30.00 30.00 100.0 and Development Grants 27 Annex 2. Outputs by Component 2.1 This Annex comprises: (i) table showing the original (OP) and revised (AF) Results Framework, and what was monitored (PAD Arrangements for Results Monitoring - ARM); and (ii) key outputs by component, OP and AF. A. Evolution of Results Framework: PDO (OP PAD) Results Framework ARM Indicators Revised Results Indicators (PAD) 19 (PAD) Framework (AF) PDO (OP and AF Loan Agreements): “Increase social and economic opportunities for the Municipalities’ rural poor by improving their access to basic socioeconomic infrastructure, thus contributing to the Borrower's objective of increasing the Human Development Index (HDI)”. PDO Sub-clauses: 20 Outcome Indicator: Outcome Indicator: Outcome Indicators 1. Integrating with and/or 1. 2500 subprojects 1. % of rural poor 1. Evidence of complementing activities of implemented and self-selecting into the improved living other Bank loans for the State maintained by Project, identifying conditions at (under the Bahia State community and implementing beneficiary household Program), particularly the associations subprojects. level PGRH and Viver Melhor 2. Proportion of Urbano II productive investments generating 2. Strengthening cross-sector full-time employment integration and providing for beneficiaries significantly more emphasis 3. ERRs for on education, health, culture, productive subprojects natural resources exceed 12% management and 4. Min. 60% of environmental sustainability Municipal Councils deliberating resource 3. Scaling-up the impact on allocation from poverty by using the skills, programs outside the social capital and experience project of the Municipal Councils 5. Ratio of funds and community associations leveraged to project to improve the relevance, funds increases over efficiency, sustainability, the life of the project targeting and outcomes of 6. Participatory non-project State and Federal decision-making being investments in rural Bahia. used to target and allocate non-project resources 19 The Results Framework shows only the primary statement of PDO and in an abbreviated form of the OP and AF Loan Agreements; the Key Indicator is an output, not outcome although it could be argued that “self-selection” into the project was a “behavior change” and thus an Outcome. 20 Shown in the PAD Main Text as part of the PDO, structured as dependent clauses joined to the main statement with “by”. These clauses do not appear in the PAD RF, or in the OP or AF Loan Agreements. 28 Intermediate Intermediate Intermediate Outcome Indicators: Outcome Indicator: Outcome Indicator: Component 1: Component 1: Component 1: 1. MIS registers SP 1. 2,500 subproject 1. Min. 800 proposals and tracks investments community implementation; implemented and subprojects 2. Periodic evaluation, maintained by implemented. and, Bank and CAR community 2. Min. 320 supervision missions; associations; communities benefited 3. CAR semi-annual in Area 1, and 480 in reports Component 2: Area 2. 2. % Community 3. Productive Component 2: associations receiving subprojects are 4. MIS tracks number training in subproject minimum 35% of total and type of training management & and generate events; operation; incremental 5. Periodic evaluation, 3. % Municipal employment and and, Bank and CAR Councils receiving income. supervision missions; training in HDI-M 4. Women receive 6. CAR semi-annual and priority-setting; 30% of all productive reports 4. % CAR staff subprojects. receiving TA and 5. min. 50% reduction Component 3: training for in incidence of 7. MIS monitors SP community waterborne disease in cost, type, geographic associations and communities with distribution and Municipal Councils. water supply relation to HDI-M; subprojects. 8. Periodic evaluation, Component 3: and, Bank and CAR 5. Baseline Study Component 2: supervision missions; 6. MTR 6. 80% of participating 9. Evaluation 7. Final Evaluation CAs have benefited framework from at least one training/TA course 7. 80% of MCs have participated in at least one training/TA course 8. STU personnel participate in min. 2 training courses per person. Component 3: 9. 2 independent audits completed and a third contracted 10. Full physical completion study completed and reviewed for MTR 11. STU has finalized on full-scale impact evaluation study and Bank has reviewed 29 B. Physical Outputs by Component: 2.2 The following shows physical outputs by component with a separate discussion under Component 1 on productive subprojects, water supply and sanitation subprojects and targeting performance, and under Component 2 on training and the institutional development of the Municipal Councils and community associations. Data is shown separately for the OP and AF, where possible/relevant. Component 1: Community Subprojects 2.3 Financing of US$ 67.5 m. (OP) and US$ 34.67 m. (AF), around 90% and 89% of total estimated cost, respectively) supported community matching grants for about 2,500 (and under the additional Financing another 800) small scale socio-economic infrastructure/productive, education, health, cultural, and environmental investments, with the productive/income generating, education and health investments intended to align with the component indices of HDI to support the Borrower’s efforts to increase it. The idea was that Community Associations (CA) would identify their most pressing needs aligned to the extent possible with HDI as a result of information campaigns conducted by the State to raise awareness of the HDI and its constituent indices. Some 407 municipalities in Bahia were expected to be covered, with 40% of project resources concentrated in the 100 poorest municipalities based on Municipal Human Development Index (HDI-M). Associations would present subproject proposals to their participatory Municipal Council which would establish and vote on investment priorities for transmittal to CAR for feasibility analysis and approval. CAR would then transfer project resources directly to beneficiary communities to implement, operate and maintain (with some exceptions) their subproject. 2.4 The project financed an aggregate 2,695 subprojects (OP 1,658 and AF 1,037), benefiting a total of 264,892 families (OP 150,172 and AF 114,271) in 396 municipalities. See 2.5 on costs. Funds for approved subprojects were transferred directly from CAR to the beneficiary associations, an enduring feature of the NE CDD projects. See table below. Table 1(a): Subprojects and Beneficiaries - OP Targets OP Projected Actual % of Target No. municipalities 407 396 97 No. families 200,000 150,098 75 No. Community Associations - 1,735 - No. community subprojects 2,500* 1,658 85 *Reduced in 2010 to 1,950 subprojects due to increased average subproject cost Table 1(b): Subprojects and Beneficiaries - AF Targets AF Projected Actual % of Target No. municipalities 407 293 72 No. families 56,000 106,890 191 No. Community Associations - 1,023 - No. community subprojects 800* 1,037 207 *Reduced in 2012 to 500 subprojects for similar reasons 30 Table 2(a): Subproject Demand and Implementation OP 21 Subproject Type Demanded Completed Health and Sanitation 1,633 1,070 Education and Culture 18 16 Income Generation 797 469 Infrastructure 264 67 Institutional Development 53 34 Environmental 5 2 Total 2,770 1,658 Source: CAR 2014 Table 2(b): Subproject Demand and Implementation AF Subproject Type Demanded Completed Health and Sanitation 11,910 675 Education and Culture 180 3 Income Generation 8,568 316 Infrastructure 3,110 30 Institutional Development 331 12 Environmental 42 1 Total 24,141 1,037 Source: CAR 2014 Table 3(a): Categories of subprojects financed by family, municipality and total value: OP Total Value of Total SPs # Type of Subproject # Families Investments Financed Municipalities (R$m.)* Health and Sanitation 1,070 64,207 331 149.784 Income Generation 469 58,681 241 48.920 Education/Culture 16 966 11 1.075 Infrastructure 67 25,044 60 5.810 Institutional Development 34 3,031 33 0.445 Environment 2 49 2 0.118 Total 1,658 150,978 NA 206.152 Table 3(b): Categories of subprojects by family, municipality and total value: AF # Total Value of Total SPs Type of Subproject # Families Municipalities Investments Financed 22 (R$m.) 21 Final data on subprojects physically completed and with accounts submitted and approved are: 1,658 OP (85% of target) and 1,037 AF (207% of target). A subproject is “completed” when the beneficiary association has submitted its statement of accounts (prestação de contas). 31 Health and Sanitation 675 37,091 223 105.378 Income Generation 316 47,702 176 42.318 Education/Culture 3 254 2 0.262 Infrastructure 30 23,974 24 3.579 Institutional Development 12 3,656 11 0.523 Environmental 1 25 1 0.172 Total 1,037 112,702 437* 152.232 *With repetition Table 4: Subprojects Financed: Families and Cost per Family and Subproject, OP and AF Aver. % Total Aver. Cost Total Subproject # SPs Total # Cost per # SPs per Family Cost Families SP R$ R$ ‘000 R$ ** Household Cistern 857 49,165 142,615.00 34.10 2,485.94 122,221.11 Household Sanitation 640 32,750 152,500.00 27.23 2,910.15 97,600.00 Ag. Mechanization 431 66,527 109,570.00 13.18 709.68 47,224.89 Small-scale Dam 60 6,524 90,092.00 1.51 828.56 5,405.56 Honey Processing 55 4,171 107,010.00 1.64 1,411.06 5,885.56 Concrete Bridge 90 45,160 94,493.00 2.37 188.32 8,504.44 Water Supply System 151 10,794 164,319.00 6.92 2,298.70 24,812.22 Manioc Mill (electric) 94 10,472 122,706.00 3.22 1,101.45 11,534.44 Other 317 39,117 111,027.00 9.82 899.75 35,195.78 TOTAL: 2,695 264,680 133,975.00 100.00 1,354.02 358,384.00 Source: CAR/MIS **Includes beneficiary counterpart 2.5 The average cost per subproject in most cases increased steadily over the course of the OP due to inflationary pressures and effects of exchange rate movements between the Real and US Dollar (see below). Under the AF, the cost of some subprojects fell from OP levels while others increased. The impact of exchange rates and cost inflation in construction materials, labor and equipment is perhaps most visible in the increased cost of water supply systems which increased from R$132,074 under the OP to R$253,109 under the OP (incremental 107%), although the number of beneficiary families per system rose from 71 to 77. The average cost of a manioc mill increased from R$98,278 under the OP to R$110,296 for the AF, an incremental increase of 12%; for a small-scale dam from R$59,641 to R$80,521 an increase of 35%; and for a concrete bridge from R$480,623 to R$495,804 an increase of 19% Targeting, OP and AF: 2.6 Targeting was envisaged in terms of a division of available resources for subprojects between two cohorts of rural poor – Areas A and B - defined by relative HDI-M. With the declining importance of the Municipal Councils in the subproject targeting and priority-setting framework, and given the importance of giving community associations a voice in the decision- making process, diverse social movements and forums were recruited by CAR to help 22 As at January 2014, Bahia had 417 Municipalities of which 10 were within the metropolitan region of the capital Salvador and ineligible for the Project. Numbers of benefited municipalities indicate repetition. 32 communities gain access to the Project. These included: State Quilombola Council (CEQ); Bahia Family Agriculture Forum (FBAF); participatory Territorial Colleges (TC); State Territorial Council (CET); State Sustainable Rural Development Council (CEDRS); State Indigenous Peoples’ Council (COPIBA); and other social movements and sector productive groups. 2.7 As shown in the tables below, targeting performance fell short of what was envisaged under either the OP or AF, for several reasons: (i) weaker organizational levels of the poorest communities which tended to be less agile, informed and connected (see Component 2 below); (ii) weakening of the Municipal Councils which had for two decades provided an open, participatory forum for facilitating inter alia, the targeting and allocation of project resources; (iii) massive local, spontaneous and drought-driven demand for project support which put pressure on CAR to approve subprojects in a more random, flexible way, skewing the targeting plan, i.e., it was difficult for CAR to maintain the intended differentiation. CAR is adamant that Bank loan resources did conform to the original targeting ratio but noted that Borrower counterpart funds did not. See tables Main Text 3.2.11. Vulnerable populations: 2.8 Women led some 844 (32%) of the total 2,639 associations and obtained a minimum 30% of all productive subprojects. Women led 70% of all manioc mill investments, 30% of agricultural mechanization, and 35-40% of other productive subprojects. However, the targeting of female-headed households was not part of the project strategy. Unicamp (2014) reported that 76% of beneficiary heads of household were male even though close to 40% of all households in Bahia are headed by women. (While this includes urban areas, rural areas usually show higher incidence of female-headed household). 2.9 Indigenous groups, among the poorest, most isolated and vulnerable, were also able to access the Project through CAR’s special efforts to build awareness, organize meetings state-wide with indigenous leaders, and establish partnerships and collaboration with entities including the National Indian Foundation (FUNAI) via its local offices, and the Indigenous Affairs Coordination of the State Secretariat of Justice, Citizenship and Human Rights. Of the 78 priority demands which resulted from the State Government’s operational plan to attend indigenous peoples, Produzir financed 28 subprojects valued at R$4.1 million benefiting 4,665 families (about three percent of all subprojects and families benefited). About half of all investments were for agricultural mechanization and the remainder for animal husbandry activities, small-scale irrigation and manioc mills. Resources for four of these investments were leveraged from the Vida Melhor Program. CAR technical staff attended/supported the quarterly meetings of COPIBA and monthly meetings of the United Movement of Indigenous Peoples of Bahia (MUPOIBA). 2.10 Quilombola communities received 40 subprojects valued at R$5.4 million with a high proportion of household water cisterns, manioc mills and sanitation systems, as well as tractors/equipment and dairy facilities. CAR and its Regional Offices organized awareness- building encounters with quilombola community leaders and their representatives and were fully cognizant of the processes essential for their inclusion in the Project. A Japanese Social Development Fund Grant (TF P118988) of US$877,614, “Leveling the Playing Field for 33 Quilombola Communities in Northeastern Brazil”, supported capacity building and small-scale subprojects in infrastructure, productive activities and training. See Appendix 3. Table 5: Targeting and Distribution of Subprojects % Total # % of Total # # Subprojects Eligible Municipalities Eligible Municipalities OP and AF Municipalities AF* Municipalities OP OP AF 00 – 05 285 76.82 242 83.86 06 – 11 83 18.23 43 13.33 12 – 17 16 3.65 5 1.40 18 – 24 5 0.78 3 1.05 25 – 33 2 0.52 1 1.35 Total 391 100.00 294 100.00 *Aggregate municipalities indicate repetition 2.10 As shown, 368 municipalities (94%) under the OP and 285 municipalities (97%) under the AF received from zero to 11 subprojects each, while 5% and 3% respectively received from 12 to 33 investments each, indicative of significant concentration of lending, a feature noted by Unicamp (2014). This does not imply leakage to the non-poor, as both the Area A and B cohorts qualified as rural poor under key socio-economic indices and are described by Unicamp (2014) as “archetypical” of the rural poor population in Bahia. It does however, suggest varying levels of capacity and organization to manage the process of gaining access to the Project. Table 6: Targeting by Distribution of Resources and Subprojects Areas # # Families Value R$m. % # Subprojects Municipalities Original Project 1658 151,978 206.152 100.0 391 Area A 389 33,759 51.438 24.95 100 Area B 1,269 118,219 151.448 75.05 291 Additional Financing 1,037 112,702 152.232 100.00 294 Area A 261 25,253 37.878 24.88 76 Area B 776 87,449 114.354 75.12 218 Source: CAR/MIS 2014 2.11 As shown in Table 6, under both stages, the Area A poorest municipalities received about 26% of available subproject financing and Area B about 74% vs. the planned 40%/60% ratio. Resources leveraged from other programs: OP and AF 2.12 The Project intended under both stages to leverage additional resources in a ratio of 1:5 (R$5.00 from other State and Federal programs for every US$1.00 of Bank loan funds) and this activity represented an important theme of the PDO as stated in the PAD. The project Municipal Councils (Municipal Councils for Sustainable Development – CMDS – in Bahia) were to be the primary vehicle for such leveraging but in practice did not play any role. Leveraging was successful but occurred exclusively through partnerships between CAR and other State and Federal programs. Further, each of those programs has its own rules, eligibility and targeting 34 requirements and geographic focus. Thus, the PDO objective of using the Municipal Councils to improve the targeting, efficiency, sustainability and impact of such programs by exposing them to, and thereby bringing them into conformity with, the Project’s methodology was not only ambitious, but implied significant institutional capacity in the MCs and a complex and multi- pronged evaluation exercise beyond the scope and timing of the Project. 2.13 As shown below, CAR leveraged through formal partnerships an aggregate R$413.48 million, about US$206.0 million (using an average USD/Real exchange rate of US$1.00 to R$2.00 for the period). Data provided by CAR show that leveraged resources financed investments benefiting an additional 127,130 poor rural families, attributed as follows: • Agua para Todos (Federal Government, water supply): 63,900 families • Productive Subprojects/Activities (Federal Government): 14,640 families • Productive Subprojects/Activities (State Government): 36,190 families • Other Projects/Programs: estimated 12,400 families Table 7(a): Programs and Resources Leveraged - OP Executed (R$ m.) Program/Activity Total Value Benef. Value: Value: Value: Contrib. State Federal Banks Development of the Rio Gaviao 3.533 - 3.534 - - Regional Development Activities (PAADR) 0.484 - 0.484 - - Sustainable Regional Development (PDRS) 0.207 - 0.207 - - PRODECAR 67.540 - 40.971 - - Credito Fundiario 1.940 - 0.627 1.313 - Social Infrastructure Projects 0.907 - 0.907 - - Production Support Programs 2.374 - 1.208 1.166 - Regional Research Efforts 0.169 - 0.169 - - Formulation of Sustainable Programs 0.013 - 0.013 - - Mata Branca Program 0.177 - 0.177 - - Sustainable Development of the Semi-Arid 33.935 - 29.901 4.037 - Water Resource Infrastructure Installation 8.668 - 8.668 - - TOTAL 119.949 - 86.867 6.512 - Source: CAR/MIS 2014 Table 7(b): Programs and Resources Leveraged - AF Executed (R$ m) Program/Activity Total Value Ben. Value: Value: Value: Contrib. State Federal Banks PRODECAR 89.665 - 65.257 - 24.408 Production Support Programs 128.278 - 72.942 55.336 - Regional Research Efforts 0.001 - 0.001 - - Mata Branca Program 3.831 - 3.831 - - Sustainable Development of the Semi-Arid 21.069 - 19.109 1.961 - Social Infrastructure Projects 1.151 - 1.151 - - Water Resource Infrastructure 48.765 - 33.574 15.191 - Quilombola Projects (Japanese Trust Funds) 0.618 - - - 0.618 35 PRONAF: Support to Family Agriculture 0.152 - - 0.152 - TOTAL 293.529 - 195.864 72.639 25.026 Source: CAR/MIS 2014 2.14 Registered demand greatly outstripped available resources, the result of CAR’s information campaigns and the pressures of severe and protracted drought on rural poor families lacking water supply. Some of this excess demand will be absorbed by the proposed new Bahia Produtiva operation and by Federal programs, particularly the water supply, sanitation and cisterns. In regard to proposals for productive investments, the Business Plans contemplated under the new project will absorb some of this demand for productive inclusion, as will the Viver Melhor Program. Separately, during the execution of Produzir, a significant number of subprojects actually approved did not receive financing. CAR has already passed many of these to other sources: BNDES, IFAD, MDS, CODEVASF, Ministry of Agrarian Development and agencies of the State Government. 2.15 Component cost: Total cost of the OP Component 1 was about US$105.72 m, 160% of the appraisal estimate with Area A investments about 50% of the reduced estimate (based on the formal revision of the overall targeted SPs to 1,950) and Area B about 108%, under the OP. For the AF: total cost was US$ 76.4 m, 219.6% of the estimate while Area A was 131% of the revised target and Area B 259%. As noted, exchange rate fluctuations and inflationary pressures explain these numbers, in addition to higher numbers of subprojects than expected under the AF. Component 2: Institutional Development 2.16 Financing of US $ 3.90 million (OP), and US $ 2.70 million (AF), around 5% and 7% of total estimated costs respectively, was intended to support: (a) technical assistance and training to support the mobilization and strengthening of the Community Associations (CAs) in their role of identifying, preparing, operating and maintaining subproject investments; (b) capacity-building for Municipal Councils (MCs) on responsibilities outlined in the Project Operational Manual, including assessment and supervision of CAs, participatory planning and the financial management of community subprojects, and environmental management of small community subprojects; (c) subproject-related training and technical assistance; (d) workshops and seminars for MCs and CAs in order to exchange experiences concerning project implementation, and to facilitate integration of other Federal and State programs for poverty reduction. The following table shows basic training outputs under both stages. Table 8: Training – Events and participants 2006-2014 Year Beneficiaries CAR Technical Team Total Events Events # Participants Events Participants 2006 136 3401 18 117 154 2007 162 5155 4 172 166 2008 110 2417 7 586 117 2009 60 2200 51 616 111 2010 108 967 18 342 126 2011 44 2928 49 742 93 2012 111 4389 29 383 140 36 2013 145 5046 22 468 167 2014 3 782 3 209 6 Total 879 27,285 201 3,635 1,080 Source: CAR MIS Outputs by recipient: 2.17 Community Associations: CAR put a major emphasis on training the associations, especially on preparing them for associative/collective activities, for the minimum technical requirements of their subproject, and strengthening their social capital to adequately implement, operate and maintain their investment and promote its sustainability. 2.18 Training was delivered in five categories: Management Support (22%); Technical and Operational Training (17%); Institution-strengthening (32%); Water Management (5%); and Socio-productive Inclusion (24%). The relatively modest proportion assigned to water management is notable given the prominence of WSS and cistern investments, and may account in part for the management issues reported in several studies. Under both phases, CAR formed partnerships with public agencies such as SEBRAE, SENAR and other specialist entities to support the training plan and to integrate Produzir with the activities of other bodies not just as a complement but also to diversify the content of training courses, both conceptually and methodologically. 2.19 Training was conducted via thematic seminars, workshops and exchanges of experiences. About 13,800 family farmers were trained under the OP and another 13,145 under the AF. Management Support was prioritized both for management of associations per se (planning, accounting, leadership, public policies etc) and of subprojects, especially their O&M. Financial management of subprojects by associations was especially challenging and received intense focus to bring them into line with State and Federal fiscal requirements. An important innovation was motivating youth to join associations because of their higher education levels and capacity to absorb more complex training. CAR also established a technical cooperation agreement with SENAR into this framework and for broader training including of CAR technicians. CAR observed that the presence of younger members in associations improved their performance. In the period from January 2011 to April 2014 (post-project), some 47% of all training was in Management Support, training youth as “rural multipliers”. Specific types of subprojects and beneficiaries involved in specialized training: 2.20 WSS and cistern beneficiaries were trained in water conservation, rationing and quality. Multiplier teams of five people per community were trained: one health agent, two young people and two association leaders. Environmental aspects were stressed including climatic variation and sources of water contamination. Water management became urgent during the protracted drought which created massive demand for water supply and related appropriate training. About 75% of all water management training was in the years 2011-2013. 2.21 Agricultural mechanization beneficiaries were trained in O&M of the equipment and in awareness of potential environmental impacts on soils and water. As with WSS, five association members were selected for training as operators and maintenance personnel, including the 37 association leadership, enabling them to understand what was involved in the rational management and use of the tractor/equipment. The trained group was obligated to pass on their knowledge to the rest of the community. 2.22 Other productive: In partnership with specialized agencies, CAR conducted courses in rustic poultry-raising, fish farming, goat and sheep-raising, beekeeping, honey and milk production, and manioc products. Inter-community meetings promoted exchanges of experiences/knowledge. 2.23 Social: Training of vulnerable, traditional groups was emphasized - quilombolas, indigenous groups, fishing communities - using meetings, seminars, exhibitions and markets to promote their culture, recover their history and promote dialogue with public agencies. Culturally-appropriate training techniques were applied where relevant. 2.24 Training partnerships: CAR established formal partnerships with SENAR, the Superintendency for Economia Solidaria, the Secretariat for Labor, Employment, Income and Sport (SETRE), Coopermulta and the Regional Institute for Small Agro-livestock Properties (IRPAA) to expand the range and content of training demanded state-wide under Produzir. 2.25 Municipal Councils: The Project was intended to provide training to the participatory Municipal Councils (MC). As discussed in the Main Text Section 2.2, the MCs began to decline after the political turnover in the State Government in 2006. However, this process was not immediate and for a period of several years, there was a transition away from the MCs and towards Territorial Colleges as well as diverse sector and social forums which picked up where the MCs left off in terms of providing support for associations seeking to access the Project. CAR maintains that some of the stronger MCs remained operational up to the time of the AF. The methodology used by the various alternative forums was participatory but they did not have the standardized procedures and one-stop-shop characteristics of the original project MCs. In 2012, CAR began a concerted effort to re-constitute the Councils (including via unification with other Councils active in the rural space) as Municipal Councils for Sustainable Development (CMDS), of which at ICR finalization there were 270 operational. It was observed that the CMDSs can play an important role in the Territorial Colleges in representing community associations’ interests, an important rationale for their reconstitution. 23 2.26 Notwithstanding the above, in the year following OP effectiveness, 267 mobilization and training events (mainly on the new Operational Manual, community organization, investment priority-setting and HDI) benefited 100% of the Municipal Councils (MC) and 7,154 members. Also, about 350 MCs participated in Territorial Development events to discuss/contribute to the participatory preparation of the State’s Multi-year Development Plan (PPA), and Economic and Environmental Zoning efforts. More recently, CAR, in conjunction with SUAF/SEAGRI, 23 In its comments on the Bank’s draft ICR, CAR expressed disagreement with key aspects of the Bank’s analysis especially concerning the decline of the Councils, stating that the Bank was informed much earlier than 2011, and that its own efforts to reconstitute the Councils actually started in 2007. The Bank team stands by the documented record on this issue, including CAR’s actions. 38 SEPLAN, EBDA and other agencies along with Territorial Colleges and municipal mayors organized 27 Territorial Conferences for Sustainable Rural Development and Solidarity with 3,240 attendees from organizations, social movements, technical specialists of public institutions, and technical assistance and rural extension providers. These conferences were critical for the effort initiated in 2012 – upon the urging of the Bank team leader - to establish alternative participatory arrangements designed to provide greater support to the associations in accessing the Project, and to re-constitute the Municipal Councils. Some 270 MCs had been restructured as CMDSs at the time of ICR finalization out of 416 municipalities (65%) and all participated in training events. 2.27 CAR: Training of CAR technical and administrative staff represented some 17% of all training events, covering operational, theoretical and technical aspects of CAR’s project management and coordination role. CAR also produced a large amount of instructional materials for training recipients, and ensured that communities were brought up to date on the results of workshops and other forums, through their leadership. Many local meetings were conducted in project-financed garages constructed for tractors. As noted above CAR was the major actor in initiating restoration of the Municipal Councils and has promoted awareness of this and other ongoing rural events through its online “Informativo Produzir”. 2.28 Due to the large number of agreements signed with community associations, 178 training events involved 2,070 CAR technical and administrative staff (including regional offices). Training covered: evaluation of subproject demands, supervision and monitoring of agreements, community organization and participatory planning, planning, economia solidaria, methodology of social work, submission of accounts and other responsibilities including updating the Operational Manual. CAR also initiated a process called “Dialogos” for the systematic exchange of information between its Regional Offices and among the various programs under its management, intended to strengthen internal communication and encourage sharing of new ideas. 2.29 Finally CAR, working with all other participating Northeast states and under the rubric of regional efforts to move forward on the marketing of products resulting from productive investments, designed a Trade Mark and logo for those products to identify them as emanating from project investments: Producao Familiar Solidaria. 2.30 Component 2 cost: The total cost of Component 2 under the OP was about 131% of the appraisal estimate, increasing to about 189% under the AF. The marked emphasis on training of the associations under both stages, plus CAR’s efforts to train the restructured MCs, as well as implementing and consequently training respectively, significantly more SPs/associations than originally planned either at AF appraisal or under its July 2012 restructuring (800 at appraisal, reduced to 500 in July 2012, with 1,037 actually implemented) explain a good part of this. Also, intensified water resources management training during the AF and under drought pressures was also an important factor pushing up the cost of this component. Component 3: Project Administration, Supervision, Monitoring and Evaluation: 39 2.31 Financing of US$ 3.30 million (OP), and US$ 1.76 million (AF), around 5% of total estimated costs at both stages, this component supported the costs (excluding salaries) of project administration, monitoring and impact evaluation. 2.32 The products of project M&E were: (i) Borrower Completion Reports for the OP and separately, the OP and AF combined; (ii) a Final Evaluation Study (Unicamp 2014); (iii) three studies by AGIR/CAR listed in Annex 9. CAR did not produce the intended beneficiary survey- based study for the first or second MTRs, although even conducting an MTR during the usually short AFs was unusual. However, CAR financed immediately post-closing, a simplified physical performance/technical study of subprojects using secondary information, interviews with beneficiaries and association leaders, and field inspection of a proportional, representative random sample of subprojects. Results were used to complement the economic and financial analysis (Annex 3). Finally, CAR maintained and upgraded its MIS and established the SACC (Monitoring and Administrative Management System), an online tool for registering and monitoring the status of demand for, execution and evaluation of training activities. 2.33 Component 3 cost: The cost of this component was 236.4% of appraisal under the OP and 886.4% of appraisal under the AF. Both are explained by CAR as the result, inter alia, of CAR’s immense effort to leverage additional, complementary resources from other programs, which greatly intensified under the AF. The administrative costs for this effort were not included in cost estimates for either the OP or AF. It is also possible, but was not verified or analyzed, that the administrative costs of CAR’s efforts to reconstitute the MCs were covered – at least in part – by State counterpart under Component 3. Appendix 1: Movements in the US$/Real Exchange Rate and Inflation, 2006-2013 2.34 Exchange rate fluctuations and inflation contributed to increased SP costs, compared to initial estimated values at appraisal. Throughout project implementation, the exchange rate varied significantly, while inflation steadily increased. The exchange rate of the Brazilian Real per US Dollar went from around R$ 2.22 in January, 2006 to R$ 2.26 in January, 2014, with significant peaks in valuation on July, 2007 (R$ 1.57) and July, 2011 (R$ 1.56). The valuation of the Brazilian Real for around two-thirds of the project implementation period (from 2006 to 2008, and 2009 to 2011) increased the unit cost of SPs above the values estimated during preparation (taking into account the estimated average unit price of subprojects in US$). Furthermore, a steady rise in domestic inflation also increased the local unit costs of materials, equipment and labor. See below. 40 100 120 140 160 40 60 80 2006.01 2006.04 2006.07 Appendix 2: 2006.1 Source: Ipeadata, Ipea. 2007.01 2007.04 2007.07 2007.1 2008.01 2008.04 2008.07 2008.1 2009.01 2009.04 2009.07 41 2009.1 2010.01 2010.04 2010.07 2010.1 2011.01 2011.04 Exchange Rate vs. Inflation 2011.07 2011.1 Average per capita household income (R$) (Source: IPEA 2012) 2012.01 2012.04 2012.07 2012.1 2013.01 Inflation 2013.04 2013.07 Exchange Rate 2013.1 2014.01 1 2 1.2 1.4 1.6 1.8 2.2 2.4 2.6 2.35 In Bahia, inequality of average income was reduced in the period 2000-2009. Household per capital income rose 62.1%, from R$137.5 to R$222.9 by 2009. Improvement was registered in both urban and rural areas, although the rate of improvement was greater in urban areas. See Graph 8 above. Population with water supply (%) (Source: IPEA 2012) 2.36 In 2009, access to water supply in Bahia, while still below national rates, was better than access for the Northeast region as a whole. However, disparity between urban and rural access remains significant. Water supply systems (piped) in Brazil increased coverage in the period from 81.4% in 2001 to 87.7% by 2009. But regional differences remain pronounced. Over 17% of the population remained without water supply by 2009 and in rural areas coverage was still only 54.5% by 2009 despite significant growth over the decade starting from just 24% in 2001 but accelerating from 2005 to 2009, adding just under 20%. See Graph 26 above. 42 Appendix 3: JSDF Grant for Quilombolas: Activities and Results # Title Activities Results Members - Quilombola housing (PNHR) - Monthly meetings at Council Headquarters Council of - Community meetings Quilombola Cisterns acquired. - Community monitoring with INCRA Associations Housing and food basket 15 - Quilombola Management Forum of the Sertão processes/monitoring - Monitoring of food baskets jointly with the Produtivo completed. communities Territory - Participation in Quilombola Community Councils in the municipalities. - Execution of the Innovation, Sustainability, and Institutional Strengthening Project for the consolidation of Quilombola disputes in State Council eight Identity Territories of the State of of Bahia – Agreement between Vida Melhor Strengthening of Quilombola Program and CAR, 2013 - 2014 Quilombola associations Communities - 3 State encounters and councils; structuring and 8 - Knowledge exchange between communities of Council Headquarters; Associations in the Arataca Settlement, with 10 State productive subprojects; of the State of Territorial leaders and purchase of a vehicle. Bahia - Execution of the Technical Advisory (CEAQ-BA) Agreement with 10 State Territories: Remaining Quilombola Communities Inclusion Project (IBRD/Government of Japan/CAR/Government of Bahia). Construction of Multiple Use Center; - Youth and adult literacy in diverse implementation of Quilombos simplified water supply - Assist in the university entrance system; purchase of examination preparatory course for cisterns for household Quilombolas water consumption; - Monthly meeting in Council headquarters housing; flour mill Council of - Community meetings construction; delivery of Quilombola - Strengthening of the Quilombola university entrance Associations communities with certifications examination preparatory of the 40 - Quilombola march: black awareness course for quilombolas; Southeast - Monitoring of projects: cisterns and distribution of food Territory of housing baskets; sale of products Bahia - Monitoring of communities jointly with through direct purchasing INCRA (compra direta); - Quilombola Management Forum institutional - Monitoring of food baskets jointly with strengthening; youth and communities adult literacy; basic - Participation in education, health and course on computers and women’s councils at the municipal level technology for youth, jointly with SENAI. 43 Regional Council of the Quilombola Communities 52 Meetings Completed in the Senhor do Bonfim Northern Bahia Region Council of the Activities other than Quilombola This Council was created more recently (but organizational were Associations within the Project period) and was selected to - financed by other of the receive institutional capacity-building and resources, as permitted by Chapada support in organizing Council meetings JSDF. Territory Council of Quilombola Activities other than This Council was created more recently (but Associations organizational were within the Project period) and was selected to of the - financed by other receive institutional capacity-building and Extreme resources, as permitted by support in organizing Council meetings South JSDF. Territory - Meetings in Quilombola communities - Quilombola housing (PNHR) Council of - Meeting in council headquarters Quilombola - Monitoring of communities jointly with Associations 20 INCRA Completed of the Médio - Quilombola Management Forum Rio de Contas - Monitoring of food baskets jointly with Territory communities - Participation in Quilombola State Councils Council of - Quilombola housing (PNHR) Quilombola - Monthly Meeting in Council Headquarters Associations 70 - Community meetings Completed of the Irecê - Monitoring of Communities alongside Territory INCRA Source: CAR 2014 44 Annex 3. Economic and Financial Analysis 3.1 The Project, under the OP and the AF, financed 2,695 community subprojects, benefiting some 265,000 families, distributed across 407 municipalities and organized into 2,758 community associations. Of the community subprojects financed and implemented, nearly 90% spanned the following seven types: water supply, mainly household cisterns; household sanitation; agricultural mechanization (i.e., tractor); small-scale dams; honey collection and processing; small-scale bridge; and manioc mills. These main types of subprojects accounted for 87% of families benefited under the Project (see Table 1). Table 1: Subprojects Financed, Families and Cost per Family and Subproject, OP and AF Aver. % Total Aver. Cost Total Subproject # SPs Total # Cost per # SPs per Family Cost Families SP R$ R$ ‘000 R$ ** Household Cistern 857 49,165 142,615.00 34.10 2,485.94 122,221.11 Household Sanitation 640 32,750 152,500.00 27.23 2,910.15 97,600.00 Ag. Mechanization 431 66,527 109,570.00 13.18 709.68 47,224.89 Small-scale Dam 60 6,524 90,092.00 1.51 828.56 5,405.56 Honey Processing 55 4,171 107,010.00 1.64 1,411.06 5,885.56 Concrete Bridge 90 45,160 94,493.00 2.37 188.32 8,504.44 Water Supply System 151 10,794 164,319.00 6.92 2,298.70 24,812.22 Manioc Mill (electric) 94 10,472 122,706.00 3.22 1,101.45 11,534.44 Other 317 39,117 111,027.00 9.82 899.75 35,195.78 TOTAL: 2,695 264,680 133,975.00 100.00 1,354.02 358,384.00 Source: CAR/MIS **Includes beneficiary counterpart Cost Effectiveness of Infrastructure Subprojects 3.2 The largest share of subproject investments financed under Produzir II and III (i.e., the OP and the AF) were small-scale infrastructure (65%), with productive investments being some 30% and social investments the remaining 5%. These infrastructure investments – primarily water supply systems, cisterns and household sanitation – are core public services which generate known, sufficient economic benefits to justify their universal provision. Hence, this ICR assesses this set of subprojects following a cost-effectiveness framework. 3.3 Several elements of Project design helped to ensure that these subprojects were the least- cost, best alternative for delivering small-scale, community infrastructure: • Demand-driven investments tend to flow to the neediest. Targeting under the OP and AF was shown to be accurate. Unicamp (2014), specific to beneficiary targeting under the OP and AF, as well as separate FECAMP studies (2004) on the larger NE portfolio of Bank-financed CDD, confirm this. • Community participation, through the Municipal Councils (and/or their participatory, representative alternates) also ensures that the investment is the best alternative for the community and that resources flow into a legitimate investment priority. 45 • The extensive use of standard designs and technical parameters: (a) protects the communities from under and over-sizing of their subprojects and minimizes future technical and/or quality problems; (b) ensures that least-cost solutions are employed and deployed at scale; and (c) lowers search and information costs by providing established patterns of initiating and completing subprojects. • Delegation of subproject execution directly to the community association generates significant cost savings relative to similar works executed by public agencies. Data from Pereira (2014) and cost data from CAR indicate that cost savings for cisterns and household sanitation were 50% and 18%, respectively, relative to market reference prices (Table 2). Such cost savings are consistent with those found in other Bank-financed CDD projects in NE Brazil. Table 2: Cost Comparison, Infrastructure Subprojects Cost (R$) Subproject Type Market Variation Produzir Reference R$ % 1. Household Sanitation 1.1. Standard Design A (unit cost) 2,475 3,000 -525 -18% 1.2. Standard Design B (unit cost) 2,475 3,000 -525 -18% 2. Household Cisterns 2.1. Cisterns (unit cost) 2,600 5,400 -2,800 -50% Source: Pereira (2014); Aide Memoire April 2013. • The Project Operational Manual requires direct, competitive contracting on all subprojects; as a rule, three bids are solicited by the community association and the least-cost bid is selected. For a sample of subprojects, Pereira (2014) found the quality of procured materials to be Excellent or Good for 37 out of 40 subprojects (93%). Table 3: Quality of Subproject Investments as rated by Beneficiaries No Subproject Type Excellent Good Fair Poor Total Response Household Sanitation 6 1 1 0 0 8 Household Cistern 10 2 0 0 0 12 Water Supply System 6 5 0 1 1 13 Agricultural Mechanization 3 0 0 0 0 3 Post-harvest processing and storage 3 1 0 0 0 4 TOTAL 28 9 1 1 1 40 Source: Pereira (2014) • Cost savings through procurement economy frequently enable benefits to be extended to additional beneficiaries not initially included in the subproject proposal. • When compared with overall operation and maintenance (O & M) of public infrastructure in Brazil, subproject O & M – the responsibility of the community association – has been 46 generally satisfactory, with a high proportion of associations paying user fees where appropriate or making other regular arrangements. Financial Analysis of Productive Subprojects 3.4 The following combines: (i) primary data analysis conducted in early 2014 by the University of Campinas (SP) Development Foundation (Funcamp); and (ii) secondary data from similar productive subprojects financed under earlier generations of the Produzir program in Bahia, as well as secondary evidence from Bank-financed CDD projects in other NE Brazil states. This approach is taken, given that data collected and analyzed in 2014 was heavily impacted by severe and persistent drought conditions throughout NE Brazil, including Bahia. Taken together, the primary and secondary data present a balanced context in which to assess the efficiency of productive subprojects. Primary data analysis: 3.5 Random sampling was conducted for the three main types of productive subprojects financed (i.e., agricultural mechanization, honey collection and processing, manioc mills). Agricultural mechanization alone accounted for some 55% of all productive subprojects; honey collection and processing and manioc mills were 9% and 5%, respectively. The resulting sample was as follows: 19 agricultural mechanization; 5 manioc flour mills; and 3 honey collection and processing. For the sample, subproject data gaps, when present, were closed through the use of proxy estimates from similar subprojects (e.g., operation and maintenance). 3.6 Net Present Value (NPV) and Financial Internal Rate of Return (FIRR). Cash flows were prepared for the subprojects to determine NPV and FIRRs. This analysis takes into account data on the investments, revenues, and expenditures needed for subproject operation. All cash flows were constructed over a 10-year horizon, which is the assumed useful life of most of the fixed investment. Data on fixed investments (e.g., construction and equipment) were verified for consistency using the Price Budget Report (Relatório de Orçamento com Preços) from the MIS system utilized by CAR. The primary data analysis strictly considered only revenue from the sale of the tractor services to both community association members and non-members. From this information, cash flows were prepared in order to reproduce the year with most subproject usage. Results are reported here by type of productive subproject. 3.7 Agricultural Mechanization: Access to the machinery was quite strong by community association members was quite strong: some 70% of members used the machinery during 2013, a total of 1,143 out of 1,652 potential beneficiary producers. In smaller associations the machinery was accessed by a larger number of members, reaching 100% of members in 8 sampled community associations. Access by non-members of the community association represented about 35% of all those who used the machines. In practice, this access contributed to improving substantially, the economic and financial evaluation indicators, reducing the idleness of the equipment and increasing the positive impact on and increased wellbeing of producers. 3.8 Overall however, the data indicate that, while access to tractors was strong, effective utilization was lower than planned, owing to several factors of which drought was prominent. Year 2013 – the reference year for the data collection – saw drought afflict these communities and thus impacted overall agricultural production. For example, five agricultural mechanization subprojects were started during the drought, i.e., after 2010, and until 2013 these associations were unable to fully utilize the tractor. 3.9 Analysis shows that the operational budgets are positive for 16 of the 19 subprojects sampled, indicating that the machinery is currently generating at least enough revenue to cover 47 operational costs. However, the current usage level of the machinery does not cover depreciation and results in a negative IRR for 11 of these 19 subprojects. Better results occurred in those subprojects where tractor utilization was more intense (i.e., more hours); in fact, tractor usage at about one-half of its technical capacity is sufficient to generate satisfactory returns, indicating that the majority of these subprojects could become viable, since in practically all cases, the machinery was markedly under-utilized (again, due largely to drought) as estimated by interviewees. 3.10 Honey Processing: Subproject investment for these three sampled subprojects was significant, at around R$170,000 each. All the associations were severely affected by the prolonged drought that began in 2010. This resulted in a severe contraction in honey production, precisely when these subproject investments were expected to contribute to expanding beekeeping/honey production in the region. The quantity of honey processed was well below installed capacity. In the case of the Valente facility, the amount produced in the year of highest production (2010) was estimated at 25 tons, which is a small fraction of the installed capacity (3 tons per day). Similarly, in Quixabeira, where the facility processed 14.3 tons in the year of highest production, installed capacity was 1.6 tons per day. 3.11 In all cases of honey collection and processing, the operational budget was positive, but with low values insufficient to guarantee recovery of the investment. Both NPV and FIRR were negative for all 3 subprojects. The prospects for recovery of honey processing activities are positive, as has been stated in previous case studies (December, 2013). Such recovery will require active intervention including technical assistance and of financial agents to regain local producer trust/confidence and to finance the re-population of bee-hives strongly hit by the drought. 3.12 Manioc flour mills: The analysis assessed 5 manioc flour mills, 4 of which were installed between 2006 and 2008 and one installed in 2012. In 2013, the associations comprised an aggregate 449 members, but only 55 (12%) had used the flour mill. This low usage can be attributed to the drought: even though the manioc plant takes longer to react to a drought due to its resilience, after drought the plant cycle takes longer to recover because of the plant’s slow growth rate, decreasing production of the raw material needed for the mills. Also, a very low number of non-members used the facilities. 3.13 The value of manioc mill investments varied significantly, reflecting the increase in costs, as well as subproject dimension. In 2013, all the subprojects generated small, positive net operational revenues, below R$500.00, except for the Vilas Unidas subproject in the Biritinga Municipality, which had net revenue of R$ 3,173.00. The positive net revenue was insufficient to compensate for investments, and the NPVs were negative at an interest rate of 2.93%. One subproject presented an IRR of 0%, while the others resulted in negative IRRs ranging from -8% to -11%. Secondary Data Analysis: 3.14 The overall poor results of the primary data analysis of productive subprojects is largely attributed to the persistent and severe drought conditions in Bahia specifically and NE Brazil more generally. As such, this ICR also presents data from previous ICRs reporting on productive subprojects in Bahia and other NE Brazil states so that a balanced picture of their efficiency can be obtained. 3.15 Under Produzir I (2001-2004), the 2005 ICR notes benefit-cost ratios are high (greater than 2.0) for the principal types of productive subprojects analyzed and that the investments are generally financially sustainable. Six productive subprojects representative of the most typical productive investments carried out in Bahia were selected for financial analysis. Activity models were constructed for these subprojects and the results are shown in Table 4. 48 Table 4: Results of Activity Models – Productive Subprojects Type of Subproject Investment FIRR NPV Net Annual Years to Recover (R$) (%) (at 10%) Income (R$) the Capital Invested Manioc Mill 18,450 15.8 5,377 3,970 5.6 Agricultural Mechanization 39,752 37.6 35,141 18,075 2.4 Small Irrigation (Type 1) 44,054 >50 327,366 90,781 0.9 Small Irrigation (Type II) 28,311 28.1 23,573 15,238 3.3 Goat Production 46,346 16.5 16,925 12,180 5.8 Fish Farm 46,104 15.6 13,739 13,162 5.6 3.16 All subprojects, in 2005 under normal yet semi-arid conditions in rural Bahia, showed satisfactory internal rates of return. Less than six years were needed to recover the investment in all cases. The net incremental annual income or value-added generated by the subprojects (at full development) ranges from R$3,970 for the manioc mill to R$90,781 in the case of one of the irrigation subprojects, with an average value of around R$26,000 for an average investment of around R$37,000. Furthermore, the modeling here – in contrast to the primary data analysis above – takes into account increased productivity and production resulting from the subproject investments. This clearly contributes to a greater net revenue stream. The FIRRs compared favorably with the real cost of borrowing, a reference for which was the Long-term Interest Rate (TJLP), 9.75%. 3.17 A comparison of financial returns to investments in two SP types (farm mechanization and honey collection and processing), common and similar in NE Brazil as a whole and four NE Brazil states with previous Bank-financed CDD projects (Table 5) shows the following: Table 5: Estimated FIRRs (%), for comparable subproject types Subproject type/ States1 NE PE PI MG SE Agricultural Mechanization 38 39 35 35 25 Honey collection and processing 17 54 63 - - 1 Northeast Region (NE). Pernambuco (PE), Paraíba (PB), Piauí (PI), Minas Gerais (MG) and Sergipe (SE) Conclusions: 3.18 The cost effectiveness of water supply systems – 63% of all investments - is strong due to design features that help to ensure that the subprojects are the least cost, best alternative for delivering small-scale community infrastructure. In the case of productive investments – about 30% of the total - the primary data analysis indicates that financial results were suppressed by severe drought affecting usage of the equipment/facilities. Operational revenues were positive in most cases but insufficient to cover replacement cost of the investment. With more intensive usage under better climatic conditions, and in the case of beekeeping appropriate technical assistance and financing, a majority of these investments could expect to become viable. To balance this assessment of efficiency, results from similar investments under the previous project in Bahia and similar projects in other Northeast states are presented, suggesting good cost-benefit ratios and financial sustainability. The overall rating for project Efficiency is Moderately Satisfactory. 49 Agricultural Mechanization 50 Honey Processing and Manioc Mills 51 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Luis O. Coirolo Task Team Leader LCSES Tulio Barbosa Consultant LCSRE Edward William Bresnyan CDD Specialist LCSES Raimundo N. Caminha Consultant LCSRE Susana Amaral Disbursement Analyst LOAG1 Maria de Fatima Amazonas Operations Officer LCSRE Jose Janeiro Senior FM Specialist LCOAA Jose Augusto Carvalho Senior Counsel LEGLA Maria Madalena dos Santos Education Specialist LCSHD Estela Neves Environmental Consultant LCSES Luis Gazoni Procurement Specialist LCOPR Supervision/ICR Susana Amaral Financial Management Specialist LCSFM Tulio Barbosa Consultant LCSAR Joao Barbosa-De Lucena Consultant LCSAR Raimundo N. Caminha Consultant LCSAR Luis O. Coirolo Consultant AFTA1 Alberto Coelho Gomes Costa Senior Social Development Spec LCSSO Eduardo Franca De Souza Financial Management Analyst LCSFM Judith M. Lisansky Sr. Anthropologist LCSSO Vinicius Lima Moura Consultant LCSWS Jorge A. Munoz Adviser AES Estela M. S. C. Neves Consultant LCSPF Anna Roumani Consultant LCSAR Fabson Vogel Financial Management Specialist LCSFM Luciano Wuerzius Procurement Specialist LCSPT Maria de Fatima Amazonas Senior Rural Development Spec. LCSAR (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY05 12.92 131.19 FY06 6.41 59.94 Total: 19.33 191.13 Supervision/ICR FY06 2.90 32.76 FY07 7.47 55.97 52 FY08 10.93 88.36 FY09 7.99 42.75 FY10 9.53 36.07 FY11 9.23 45.82 FY12 8.50 45.16 FY13 4.26 41.47 FY14 5.88 54.24 Total: 66.69 442.30 53 Annex 5. Beneficiary Survey Results 5.1 The Bahia Integrated Project: Rural Poverty financed an impact evaluation at end-project, conducted by the Institute of Economics, University of Campinas (Unicamp), State of Sao Paulo. The methodology for this study is described below, followed by a summary of the study’s main findings and conclusions. Methodology: 5.2 The study employed a quasi-experimental design to delineate the main outcomes of project interventions. Definition of the sampling plan for research purposes included two groups: the universe of households which benefited from the Project (Treatment Group - TG) and a group of households which sought to access the Project but did not do so (Control Group - CG). The Project did not conduct a baseline study. 24 5.3 Three criteria were used to compose the study universe: (i) selection of subprojects signed after January 2005 and concluded by July 2012, avoiding contamination with subprojects financed by the previous project and the most recent still being completed. The beneficiary sample (Treatment Group) did not represent the universe of families benefited but eliminated families whose subprojects were incomplete at the sample cut-off date. An incomplete subproject was considered unable to demonstrate any results, let alone positive or negative ones, and was considered likely to provoke distortions in the analysis. Further, the inclusion of then recently- concluded subprojects could also provoke distortions given the fact that subproject impacts are not instantaneous in many cases, but such subprojects were a normal cohort of the sampling universe; (ii) filtering out subprojects incompatible with study objectives, e.g., Digital Center, Institutional Development and Concrete Bridge. Unicamp maintains that while such subprojects were important, their evaluation required a different methodology to that used for the TG and CG; and (iii) excluding subprojects located in the metro area of Salvador, the capital. This process resulted in a study universe of 1,359 subprojects. 5.4 An important objective of the study from a quantitative viewpoint was to evaluate project impacts based on specific indicators by comparing their estimated values for the two reference universes/groups. There were however, certain difficulties involved in making this comparison, magnified by the absence of a baseline study to permit a direct comparison of the evolution of the indicators in the two populations over time. Also, since the Treatment Group universe was dispersed among approximately 400 municipalities in a large State, Unicamp was unable to conduct its research using either a simple or stratified random sample due to binding constraints in terms of costs, distance, logistics and budgets. Unicamp therefore resorted to a TG sample based on clusters of geographically contiguous municipalities (amostra por conglomerados) which they maintain was the only viable option in the circumstances. 24 The PAD states that the analysis of implementation would rely on a database of subproject information from the project MIS. No formal database study was planned. The impact evaluation or exit study was to use repeated surveys of beneficiaries, non-beneficiary households and beneficiaries of similar projects with centralized delivery mechanisms. The University of Campinas (FECAMP) conducted a baseline study of 702 poor rural families in Bahia in 2005, assessing their situation before they participated in Produzir I. The objective is stated as being to establish a baseline for future evaluation. It was considered for Unicamp’s 2014 evaluation but certain issues prevented its use. 54 5.5 Such sampling strategy is considered methodologically satisfactory given the constraints in budget, time and logistics. Despite it having less precision in relation to a simple or stratified random sample, it has a comparative efficiency advantage in cost vs. precision. Unicamp stresses that this was not one option but the only option available. Taking into account sampling efficiency, a conglomerate sample must have clusters as heterogeneous as possible. 25 To form the conglomerates, municipalities were randomly sampled and aggregated in groups, then subprojects were randomly-selected, and finally a set of households per subproject was randomly-established. The CGs were formulated randomly within the same municipalities from families which did not receive a subproject and was intended to result in households (TG and CG) with similar socio- economic profiles to compare project effects. 5.6 The household sample within the TG was defined randomly through a three-stage process: (i) random selection of conglomerates formed from various adjacent municipalities in each homogeneous meso-region; (ii) random selection of up to eight subprojects within the conglomerate; and (iii) random selection of beneficiary households within each subproject. The Treatment Group is described as “pre-stratified”, i.e., using as strata the homogeneous meso- regions defined by IBGE (Brazilian Institute of Geography and Statistics). The final selected Treatment Group for research comprised: 675 households in 42 municipalities and covering 132 subprojects. 5.7 Households in the Control Group sample were selected from within the same municipalities utilized for the Treatment Group. In these municipalities, some communities were selected and then a sample of families selected from within these communities. The objective of the CG was to obtain a minimum group of households comparable to the TG considering the conditions/criteria for entering the Project. The CG was defined after the TG, consequently, the stratification of municipalities by homogeneous meso-region, their agglomeration into clusters and the first stage of selection correspond to the TG. The next stage was to select randomly communities whose proposals were not attended by the Project or which, for one/other reasons, were still waiting for a subproject. From these were selected the sample households, the final stage of sampling. The final selected Control Group for research comprised: 411 households, in 39 municipalities and covering 77 subprojects. Project coverage/targeting of rural poverty by municipality: 5.8 Unicamp explored the amount of resources allocated within the Project, number of households, amount spent per household, and total amount spent per meso-region 26. • The main finding is that project resources were not necessarily allocated according to the original vision; regions with higher concentrations of poverty did not always receive more funds. • As a result, the distribution of resources does not appear to have been be optimally targeted, despite the Area A and Area B targeting framework established at appraisal and continued by the Additional Financing. 27 25 The ideal would be to form clusters resulting in maximum variation of the variables studied within each cluster. 26 Geographical stratification done by the Brazilian Institute for Geography and Statistics (Instituto Brasileiro de Geografia e Estatística – IBGE). 27 CAR confirms/explains this finding in its Borrower Completion Report (2014), referring to levels of community organization and intense pressure during the 3-year drought to service spontaneous demand from communities, especially for water supply. 55 • Unicamp suggests that regional targeting probably responded more to community demand and/or from local agents trying to secure resources for areas previously defined with high incidence of poverty, i.e. drawing resources to certain areas. • Unicamp does not suggest that there was leakage to non-poor families, describing project beneficiary families as “archetypical” of poor rural families in Bahia. • See Main Text Section 3.2. The pressure of three years of drought resulted in massive levels of spontaneous demand for water supply and CAR had difficulty “holding the line” on targeting strictly as understood at appraisal. Profile and evolution of beneficiaries/non-beneficiaries: • Average size of beneficiary families was 3.7 persons (higher than non-beneficiaries) with 52% of working age and 22% below the age of 14. Average age of head of household was 52 (both beneficiaries and non-beneficiaries). • Majority of the younger population (7-14 years) could read, while 50% above the age of 40 declared themselves illiterate. • Otherwise, both treatment and control groups presented similar household, gender, educational level per age range compositions. • Change in youth occupation over the project period was not significant; the project did not set out with a strategy for youth (see Main Text Section 2) but did test/develop important concepts of young people as future leaders and motivators. • Most beneficiaries of working age were in agriculture, especially those from 41-60 years but clearly younger age groups among beneficiaries and non-beneficiaries were diversifying their sources of employment. • About 80% of both groups continued to work as self-employed laborers/on their own account. Some 90% of beneficiaries and 85% of non-beneficiaries worked in the rural zone. There was little evolution in the project period within either group. • The number of male beneficiaries was slightly higher than women and 76% of beneficiary heads of household were male. • Unicamp suggests that the Project lacked a strategy for favoring female-headed households, although women were major participants in project activities.28 Effects of drought on income and assets: • Drought affected the majority of households in both sampled groups (72% and 66% respectively) making a precise analysis of income more difficult. • For beneficiaries of agricultural mechanization (tractors and equipment) and manioc mills, subprojects more sensitive to drought, percentages affected by drought reached 88% and 71% respectively. While for 14% of respondents the drought started in 2009, the period 2010-2012 saw severe drought affecting a much wider area. • The study attempted to reconstruct income pre-drought, but respondents were unable to recall with any precision their income in 2009 or 2010. • This is natural given the fragmented structure of their income sources: temporary work, subsistence production, irregular sales at local markets, family remittances, pensions and 28 As noted in Main Text Section 2, the PAD did not contain a gender strategy although in practice – as noted in the Borrower Completion Report (CAR 2014) - women’s participation in the Project was significant: some 32% of the total 2,639 associations were led by women. 56 cash transfers. Perceptions of the evolution of income are described by Unicamp as more “robust” • Average area of land held by beneficiaries remained at 14 ha between the two periods (26 ha for beneficiaries of tractors/equipment and 4ha for those obtaining sanitation). The average area for the Control group was 11 ha. • 48% of beneficiaries reported that their assets increased in the period (vs. 39% of non- beneficiaries): for tractor and sanitation beneficiaries respectively, the percentage was 51% and 42%. • Only 13% of beneficiaries reported asset decline (mainly drought-related) vs. 26% of non-beneficiaries. • The percentage of non-beneficiaries identifying loss of assets was significantly higher than beneficiaries due to the improved drought resistance of the latter group from project investments in water supply systems and cisterns. Income level and source: • Data from the period December 2012 to 2013 was used to analyze income results.29 • Respondents’ perception of income post intervention in 2013 compared to 2009 (better, unchanged, worse off) was as follows: Treatment Group 48.7% were better off, 37.1% remained unaffected, and 14.2% were worse off; and, Control Group: 42.3% better-off, 27.3% unaffected, and 30.4% worse off. • This suggests that the Project assisted in stabilizing the financial situation of beneficiaries. • Income from agricultural mechanization and manioc mills - the main types of productive subprojects - averaged R$15,902 and R$14,748 respectively, while beneficiaries of sanitation showed the lowest incomes post-intervention (R$10,972) being among the poorest and having received a social not productive investment. • Some 43% of income came from pensions and the conditional cash transfer program (Bolsa Familia), illustrating the high dependence of such families on social programs. Location and living conditions: • The two sample groups live in similar types of housing, in rural areas. • Electricity availability went from 80% to quasi-universal access by 2013. • Sanitation facilities within residences increased significantly, from 54% to 85% for beneficiaries and from 57% to 80% for non-beneficiaries. • Sewage systems increased for beneficiaries and non-beneficiaries but more so for the former, due to project investments in sanitation subprojects. 29 The ICR notes first, that the lack of a well-constructed baseline and weak Results Framework created difficulty regarding income data, hence the use of December 2012-2013 data. Other data sources could perhaps have been explored to obtain an average estimated income prior to project implementation (using the Pesquisa Nacional por Amostra de Domicilio, for instance) for persons with characteristics similar to the beneficiaries (educational level, household conditions). Second, the perception questions regarding changes in income are not equivalent: in the Control Group they were asked to compare 2009 to 2013, while the Treatment Group was asked to compare before/after subproject investments. Temporal effects (when the investments occurred) may skew the comparisons, especially considering the severe drought from 2011 on. 57 • Availability of household water cisterns and public, piped water supply was significantly higher in the homes of beneficiaries. • Even so, water trucks still played an important role for both Treatment and Control Groups: in 2013, a drought year, 66% of beneficiaries and 75% of non-beneficiaries still had water trucks as an important source of water supply, demonstrating the major difficulty in resolving water supply issues in the rural Northeast, despite these projects. • Household comfort improved for 61% of beneficiaries with 79% attributing this to the project, i.e., 57% of beneficiary households considered that the project was connected to that improvement. Some 51% of the CG also reported better household comfort. • 72% of beneficiary households perceived improved quality of life and that the Project was connected to this improvement. Only 53% of non-beneficiaries observed improved quality of life. Food security: • Food security was captured in two indicators: (i) number of beneficiaries per type of food security, and (ii) household perception of food security. For the first, there appears to be no statistically significant difference between the two groups in terms of initial status and evolution by 2013. • For the second, despite the perception of improvement of food security being relatively equal between treatment and control groups (52.2% and 48.1%, respectively), the worsening of food security was significantly higher for non-beneficiaries (13.9%), while only 3.0% of project beneficiaries stated the same. • Unicamp suggests that income transfer programs, especially pensions and Bolsa Familia, may have played a significant role in improved food security and that beneficiary improvements cannot be attributed exclusively to the Project. Health: • Project interventions improved the health situation of beneficiaries. • 54% of beneficiaries perceived improved health and said that the Project was an important factor; the Treatment Group said that 56.4% improved, 36.7% remained the same, and 6.9% had worsened. • 37% of non-beneficiaries reported, comparing 2009 with 2013, that their health had improved; the Control Group reported that 36.9% had improved, 44.0% remained the same, and 19.1% worsened. • It is evident that sanitation infrastructure leads to immediate health improvement in terms of water borne disease for municipalities with no or low ex-ante sanitation conditions. (See water-related health discussion below). Water supply: • Prior to receiving project investments in water supply and cisterns, most beneficiaries personally carried water to their homes in cans (78%) or using animals (54%) and the water did not last long (64%). • Post-project, 68% said they now have water year-round and only 8% said they have it short-term. • 92% of beneficiaries said the Project resolved the problem of water availability but for 27% the problem was only partially resolved. • Notably, 71% of beneficiary households were affected by drought – only 10% were not. 58 • Beneficiaries perceived the project water interventions as positive and important: 96% of households said the project had contributed to drought resistance. • 96.1% of beneficiaries believed the quality of the water they accessed had improved. • 58.1% believed that their water subprojects improved child and elder health. • The most frequent problem mentioned was inadequate maintenance (44% of households), reflecting the inability of some associations to manage the systems after their installation, followed by the amount of water supplied (24%) which can be explained by drought and suggests the need for water rationing in such periods. • Water quality improved for 45% compared to pre-project and was being used primarily for domestic consumption (97%) and for household cleanliness (36%). Use for agricultural purposes/cultivation was rare. • Availability of treated water rose from 37% pre-project, to 50% post-project. • Health effects: 79.2% of WSS beneficiaries stated that the Project had improved their health. • Water borne disease was reduced in the project period: Hepatitis (9% to 1%); Schistosomiasis (13% to 5%); parasites (41% to 23%); and Diarrhea (46% to 35%) • Orthopaedic illnesses resulting from years of carrying water did not decline much (48% to 44%) suggesting that such illness may have other/additional causes. Sanitation: • Sanitation improvements had positive impacts on a range of personal hygiene habits of households including hand-washing and use of hygiene products. • Health of children and the elderly improved (96%) • Main problem cited was incomplete subprojects in 32% of households – parts missing and faulty installation, and lack of water in 20%. Delays and dirty water were also cited. 5.10 The ICR asserts that while the lack of comparison between groups limits the above analyses, information on health and water corroborates the idea that improved quality of life, in terms of access to water and health improvements, was a direct result of Project interventions. Social Capital • Unicamp used proxies for social capital: participation in associations; participation in meetings within associations; number of association meetings; how well-informed the beneficiary is, in regards to association initiatives; perception of benefits from participating in associations; perception of beneficiaries in regards to problems between communities and associations; and perception on project. • 76% of beneficiaries and 66% of non-beneficiaries had already participated in an association, union, community working groups and political parties at some point in their lives and 77% had participated specifically in associations prior to the Project. • This increased to 93% immediately after the Project which would be expected given the institutional characteristics of the Project. • Participation of non-beneficiaries in associations was 69% in 2009, rising to 76% by 2013 • 69% of beneficiaries said they had participated directly in selecting their investment; 59 • About 41% admitted that they were not well-informed about association activities – the level of mobilization and social capital formation is uneven – but about half of this cohort considered that participating in the association brought benefits. • The perception of conflicts between communities and associations was low (4.8%), and overall the beneficiaries considered the project as good or very good (95.7%, being 41.3% former and 54.4% the latter). • 100% of associations hold such meetings, indicating that the expectation of project benefits is a mobilizing factor. • 39% of beneficiaries said they had been to all meetings of their association in 2013 and 15% said they had not attended any meeting in that year. Subprojects supporting income generation: • Drought effects were “devastating” on Produzir II especially for income generating subprojects, and even for those which were quite well established/consolidated and had potential to generate reasonable returns and were receiving good maintenance. Agricultural mechanization (tractors and equipment): • 88% of beneficiaries were severely affected by drought, greatly reducing the positive impact of tractors on production. • 45% said their income had improved post-subproject and that they believed the Project was a contributing factor, (more timely land preparation/planting). • 40% of beneficiaries said their area under cultivation increased, and 30% diversified production. There was little diffusion of new technological practices in agriculture. • 61% of beneficiaries said their production destined for market did not change, however production destined for home consumption did increase for 38% of surveyed households. • 80% said that time spent by household members on plowing, grading, planting and other related activities had decreased. • 70% said this saved time was used for other agricultural activities and 44% said for non- farm activities as well, while 46% used the time for rest. • 82% of households reduced their expenditures on labor costs. • Mechanization proved an attraction for young people to remain on-farm and not out- migrate, and alleviated the work burden on household members. • The study highlights that despite more advanced mechanization via the tractor, there was little diffusion of other inputs such as fertilizers and pesticides which would have been promoted if access to rural extension and credit were better. • Labor productivity did not increase but nor did mechanization induce increased beneficiary/family unemployment. • While reports of soil/environmental issues were modest (averaging 12% of beneficiaries), agricultural mechanization remains a concern, especially in dry, fragile Northeast soils. Manioc mills: • Beneficiaries of manioc mills were also badly affected by drought (manioc is normally fairly drought-resistant); of 24 beneficiaries in this sample, 17 said they were much- affected. • 54% said their incomes had increased and that they believed the Project was an important factor, a better result than non-productive investments. 60 • Despite the technological limitations and that such subprojects address only the transformative stage of an agricultural product it was notable that 60% of households using manioc mills said they had increased their production of manioc and of the flour and manioc chips (fecula). • 70% said the quality of their product had improved. • Time taken in transporting manioc decreased and time taken in diverse activities associated with manioc flour/fecula production was also reduced and used for other agricultural activities. • 40% said their expenses associated with manioc flour production had decreased. • Case studies showed that the fees charged by beneficiary association-owned manioc mills were lower than those charged by outside mills, reducing production costs for members. Main lessons defined by Unicamp (2014): • Successful subprojects are the result of a long process of maturation which cannot be accelerated – in many cases - by Produzir-style interventions. • Income generation subprojects should be taking drought – of varying intensity and duration - into account via protection and mitigation mechanisms. • Subproject weaknesses are mostly avoidable through better conception, design, feasibility analysis and execution. • While perhaps controversial, the most successful subprojects (Unicamp’s examples were in dairying exclusively) are those which include some beneficiaries who do not conform precisely to the project’s target population, i.e., more socio-economically heterogeneous associations/cooperatives. The superior experience, training/education and political insertion of some members can enhance the dynamism and success of the overall venture but care is needed to ensure full participation of poorer members. • Unicamp maintains that despite fragmentary evidence of negative political intervention in some cases, generally the participation of legitimate political agents in the process of helping an association obtain access to the Project was important in the process of securing CAR’s approval. • The time involved from the demand to approval and implementation could be protracted and in some cases was two years or more. • Even the simplest of subprojects which function well need to be well-analyzed before they are reproduced/scaled up. Most subprojects are strongly-dependent on local conditions either historical or of the beneficiary group, local leadership, extent of mobilization, local traditions, role of local social movements, and transferability of experiences from one to group to another. Such conditions can vary from area to area. • Sustainability is questionable in some cases due to O&M issues, and CAR’s expectations of results were/are too lofty in relation to conditions on the ground. 61 Annex 6. Stakeholder Workshop Report and Results 6.1 The following is a summary record of the proceedings of a CAR-sponsored post-closing seminar/workshop held in March, 2014 to disseminate and discuss the findings of the Unicamp (2014) Final Evaluation and discuss the then-proposed new Bank-supported project. 6.2 In order to complete and finalize the Produzir Program, seeking “transparency, participation and decentralization”, and despite the challenges of the project design in Bahia, CAR publicly presented the scientific evaluation conducted by the UNICAMP/FUNCAMP research team on the impacts of project activities on the lives of families and rural communities, the Project’s target population. 6.3 On March 19-20, 2014 in Salvador, an event entitled “Seminar on New Challenges” was conducted involving some 545 participants including public authorities/agencies and social agents and movements. Many representatives participated: community associations; indigenous and ethnic groups such as the Tuxá, Pataxó, Pataxó Hã Hã Hãe and Tupinambá; CAR technicians, representatives of the World Bank, UNICAMP/FUNCAMP, and managers of State public agencies. The objective of the event was to close out the cycle of activities financed by the Produzir Program, present the new, proposed Bahia Produtiva operation and start to build partnerships for its execution. CAR’s choral group “Canto da Terra” opened the event and the proceedings were in the form of panels. DAY 1 Panel 1: The focus was a presentation on the Bahia Produtiva Project, a new operation sponsored by the State of Bahia and at an advanced stage of negotiations with the World Bank. The project represents a new challenge for CAR whose responsibility is the socio-economic development of the State, especially its rural areas and family agriculture, agro-ecology, food and water security, as well as its essential complements – socio-productive infrastructure and market access. Panel 2: Various agencies presented proposals for partnerships with the new operation: SEAGRI/SUAF; SEAGRI/EBDA; SEAGRI/CDA; SEDES and SETRE/CESOL. Observation: During the seminar, space was allocated to a discussion of Social and Environmental Safeguards under Bahia Produtiva which have already been submitted for public comment through an Edital published in the period March 10-21, 2014. The documents submitted were the Sustainable Development Plan for Indigenous Peoples, the Socio-Environmental and Sustainable Development Plan, and the Involuntary Resettlement Safeguard. DAY 2 Panel 1: With a positive evaluation of project performance, UNICAMP/FUNCAMP presented the results achieved by Produzir, and its activities to fight poverty and exclusion in 407 municipalities of Bahia. Themes covered included Rural Poverty in Bahia: Performance and Achievements under Produzir III; Study Methodology – Sample and Indicators; Report on Field Research; and a plenary session with clarifications, comments and debate about the external evaluation research. 62 Panel 2: Evaluating the Focus of Produzir, Unicamp/Funcamp discussed the approach to the socio-economic profile of beneficiaries, the Project’s impacts and principal conclusions and recommendations. In the plenary session, participants were able to clarify, comment and debate all aspects of the investments and results achieved by the Project’s execution. Successful, representative experiences were also presented, along with a video and testimonials from beneficiaries and researchers from Federal universities (São Carlos and Uberlândia), and State (Campinas, Paulista and Southwest Bahia (UESB)), present at this workshop as well as members of the UNICAMP/FUNCAMP team. Statements by beneficiaries: “Non-one has done as much for indigenous communities as this Government. Indigenous peoples need projects to improve their lives. We need more water cisterns. The challenge for country folk is water to produce and drink”. Cacique (Chief) Anselmo, Tuxá For the Chief Maria do Carmo Quirino, of the Reserva Aldeia Patiburi, in the Municipality of Belmonte, in the extreme south of the State, before the implementation of their subproject, indigenous people were considered “doctors of sheep-farming”. “I am here to confirm the success of this sheep-herding subproject, due to which there has been a change in our lives. This is an activity which involves the whole family. I believe that in the future my children and relatives are going to be able to stay in our village. I am sure also that with Bahia Produtiva, we can successfully advance.”. The President of the Association of Beekeepers of the Capão Valley, in the Palmeiras Municipality, Chapada Diamantina Region, Pedro Constan, noted that support from Produzir was fundamental for their association to develop beekeeping/honey processing by the local community. “These investments by CAR are very welcome in associations which have them and have developed this work. Recently, we were also considered by CAR as a pathway, through the State’s Vida Melhor Program, which will help us a lot with transport and create dynamism in our activity ”. The President of the Association of Handicrafts of Caldas do Jorro, Municipality of Tucano, Maria de Lourdes Silva, also gave thanks for the benefits received from the Produzir Program. “We are very blessed through CAR and the World Bank, because we work with handicrafts and were marketing our wares from huts which every day we had to stock up and then take down”. Through the CAR project, the association received 48 “booths”, and in 2012 the subproject was expanded with another 28 booths. “Today we have over 70 booths and this advance is a reference point for our municipality, because it benefited 70 people directly and another 200 indirectly”, she said. 63 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR A. Executive Summary, Borrower Completion Report 1. ORIGINAL PROJECT The Bahia State Integrated Project: Rural Poverty – Produzir (OP and AF), operated by the Development and Regional Action Company (Compania de Desenvolvimento e Ação Regional – CAR), subordinated to the Secretariat for Development and Regional Integration of the State of Bahia (Secretaria de Desenvolvimento e Integração Regional do Estado – SEDIR), was aligned with the Brazil Country Assistance Strategy (CAS). The CAS identified four major challenges that Brazil faced in achieving inclusive and sustainable growth: improving macro foundations, equity, competitiveness and environmental sustainability. The operating model adopted by Produzir to support the rural population assumed that mechanisms that include increased participation of civil society tend to generate better results as enhanced social capital and increase the capacity of communities to mobilize internal and external resources enables their development. In this sense, the financing of community subprojects is presented as a catalytic instrument for mobilization and organization of these communities. Municipal Councils, the main institution responsible for submitting project proposals to CAR, were slowly dismantled throughout project implementation due to external political forces. Nonetheless, CAR technicians still followed subproject proposal by participating in local community meetings. 1.1. Physical and Financial Performance of Subprojects In the period from 2006 to 2014, 2,695 subprojects were implemented, benefiting a total of 264,680 families in 407 municipalities, with a total investment of over US$ 216.0 million. Investments in Support to Health and Sanitation (WSS) were the most demanded and implemented types of subprojects at around 63%. Such dominance can be attributed, in part, to severe drought and consequent need for WSS. Productive subprojects corresponded to about 30% of total number of subprojects and were mainly focused on increasing revenue rather than generating jobs. The main types of WSS investments were: household cisterns; water supply systems; sanitary quality. The main types of productive subprojects were: agricultural mechanization and small agro-industries. 1.2. Advances and Innovations Regular Community Meetings In order to continue promoting social capital, and despite MC dismantling, throughout project implementation, CAR technicians closely followed community meetings, advising investment decisions when necessary. Other councils and social movements and forums steadily took over these functions, especially under the AF. As an association could only acquire a single investment at a time, and usually a community applied for a single investment, the close presence of a technician allowed for more precise and prioritized decision-making processes. 64 Capacity Building Throughout project implementation, diverse associations demonstrated difficulties in managing legal and financial aspects of their agreement with CAR. Other capacity building initiatives led to an increase in subproject sustainability (see Annex 2): cistern preservation and rational use of water and quality water storage; management of agriculture equipment and machinery; specific courses on different productive initiatives (aquaculture; goat and sheep farming; apiculture; honey production; starch, flower and cookie production; inter alia). Traditional Communities (IPs and Quilombolas) An action plan to include IPs was elaborated after three encounters with indigenous leaders, when the project was presented and discussed (basic guidelines; access points; etc.), seeking to identify the real current situation of the IPs throughout the State. The Plan contemplated all IPs territories, an investment sum of around US$ 1 million, and 25 community subprojects. The project supported Quilombola Communities in alignment with Projeto Quilombola and the Federal Constitution’s principles, rules and guidelines that acknowledge this group’s right to self- determination and original territory occupation and right of use of the natural resources on their territories. 142 agreements were signed, benefiting 12.486 families with an investment sum of around US$ 5 million. Economia Solidária In partnership with the Secretariat of Labor, Income and Sports (Secretaria de Trabalho, Emprego, Renda e Esporte – SETRE), CAR executed subprojects that promoted economia solidária: 76 agreements were signed, benefiting 7.288 families in 15 municipalities, with an investment sum of around US$ 1 million. 2. ADDITIONAL FINANCING The second phase of Produzir continued the strategies adopted under phase I of this project while introducing the following new practices: (i) capacity building in FM focused on youth; (ii) restructuring of the results framework and monitoring to encompass capacity building activities; (iii) counterpart financing of diverse initiatives to account for an emergency state due to the severe drought. In 2010, the most severe drought in over 50 years afflicted the Northeastern Region of Brazil. Bahia is comprised of 2/3 semi-arid territories, which led to a dramatic increase in WSS subproject demand throughout the AF. 2.1. Physical and Financial Performance of Subprojects In the period of 2011 to 2014, 987 subprojects were implemented, benefiting a total of 106,890 families in 422 municipalities, with a total investment sum of US$ 62.3 million. Like the OP, during AF investments in Support to Health and Sanitation (WSS) were the most demanded and implemented types of subprojects (63% of all subprojects and investments). Such dominance can be attributed, mainly to severe drought and consequent need for WSS investments. Productive subprojects corresponded to 32% of total number of subprojects (29% of all 65 expenditure) and were mainly focused on increasing revenue rather than generating jobs. The main types of WSS and productive subprojects are the same as previously described. 2.2. Advances and Innovations Information System – SACC containing Capacity Building Initiatives The monitoring and information system used throughout project implementation (SACC) was modified during AF to include capacity building activities. This allows implementing agents to monitor and follow beneficiary capacity building as a separate action, incorporating the significance of success capacity building and training on investment outcomes. Youth Integration into Productive Subprojects Throughout OP difficulties arose regarding beneficiary FM. The youths within beneficiary associations were motivated to undergo specific capacity building to increase their community integration. The courses’ main activities were related to basic computer skills and therefore naturally appealed to the community youths. The overall results were positive and promoted social integration of youth in productive initiatives. 3. LESSONS LEARNED AND CHALLENGES The main lessons learned throughout Project implementation are:  Alignment of diverse public initiatives: Partnership with other public institutions optimized results in terms of poverty reduction; allowed for leveraging of diverse public resources; strengthen and expanded specific project interventions.  Capacity building: Previous capacity building initiatives to the project subproject intervention is essential to guarantee success. Furthermore, social capital capacity building initiatives is fundamental at MC level.  Initiatives to impulse improvement of institutional management: Institutional strengthening of MCs and CAs, alongside NGOs, to increase management capacity and vision on monitoring of public policies in rural context by: (i) promoting territorial and state encounters; (ii) best practice competitions; (iii) systematic advisory system; (iv) knowledge exchange between different stakeholders.  Local cultural changes to promote social capital: Beneficiary association attitude may change when new mechanisms of community and productive life arise, leading to an increase in autonomy in beneficiary organizations. As such, a new methodology based on participatory instruments; collective construction; appreciation of prior agricultural knowledge of beneficiaries; and development of social technologies is fundamental for the improvement of quality of life in beneficiary locations.  Technical Assistance: Technical advice for the beneficiaries is important throughout all phases: initial proposal formulation, implementation, results monitoring and consolidation of initiatives developed throughout the project.  M&E: Importance of building Monitoring and Evaluation processes to systematically follow actions and allow for qualitative and quantitative information, as well as information management. 66  Beneficiary ownership of subproject through financial management: Capacity building and technical assistance to strengthen beneficiary FM by establishing a systematic technical assistance strategy. The main challenges faced by the project:  CAs: Strengthening of CAs project proposal and implementation, with special activities focused on subproject maintenance, management, and operationalization.  Capacity Building: Continued beneficiary capacity building activities should be incorporated into future project design.  Technical Assistance: Technical assessment in Financial Management provided to the CAs must be consistent to allow for more institutional control of resource management.  CMDS: Municipal councils need to be restructured and sustainable. B. Borrower’s letter commenting on the Bank’s draft ICR: 67 68 69 70 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders NA 71 Annex 9. List of Supporting Documents Project Appraisal Document (PAD), February 4, 2005 (no Report No.) Legal Agreements (Loan 7327-BR, Loan 7732-BR) Project Paper for Additional Financing (Report No. 44717-BR, June 23, 2009) Borrower Completion Report: Original Project (CAR, October 2010) Borrower Completion Report: 2006-2014 (CAR, March 2014) Supervision Implementation Supervision Reports (ISRs) Supervision Aide Memoires Financial Management and Procurement Supervision Reports Economic and Financial Analysis (Funcamp, 2014) Final Evaluation Report (Unicamp 2014) Avaliação de Resultados do Subprojetos de Cisternas Domiciliares, Produzir II, AGIR/CAR, December 2009 Subprojetos Comunitários de Beneficiamento de Café: Avaliação de Resultados/Relatório Técnico, AGIR/CAR, December 2009 Avaliação de Resultados: Subprojetos de Mecanização Agrícola, AGIR/CAR, December 2009 Situação Social nos Estados: Bahia, Institute for Applied Economic Research (IPEA), 2012 Relatório Técnico: Estudo Simplificado de Desempenho Físico do Produzir, M. Pereira/CAR, July 2014 72 MAP: State of Bahia, Brazil 73