Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) Report Number : ICRR0021510 1. Project Data Project ID Project Name RW:Land Husband,Water Harvest,Hill P114931 Irrig Country Practice Area(Lead) Additional Financing Rwanda Agriculture P124785,P147543 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-46740,IDA-53600,TF- 30-Jun-2014 85,284,461.56 10953,TF-11435 Bank Approval Date Closing Date (Actual) 22-Dec-2009 29-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 34,000,000.00 21,065,000.00 Revised Commitment 34,000,000.00 21,065,000.00 Actual 85,284,461.56 21,065,000.00 Prepared by Reviewed by ICR Review Coordinator Group Keith Robert A. Oblitas John R. Eriksson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives To increase the productivity and commercialization of hillside agriculture in target areas in the Recipient’s territory. Source: Financing Agreement, February 8, 2010 Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components A. Capacity Development and Institutional Strengthening for Hillside Intensification. (Estimated cost at Appraisal: US$ 13.85 million. Actual cost at closure: US$ 19.47 million.) Developing the capacity of individuals and institutions for improved hillside land husbandry, stronger agricultural value chains, and expanded access to finance, through four sub-components: (i) Strengthening Farmer Organizations; (ii) Extension; (iii) Marketing and Finance; and (iv) Capacity Development and Institutional Strengthening of the Ministry of Agriculture and Animal Resources (MINAGRI) and its agencies. Activities financed under Component A included institutional strengthening through mobilization, community outreach, training, extension and marketing capabilities. B. Infrastructure for Hillside Intensification. (Estimated cost at Appraisal: US$ 20.75 million. Actual cost at closure: US$ 102.08 million.) Infrastructure (in three sub-components) for: (i) land husbandry; (ii) water harvesting; and (iii) irrigation. Activities under the component included soil conservation with technologies depending on topographic slope, e.g. terracing, contour bunding, green manure, downstream reservoir protection, fencing, planting of perennials, water harvesting, and conveyance structures for irrigation. C. Implementation through the Ministerial SWAp Structure. (Estimated cost at Appraisal: US$ 10.47 million. Actual cost at closure: US$ 13.72 million.) Management to enable effectiveness of project activities within the new Sector Wide Approach (SWAp) structure for implementation of programs and projects at MINAGRI and the Land Husbandry, Water Harvesting and Hillside Irrigation Project (LWH ). Broader ministerial activities under the SWAp, included: technical assistance, training workshops, surveys, studies, strengthening M&E, management information systems, and equipment. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Costs: Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) Total project costs estimated at appraisal were US$ 45.1 million and actual costs at closure were US$ 135.2 million – 200 percent more than the appraisal estimate. The primary reason was due to a substantial increase in project activities for most of the project’s components and sub-components. In particular, the infrastructure program under component B, the largest project component, increased five- fold. Financing: As approved at appraisal, financing was to be through an IDA Credit of US$ 34 million, but this was considerably augmented during project implementation through additional funding: IDA provided an additional US$ 35 million (effective March, 2014), US$ 50 million was provided under the Global Agriculture and Food Security Program (effective June, 2011), a grant of US$ 13.3 million was provided by USAID (effective May, 2012), and a grant of US$ 7.8 million was provided by CIDA (effective May 2012). In combination the additional IDA Credit and development community funds amounted to US$ 105.8 million. Together with the original IDA Credit of US$ 34 million, the total financing provisions were US$ 139.8 million. This excludes the borrower contribution. Borrower Contribution: The original borrower contribution is stated in the ICR at US$ 23 million. The actual contribution is not shown in the ICR’s financing table. Restructurings: The project was restructured in 2011, 2013, 2015 and 2017 (all retructurings were Level 2). There were no changes to the project’s Development Objectives, and only minor adjustments to Component activities, but there were a number of implementation and disbursement changes, and increases in project targets and project size based on the larger financing made available during project implementation. Main restructuring features were: 2011: Adjustments were primarily to accommodate the US$ 50 million financing from the GAFSP and the US$ 35 million additional financing from IDA, to which was added the contributions from USAID and CIDA. Together, the total additional financing (including IDA) was US$ 105.8 million. Accommodating these financing increases, the disbursement profile was updated, adjustments made to disbursement categories, project targets were increased, and the project closing date extended (from June 2014 to December 2015). A basket approach to project funding was also set up to enable multi-agency financing from a common pool. 2013: Following the mid-term-review and additional IDA financing, a reallocation was made from components A and C to component B to enable a larger infrastructure program. Baseline values for productivity indicators were revised downwards due to initial overestimates of the baseline situation at appraisal (the Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) ICR advises that at appraisal only national level figures were available and these were adjusted during implementation based on site-specific Implementation targets which were increased to reflect the larger funding available. 2015: A reallocation between disbursement categories to accommodate changes in disbursement amounts. 2017: A last extension of the project closing date, to June 2018. Additional to these formal restructurings, there was a change in management, through merger of the implementation units of LWH and the Rural Sector Support Project followed by transfer of the combined implementation unit from the Ministry of Agriculture to the Rwanda Agriculture and Animal Resources Development Board. This was to be in line with Rwanda’s policy to transfer implementation activities from the ministries to implementing agencies. Dates: LWH was approved on December 22, 2010, with a projected closing date of June 30, 2014 - an estimated project period of just over 4 years. Actual closure was on June 2, 2018 - a project period of eight years, double the appraisal estimate (refer above and section 4). 3. Relevance of Objectives Rationale Development Scenario. The two Objectives of the Rwanda Land Husbandry, Water Harvesting and Hillside Irrigation Project (LWH) – to increase: (i) productivity and (ii) commercialization of hillside agriculture – were centrally situated within the agricultural and overall development strategies of the Bank and the Government of Rwanda. The country’s post-conflict recovery had seen rapid GDP growth (8 percent per annum between 2000 and 2009) and the poverty headcount ratio had fallen from 59 percent in 2000 to 45 percent in 2010. But growth was starting to slow down, not least in the agriculture sector where increases in value added had fallen to 5.5 percent per annum. Agriculture was the fundamental driver of economic growth and poverty alleviation. In 2009 the farming sector contributed 30 percent of the country’s economy, about half of export revenues, and 81 percent of employment. Poverty was mainly rural (at 67 percent) and tied to subsistence farming. Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) In order to raise agricultural growth and accelerate poverty alleviation, both the Government’s development strategy – the Strategic Plan for the Transformation of Agriculture in Rwanda Phase II, 2009-2012; and the Bank’s Country Assistance Strategy for Rwanda (FY09 – 12), stressed the need to enhance agricultural growth as a means to drive the economy forwards and reduce poverty. The Government’s development strategy – as articulated in its succession of Strategic Plans, has consistently emphasized rural transformation to a modernized agricultural economy, containing much of the elements espoused under the project – notably, addressing poor yields through modernization of production, and the commercialization of agriculture by providing access to finance and market linkages. The Fourth Strategic Plan issued in 2018, approximately at the time of project closure, continues and intensifies the above thrusts. The Bank’s Country Assistance Strategy of FY 09-12, coincident with the beginning of the project, and the Country Partnership Strategy for FY14-18, the document relevant at project closure, have broadly similar thrusts and support Government’s strategy. The FY14-18 CPS sees agriculture as a key focus area, and explicitly emphasizes intensifying productivity and promoting a more commercially based sector. The Project’s Theory of Change is well aligned to the two objectives. By increasing productivity and commercialization, the incomes and welfare of the rural population, and an enhancement of the agriculture sector’s contribution to the economy, could be expected. Project actions were in most respects knowledgeably identified for likely impact, and designed practically to achieve the corresponding outputs, though some sub-components (e.g. irrigation), achieved targets but were less cost-effective than other activities like terracing. Such differences can be expected in a multi-component project, and, in general, the design provided a good base for the project’s implementation, and, in turn, for achieving the project's objectives. The realism of LWH's Theory of Change was proven by project achievements. For most project outputs and for assessing project outcome, achievements were well beyond appraisal targets. This might suggest that the "bar" for achievements was set too low, but at appraisal the targets would reasonably be considered already ambitious relative to the baseline situation. The actual increase in productivity under the project was over 10 times baseline productivity for irrigated agriculture and over seven times the baseline for non-irrigated agriculture, and commercialization of agricultural production went from 35 percent of production to 81 percent. The ICR does not indicate exogenous variables (e.g. weather, pricing) which would have significantly affected project achievements.relative to its theory of change. Thus, the realism of the theory of change established at appraisal was realistic. The Relevance of LWH's Objectives was High. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) Objective 1 Objective Increase Agricultural Productivity Rationale The project's first Objective - to increase agricultural productivity - was addressed through a variety of measures to improve farming practices and provide supporting infrastructure. Actions covered crop husbandry, cropping patterns, varieties, and crop mix; use of fertilize; upgrading of soils; sustainable land management; terracing; increased irrigated area including establishment of water user associations; provision of agricultural extension services; technical and organizational training; and provision of credit. Key Outputs (achievements) are summarized below: 1. The proportion of male farmers using improved farming methods rose to 93% compared with 30% baseline and 90% targeted. Achievements for female farmers were 90% compared with 25 % baseline and 90% targeted originally (at appraisal) 2. The area developed for irrigation reached 2,555 ha, as against the original target of 900 ha and the revised (including the extension period) target of 2368 ha. 3. 55,000 kitchen gardens were established, compared with 15,000 originally targeted and a revised target of 19,000. 4. The cost recovery ratio for operations and maintenance of water user associations (WUAs) reached 66% compared with the original and unchanged target of 40%. Outcome At project design stage, three monitorable indicators were chosen to assess the outcome of the project’s first objective. At project completion, productivity of irrigated land was over 10 times baseline productivity and was 230 percent higher than the original target at appraisal. Non-irrigated agriculture also had high increases. Productivity was seven times the baseline productivity, and 150 percent greater than the targeted productivity set at appraisal. There was also an increase in the number of beneficiaries, which increased threefold compared with the target. Results are shown below: 1. The Productivity of Irrigated Areas reached 5,600 US$/ha compared with targets of US$1,700 (original) and US$2,800 (revised), and baseline (pre-project) productivity of US$ 492/ha. 2. The Productivity of Non-Irrigated Hillside Agriculture reached 3,500 $/ha compared with the original target of 1,400 US$/ha and the revised target of $2,500/ha, and the baseline productivity of US$ 469/ha. 3. The Number of Beneficiaries of the project was 310,000 of which females were 150,000. The target number of beneficiaries was 105,000, of which the target for women’s participation was 52,000. (A survey conducted by the World Bank’s Development Impact Evaluation Unit (DIME) also shows impacts on agricultural production. The survey commented that: “households in LWH project sites witness large and Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) statistically significant impacts on agricultural production indicators that can directly be attributed to project interventions.”) Overall Efficacy of Objective 1. LWH’s productivity targets were exceeded - by an estimated 230% for irrigated lands, and by 150% for non- irrigated land - and formed a robust base for the outcome of the project; and the number of project beneficiaries increased threefold beyond expectations (comparison is with the original targets (4 years) against achievements at eight years.). However, these figures are indicative rather than based on known data collection processes, about which the ICR provides little information (section 9 provides commentary on LWH evaluation processes). Nevertheless, significant changes in productivity can be expected from the intensive agricultural extension and infrastructure introduced under the project,, and are not inconsistent with international experience of agricultural yields. Accordingly, the reported data is taken to be sufficiently accurate to provide a broad picture of the project's efficacy. The Efficacy of the objective to Increase agricultural productivity is rated Substantial.. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Increase the Commercialization of Hillside Agriculture Rationale An extensive program to increase commercialization of agriculture was mounted, including comprehensive training across the spectrum of activities that could support development of market chains, promotion of rural finance and credit, and construction of marketing infrastructure. Outputs for Objective 2 The main outputs under objective 2 were: training in group formation and marketing, establishing cooperatives and market channels, post-harvest storage and processes; and the use of credit and other financing sources. Quantified results from available data are: 1. 17,000 farmers were trained in group formation; 3,000 farmers were trained in horticultural production and marketing; and 7,000 farmers were trained in post-harvest handling.(no targets provided). Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) 2. The number of financial institutions participating in the project was 45. The target was 12 (original) and 25 (revised). 3. Several finance channels were developed, including agricultural production credit, harvest credit, and credit for investments developing marketing value chains. (no target/achievement data). 4. 27 cooperatives and 3,270 self-help groups were established, with 47% women. (targets not provided).. Outcome of Objective 2 The degree to which agriculture was commercialized (monitorable indicator 1 below) was the central indicator of the outcome of this objective. Data such as the opening up of financial markets and increases in the number of farmers establishing bank accounts (monitorable indicators 2 and 3) indicate development of a rural market economy which would be consistent with the increases in commercialization reported in the monitorable indicator for commercialization. 1. The percentage of total agricultural production that was commercialized rose to 81 percent as against the baseline situation (35 percent commercialized), the original target (60 percent commercialized) and revised target of 70 percent commercialized. 2. Usage of formal financial institutions rose to 92 % for males and 89% for females as against targets of 40% (original) and 90% (revised) for males, and 36% (original) and 85% (revised) for females. 3. 57,000 farmers (41% women) opened bank accounts. (no targets established). Efficacy of Objective 2 LWH undertook a comprehensive set of actions to increase the commercialization of agriculture, ranging from training for marketing infrastructure and group formation, and facilitating development of financing institutions. Commercialization increased during the project period from 60% baseline to 81% achieved. There is little commentary in the ICR on the quality and reliability of M&E, leaving questions on the reliability of some project results, especially for commercialization (section 9). Nevertheless the magnitude of the ICR estimates of increases in commercialization during the project period is large and could reasonably be ascribed primarily to the concerted effort under LWH to develop agriculture’s market economy. Additionally, the increases in banking activity are consistent with increased commercialization. The Efficacy of the objective to Commercialize the Agricultural Sector is therefore rated Substantial. Rating Substantial Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) PHREVDELTBL PHOVRLEFFRATTBL Rationale Given that the efficacy of each of the two sub-objectives is rated Substantial, overall efficacy is rated Substantial. :" Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Efficiency will be assessed from the perspectives of Economic Viability, Implementation Efficiency, and Cost Effectiveness: Economic Viability The ICR estimates the project’s economic rate of return to be 55%, which is moderately sensitive to varying costs and benefits - a 20 percent increase in costs together with a 20 percent decrease in benefits yielding an ERR of 39 percent. The calculation is based on data from the Project Implementation Unit, the Development Impact Evaluation Unit, M&E reports, and field visits. Environmental benefits were also assessed, based on estimates of carbon sequestration, but the relative size of sequestration benefits is not clear. The ERR estimated in the ICR (55%) is substantially higher than the ERR calculated at appraisal of the LWH (29%), likely due to the higher achievements in productivity of LWH against the targets set at appraisal The ERR estimate may be on the high side due to some analytical assumptions - for instance, benefits from farmer organizations would already be expressed in general crop yield and value benefits, and an assumed life of 20 years is long for some infrastructure such as small-scale irrigation or terraces. Nevertheless, the productivity and value increases under the project could be expected to yield an ERR above the cost of capital. Implementation Efficiency LWH’s physical achievements and productivity increases can, as a crude indicator of implementation efficiency, be compared with the increases in project duration and financing. Thus, the yardstick could be the doubling of the project period, and the fourfold increase in financing. Under the project, the number of beneficiaries increased to three times the targeted increase; the productivity of irrigated and unirrigated land grew by 230 and 150 percent respectively, compared with targeted productivity and the share of commercialized production in total agricultural output rose from 35% baseline (and 60 targeted) to 89 percent. Accordingly, the very Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) substantial increase in the project period and in project funding does not appear to have unduly compromised either the pace of development or the increase in productivity. Cost Effectiveness. Comparative data on costs of similar projects are not provided in the ICR. Changes in cost-effectiveness over the project life, can, nevertheless, be assessed, At project completion, the total costs of the project’s field components (i.e. components A and B and excluding the broader institutional development costs under component C), were US$ 121.6 million and the number of direct project beneficiaries was 310,000, an average cost per direct beneficiary of US$ 392. A number of the project’s investments would have required on-site works (e.g. irrigation, catchment investments for sediment control and post-harvest facilities). At appraisal, costs per beneficiary were estimated to be US$ 330/beneficiary. Hence,at least as concerns actual cost- effectiveness versus appraisal estimates, cost-effectiveness was broadly the same. Overall Efficiency Although the two estimates available of LWH’s economic viability have different results, both show significant gains in economic returns. Available data indicate likely cost-effectiveness, and the project’s considerably longer implementation period is matched by the threefold increase in the project’s physical achievements. Overall Efficiency is assessed Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  29.00 Not Applicable 100.00 ICR Estimate  55.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome LWH was Highly Relevant. It responded to Rwanda’s strategic need to enhance welfare and reduce poverty in rural areas, and to drive GDP growth forwards. The project’s twin objectives – to increase productivity and commercialize the sector – were both key elements in the economic growth and poverty alleviation strategy. The Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) Efficacy of both Objectives was Substantial – productivity increased by 230 and 350 percent respectively on irrigated and non-irrigated land, and, based on the information available, the proportion of production commercialized is estimated to have grown from 60 percent to 89 percent. Efficiency was also Substantial. LWH was, economically viable and cost-effective, and the doubling of the project period was matched by commensurate increases in implementation. Overall, LWH’s Outcome was Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Interest in the project is widespread across various actors - farmers, entrepreneurs, government and development agencies - due to positive experience in the implementation of project activities. An important issue, however, is the organizational and financial health of the irrigation water user associations. Only 66% of farmers pay their water fees, and construction at the largest irrigation site was only completed in June 2018, at the end of the project, leaving little time for training and technical assistance to farmers and their WUAs. The ICR (pages 26 and 27) considers that, in general, WUAs are weak in financial, technical, and institutional capacity, and also comments that: “Capacity building efforts and technical assistance will need to continue.” As irrigation is one of the larger project components, the project's development outcome entails significant risks. 8. Assessment of Bank Performance a. Quality-at-Entry Project objectives were well aligned to Rwanda’s strategic needs for economic growth and poverty alleviation, and with the appraisal report’s theory of change which presented a generally clear institutional and technical pathway to reach these objectives. Project preparation was informed by the experience of an earlier project. The monitorable indicators were limited in coverage, especially for the commercialization objective which covered the extent of commercialization but without indication of sources of commercialization, quality or enhanced value. Project design was detailed and proved a mostly effective base for implementation, enabling only minor changes throughout the project period, including the extension period. But irrigation required adjustments in the scheme selection process – its development was initially slow as planned locations of irrigation systems included some schemes not suitable for hilly terrain. Land husbandry improvements alone would have been more suitable in such conditions. An adaptive approach was therefore adopted with decisions on the type of infrastructure based on watershed characteristics. Performance improved, and by project closure, with more favorable conditions for irrigation, the area irrigated exceeded initial targets by 180%. Project risks were tabulated in the PAD which indicated what was being done to reduce component-wide Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) risks, or what the project would do to tackle such risks. Although there were some shortfalls, overall quality of design was good, as attested by: first, the way that the project was aligned with Rwanda’s and the Bank’s policies and integrated within the rural sector’s institutional structure. And second, by the detailed design that provided a practical guide during implementation and required few changes. These features did not significantly change during the extension period, facilitating taking up the much larger program adopted. The Quality at Entry was Satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision Supervision was regular and made more effective by the location of the Task Team Leader in the country office, thereby enabling close interaction between the TTL and the project implementers and policy makers. Basic supervision activities such as financial management, procurement, and progress in implementation, were assiduously carried out, and adjustments were made to a component’s implementation when bottlenecks arose. Interchange of experience with other Bank projects, such as the Rural Sector Support Project provided mutually useful exchange of ideas. Team staffing included contributions from appropriate specialists. The integration of LWH as part of Rwanda’s combined Sector Wide Approach (SWAp), wherein the project was a key factor in Government’s intentions to scale up the agricultural and land and water husbandry program, was largely effective, despite the major increase in the dimensions and funding that the task team took on. The Bank was also effective in attracting development partners in project funding and implementation, and, in association with Government, coordinatiing of activities between partners, and establishing a "common basket," funding mechanism combining resources to a single funding channel. Overall Bank Performance Bank performance, both at design stage and during supervision, carried a complex project forwards, broadly achieving the project’s objectives. LWH’s design required little change during project implementation. Though risky, the radical increase in project funding, was accommodated without undue loss in quality. The Bank team played a lead role during implementation, facilitating adjustments as issues or bottlenecks occurred. The overall Quality of Bank Performance was Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) 9. M&E Design, Implementation, & Utilization a. M&E Design M&E was based on a clear Theory of Change with well-defined impact pathways and generally practical indicators for monitoring changes. M&E was embedded into the Ministry of Agriculture’s monitoring system, providing better means for consistency over time and across activities. Farmers’ self-help groups and cooperatives were used as a primary means of collecting data. Beneficiary surveys were also used. b. M&E Implementation Monitorable indicators over the project period were not changed, except in physical targets which were increased to reflect the larger size of the program resulting from the additional financing. The indicators enabled comparisons over time of project outputs, and, to a lesser extent, impacts and outcomes. Monitoring and analysis of outcomes could have been further elaborated, to add to the understanding of what the outcome was. For instance, for the commercialization objective – the only indicator chosen was the changes in “commercialization.” Yet only to know that agriculture had been “commercialized” conveys little idea of what happened and why. Greater elaboration of the types of crops and interventions which were particularly impactful, and of the main sources of commercialization – e.g. cooperatives, private businesses, individual farmers - would have provided additional operationally useful information. Also, monitorable indicators for outcomes could have been buttressed by other information that might be expected to be consistent with the primary indicator – for instance, from different sources, or from tracking variables with proxy value to the main indicator. There is little commentary in the ICR on the quality and reliability of M&E data (for instance, as relevant, brief remarks on sources of data, survey design, sampling, capacity of enumerators, etc.). Such information would have helped ascertain the reliability of some key project results, especially for commercialization (section 4). c. M&E Utilization M&E operated within a performance-based culture receptive to M$E findings. Data was used for operational decision making by government policy makers, and in Bank supervision missions.The mainstreaming of M&E into the overall LWH program enabled project data to be integrated into Rwanda's Sector Wide Approach.for the agriculture sector, enhancing utility in this broader program. Data from the independent DIME survey (section 4) could have been more integrated into the overall evaluation framework. M&E Quality Overall Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) The M&E system provided a base for project implementation, but with some weaknesses in monitoring of outcomes. Considering the M&E program as a whole though, the overall Quality of M&E is rated Substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental The project was category B and triggered the following safeguards: Environmental Assessment OP/BP4.01); Involuntary Resettlement ( (OP/BP 4.12); Natural Habitats (OP/BP 4.04); Forests (OP/BP 4.36); Pest management (OP 4.09); Physical Cultural Resources (OP/BP 4.11); Safety of Dams (OP/BP 4.37); and Projects on International Waterways (OP/BP 7.50). An Environmental and Social Framework and a Pest Management Plan were prepared. The Framework was needed as locations and types of investments could not be predicted in advance, hence the Framework provided overall guidance for environmental and social screening of the sub-projects. Where an investment was large or diverse, a dedicated environmental and social assessment was made. Although the issues and actions for the environmental and social aspects of the project are well covered in the ICR, and preparation of a Pest Management Plan is mentioned, for the majority of safeguards, there is no or minimal information on the situation and any actions taken. Five of the safeguards triggered are in this situation (natural habitats, forests, physical cultural resources, safety of dams and projects on international waters). Even if some of these are found to be not relevant to LWH, brief reference to the safeguard and why there are no issues to be addressed, would have been desirable, or where some issues are identified, appropriate actions could have been identified. Social As the sites of works could not necessarily be predetermined under a community driven project, a Resettlement Policy Framework was prepared, and screening and review procedures were issued to provide guidance on resettlement. For sites identified as requiring a comprehensive resettlement program, a Resettlement Action Plan was prepared. Six irrigation sites had such RAPs, but resettlement policy was also applied for compensating farmers affected by terracing and other small infrastructure. There was a total of 2,135 Project Affected Households, of which 117 required relocation. Compensation and job opportunities were provided in advance of any civil works. Equitable compensation was facilitated by the requirement that land titles and bank accounts of married couples had to be shared between both partners. And locally led grievance committees were established for resolving disputes between project affected persons. Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) b. Fiduciary Compliance Financial Management Once financial staff had gained experience, accounting was up to date and LWH’s financial management, which had been rated moderately satisfactory during the first two years of the project, was rated satisfactory throughout subsequent project implementation. Financial statements and audit reports were provided in timely fashion and all audits were unqualified. Disbursements were also timely. Good internal control and adequate staffing contributed to this generally good performance. Procurement Procurement was carried out following Bank procedures, though with implementation issues initially - in particular, delays in design studies, and cases of non-responsive bids. The latter was due to capacity constraints of local contractors, which were too small to handle the financing needs for larger contracts. Other issues were unrealistic procurement plans and uneven contract management. Nevertheless, the ICR does not refer to any irregularities, the issues being more a matter of weak administration. The procurement problems were progressively resolved, with intensive supervision from Bank staff, over the first several years of the project, and do not appear to have significantly delayed project implementation. By the fourth year of the project, procurement was rated satisfactory, although it was downgraded to moderately satisfactory in 2017, due to understaffing in the procurement unit. In the event, the existing staff were able to shoulder the workload, made easier by closure of other projects with procurement managed by the LWH procurement unit. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons 1. A major extension of a project’s implementation period might be appropriate if the extension is for strategic reasons, is expected to be economically viable, is adequately resourced. and is practically implemented. The unusual step of doubling the project period was: to maintain the momentum of the field development program, which was still acquiring experience, to facilitate the involvement of financing partners in Rwanda's sector-wide agricultural development program; and, to further consolidate institutional links. An alternative would have been a new project providing opportunity for a fresh look at the development program, but the LWH’s extended program was effectively implemented, contributing to the broader and longer-term goals of Rwanda's rural development program. 2. Success with a long-term project extension is facilitated by a clear strategy and technical features. Amongst the factors driving the success of the project’s additional 4 ½ year program were: the extension benefitted from the experience of LWH’s first four years; technical and institutional features were already tested and remained largely the same; Government and the Bank were dedicated to a sector-wide rural development program; and other donors were willing to support LWH's program without variance in project implementation features, including combining their funds with the Bank into a common financing basket. 3. Monitoring should place particular focus on monitoring the outcomes of the development objectives. For assessing the outcome of the commercialization objective, the only direct measure of outcome was the share of “commercialized” products as a percentage of total farm production. It would have been desirable to know how commercialization was defined, how it was measured, what commercialization activities were most viable, and what particular crops and marketing or processing activities were involved. Other supporting evidence. proxy data, or additional monitorable indicators, would also have been desirable. 4. A flexible approach in the proportion of different components and activities in rural development projects provides scope for enhancing outcomes. LWH’s field development program was adjusted during project implementation to increase the proportion of the land husbandry activities, thereby reducing the proportion of the larger irrigation schemes, which provided lower returns and were more difficult for farmers to manage. Local conditions were taken as the determining factor for developing each watershed. 13. Assessment Recommended? Yes Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW:Land Husband,Water Harvest,Hill Irrig (P114931) Please explain As part of an assessment of the experience in watershed improvements of several projects to ascertain factors that can contribute to success. 14. Comments on Quality of ICR The ICR is thorough, thoughtful and candid. It is particularly strong in its strategic orientation, which provides context for the report. Supporting information is also generally provided. And the lessons are practically oriented, and well anchored in the project’s experience. The boxes in the text, and photographs at the end of the report add understanding to the project’s achievements (several additional boxes or small tables in the efficacy section highlighting some of the output achievements would also have been useful). Significant changes and key factors affecting implementation and outcome are clear and in sufficient detail. There are several areas where the ICR could have been improved : (i) information on the sources of data and its reliability (critical for assessing the project’s outcome) are largely missing; and there could have been greater use of other relevant sources of data, such as the Bank’s DIME survey; (ii) the reasons behind the decisions for scaling-up, and for categorizing the project as a SWAp could have been elaborated; (iii) the project’s situation or role within the overall lending program, both pre-project and planned, was only cursorily described; and (iv) any actions taken regarding safeguards beyond environmental, social and pest management aspects could have been noted. Overall, though, the ICR is an informative and analytical review and its Quality is Substantial. a. Quality of ICR Rating Substantial Page 17 of 17