Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Report Number: ICRR0022303 1. Project Data Project ID Project Name P122486 IN: Karn Wtrshed II Country Practice Area(Lead) India Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-50870 31-Dec-2018 40,181,396.71 Bank Approval Date Closing Date (Actual) 06-Sep-2012 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 60,000,000.00 0.00 Revised Commitment 45,000,000.00 0.00 Actual 40,181,396.71 0.00 Prepared by Reviewed by ICR Review Coordinator Group Richard Anson J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO), as stated in the Financing Agreement (FA, 2012) and the Project Appraisal Document (PAD, 2012), for this India Karnataka Watershed Development Project II (KWDP II), was to “demonstrate more effective watershed management through greater integration of programs related to rainfed agriculture, innovative and science-based approaches, and strengthened institutions and capacities”. KWDP II was part of the national Integrated Watershed Management Program (IWMP). Page 1 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) While the KWDP II had three restructurings during implementation, the PDO remained the same. There were some adjustments in some of the performance indicators and targets with respect to the PDO and some of the components, as described in Section 2e below, where it is argued that a split evaluation of objectives be undertaken in this review. For purposes of assessing the extent to which the PDO was achieved in Section 4, this review will parse the PDO into three objectives: Objective 1: to demonstrate more effective watershed management through greater integration of programs related to rainfed agriculture; Objective 2: to demonstrate more effective watershed management through innovative and science-based approaches; and Objective 3: to demonstrate more effective watershed management through strengthened institutions and capacities. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components Component 1: Support for Improved Program Integration in Rainfed Areas (Original allocation: US$22.13 million; Actual: US$17.86 million): Five activities, namely; (i) developing a Land Resource Inventory (LRI) of the pilot micro-watersheds (MWSs) and the decision support systems (DSSs) for improved integration of watershed management; (ii) developing a spatial digitized library; (iii) conducting hydrological assessments of selected catchments, and generating and disseminating water availability maps to farmers; (iv) developing MWS master plans for soil and water conservation investments, integrating with plans in the Integrated Watershed Management Program (IWMP) and the National Rural Employment Guarantee Scheme (NREGS); and (v) strengthening community-based monitoring and documentation. Component 2: Research, Development, and Innovation (Original allocation: US$19.57 million; Actual: US$21.75 million): This component was comprised of four four activities; (i) conducting applied research studies on several strategic topics; (ii) conducting hydrological monitoring and piloting community-based groundwater management; (iii) developing or adapting planning and training tools, maps, and climate information for use by farmers; and (iv) strengthening the institutional arrangements for rainfed agriculture and watershed management research and technology transfer. Component 3: Institutional Strengthening (Original allocation: US$9.14 million; Actual cost: US$1.61 million): Four activities, namely; (i) developing and delivering training modules, awareness raising activities, and structured exposure visits on extension, integrated watershed management, and value addition for key Page 2 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) actors; (ii) training of key Government staff on in-house M&E systems and conducting exposure visits for state-level stakeholders; (iii) improving the capacities of farmer contact centers (RSKs) in project areas; and (iv) upgrading government training centers’ servicing activities and strengthening their information systems. Component 4: Strengthening Horticulture in Rainfed Areas (Original cost: US$27.57 million; Actual cost: US$14.93 million): Five activities, namely; (i) demonstrating productivity improvement in annual and perennial horticulture crops, nutrition gardens, crop diversification, and crop-soil-water relationships; (ii) strengthening model nurseries in the existing Department of Horticulture (DoH) farms; (iii) establishing soil and leaf tissue analysis laboratory facilities and providing training to farmers on their use; (iv) conducting feasibility studies, participatory value chain investigations, and field demonstrations of low-cost post-harvest equipment and expanding the DoH skill development programs on the production post-harvest management, value-addition, and market linkage support; and (v) establishing producer companies. (e) Component 5: Project Management and Coordination (Original cost: US$7.29 million, Actual cost: US$3.79 million). Ensuring effective and efficient project management by financing (i) specialized consultants; (ii) incremental administration costs; (iii) a communication strategy; and (iv) third-party M&E. The project was to implement watershed management planning and capacity-building activities in two phases: support a small number of sub-watersheds in the first phase, and apply lessons learned and experience from the first phase in additional sub-watersheds under a second phase. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates (i) Project Costs: The total project costs at approval was US$85.7 million. The actual project costs at closing was US$59.94 million (or about 70% of original total costs). The difference was due to three factors: a cancellation of US$15 million of the IDA credit (during the December 2018 restructuring); SDR depreciation against the US dollar of almost US$5 million; and a decrease of Government counterpart funding of US$5.94 million. (ii) Financing: At approval, the IDA credit (IDA No. 50870) was US$60.0 million, with a Government counterpart financing commitment of US$25.7 million, totaling US$85.7 million. By the end of the project, total financing had decreased to US$59.9 million (IDA: US$40.18 million; Government: US$19.76 million), due to the 3 factors cited above. (iii) Borrower/Recipient Contribution: At approval, the Borrower counterpart contribution was supposed to have been US$25.7 million, with a final contribution of $19.76 million (or 77%) (ICR, Annex 3). The main reason for this decreased contribution was due to reduced project costs for various components. (iv) Dates: The project was approved on September 6, 2012, became effective on April 23, 2013. A mid- term review was carried out in November 2016. The original closing date was December 31, 2018, with the final closing date being December 31, 2019. Restructurings: Significant Changes During Implementation, Revised Allocations Among Components and Revised Indicators/Targets (see ICR Annex 7 for details of the main changes) Page 3 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) The project underwent three Level-2 restructurings: in July 2015, in October 2017, and in December 2018. While the PDO was not revised, PDO indicators and targets were revised in the first two restructurings (see ICR, paras. 12 - 24 and Annex 7). Before any project restructuring, the four PDO indicators were as follows (ICR, paras 8 and 9):  Percentage of micro-watersheds with improved convergence and integration (target 70%);  New science-based approaches and tools adopted into wider watershed operations (number);  Improved M&E capability in Watershed Development Department (WDD) (M&E cell established and functional); and  Incremental change in agricultural and horticultural productivity in project areas for selected crops (ton/hectare) (target 70%). The main revisions in components and changes in PDO indicators during restructuring were as follows (ICR, paras. 14 and 16): First restructuring in 2015: The changes made were (a) three new activities were introduced under component 2, (b) the PDO indicator (percentage of micro-watersheds with improved convergence and integration) was changed from only hydrological assessments to include the digitized library, the DSS and the portal; and (c) for the PDO Indicator (increase in agricultural and horticultural productivity) the target was reduced from 75% to 25%. These changes resulted in a reduction in the level of ambition of the project's PDO. Second restructuring in 2017: The changes made were (a) various activities were added and dropped, (b) the major change was abandoning the four original PDO indicators and replacing them with the following two:  Data from science-based approaches adopted by other departments/institutions in the state for planning (number).  Sub-watershed management plans based on innovative science-based approaches and community consultation (number). These changes resulted in another reduction in the level of ambition of the PDO. Third restructuring in 2018 made the following changes (a) value addition support was shifted from farmer- based organizations to farmer-producer organizations (FPOs), also resulting in some adjustments in costs; (b) there was cancellation of $15 million from the IDA credit and an extension of the closing date by 12 months (to December 31, 2019); and (c) the scope of the project increased with respect to two aspects: the size of the saturation area where the project was to finance implementation of the newly developed watershed management plans (DPRs) increased from 11 to 87 micro-watersheds, and from 9 to 11 sub- watersheds; and the inventory scale increased from 935 micro-watersheds to 1,931 micro-watersheds, with the addition of four districts to the project (from 7 to 11 districts) (see ICR, para. 20, and Annex 7). These changes resulted in no reduction in the level of ambition of the PDO. These revisions in components, PDO indicators and targets, especially those which were part of the second restructuring, resulted in a two stage reduction in the level of ambition of the project, caused by the revised indicators, targets and achievements (summarized in Annex 7 of the ICR). It was for these reasons that this review will undertake a split rating of objectives in Sections 4 and 6 based on the project's restructuring in Page 4 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) 2015 and 2017. It should be noted that the ICR assessment of the project's achievements in Section II B and its split rating analysis of objectives recognizes only the 2017 restructuring because "it was the most profound revision of the results framework" (para 28). 3. Relevance of Objectives Rationale The project objectives were relevant to addressing the critical problems of achieving sustainable and effective watershed development in the State of Karnataka, namely: (a) Ineffective watershed management planning and implementation; (b) Ineffective integration of national support programs; (c) Outdated and generic approaches to watershed management; and (d) Insufficient capacities of central and Karnataka state extension services, and of farmers. Accordingly, the overall PDO was to achieve more effective watershed management through three strategic sub-objectives. The objectives were strongly aligned with the Government of India (GOI) 12th Five Year Plan for 2012–2017, which called for inclusive growth through increasing agricultural production and food security, with attention to rainfed agriculture. The project was particularly relevant to supporting core elements of the Plan’s priorities, especially regarding the integrated water management program (ICR, para. 5). As part of Government’s national watershed management program, Government had identified three weak, but imprecisely defined, watershed management aspects (integration with other programs supporting rainfed agriculture, the extent of innovative and science-based approaches, strengthened institutions and capacities) which were considered to be major gaps which needed to be addressed by the project. Hence, these gaps motivated Government’s request to the Bank for financing this project. The project's objectives also contributed to the fulfillment of the World Bank Group’s FY13–FY17 Country Partnership Strategy (CPS) for India, especially with respect to the improvement of rainfed agriculture’s productivity by supporting the implementation of the national IWMP; helping farmers diversify production, lower the cost of production, and identify stable markets for their goods; supporting self-employment opportunities; and developing in-demand skills, particularly for women. The Project also was strongly aligned with and supported the Bank's subsequent Country Partnership Framework/CPF for India (FY18 - FY22) with respect to a focus area of "Promoting resource-efficient growth", objective 1.1 and three cross- cutting themes: climate-smart engagement; addressing gender gaps; and harnessing high impact technologies. The World Bank had a history of partnership with India in supporting their watershed development program, supporting 11 watershed development projects, since 1981, for a total of US$808 million. Projects pioneered and scaled-up participatory watershed development planning integrated with livelihood improvement and improved monitoring and evaluation (M&E). KWDP II was a follow-on intervention to the Karnataka Watershed Development Project (KWDP), which closed in FY09 and was implemented by the Government of Karnataka (GoK) Watershed Development Department (WDD). The GoI requested a different model of support which would ensure that watershed management was Page 5 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) implemented more effectively, and would address key weaknesses in watershed management (ICR, paras. 3 and 4). In summary, this project's objectives addressed highly relevant watershed management issues and the objectives were also highly aligned with both Government and World Bank development strategies for India. The relevance of the project's objectives is therefore rated high. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To demonstrate more effective watershed management through greater integration of programs related to rainfed agriculture. Rationale While the project’s original design included a results framework (RF) in the PAD, it did not include a theory of change (ToC) because it was not required at the time the PAD was written. The ICR reconstructed a ToC for the project which was consistent with its strategy, objectives and components, especially with respect to the overall outcome of promoting more efficient watershed management through the three sub- objectives (ICR, Figure 1, paras. 6 & 18). The ICR highlights the rationale for addressing watershed development in rainfed areas (ICR, para. 1), with high concentrations of environmental stress and rural poor. Theory of Change: The ToC emphasizes integration of micro watershed (MWS) master plans for soil and water conservation investments developed for this project with those developed by the India-wide Integrated Watershed Management Program (IWMP) and the National Rural Employment Guarantee Scheme (NREGS). As noted already, poor integration of the national rainfed support program with other schemes was a weak aspect of watershed management (ICR, para 4). Outputs: Number of micro-watershed master plans (MWMPs) integrated with larger scale sub-watershed assessments: Original Target: 935 not achieved. The convergence target with the IWMP and NREGS of 70% was not achieved as planned because the IWMP ceased to exist in 2015 and the mobilization of the NREGS resources until then had been slow (ICR, para 29). Outcomes: Page 6 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Percentage of micro-watersheds with improved convergence and integration: Original Target: 70%; Revised Target: 25%; Original target not achieved when the project closed for the same reasons as the output was not achieved. The efficacy with which Objective 1 was achieved is rated modest Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective To demonstrate more effective watershed management through greater integration of programs related to rainfed agriculture, but with a reduced level of ambition resulting from the restructuring in 2015. Revised Rationale Theory of Change. The ToC was the same as for Objective 1, but with a reduction in the watershed management planning and implementation to achieve integration and convergence with IWMP and NREGS. The a reduced ambition reflected a deeper understanding of the long time it took to achieve the envisioned integration and convergence (ICR, para. 18); Outputs (ICR, para. 34, Annex 7): As for Objective 1, and in addition, the following outputs information was recorded in the ICR: The target for the number of micro-watershed master plans integrated with larger scale sub-watershed assessments at the end of the project was reduced from 935 to 931. The ICR comments that "The target was not upscaled to 1,931 with the addition of four districts in October 2017 (see below) due to an oversight" (Annex 7, page 75) A reduction of the time taken for net planning at the micro-watershed (MWS) level was introduced in July 2015 "to reflect the efficiency introduced into the planning process", but the new standard was dropped in October 2017 "because of concerns about its measurability" (Annex 7, page 75). Outcomes. As noted above, the percentage of micro-watersheds with improved convergence and integration at the end of project target was reduced from 70 percent to 25 percent. The reduction was made because of the expected shift in the IWMP. The ICR does not indicate whether 25% was achieved. Because of the absence of any evidence of progress toward the objective, the efficacy with which Objective 1 Revision 1 was achieved is rated modest. Revised Rating Modest Page 7 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) OBJECTIVE 1 REVISION 2 Revised Objective To demonstrate more effective watershed management through greater integration of programs related to rainfed agriculture, but with a reduced level of ambition resulting from the significant restructuring (substantial changes in PDO indicators) in 2017. Revised Rationale Theory of change. The ToC was the same as for Objective 1 except that, according to the ICR, the IWMP had been terminated and "since the IWMP resources were no longer available, it was decided to use the project resources to finance implementation of DPRs within several MWSs (called a saturation area), thus demonstrating the innovations and technologies supported by the project". The "PDO indicator which measured the integration outcome was reduced in scope to measure adoption of science-based data by departments/institutions in the state for watershed planning". In addition, the PDO indicator measuring improved agricultural productivity was dropped because yield increases were not expected before project closure (ICR, para 19). Outputs (ICR, Table 4 and Annex 1, page 46) Departments and institutions provided access to information from the digital portal and decision support system (DSS): Target: Yes (all 5 Depts.); Actual: Yes: Percent of Target: 100% Establishment and operationalization of a public portal, as interface between digitized library, Decision Support System (DSS) tools and users; Organization of international conference to showcase the Land Resource Inventory (LRI) work; Hosting of five day knowledge sharing workshop for key representatives from 9 states; Development of three demonstration models on soil fertility management in MWSs; integrated nutrient management; water harvesting and improved production technologies. Outcomes: Data from science-based approaches adopted by other departments/institutions in the state for planning. (Target: 5; Actual: 5: Percent of target: 100%. Substantive outcomes generated by the science-based approaches to planning included (a) the Department of Agriculture used the LRI data and crop planning DSS to determine the suitability of crop introduction in watersheds; and (b) the Rural Development and Panchayat Raj Department (RD-PR) used hydrological monitoring data and the farm pond and check dam planning DSS to determine the need for check dams in Karnataka. Other important outcomes are listed in the ICR (para 34) The extent to which Objective 1, Revision 2 was achieved is rated substantial Revised Rating Page 8 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Substantial OBJECTIVE 2 Objective To demonstrate more effective watershed management through innovative and science-based approaches. Rationale Theory of Change: Problems which constrain the effectiveness of watershed management in the target areas were identified as high levels of poverty, as well as ineffective and outdated watershed management planning and implementation. In response, the project design established priority activities which would address these constraints through generating, testing and implementing enhanced innovative and science-based approaches to achieve more effective watershed management. These investments provided the basis for a range of approaches and outputs which resulted in a number of important outcomes, including preparation and implementation of: enhanced/”new generation” watershed management plans, supported by improved Land Resource Inventories (LRIs); establishment of digitized information to support planning; generation/piloting of improved production and post-harvest technologies at the farm level; and access to improved weather advisory information. Outputs (based on ICR, Annexes 1 and 7): Number of new hydrological monitoring systems installed and functional (advanced and community monitored): Equipment Units: Original Target: 150; Revised Target: 120; Actual: 179; Percent of Revised Target: 149%; Advanced Monitoring Stations: Target: 14; Achieved 14: Percent of Target: 100%; Community Monitoring: Target: 25; Achieved 25: Percent of Target: 100%; Number of MWSs with digitized database library fully operational: Target: 935; Actual: 2534: Percent of Target: 270%; Number of Demonstrations of Improved Technologies for processing and value chain improvements in selected horticultural crops: Target: 108; Actual: 60; Percent of Target: 56%; Number of Farmer Contact Centers for improved information for farmers on climate change and risk management: Target: 73: Actual: no data in the ICR Number of commodities with value addition and market system improved: Target: 3; Actual: no data in the ICR Outcomes (a) The ICR (para 30) states that the outcome, as measured by the original PDO indicator, (new science- based approaches and tools adopted into wider watershed operations) was "partially achieved". It is noted, however, that Table 2 in the ICR which shows the target for the number of new science-based approaches and tools adopted into wider watershed operations as 2 compared with an actual achievement of 4 based on the revised definition of this indicator "before restructuring" in 2017 and not before restructuring in 2015. Page 9 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) (b) Incremental change in agriculture/horticulture productivity in project areas for selected crops (MT/ha.) - no data in the ICR Based on the shortfalls in outcomes for this objective, measured with the relevant indicator before the 2015 restructuring, the efficacy with which Objective 2 has been achieved is rated modest Rating Modest OBJECTIVE 2 REVISION 1 Revised Objective To demonstrate more effective watershed management through innovative and science-based approaches, but there was a reduced level of ambition because the indicator calling for an increased productivity of agricultural and horticultural crops was reduced from 75% to 25% (Restructuring in October 2015). Revised Rationale Theory of Change: While based on essentially the same theory of change, revision 1 Introduced key innovative tools which were developed and implemented under the project, but also reduced the level of ambition of a key performance indicator (i.e. increased productivity of target crops). Outputs Number of new hydrological monitoring systems installed and functional: Advanced Monitoring System: Revised Target: 120; Actual: 179; Percent of Revised Target: 149%; Number of MWS master plans where innovative and relevant science-based knowledge is used to develop Land Resource Inventories: Target: 1,931; Actual: 2,534; Percent of Revised Target: 131%; Number of MWSs with digitized database library fully operational: Revised Target: 1,931; Actual: 2,534; percent of revised target: 272%; Number of MWSs with pilot demonstrations on watershed treatments saturation: Revised Target: 87; Actual: 89: Percent of Target: 103%; Number of pilot demonstrations of water harvesting and crop production technologies: Target: 9: Actual: 11: Percent of Target: 122%; Introduction of peri-urban watershed planning and management pilots. Outcomes: The designated PDO indicator was "Number of sub-watershed management plans based on innovative science-based approaches and community consultations": Target: 9; Actual: 11; Percent of Target: 122%; Page 10 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Incremental increase of yields (of 25%) for agricultural and horticultural crops was not achieved (and productivity progress was not cited) This ICRR does not accord the same importance as the ICR to the "number of sub-watershed management plans based on innovative science-based approaches and community consultations" because this indicator does not measure the outcome of Objective 2/Revision 1. While this indicator measures a process, it does not "demonstrate more effective watershed management through innovative and science-based approaches". While productivity increase (usually a sound PDO indicator) is often not achieved before project closure, some progress towards "increased productivity of agricultural and horticultural crops" may have justified a "substantial" efficacy rating. However, the ICR provided no evidence of increased productivity. Therefore, in the light of the above points with respect to the two outcome indicators, the efficacy with which this Objective 2 Revision1 was achieved is rated "modest". Revised Rating Modest OBJECTIVE 2 REVISION 2 Revised Objective To demonstrate more effective watershed management through innovative and science-based approaches, but while science-based approaches were retained, the aim to achieve incremental increase in agricultural and horticultural yields was abandoned (restructuring in 2017). Revised Rationale Theory of Change: Overall , most elements of the ToC for this objective remained the same, while incorporating improved tools, but the PDO indicator involving incremental agricultural and horticultural yields was eliminated, due to a change in the national watershed management program and project implementation delays. Outputs: (In addition to the outputs already listed above under Revision 1, unless noted below.) Community monitoring breakdown indicator: Dropped (due to termination of the national IWMP); Improved information for farmers on climate change and risk management: Dropped (measured by next output Number of farmers in each MWS willing/intending to adopt at least two climate-adaptation and/or climate mitigation practices recommended by LRI knowledge and sub-watershed management plans: Target: 70% of farmers; Actual: 78%; Percent of Target: 111%; Value addition and marketing system improved: dropped (but merged with another indicator); Peri-urban watershed planning and management pilot: dropped, due to delayed launching; Page 11 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Number of sub-watershed in which soil and water conservation investments were made, based on management plans: Target: 11: Actual: 11: % of Target: 100%; Outcome: New science-based approaches and tools successfully adopted into wider watershed operations. The ICR concluded that as a result of the innovations and capacities introduced into watershed management planning at the MWS level, soil and water conservation planning and execution, crop planning, nutrient management, and livelihood interventions would be more effective and would demonstrate strong results while better utilizing government resources (Annex 7, page 74). The efficacy with which Objective 2, Revision 2 was achieved is rated substantial Revised Rating Substantial OBJECTIVE 3 Objective To demonstrate more effective watershed management through strengthened institutions and capacities, but the PDO indicator (“M&E capability of the WDD” as a measure of institutional capacity) was dropped because “it measured only a narrow capacity element in the WDD” (ICR, Annex 7). Rationale Theory of Change: Based on an assessment of institutional capacities of central and extension services, there was a need to identify priority activities which would enhance the capacities of the relevant agencies and farmer groups, at central and provincial levels to prepare, implement and monitor/evaluate enhanced approaches to watershed management, with a result focus. The ToC (ICR, Figure 1) illustrates how the institutional development objectives were achieved through the inter-connections between the various inputs, outputs and institutional outcome, including interventions at the farmer group level. The restructuring in either 2015, 2017 or 2018 did not affect the level of ambition for Objective 3 and hence this review does not consider any revisions of this objective The main relevant outputs and outcomes achieved and not achieved with respect to Objective 3 during this project's implementation are summarized below. Outputs (ICR, Tables 3 and 6, paras 32,33, 37,38, and Annexes 1 and 7): Percentage of farmers in each micro-watershed willing/intending to adopt at least two climate-adaptation and/or climate-mitigation practices, recommended by LRI knowledge and sub-watershed management plans: Target: 60%; Actual: 74%; % of Target: 123%; note: Annex 7 states this indicated was dropped but provides no reason. Page 12 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Percentage of farmers trained in each sub-watershed intending to adopt practices that are based on integrated land resource knowledge generated by the project: Target: 70%; Actual: 78%; % of Target: 111% Number of Watershed Assistants trained on use of LRI data for assisting farmers: Target: 60; Actual: 172: % of Target: 350%. Percentage of farmers expressing satisfaction with the LRI-based assistance provided by Farmer Centers. Target: 75%; Actual: 93%; Percent of Target: 124%; Number of strengthened women's self-help groups (SHGs) supported: Target: 250; Actual: 360: Percent Target: 144%; Number of technical staff trained on use of LRI data for assisting farmers: Target: 60; Actual: 172; % of Target: 287%; Farmer Producer Organizations (FPOs) replaced the previous Farmer Based Organizations (FBOs), to provide more effective support, covering several activities, which contributed to the overall objectives more effectively, involving improved value chains, processing facilities, development of business plans, and logistics infrastructure Strengthened women self-help groups supported .Target: 250; Actual: 360; Percent of Target: 144%; Number of laboratories established and operationalized: Target: 2; Actual: 2; Percent of Target: 100% Outcome Improved M&E Capability in the Watershed Development Department (Target: M&E cell produces M&E reports on regular basis: Actual: M&E reports were produced on a regular basis). While the ICR (para. 32) states that this outcome "as measured by the original indicator was partially achieved” by end of project it was 100% achieved (ICR, Annex 1). Nevertheless, by the second restructuring in 2017 this indicator had been dropped because, according to the ICR, it measured a narrow capacity element in the Watershed Development Department (WDD). Percentage of farmers adopting improved conservation practices & production technologies: Target: 60%; Actual: 89; Percent of Target: 148%. Percentage of farmers adopting Improved horticulture techniques: Target: 60%; Actual: 28%; Percent of Target: 47%; Based on the achievement of the PDO indicator, other indicators and the institutional improvements this review rates the extent to which Objective 3 has been achieved as substantial. Rating Substantial Page 13 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The efficacy of the extent to which the original objectives were achieved was modest. This was because there was very little progress made towards Objective 1, namely the integration of the activities in this project with those of the national Integrated Watershed Management Program (IWMP) and the National Rural Employment Guarantee Scheme (NREGS). For the original Objective 2 the project made only partial progress on "new science-based approaches and tools adopted into wider watershed operations". For Objective 3, however, for which there were no revisions, efficacy was rated substantial . Nevertheless because of the original poor performance vis a vis Objectives 1 and 2, the overall efficacy of the original objectives was rated modest.. Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale The overall efficacy of the objectives at their first revision was also modest. The basis for this rating was as follows. With respect to Objective 1 Revision 1, efficacy was rated negligible because, having reduced the percentage of micro-watersheds with improved convergence and integration from 75% to 25%, there was no evidence that this much lower level of ambition was achieved. The efficacy of Objective 2 Revision 1 was rated modest because, although the first restructuring had reduced the target for increased productivity of agricultural and horticultural crops from 75% to 25%, there was no evidence that the lower productivity was achieved. As noted above, although the number of sub-watershed management plans based on innovative science-based approaches and community consultations was exceeded by 22% this was not an outcome. There was no revision of Objective 3 and hence the efficacy with which this objective was achieved remained the same, namely substantial. Based on these ratings the overall efficacy of the objectives at their first revision was rated modest. Overall Efficacy Revision 1 Rating Primary Reason Modest Low achievement OBJR2_TBL OVERALL EFFICACY REVISION 2 Overall Efficacy Revision 2 Rationale The efficacy with which Objective 1 Revision 2 was achieved was rated substantial by this review because new science-based approaches and tools were being successfully adopted into wider watershed operations and substantive outcomes were being generated. .The evidence regarding the achievements of Objective 2 as a result of Revision 2 was that data from science-based approaches had been adopted by other departments/institutions in the state for planning, and that consequently at the MWS level, soil and water conservation planning and execution, crop planning, nutrient management, and livelihood interventions would be more effective and were expected to generate strong results. As noted earlier, there were no Page 14 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) revisions for Objective 3 for which efficacy was rated substantial during the project implementation period. The overall efficacy of these achievements is rated substantial by this review. Overall Efficacy Revision 2 Rating Substantial 5. Efficiency Overall, the project performance and results demonstrated an efficiency rating of substantial, based on the various evidenced-based tools applied and presented in the ICR (paras. 44 & 45, and Annex 4, also informed by the project’s impact evaluation study). This detailed assessment in the ICR shows that the project cost- benefit analyses demonstrated improved efficiency, especially following the project restructurings, compared to the values at appraisal. In the cost-benefit analysis, the project compared well with past watershed projects, and considerably better than the predecessor KWDP. Using conservative assumptions and reliable datasets, the economic rate of return (ERR) at completion was estimated in the ICR to be 27.4 percent, compared to the two ERRs estimated at appraisal (a) ERR of 19.7 percent, if other complementary incremental investment funds (from NREGS program) are included in the project investments; and (b) ERR of 17 percent, with only the project investments from the overall “parent” IWSM program. Also, the ICR financial rate of return (FRR) was estimated to be 18.8 percent, slightly above the estimated FRR of 17.2 percent estimated at appraisal. As the ICR states, the ERR is, of course, sensitive to reductions in income for farmers due to the weather (ICR, para. 44), but the analysis in Table 4.3 in Annex 4 of the ICR shows that the estimated percentage decline in the ERR due to the weather is lower than the estimated percentage decline in farm income due to weather. The ICR does not comment on this. With respect to other cost-effectiveness measures, the ICR showed positive results due to the project: (a) The total project financial cost at current prices per project farmer was India Rupees (INR) 11,625 over the project period compared to INR 24,194 at appraisal; (b) The cost per hectare of designing and implementing watershed management interventions was INR 14,786 during the project, compared to INR 20,479 in non-project watershed management planning and implementation under the IWMP, showing a drop of 28 percent because of project interventions (para. 44); and (c) The project showed significant reductions in the duration of the normal watershed project cycle used in Government programs, thereby reducing watershed treatment/management costs; these reductions resulted from the project-supported generation and availability of the land resource inventory (LRI) data and from the large inventory of micro-watersheds for which LRI data is now available. The implementation efficiency is also reflected by project actions and corresponding efficiency measures (ICR, para. 45): (a) The ICR stated (para 45) that the project caught up with the delays experienced during the first two years with a one-year extension, while expanding significantly the: project's geographical scope (from 7 districts to 11, and from 935 MWSs to 2,534 MWSs); scope of the digitized library, data portal, and DSS tools, to include additional features; and the beneficiary reach (11 FPOs compared to 2 planned, and 360 SHGs compared to 250 planned). Further, available project resources were used to implement DPR interventions in a saturation Page 15 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) zone—an activity which was not budgeted originally, due to project efficiency measures and other aspects, /actions, including: several key items purchased at a lower price; and depreciation of the Indian rupee; (b) The ICR also noted in paragraph 45 that the project generated efficiencies for the GoK in watershed management planning and implementation (ICR), and also demonstrated cost efficiency in implementation (e.g., management and coordination costs were 6.47 percent of the IDA amount, compared to 8.50 percent allocation at appraisal; and 6.32 percent of both the IDA and government allocations compared to 7.29 percent at appraisal, which was commendable considering the additional year of project implementation); (c) The project also contributed to more efficient targeting of soil and watershed investments once the DPR was completed, based on the different Decision Support System (DSS) tools that were developed under the project. The ICR does recognize some inefficiencies, although not major, mentioned in para. 45: Some inefficiencies were observed in the advertisement of procurement items above US$10,000 in national and local newspapers despite a World Bank threshold of US$100,000, which was subsequently rectified, in accordance with Bank procurement guidelines. There was also frequent turnover of senior officials during the first two project years and of procurement staff throughout implementation, which contributed to implementation inefficiencies. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  17.00  Not Applicable 0 ICR Estimate  27.40  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The Outcome Rating is based on a split evaluation of the performance of the project's objectives, for reasons stated in Section 2e, and influenced by the features of the three restructurings during the implementation period, such as reduced levels of ambition for some of the key indicators and their targets, albeit the project's objectives remained the same. Based on the project’s high relevance, a range of ratings for efficacy, and substantial efficiency this review concludes that the project’s overall outcome is rated “Moderately Satisfactory”. Table 1 provides the basis for this assessment. The broad evidence used for the ratings of the three core elements cited above as determining the project's overall outcome rating are as follows: Page 16 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) (1) High rating for relevance of the PDO, based on the project's strong alignment with (a) Government national and state-level policies and strategies for enhancing watershed development, especially in watersheds supporting highly populated rainfed agriculture areas; and (b) IDA’s country partnership strategy/framework, while building on the lessons of previous extensive investments by the government in watershed management programs in many states; (2) Modest and Substantial ratings for efficacy, based on the evidence of the project’s achievements in meeting the three objectives defined in Section 2a of the review and in light of performance indicators which were amended by project restructuring during project implementation. (3) Substantial rating for efficiency, considering the economic and financial achievements reflected in positive economic and financial rates of return together with an overall efficient implementation performance, based on the various cost-effectiveness measures presented in the ICR. Table 1: Overall Outcome Rating Rating Aspects and Objectives at Objectives at Original Objectives Dimensions First Restructuring Second Restructuring 1) Relevance of High Objectives 2) Efficacy (with same objectives, but revised _ _ _ indicators/targets) Objective 1: Greater integration of Modest Modest Substantial programs related to rainfed agric. Objective 2: Innovative and Modest Modest Substantial science-based approaches Objective 3: Strengthened Substantial Substantial Substantial institutions and capacities 2) Overall Efficacy Modest Modest Substantial 3) Efficiency Substantial Moderately Moderately Outcome Rating * Satisfactory Unsatisfactory Unsatisfactory Outcome Rating 3 3 5 Value Amount Disbursed 2.79 12.57 24.84 ($M) Disbursement (%) 6.9 31.3 61.8 Weight Value 0.21 0.94 3.09 Weighted Total Rating 4.24 (rounded to 4.0) Page 17 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Overall Outcome Moderately Satisfactory Rating *For details, see ICR, paras. 25-45, Annex 7, and the disbursement table on p.2). a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome There is moderate risk to sustaining the project’s outcomes and contribution to the project’s expected impacts. The ICR provides sound evidence and rationale for this conclusion, while also highlighting various mitigation measures promoted by the project during implementation (ICR, paras. 76 and 77), including: (a) The 14th Project Empowered Committee (PEC), the project’s Steering Committee, held a meeting in July 2019 (six months before the project was due to close), and approved an exit strategy to ensure continuous updating and maintenance of key project interventions (including: LRI data and training; digitalized systems, DSS, hydrological monitoring, key laboratories, supported by a regular budget to be allocated annually; training provided to extension staff and farmers to use and maintain project interventions; continuation of animal husbandry demonstrations, and regular maintenance of project investments, supported by a regular budget and local level sub-watershed committees. (b) Evidence that the GoK’s new approach for watershed management planning and implementation is expected to be maintained and scaled up within the state and in other states, as reflected by budgetary allocations to relevant activities and complementary projects, and the state government's "exit and sustainability strategy" for KWDP II (ICR, para. 77); and (c) Building on the practical technical and institutional findings and lessons from KWDP II, the Indian Government, including active participation by key counterparts from the Government of Karnataka, the Bank has supported the formulation of a follow-on program, currently under preparation, called: "Rejuvenating Watersheds for Agricultural Resilience Through Innovative Development", or "REWARD" (with a Government program document and draft Project Appraisal Document already prepared). The follow-up program will address the above risks, while helping to scale-up and sustain the interventions and benefits emerging from KWDP II. 8. Assessment of Bank Performance a. Quality-at-Entry Page 18 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) This project design was based on successful experiences and lessons from previous World Bank- financed investments in watershed development in India (for cumulative investments of about US$800 million). Based on the evidence in the ICR (para 73) the quality-at-entry for this project was arguably "satisfactory". Specifically the ICR provided the following evidence. (i) Effective Role/Approach of Bank Design team: According to the ICR the project was designed by a team comprising the required multi-disciplinary expertise and experience; this team was supported by the Region’s quality assurance unit and various experts from the FAO Investment Center, thereby helping to ensure integration of relevant cross-country lessons/good practices. The Bank’s team worked closely with the Government’s counterpart team to apply lessons learned and good practices from the first KWDP, while establishing a new model of watershed management support, which was highly innovative and relevant to the GoI/GoK and World Bank strategies. Also, the Bank’s team carried out multiple missions and meetings with stakeholders both at the national and state levels to make sure that the design was technically sound and widely endorsed by key stakeholders. (ii) Sound Bank Fiduciary Assessments: The World Bank adequately fulfilled its fiduciary role by: carrying out procurement and financial management assessments of the implementing agencies; ensuring the development and disclosure of necessary safeguard documents before appraisal; ensuring that funding was available to generate missing background information and knowledge, and to conduct a socioeconomic assessment of and consultation with affected communities. However, a number of quality-at-entry weaknesses created significant problems for this project. From the start, the three elements of the PDO were imprecisely defined with over ambitious expectations of their impact . As elaborated in the discussion of M&E in the next section of this review, this led to a results framework which was weak because it was not clear what was meant by and how to measure the achievement of “integration and convergence”, “innovative and science-based approaches to watershed management”, what were the requirements for achieving “effective watershed management through strengthened institutions and capacities”, and how to calibrate appropriate indicators. Evidence of the lack of clarity of the original three-part PDO was that PDO indicators and targets were revised during implementation and ultimately abandoned and replaced in a series of restructurings. Finally, as noted by the ICR, there were no mitigation measure proposed in the PAD for the changes that occurred (and might have been foreseen by local Bank staff) in the national IWMP (para. 73). On balance this review rates this project's quality at entry as moderately satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The quality of supervision support was satisfactory, based on the evidence provided in the ICR (para. 74), and other supporting project documentation, especially Implementation Support Reports/ISRs, which included: Page 19 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) (i) two to three implementation support missions annually, including technical and fiduciary reviews with relevant stakeholders, as well as Aide Memoires, ISRs, with justified and evidenced-based project performance ratings; (ii) official communications between central and state government authorities and Bank management, addressing key issues, agreed actions and improved results; and continuity of the Bank’s TTL/technical team, from concept to closing, which also enabled application and preservation of institutional memory; (iii) Bank project team initiatives to share and apply good practices and learning from other countries; (iv) provision of training of key officials in fiduciary practices, and using the Bank supervision missions to ensure sound fiduciary practices, compliance and proper use of Bank proceeds; (v) the Bank’s project team used three project restructurings to adjust the project to national and local contexts, overcome delays, and to use project resources more effectively and efficiently; and (vi) the Bank team ensured that a sound exit strategy was formulated and approved by the GoK before project closure (ICR, para. 76). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The ICR noted that the project’s M&E design provided a basis to guide implementation and served as a tool to track and generate expected results, based on the following evidence (ICR, para. 64); (i) the project was designed to support an effective M&E system, including the assessment of ongoing progress and results monitoring, thematic studies, and impact evaluations by third-party agencies; (ii) the KWDP’s M&E cell was re-established to gather information from stakeholders and executing entities and produce consolidated semi-annual progress reports; and (iii) adequate IDA and government funds were allocated from outset for the execution of the M&E system. Page 20 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) However, the project Results Framework presented several weaknesses: (i) the PDO-level indicators included a measurement of the long-term outcome (increased productivity), going beyond the PDO; (ii) the PDO-level indicator for the increased institutional capacity outcome measured a narrow capacity aspect (M&E capabilities); (iii) several central project-funded activities were not captured through indicators (e.g., LRI and support to SHGs and the GIS laboratory); (iv) linking results to measurable outcomes was complicated due to the unclear PDO and their indicators, also unclear component structure, which the M&E system had to address during implementation. Overall, the project restructurings: improved the ability to measure the most relevant outputs and outcomes; adjusted measurements and targets, based on increased knowledge of costs, timelines, and national and local settings; and added indicators to measure additional results. However, the revised Results Framework did not adjust all the relevant targets. b. M&E Implementation The ICR stated (para 65) that the M&E system/activities were implemented as planned, as follows; (i) the M&E cell collected data on project-supported activities on an ongoing basis and, in 2016, when the project began to show results, third-party agencies began to conduct impact evaluations; (ii) semi-annual progress reports provided detailed information on the progress of all activities, while the evaluations assessed progress based on enhanced Results Framework indicators; and (iii) thematic studies were prepared on a variety of project-financed interventions (e.g., benefits of livestock/feed management activities supported by the project). The monitoring of demonstration sites did include arrangements for the tracking and assessment of productivity changes for various crops, which provided useful data for the follow-up activities of this project. However, as mentioned in Section 4 of this review, the indicator tracking agricultural and horticultural yields was eventually abandoned. c. M&E Utilization The ICR (para 66) concluded that there was effective utilization of the information generated by the M&E system, as follows; (i) the information presented in the progress reports and impact evaluations informed the World Bank team on project progress and aspects requiring and basis for restructuring decisions; (ii) supported decision-making during semiannual PEC meetings; (iii) informed quarterly interim unaudited financial reports (IUFRs), work plans, and budgets; (iv) provided information for the technical project partners conducted monthly monitoring meetings to discuss the physical and financial status of activities, and the PTC met every two to three months to discuss activities’ technical soundness; (v) as stated above, there were arrangements for tracking of the productivity trends in the project-supported demonstration plots, which provided useful inputs for the follow-on “Rejuvenating Watersheds for Agriculture Resilience through Innovative Development (“REWARD” - see ICR, para 77); and (v) project information was used by the GoK to produce quarterly electronic newsletters, showcasing progress and early success stories to government stakeholders. M&E Quality Rating Substantial 10. Other Issues Page 21 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) a. Safeguards The project was classified as environmental category B (partial assessment), and triggered the Environmental Assessment (OP/BP 4.01) and Pest Management (OP/BP 4.09) policies due to potential impacts of horticultural activities. When soil- and conservation-related works in the saturation area were added, the Environmental Assessment category was maintained without triggering additional environmental policies. Social policies were not triggered because land acquisition and impact on tribal communities were not expected. The required safeguard instruments, a project-level Environmental Management Framework (EMF), and a supplementary EMF for horticulture activities were prepared and disclosed ahead of project appraisal. With respect to safeguard policies, the project was in compliance, including: (i) the provisions of the EMF, including for pest management, were implemented, and works in saturation areas were carried out using international good practice construction management measures; (ii) the DoH installed fire safety equipment in the soil, water, and leaf laboratory and post- harvest facilities, and on-site fire safety training was provided to the facilities’ users; (iii) grievance redress was implemented, using the Government’s systems. All applications and complaints (37 in total) were recorded and addressed satisfactorily (ICR, para. 69). b. Fiduciary Compliance (i) Financial Management (ICR, para. 71): The GoK’s financial management system was used in full compliance, as reflected by the following actions: (a) quarterly IUFRs and annual external audits were produced by both the WDD and DoH until FY15, and merged into single reports/audits at the request of the World Bank afterward; (b) audits were submitted on time and audits’ findings were addressed timely by the GoK; and (c) staffing was adequate, with low turnover and no issues were identified, except an erroneous payment to government staff in 2015 using the IDA Credit, which was resolved. (ii) Procurement Aspects (ICR, para. 70): The project complied with procurement procedures. Weaknesses during implementation included: (a) delayed processes and frequent turnover of procurement staff; (b) notable procurement delays included the development of the digitized library, DSS tools, and portal, which was resolved during implementation, with the new counterpart team in place (fall of 2016). No mis-procurement was reported during implementation. c. Unintended impacts (Positive or Negative) The ICR highlighted 6 other positive aspects, contributed by the project. While recognizing some attribution challenges with respect to the precise role and contributions of this project, the nature/scope of these other benefits are summarized below, based on evidence presented in the ICR (paras. 47 to 52). (i) Gender (para. 47): Although gender was not an explicit part of the PDO and components, it provided various tangible gender benefits, as follows (ICR, para. 47): (a) the project provided women and men farmers equal access to extension services’ advice and inputs; (b) enhanced women’s voice in the preparation of watershed management plans through participatory and inclusive local consultations; (c) supported job creation for marginalized women in the communities by providing training on income- Page 22 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) generating activities to women SHGs (i.e., 2,194 women SHG members benefited from training, 22 percent of whom were landless); (d) usefulness of women trainees in empowering them and providing them with new employment opportunities. (ii) Youth (para. 48): Youth were engaged in the project as students (52) using the LRI data to conduct postgraduate-level research, and 586 young graduate students were trained on the LRI fieldwork, conducting soil profiles in laboratories, preparing GIS maps and atlases, and conducting socioeconomic surveys. These skills are expected to help them find jobs in Karnataka’s highly competitive job market. (iii) Institutional Strengthening (para. 49): In addition to the achievements described under Objective 3, several institutional changes took place, due to the project, especially changes reflected in the GoI’s new watershed management guidelines, including: (a) an increase in cost norms to accommodate a wider range of watershed management aspects; (b) a reduction in the planning and implementation period from 7 to 4–5 years; and (c) the use of science-based planning, management, and monitoring based on evidenced-based data. In addition, the GoK is launching a ‘center of excellence’ to build the capacity of FPOs in Karnataka in a way similar to that undertaken by the project, to be incorporated into the state’s FPO Policy (2018). (iv) Mobilizing Private Sector Financing and Partnership Agreements (para. 50): The project did not intend to mobilize financing from the private sector. Nevertheless, it created partnerships between project supported FPOs and horticulture commodity vendors and companies. Four functional partnerships were formed under a private-public integrated horticulture development marketing model, where companies provided government-subsidized training and inputs and post-harvesting technical services and bought outputs from the FPOs. This scheme secured supply and markets, and also increased the post-harvest value of commodities. Several FPOs also registered as vendors with the Agricultural Produce Market Committee and signed agreements with large retail companies, mainly for horticulture products. (v) Poverty Reduction and Shared Prosperity (para. 51): The project’s activities contributed to poverty reduction, especially by vulnerable groups, as reflected in the following evidence presented in the ICR: (a) innovations and training extended to farmers enhanced their ability to select optimal crops, cropping patterns, and soil and water conservation actions and inputs and adjust agricultural activities and practices to potential climate change impacts; (b) an impact study in 9 of the 11 project-supported sub-watersheds showed that the average annual income of families increased by 18 percent, compared to an 11 percent increase in control areas; (c) the average value of durable assets owned by communities increased by 31 percent, compared to a 23 percent increase in the control areas; (d) saturation activities were carried out in areas where small and marginal farmers resided; (e) an impact study specific to this area showed a 33 percent increase in the average household income primarily because of a change in cropping patterns and introduction of new crops and technologies; (f) in 2018, the project shifted its horticulture support focus, from larger farmers to small and marginal farmers in FPOs to enhance equity; (g) at project closure, the DoH reported 9–10 percent average profitability of the new FPOs, which was considered positive compared to mostly unprofitable FPOs across India; and (h) the project generated a total of 1,150 short- term and 1,611 long-term jobs, including for farmers, SHG and FPO members, students, and new extension and demonstration personnel. (vi) Other Unintended Outcomes and Impacts (para. 52): There are 3 notable other impacts arising from the project, which are also generating wider spill-over benefits: (a) the project resulted in a consortium of 14 expert state-based agencies, the first of its kind in Karnataka. The group exchanges among state agencies technical information on key aspects of watershed management and agricultural Page 23 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) development; (b) the project also generated a direct dialogue between the technical partners and farmers because of their interaction during the development of the LRI (which has won several awards); and (c) the solar vending vans that were provided to the FPOs have proven useful in the context of the COVID-19 lockdown as produce is being supplied directly to consumers. d. Other Not Applicable 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment There were lower ratings for efficacy in this review at various Outcome Satisfactory Moderately Satisfactory stages of the project's implementation compared with efficacy ratings in the ICR. The three restructurings reflected quality at entry weaknesses, including the Bank Performance Satisfactory Moderately Satisfactory unclear objectives and the poor framing of PDO indicators and realistic targets. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR presents seven relevant lessons and their aspects which have broader application beyond this project. According to this review, the key lessons and broader aspects are summarized below (ICR, paras. 74 – 84). (a) Science-based approaches can be a central element in the next generation of integrated watershed projects. The project successfully demonstrated a new model for integrated watershed development support, based on stronger science-based evidence. Previous World Bank-supported watershed development projects in India, including the first KWDP, pioneered and scaled up participatory watershed improvement and improved M&E. KWDP-II improved the effectiveness of integrated watershed management by developing, through technical partners, better science-based planning, decision support tools, and the capacities to use them. The GoK now has better systems and capacities to conduct improved watershed development and disseminate this knowledge to other states and countries. This enhanced model of watershed development is of particular relevance in states/countries where development planning is integrated with livelihood interventions, Page 24 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) and where integrated watershed management programs can be supported by the required technical and financial resources. (b) A Land Resource Inventory (LRI) is a valuable tool for managing watersheds effectively. The project was successful in developing a LRI at the micro-watershed (MWS) level and combining it with additional layers of data in a digitized library and using decision support system (DSS) tools. This multi-layered database resulted in a 35–40 percent reduction in watershed planning time and a 70 percent reduction in planning costs per detailed project report and per hectare. It also made watershed management interventions more relevant to the specific conditions in the field and made better use of government resources because of the targeting accuracy at which investments could be made. The lesson is that future watershed management projects could benefit by adopting this LRI tool and approach, as a core element in their design, adjusted as needed for local conditions, capacities, and data availability. (c) The LRI can also integrate/converge government agencies’ interventions around a useful extension tool. The establishment of the portal and the dissemination workshops carried out within the GoK allowed government agencies to access the digitized library and make use of the information to plan their watershed development interventions in an integrated manner. The wide outreach to farmers through RSKs enabled farmers to access plot- level information from their mobile phones and use it to better manage their lands in terms of crop and soil conservation techniques, selection of crops and inputs, and timing of interventions. The lesson is that agricultural development projects, through the extension component, consider the benefits, rationale of and most suitable approach for establishing LRIs as sound measures of integration and extension, while contextualizing/adapting them to local conditions. (d) Establishing a “consortium” of qualified local scientific and technical partners can be an effective arrangement for supporting the Government in the design and implementation of watershed management investments. Instead of hiring consulting firms, this project partnered with a group of highly capable state-based scientific and technical agencies, which took an active role in the design, development, and implementation of key project activities. This project experience demonstrated that this structure had several advantages, pertinent to the context of watershed management. . For example, the consortium approach, contextualized for each case, can lead to better coordination of implementation and enhanced cost savings in project management. (e) In countries where government staff turnover is frequent, the project shows the benefit of allocating additional implementation time to accommodate repeat trainings and learning curves, and to work out a delegation of the procurement clearance authority. Implementation delays occurred due to frequent turnover of senior officials and of trained procurement staff in key departments (WDD and DoH). A key lesson was the value of issuing a Government Order which gave the procurement clearance authority to other key actors (in this instance, the WDD Commissioner, PEC, and PPMU). The project also showed the benefit of providing a one-year extension to complete key project activities and accommodate the GoK staff and management learning curves. (f) Using existing government training centers can reduce capacity building costs. The project showed that it was able to take advantage of existing and new government training centers/facilities for much of the capacity and institution building. This approach illustrates the lesson of saving Page 25 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) project funds compared to using and paying for private sector suppliers, provided the government centers are high quality and can develop suitable programs. (g) Since national programs may change over time (because of changes in the Government and/or setting of new national priorities), this project showed that projects which rely on the implementation of national programs should remain flexible to accommodate unforeseen changes. The 2014 general elections resulted in the (apparently) unforeseen closing of the Integrated Watershed Management Program (IWMP) and the launch of the Prime Minister Krishi Sinchayee Yojana Plan (PMKSY - another government initiative), which threatened the financing and implementation of physical conservation investments. This challenge was eventually overcome by making use of and/or reallocating available project resources to implement essential physical (or other) interventions in sample micro watersheds( MWSs). 13. Assessment Recommended? No 14. Comments on Quality of ICR Overall, the quality of the ICR is "Substantial". The ICR is well written, consistent with the ICR guidelines, analytical (including the inclusion of two versions of the Theory of Change/ToC (at design and following the project's last restructuring) and sound Economic and Financial Analysis/EFA aspects), candid, results-focused, and generally supported by adequate evidence to justify the assessment and proposed ratings. The main evidence integrated and included the results of the analyses of data presented in the: Results Framework (Annex 1); changes in the Results Framework and corresponding indicators, targets and rationale, arising from the 3 restructurings (Annex 7); and the Project’s Impact Evaluation Studies (two independent third party studies, in addition to the project's final and comprehensive M&E report; all three reports are consistent and complementary). While the three project restructurings improved the clarity and corresponding performance indicators (outcome and output levels), consistent with adjustments in the project’s theory of change/ToC, the ICR presented some aspects of the RF (Annex 1) in which various elements were not properly aligned and formulated (e.g., some indicators were presented as outcomes, when actually they were outputs; some outputs/outcomes where presented under a particular outcome/objective, rather than being presented under the appropriate objective/outcome). However, Annex 7 of the ICR provided a useful summary of the changes of the key performance indicators and their rationale for each of the three restructurings, hence, providing further clarity, especially to help assess the project's efficacy, and the split evaluation. There were some inconsistencies in some of the revised indicators in the text and in Annex 7. While it is recognized that these indicators were system generated, they could have been corrected during the project's restructurings to become more appropriate and precise indicators, making clear distinctions between outputs and outcomes. Also, the ICR, Annex 1, under outcomes 2 and 3, included outcome and output indicators for components 4 and 5, which accounted for significant project costs, and contributed to the 3 core objectives (ref. "Strengthening Page 26 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: Karn Wtrshed II (P122486) Horticulture in Rainfed Areas" and "Institutional Development"). It was useful that Annex 7 also presented the original and revised key output and outcome indicators for these two key components 4 and 5. a. Quality of ICR Rating Substantial Page 27 of 27