Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Report Number: ICRR0022333 1. Project Data Project ID Project Name P133327 PAMP II Country Practice Area(Lead) Tajikistan Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-56970,IDA-D0810,IDA-H8230,TF- 28-Feb-2018 56,389,047.10 13997 Bank Approval Date Closing Date (Actual) 29-Nov-2012 28-Feb-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 18,000,000.00 27,900,000.00 Revised Commitment 57,899,547.24 27,900,000.00 Actual 56,462,038.10 27,900,000.00 Prepared by Reviewed by ICR Review Coordinator Group Kishore Laxmikant Vibecke Dixon Ramachandra Jammi IEGSD (Unit 4) Nadkarni 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objectives of the Project are to: (i) provide employment to food-insecure people through the rehabilitation of irrigation and drainage infrastructure; (ii) increase crop production in response to improved irrigation and drainage infrastructure; and (iii) support the development of policies and institutions for water resource management, as a means to improve food availability and food access for low-income people in poor rural areas supported by the project. (Schedule 1 of the Financing Agreement dated February 5, 2013). Page 1 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) For the ICRR, the PDO are parsed into the following: Objective 1: To improve food availability and food access to low-income people in poor rural areas supported by the project. Sub-Objective a: To provide employment to food-insecure people through rehabilitation of irrigation and drainage infrastructure in the areas supported by the project Sub-Objective b: To increase crop production through rehabilitation of irrigation and drainage in the areas supported by the project Sub-Objective c: To support the development of improved policies and institutions for water resource management in Tajikistan Under a Level 1 restructuring in June 2015, a substantive change made in the PDO was the withdrawal of the part pf the original PDO "as a means to improve food availability and food access for lower-income people in poor rural areas supported by the project". The PDO were revised as follows: "The objectives of the project are to: (i) provide access to temporary employment to food-insecure people through rehabilitation of irrigation and drainage infrastructure; (ii) increase yields of selected crops in response to improved irrigation and drainage infrastructure; and (iii) strengthen the capacity of Tajikistan to undertake integrated water resource management. " (Schedule 1 of the Financing Agreement dated June 22, 2015). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 22-Jun-2015 c. Will a split evaluation be undertaken? Yes d. Components (Reference - PAD, paras. 22 to 30). Component 1: Public Works and Rehabilitation of Irrigation and Drainage Infrastructure (estimated cost US$35.82 million; actual cost US$44.35 million) Subcomponent 1a: (Employment Generation for Food Insecure Households through Public Works): This subcomponent was to support the employment of food-insecure people from the project districts to be employed for manual cleaning of secondary and tertiary irrigation canals. Subcomponent 1b: (Mechanized and Other Works): This subcomponent was to support rental and purchase of machinery (excavators, bulldozers, flushing equipment, etc.); purchase and installation of irrigation gates, Page 2 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) irrigation canal and structures repair works; pipeline network repairs; strategic pump station and drainage repairs. Subcomponent 1c: (Flood Channel Emergency Works): This subcomponent was to support emergency restoration of a flood channel that traverses and protects the city of Kulyab, and reduces risks for agricultural land and irrigation systems supported by the project in Vose district. Component 2: Assistance in Water Resource Management, including Technical Assistance for Policy and Institutional Reform (estimated cost US$8.07 million; actual cost US$9.14 million). This component was to provide technical assistance to the Ministry of Amelioration and Water Resource Management (MAWRM) and other relevant institutions for reform of Water Resource Management (WRM). Subcomponent 2a: (National Level Policy, and Legislative and Institutional Formulation): This subcomponent was to support the Government of Tajikistan (GoT) to: (i) further develop the legal basis for Integrated Water Resource Management (IWRM) and Water Sector Reform (WSR); (ii) transform MAWRM into a Ministry for Water Resources (MWR); (iii) prepare a National Integrated Water Resource Management Strategy for Water Resource Management (WRM): and (iv) establish a Water Information Center (WIC). Subcomponent 2b: (River Basin Planning): This subcomponent was to assist the GoT to implement river basin management in the Kofernihon river basin. Subcomponent 2c: (Develop and Strengthen Drainage Institutions): This subcomponent was to assist the GoT to: (i) build independent, financially autonomous irrigation and drainage service providers responsible for management and maintenance of off-farm irrigation and drainage infrastructure and water delivery to water user associations; (ii) support the transformation of existing region and district level institutions into the new institutional framework; and (iii) establish and strengthen the capacity of these institutions through provision of technical assistance, goods, works, and training. Component 3: Project Management (estimated cost US$2.01 million; actual cost US$2.90 million) This component was to provide support to the Project Management Unit (PMU) responsible for project implementation and coordination, financial management, procurement, communication and awareness programs, environmental management and safeguards, and monitoring and evaluation. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The estimated cost at appraisal was US$45.90 million. With the Additional Financing approved in 2015, the project scope was scaled up and the estimated cost was increased to US$57.90 million. The actual cost at completion was US$56.46 million (ICR Data Sheet). Project Financing: At appraisal, the financing approved was an IDA grant of US$18.00 million and a GAFSP (Global Agriculture and Food Security Program) grant of US$27.90 million for a total of US$45.90 million. Under the 2015 Additional Financing, an additional IDA credit of US$6.60 million and an IDA grant of Page 3 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) US$5.40 million, were approved. At completion, actual disbursements were: IDA credit and grants US$28.56 million (95% disbursement) and GAFSP grant US$27.90 million (100% disbursement) for a total of US$56.46 million (ICR Financing table in ICR Data Sheet). Borrower Financing: There was no contribution from the Borrower in regard to project financing. Co-financing: The project was supported by a CAEWDP (Central Asia Energy and Water Development Program) grant of about US$318,000 which was used to provide technical assistance to MAWRM, MEWR and ALRI. This amount was not reflected in the project cost and project financing as stated in the ICR data sheet. Dates: The original closing date was February 28, 2018. Under the June 2015 restructuring, It was extended by 24 months to February 28, 2020 to allow for completion of the scale up activities after the Additional Financing. Restructurings: The project underwent three restructurings. There was one substantive change in the PDO under the first restructuring in 2015. The changes did not affect the original theory of change (TOC).. First Restructuring (June 22, 2015 - amount disbursed US$15.09 million - Restructuring Paper dated June 22, 2015): This was a Level 1 restructuring. The purpose of the restructuring was to provide Additional Financing (AF) in the amount of US$12 million through an IDA grant. The AF was in response to a request from the GoT for assistance in dealing with a downturn in the Tajikistan economy. The AF was to be used to enable the scale-up of the Public Works component to support measures to create employment for low- income and food-insecure people in an additional six districts in Khatlon province and Districts of Republican Subordination (DRS). The closing date was extended by 24 months from February 28, 2018 to February 28, 2020 to allow time for completion of the activities added under the AF. In addition, to reflect changes and experience during implementation, changes were made in the PDO, PDO Indicators, and Intermediate Results Indicators (IRIs) as follows: Reformulation of the PDO: One substantive change in the PDO was the withdrawal of the part of the original PDO "as a means to improve food availability and food access for lower-income people in poor rural areas supported by the project". ) The revised objectives (as stated in Schedule 1 to the Financing Agreement dated June 22, 2015 for the Additional Financing) were to: (i) provide access to temporary employment to food-insecure people through rehabilitation of irrigation and drainage infrastructure; (ii) increase yields of selected crops in response to improved irrigation and drainage infrastructure; and (iii) strengthen the capacity of Tajikistan to introduce Integrated Water Resource Management (IWRM). " (The underlining indicates the changes as compared to the original PDO). Changes in PDO and Intermediate Results Indicators: The rationale for changes in project indicators was to: (i) better align them with the revised PDO; (ii) adjust project outcomes to the new Country Partnership Framework (CPF) that had been approved; and (iii) reflect the impact of the Additional Financing on project targets (Restructuring Paper, para. 35). PDO Indicators: To reflect the scaling-up under the AF, targets were increased for the following indicators: (i) direct project beneficiaries; (ii) number of person-days worked by participants in public works programs; (iii) increase in wheat and vegetable yields on rehabilitated irrigated land; (iv) number of water user associations (WUAs) created and/or strengthened; and (v) number of WUA members. In regard to the National Strategy for IWRM and the Kofarnihon River Basin Plan, changes were made to reflect the organizational changes in the concerned ministries/agencies. These were: the Ministry of Amelioration and Page 4 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Water Resource Management (MARWM) was replaced by a newly formed Ministry of Energy and Water Resources (MEWR). The Agency of Land Reclamation and Irrigation (ALRI) was made responsible for irrigation and service delivery (Restructuring Paper, para. 35). The proposal to create a Water Resources Information Center was replaced by a more decentralized Water Information System (WIS). (Restructuring Paper, para. 35). Intermediate Result Indicators: To reflect the scaling-up under the AF, targets were increased for the following indicators: (i) public works beneficiaries disaggregated by gender and vulnerable groups; (ii) area provided with irrigation and drainage services; (iii) water users provided with irrigation and drainage services; (iv) length of canals cleaned; (v) length of drains cleaned; (vi) key hydraulic installations, controls and structures renovated. To reflect changes in the organizational structure of concerned ministries and agencies, changes were made in regard to the concerned indicators, including deletion of the indicators referring to "mirobs" - the agencies intended to support off-farm maintenance and management in regard to irrigation and drainage. The indicator in regard to establishment of a Water Resources Information Center (WRIC) was replaced by establishment of a Water Information System (WIS). Changes in Components/Costs: There were no changes in components. Costs were increased to reflect the Additional Financing. Second Restructuring (November 6, 2015 - amount disbursed US$18.42 million - Restructuring Paper dated November6, 2015): This was a Level 2 restructuring. The rationale was to respond to the GoT's request for support in financing additional flood emergency works outside of the Kulob district that were necessitated by new flood emergencies in 2015. This was done through a reallocation in the amount of US$4.2 million between components. Associated changes were made in the the Results Framework (RF) and project components/costs including dropping of the reference to Kulob City.. Third Restructuring: (July 23, 2018 - amount disbursed US$49.22 million - Restructuring Paper dated July 23, 2018): This was a Level 2 restructuring. It was made to carry out reallocations between disbursement categories to enable utilization of un-committed amounts under the IDA and GAFSP grants. 3. Relevance of Objectives Rationale Country and Sector Context: Tajikistan, with a population of 7.6 million, remains one of the smallest and poorest countries among the Central Asian economies. Despite periods of economic growth, at the time of project appraisal in 2012, an estimated 46% of the population lived below the poverty line. The agricultural sector accounted for 21% of the GDP and 64% of the employment in the country. Severe limitations on employment opportunities led to migration of labor to Russia. An estimated 77% of the poor segments of the population were dependent on agricultural activities. Tajikistan's economy is highly vulnerable to external shocks. It has a high dependence on imports of oil and cereals which leads to rising prices for the commodities. The economy also has a high exposure to risks of natural disasters including droughts, floods, and landslides, making agriculture a significantly vulnerable sector After the land reforms carried out by the GoT, the agricultural sector has over 60,000 mainly small-scale private farms. At appraisal, an Page 5 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) estimated 50% of the rural population lived in poverty while 30% were considered food-insecure. Out of a total of about 880,000 hectares (ha) of arable land, about 720,000 ha was irrigated, but only about 515,000 ha were in use due to the low performance of the irrigation sector. Issues included (i) severe deterioration of the irrigation and drainage infrastructure due to lack of public funds; (ii) irregular maintenance of secondary and tertiary irrigation canals due to lack of clear responsibility assigned to agencies; (iii) low access to rural finance; and (iv) slow progress with land reform and water resource management. (ICR, paras. 1 and 2). Alignment with National Priorities: The project development objectives were aligned with the national priorities at the time of appraisal, and the original and subsequently revised objectives continue to remain so under the current priorities. At the time of appraisal, the GoT launched an Agrarian Reform Program for 2012 to 2020 which recognized farm productivity as Tajikistan's major domestic food security challenge. This was also recognized under the GoT's National Development Strategy (NDS) for 2006 to 2015 (ICR, para. 6). The original and subsequently revised objectives continue to be consistent with the development priorities under the GoT's NDS for 2016-2030 that is currently in effect, including (i) development of human capital, ensuring food security, and people's access to good quality nutrition, and (ii) strengthening the country's institutional capacity (CPF FY2019-2023, Table 3). Alignment with Country Partnership Framework (CPF): The project development objectives were aligned with the CPF in effect at the time of appraisal, and both the original and subsequently revised objectives continue to remain so under the current CPF for FY2019-2023. The CPF calls for a strong emphasis on a preventative approach to addressing the country's identified fragility risks, including specific interventions proposed to strengthen resilience in regions most at risk from unemployment and poverty (CPF, para. 45). The PDO are consistent with Pillar 1 (investing in human capital and building social resilience) and Pillar 2 (improving effectiveness of public institutions for sustainability (CPF, Executive Summary, para. vi). The CPF includes Objective 3: Improving Resilience of Residents in Local Communities. This is to be implemented in GBAO and Khatlon provinces through operations targeting selected socially and economically marginalized groups (CPF para. 61). . Prior Bank Experience: The project is a successor project to the Public Employment for Sustainable Agriculture and Water Resources Management Project (PAMP I) which was implemented in 2010-2011 to address emergency food security. The current project (PAMP II) builds on the results and experience under the earlier project. The project focuses, in particular, on the more vulnerable districts, and places greater emphasis on improvement of irrigation and drainage infrastructure as well as on policy and institutional reform. The Original Objectives: The overarching original objective of improving food availability and food access to food-insecure people in the poor rural areas supported by the project was highly relevant in the context of the national priorities and those supported under the CPF currently in effect. The PDO was pitched at the right level and it adequately reflected a solution to a development problem aiming at improved food availability and improved access to food. The means identified to achieve this objective (increase in temporary employment, increase in crop production, and strengthening of Tajikistan's capacity for water resources reform) were also appropriate to the achievement of this objective. The Revised Objectives: In the restructuring of the project, the PDO was revised to: "(i) provide access to temporary employment to food-insecure people through rehabilitation of irrigation and drainage infrastructure; (ii) increase yields of selected crops in response to improved irrigation and drainage infrastructure; and (iii) strengthen the capacity of Tajikistan to undertake integrated water resources Page 6 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) management. " (Schedule 1 to the Financing Agreement dated June 22, 2015). This left the project without a PDO that would address a development problem, as the remaining sub-objectives, which in the original objective had correctly been identified as a means to achieve the development objective, are all at a lower level in the logical chain and none of them reflect a potential solution to a development problem. However, while there still was clear alignment between the project's development objectives and the country- and World Bank- strategies, the relevance of the revised objectives is pitched at a level that does not adequately reflect a potential solution to a development problem. A significant shortcoming here is the lack of clarity in the PDO formulation around what development outcomes would be achieved through the achievement of the objectives. "Provision of temporary employment" is at an input level in the logical chain in that it may be regarded as a project activity. Focusing on "provision of temporary employment" alone is not outcome focused and does not help in understanding what development outputs were expected as a consequence of the project. Temporary increased access to food (due to temporary increased income) would be a potential and logical outcome of this, but it is not reflected in the PDO formulation. The objective "increase in yields of selected crops" is an intermediate level indicator which could potentially contribute to improving food availability and food security for low-income people in the project area, but, again, this was not articulated in the PDO. The objective "strengthen the capacity of Tajikistan to introduce integrated water resource management "is an intermediate level indicator in the logical chain and not outcome focused in regard to delivering the outcome of increased food availability and food security or other relevant outcome (increased resilience to natural disaster risks). Increased income and improved health are longer term or higher level targets, but identifying them and tracking them is an important aspect of a successful development operation, which the revised PDO do not do. Based on the above, the relevance of the original objective was in line with the intended development impact and thus could be rated High while the revised objective is limited in scope and unlikely to sufficiently respond to the development challenge. Thus, it is rated Modest. In line with the guidelines agreed between IEG and OPCS, that the PDO formulation which is used at the end of the project is the one to be rated, the relevance of objectives is rated Modest. Rating Relevance TBL Rating Modest 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve food availability and food access to low-income people in poor rural areas supported by the project. Rationale Theory of Change (TOC); A TOC was not articulated in the PAD. The ICR (para. 7) provides a TOC which indicates that the combination of measures to be taken under the project would improve food security and Page 7 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) enhance rural development and livelihoods. In the short term, investments in public works programs (inputs), including manual cleaning of drains, would provide employment to food-insecure people (outputs) which would increase their income and improve access to purchased food (outcomes). In the medium-term, investments in rehabilitation irrigation and drainage infrastructure and flood emergency works (inputs) would increase and improve the areas equipped with irrigation and drainage services, and thereby enable increased crop production (outputs). This would increase food availability to to the people in the project-supported areas (outcomes). In the longer-term, the provision of technical assistance and capacity building to the GoT and MAWRM for water sector policy reform, including development of an IWRM strategy, and to water user associations (WUAs) and other institutions (inputs), would result in improved policies and strengthened institutions for undertaking sustainable water resources management (outputs). This would improve sustainability of irrigated agricultural land and help increasing agricultural production, thereby improving food security and enhancing rural development and livelihoods (outcomes). Although food availability and food access were the principal outcomes to be achieved, no relevant PDO level indicators were included to enable an assessment of achievement of these intended outcomes. The efficacy of Objective 1 is assessed on the basis of the following sub-objectives Sub-Objective a: To provide employment to food-insecure people through rehabilitation of irrigation and drainage infrastructure in areas supported by the project. The TOC was that the project would finance investments (inputs) for carrying out of public works programs, including manual cleaning of on-farm irrigation canals, that would result in providing employment to food-insecure people (outputs). This would provide additional income to the participating households, thereby increasing their access to purchased food (outcome). However, no outcome level indicators were included under the project for tracking increase in household incomes and improved access to food. The outputs and outcomes listed below in this section and in following sections are based on the ICR Results Framework and paras. 30 to 33). Outputs: 1.Number of project beneficiaries from public works (baseline 0; original target 22,000; achieved at restructuring 4,372; actual achievement 30,005; achievement level 136%) Outcomes: 1. Number of person-days worked by participants in public works programs (baseline 0; original target 880,000; achieved at restructuring 148,850; actual achievement 1,062,927; achievement level 121%) of which:  female participants (baseline 0; original target 132,000; achievement at restructuring 29,770; actual achievement 253,561; achievement level 192%)  severely food-insecure participants (baseline 0; original target 176,000; achievement at restructuring 9,904; actual achievement 211,517; achievement level 120%) Assessment: The original output targets in regard to the public works activities and the number of beneficiaries were increased under the first restructuring (AF) undertaken in June 2015 to reflect the increase in project scope by adding six more districts. Consequently, the original targets were overachieved at project Page 8 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) closing. However, it should be noted that the outcome indicator (number of person days worked) does not indicate the degree of achievement of the intended outcomes (increased household income, improved access to food). The project's M&E system did not include relevant indicators for measuring these intended outcomes. Some relevant data have been provided in the Final Report on Impact Assessment (dated February 2020) prepared by the Project Management Unit (PMU) at the end of the project. These indicate that, based on beneficiary surveys for a sample of the beneficiary population, on average, the households reported an additional income equivalent to about US$197 or about 1/8th of the average household income during the year. The Report indicates that, after increasing between 2013 and 2016, the ability to purchase food items seemed to decline since 2016, and this could be correlated to rising food prices and non-food expenditures, and decline in remittances from abroad. The Report also indicates that, based on the World Food Program's (WFP) Food Consumption Score (FCS), the FCS improved between 2016 and 2019, but remained below the baseline level of 2013. The results were similar across both project and non-project districts. Overall, there is a lack of clarity as to the project's impacts in regard to food availability and food access. Sub-Objective b: To increase crop production through rehabilitation of irrigation and drainage infrastructure in areas supported by the project. The theory of change (TOC) was that project outputs, including the rehabilitation of irrigation and drainage infrastructure, manual and mechanical cleaning of irrigation canals and drains, together with carrying out of works to address flood emergencies, would improve the level of irrigation and drainage services in the project supported areas. These activities would be supported by strengthening of the agencies and institutions involved in the process of maintenance and management of the infrastructure, including the water user associations. Together, this would enable an increase in agricultural productivity resulting in increased crop production (intermediate results indicator - but treated in the project as an outcome). This would contribute to increased food availability (outcome) for which the project did not have relevant PDO level indicators. Outputs: 1. Length of canals cleaned (baseline 0; original target 5,200 km; achieved at restructuring 1,280 km; actual achievement 8,571 km; achievement level 165%) 2. Length of drains cleaned (baseline 0; original target 1,300 km; achieved at restructuring 242 km; actual achievement 1,112 km; achievement level 86%) 3. Key hydraulic structures, installations, and rehabilitation equipment provided (baseline 0; original target 5,800 items; achieved at restructuring 988 items; actual achievement 5,441 items; achievement level 94%) 4. Area provided with improved drainage services (baseline 0; original target 190,000 ha; achieved at restructuring 48,457 ha; actual achievement 251,528 ha; achievement level 132% Outcomes: 1. Increase in crop production - (original target 10%; actual achievement - greater than 10%; achievement - greater than 100%). Page 9 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) 2. Number of direct project beneficiaries (baseline 0; original target 772,000; achieved at restructuring 308,607; actual achievement 1,393,276; achievement level 180%) Assessment: The outcome indicator used was the increase in cereals/food/fodder crops yields on rehabilitated irrigated land. This indicator is more appropriately an intermediate results indicator. The targeted increase in yields was 10%. The ICR (para.35) reports that, between 2010-2012 (baseline) and 2017 to 2018 (final), based on a sample of 10 districts supported under the project, yields for all cereals increased by about 25.2% (including for wheat by 22.3%); for all vegetables by 15.1%; and fodder crops by 22.3%. Yields for cotton declined by 12.6%. According to the Final Impact Assessment Report, 57% of households in the project area (and 54% in non-project areas) reported that the amount of irrigation water increased. In 9 out of the 17 project districts, households reported that access to, and availability, of irrigation water was a principal reason for the increase crop production. The ICR reports that beneficiary surveys have indicated that not all the increase in yields can clearly be attributed to the project since farmers stated that availability of other inputs like fertilizers and seeds also played a significant part. However, based on this and other information, the ICR assesses that, overall, the increase in yields for crop production was higher than 10% and exceeded the target of 10%. The original outcome target in regard to the number of direct project beneficiaries was increased at the June 2015 restructuring (AF). the original target was overachieved at project closing. However, there were no PDO level indicators to assess achievement of this outcome. Sub-Objective c: To support policies for improvement of water resource management in Tajikistan. The theory of change (TOC) was that provision of technical assistance (input) to the GoT and ministries/agencies concerned with policy development and implementation, including assistance for formulation of a legal basis for water sector reform and Integrated water resource management (IWRM), and assistance for related training and capacity building, would result in improved legislation and regulatory framework, supporting water sector related policies, and strengthened institutions and agencies involved in water resource management (outputs). The main institutions involved were MEWR, ALRI, the River Basin Organizations (RBO), River Basin Commissions (RBC), and water user associations (WUAs). The combination of measures would contribute to improved water resource management, and the expansion and sustainability of irrigation infrastructure and services, leading thereby to increased crop production and improved food availability and security (outcome). Outputs: 1. Amendments to water legislation were drafted including two key items: Water Code of the Republic of Tajikistan and Law on Water User Associations. Both drafts were finalized in 2019 and adopted by Parliament in 2020, thereby exceeding the targets (ICR para. 42) 2. A National Water Information System (NWIS) was established. Responsibilities for data collection and information-sharing between different agencies were set under a Water Information System (WIS) Concept Paper. (ICR para. 44). 3. Training in IWRM was provided to concerned staff in MEWR/ALRI/WUAs (original target 50; revised target 50; actual achievement 316; achievement level 632%). Outcomes: Page 10 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) 1. A National IWRM strategy for the period 2017 to 2025 was prepared and agreed with MEWR (original target - yes; revised target - yes; actual achievement - yes; achievement level 100%). The IWRM strategy is expected to contribute to the development of a National Water Strategy up to 2030 by the GoT. (ICR para. 43) 2. A Kofernihon River Basin Management Plan (RBMP) was prepared and agreed with MEWR (original target- yes; revised target - yes; actual achievement - yes; achievement level 100%). This RBMP is the first of its type in the country and is expected to serve as a model for similar other RBMPs to be prepared in future. (ICR para. 43). 3. Operational water user associations (WUAs) were created and/or strengthened (original target 95; revised target - 135; actual achievement 130; achievement level 96%) of which:  WUAs created (original target 20; revised target 30; actual achievement 20; achievement level 67%)  WUAs strengthened (original target 75; revised target 105; actual achievement 110; achievement level 105%) Assessment: The output targets in regard to strengthening of Tajikistan's capacity for water management were achieved. In the context of the project's PDO, the outcome targets set were more appropriately output or intermediate results indicators and did not capture the intended outcome of improved food availability and food access. After early delays, the targeted amendments to water legislation were completed and adopted. Training was provided to strengthen capacities at the key agencies (MEWR and ALRI). The capacities of the WUAs were strengthened through required changes in their legal status and definition of responsibilities. However, despite a reported improvement in fee-setting and collection performance, the ICR reports that, on average, the WUAs are able to cover only about 50% to 60% of their operations & maintenance costs (ICR paras. 47 and 86). Overall assessment of efficacy of Objective 1: As discussed above, targets in regard to outputs and intermediate results indicators were achieved or substantially achieved under each of the three sub-objectives under Objective 1. However, a significant shortcoming in design of the PDO indicators was that the outcome indicators used were more appropriately output or intermediate results indicators. There was no indicator that specifically addressed the issue of improved food availability and food access to food-insecure people in poor areas supported by the project. Two core indicators related to the GAFSP grant had been included at the start of the project: (i) increase in household incomes for project beneficiaries; and (ii) reduction in the proportion of the target population below the minimum level of dietary energy. These indicators were dropped due to difficulties in measurement (ICR paras. 51 and 52). The lack of relevant indicators precluded the assessment of achievement of the overarching objective under this PDO. The ICR ((para.30, Table 2) acknowledges that achievement of this objective was Modest pre-restructuring. The objective was dropped in the revised PDO under the June 2015 restructuring. The efficacy for Objective 1 is rated Modest. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective Page 11 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) There was no revised Objective 1. The PDO changes led to three separate objectives building from the previous sub-objectives. Revised Rationale N/A Revised Rating Not Rated/Not Applicable OBJECTIVE 2 Objective There was no Objective 2 under the original PDO which had a single objective with three sub-objectives. Rationale Not applicable since there was no Objective 2 under the original PDO. Rating Not Rated/Not Applicable OBJECTIVE 2 REVISION 1 Revised Objective The original Objective 1 was dropped under the June 2015 restructuring for Additional Financing and replaced by three separate objectives. The revised Objective 2 is "To provide access to temporary employment of food-insecure people through rehabilitation of irrigation and drainage infrastructure in poor rural areas supported by the project". Revised Rationale Under the June 2015 restructuring, a change made to the original objective was to replace the term "to provide employment to food-insecure people" by the term "to provide access to temporary employment for food-insecure people". The theory of change (TOC) was that the project would finance investments (inputs) for carrying out of public works programs, including manual cleaning of on-farm irrigation canals, that would result in providing temporary employment to food-insecure people (outputs). This would provide additional income to the participating households, thereby increasing their access to purchased food Page 12 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) (outcome). However, no outcome level indicators were included under the project for tracking increase in household incomes and improved access to food. Outputs: 1.Number of project beneficiaries from public works (baseline 0; original target 22,000; revised target 32,000; actual achievement 30,005; achievement level 94%) Outcomes: 1. Number of person-days worked by participants in public works programs (baseline 0; original target 880,000; revised target 1,255,000; actual achievement 1,062,927; achievement level 85%) of which:  female participants (baseline 0; original target 132,000; revised target 188,250; actual achievement 253,561; achievement level 135%)  severely food-insecure participants (baseline 0; original target 176,000; revised target 251,000; actual achievement 211,517; achievement level 84%) Assessment: The output indicator in regard to number of beneficiaries as well as the outcome indicator regarding number of person-days - both overall and also in relation to the disaggregated categories - gender (female) and vulnerable groups (severely food-insecure people) - were achieved or substantially achieved. As discussed in Section 3 on relevance of objectives, the outcome indicators used in the project were more appropriately intermediate results indicators and did not capture the project's impacts in regard to achievement of additional income, and the higher level outcomes of improved food availability and food access, for low-income households in poor rural areas supported by the project. However, given the achievement or substantial achievement of the targeted outputs and outcomes, the efficacy for Revised Objective 1 is rated Substantial. Revised Rating Substantial OBJECTIVE 3 Objective There was no Objective 3 under the original PDO which had a single PDO with three sub-objectives. Rationale Not applicable as there was no Objective 3 under the original PDO. Rating Not Rated/Not Applicable OBJECTIVE 3 REVISION 1 Page 13 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Revised Objective The revised Objective 3 was added under the June 2015 restructuring (AF) as follows: "To increase yields of selected crops in response to improved irrigation and drainage infrastructure in areas supported by the project." Revised Rationale Under the June 2015 restructuring, the original formulation of the objective was changed from "to increase crop production" to " to increase yields of selected crops". The change was based on experience during implementation that farmers realized higher value from some crops than from others. The theory of change (TOC) was that project outputs, including the rehabilitation of irrigation and drainage infrastructure, manual and mechanical cleaning of irrigation canals and drains, together with carrying out of works to address flood emergencies, would improve the level of irrigation and drainage services in the project supported areas. These activities would be supported by strengthening of the agencies and institutions involved in the process of maintenance and management of the infrastructure, including the water user associations. Together, this would enable an increase in agricultural productivity resulting in increased crop production (intermediate results indicator - but treated in the project as an outcome). This would contribute to increased food availability (outcome) for which the project did not have relevant PDO level indicators. Outputs: 1. Length of canals cleaned (baseline 0; original target 5,200 km; revised target 8,820 km; actual achievement 8,571 km; achievement level 97%) 2. Length of drains cleaned (baseline 0; original target 1,300 km; revised target 1,300 km; actual achievement 1,112 km; achievement level 86%) 3. Key hydraulic structures, installations, and rehabilitation equipment provided (baseline 0; original target 5,800 items; revised target 5,800 items; actual achievement 5,441 items; achievement level 94%) 4. Area provided with improved drainage services (baseline 0; original target 190,000 ha; revised target 260,000 ha; actual achievement 251,528 ha; achievement level 97%) 5. Length of flood channels rehabilitated along Tebalai river (baseline 0; original target - none; revised target - 5 km; actual achievement 5 km; achievement level 100%) Outcomes: 1. Increase in crop production - (original target 10%; revised target - greater than 10%; actual achievement - greater than 10%; achievement - 100%). 2. Number of direct project beneficiaries (baseline 0; original target 772,000; revised target 1,385,000; actual achievement 1,393,276; achievement level 101%) Assessment: The outcome indicator used was the increase in cereals/food/fodder crops yields on rehabilitated irrigated land. This indicator is more appropriately an intermediate results indicator. The targeted increase in yields was 10%. The ICR (para.35) reports that, between 2010-2012 (baseline) and 2017 to 2018 (final), based on a sample of 10 districts supported under the project, yields for all cereals increased by about 25.2% (including for wheat by 22.3%); for all vegetables by 15.1%; and fodder crops by 22.3%. Yields for Page 14 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) cotton declined by 12.6%. According to the Impact Assessment Report, 57% of households in the project area (and 54% in non-project areas) reported that the amount of irrigation water increased. In 9 out of the 17 project districts, households reported that access to, and availability, of irrigation water was a principal reason for the increase crop production. The Impact Assessment Report indicates that, overall, crop production increases in project areas were higher than in non-project areas. The ICR reports that beneficiary surveys have indicated that not all the increase in yields can clearly be attributed to the project since farmers stated that availability of other inputs like fertilizers and seeds also played a significant part. However, based on this and other information, the ICR assesses that, overall, the increase in yields for crop production was higher than 10% and exceeded the target of 10%. The original outcome target in regard to the number of direct project beneficiaries was increased at the June 2015 restructuring (AF). The revised target was substantially achieved at project closing. Given the lack of clarity in regard to the degree of attribution of the increases in yields to the project interventions, the efficacy of Revised Objective 2 is rated Substantial with moderate shortcomings. Revised Rating Substantial OBJECTIVE 4 Objective There was no Objective 4 under the original PDO which had a single objective with three sub-objectives. Rationale Not applicable as there was no Objective 4 under the original PDO. Rating Not Rated/Not Applicable OBJECTIVE 4 REVISION 1 Revised Objective Objective 4 was added at the June 2015 restructuring.(AF) as follows: "To strengthen the capacity of Tajikistan to introduce Integrated Water Resource Management". Revised Rationale Under the June 2015 restructuring, the formulation of the objective was changed from the existing "to support the development of policies and institutions for water resource management" to "to strengthen the capacity of Tajikistan to introduce integrated water resource management'. The change was in response to the desire of the GoT to emphasize IWRM as a main feature in water resources management. The theory of change (TOC) was that provision of technical assistance (input) to the GoT and ministries/agencies concerned with policy Page 15 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) development and implementation, including assistance for formulation of a legal basis for water sector reform and Integrated water resource management (IWRM), and assistance for related training and capacity building, would result in improved legislation and regulatory framework, supporting water sector related policies, and strengthened institutions and agencies involved in water resource management (outputs). The main institutions involved were MEWR, ALRI, the River Basin Organizations (RBO), River Basin Commissions (RBC), and water user associations (WUAs). The combination of measures would contribute to improved water resource management, and the expansion and sustainability of irrigation infrastructure and services, leading thereby to increased crop production and improved food availability and security (outcome). Outputs: 1. Amendments to water legislation were drafted including two key items: Water Code of the Republic of Tajikistan and Law on Water User Associations. Both drafts were finalized in 2019 and adopted by Parliament in 2020, thereby exceeding the targets (ICR para. 42) 2. A National Water Information System (NWIS) was established. It assigns responsibilities for data collection and information-sharing between different agencies (ICR para. 44). 3. Training in IWRM was provided to concerned staff in MEWR/ALRI/WUAs (original target 50; revised target 50; actual achievement 316; achievement level 632%). Outcomes: 1. A National IWRM strategy for the period 2017 to 2025 was prepared and agreed with MEWR (original target - yes; revised target - yes; actual achievement - yes; achievement level 100%). The IWRM strategy is expected to contribute to the development of a National Water Strategy up to 2030 by the GoT. (ICR para. 43) 2. A Kofernihon River Basin Management Plan (RBMP) was prepared and agreed with MEWR (original target- yes; revised target - yes; actual achievement - yes; achievement level 100%). This RBMP is the first of its type in the country and is expected to serve as a model for similar other RBMPs to be prepared in future. (ICR para. 43). 3. Operational water user associations (WUAs) were created and/or strengthened (original target 95; revised target - 135; actual achievement 130; achievement level 96%) of which:  WUAs created (original target 20; revised target 30; actual achievement 20; achievement level 67%)  WUAs strengthened (original target 75; revised target 105; actual achievement 110; achievement level 105%) 4. Registered members of WUAs (original target - not included; revised target 22,000; actual achievement 45,163; achievement level 205%) of which: female (original target - not specified; revised target 10%; actual achievement 15%; achievement level 150%) Assessment: The output targets in regard to strengthening of Tajikistan's capacity for water management were achieved. In the context of the project's PDO, the outcome targets set were more appropriately output or intermediate results indicators and did not capture the intended outcome of improved food availability and food access. After early delays, the targeted amendments to water legislation were completed and adopted. Page 16 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Training was provided to strengthen capacities at the key agencies (MEWR and ALRI). A pilot project was carried out in the Lower Kofernihon River Basin involving collaborative action between ALRI, River Basin Organizations (RBOs) and WUAs. The Impact Assessment report indicates that the results were positive and confirmed by beneficiary surveys in regard to improved quality of irrigation services. This could provide a model for similar projects in other river basins. The project supported the introduction of Tajikistan's first cashless user fee billing and collection system for provision of irrigation water and services. This is expected to be rolled out to other agricultural districts. The capacities of the WUAs were strengthened through required changes in their legal status and definition of responsibilities. However, despite a reported improvement in fee-setting and collection performance, the ICR reports that, on average, the WUAs are able to cover only about 50% to 60% of their operations & maintenance costs (ICR paras. 47 and 86). Based on the substantial achievement of the targeted outputs and outcomes, the efficacy of Revised Objective 3 is rated Substantial. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Under the original Objective 1, For Sub-Objective a the output targets in regard to provision of temporary employment and infrastructure rehabilitation were achieved. The targeted outcome in regard to the number of person-days worked was also achieved, both overall and also in regard to the disaggregated categories of gender (female) and vulnerable groups (severely food-insecure). For Sub-Objective b, the ICR (para. 35) reports that the targeted increase of 10% in crop production yields was exceeded, but also reports that it is not clear to what extent this can be attributed to the project. The ICR (para. 72) further reports that there were significant shortcomings in measurement of some of the related indicators. For Sub-Objective c, the targeted outputs and outcomes were substantially achieved. However, for lack of relevant indicators, the achievement of the overarching objective of improved food availability and food access to food-insecure people in poor rural areas supported by the project is not clear. Based on the foregoing, the overall efficacy for Objective 1 is rated Modest. Overall Efficacy Rating Primary Reason Modest Insufficient evidence OBJR1_TBL OVERALL EFFICACY REVISION 1 Page 17 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Overall Efficacy Revision 1 Rationale For Revised Objective 1, efficacy is rated Substantial. The output targets in regard to provision of temporary employment and infrastructure rehabilitation were substantially achieved. The targeted outcome indicator - number of person-days worked was substantially achieved, both overall and also in relation to the disaggregated categories - gender (female) and vulnerable groups (severely food-insecure people). For Revised Objective 2, efficacy is rated Substantial with moderate shortcomings. The ICR (para. 35) reports that the targeted increase of 10% in crop production yields was exceeded, but also reports that it is not clear to what extent this can be attributed to the project. (ICR para. 72). The ICR) further reports that there were significant shortcomings in measurement of some of the related indicators (ICR para. 72). For Revised Objective 3, efficacy is rated Substantial. The targeted outputs and outcomes were largely achieved. On balance, for the revised PDO, the overall efficacy rating is Substantial with moderate shortcomings. Overall Efficacy Revision 1 Rating Substantial 5. Efficiency Administrative and Operational Efficiency Project Cost: The project was completed within the allocations made at appraisal and Additional Financing. Output and outcome targets as set under the project were achieved or substantially achieved (although there were deficiencies in regard to measurement of achievement of the main intended outcome of improved food availability and food access). The estimated project cost, including Additional Financing, was US$57.90 million; the actual cost at completion was US$56.46 million. Project Duration: The originally planned project implementation period was extended by 24 months to allow for completion of the increased project scope and new activities included under the Additional Financing in June 2015. Economic and Financial Efficiency Cost-Effectiveness: The planned project outputs were overachieved or substantially achieved. Thus, cost effectiveness, in terms of output delivered per US$ of project cost met or exceeded the expectations at appraisal and Additional Financing.. No cost-effectiveness analysis was carried out at appraisal. Economic Analysis: At appraisal, the key indicators of economic efficiency adopted were Economic Net Present Value (ENPV), Economic Internal Rate of Return (EIRR)., and the Benefit-Cost Ratio (B-C R). The ICR (paras. 54 and 55 and Annex 7) provides a post-completion re-estimation of the project's economic benefits, summarized in the following table: Page 18 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Comparison of Appraisal vs Post-Completion Economic Indicators Appraisa Additional Post- Indicator l Financing Completion Area rehabilitated (ha) 190,000 236,600 251,433 Net increase in crop yields 10% 10% >10% (%) ENPV (US$ million) 30.2 27.2 21.9 EIRR (%) 26.1% 27.0% 41.5% Benefit-Cost Ratio 5.82 7.14 2.83 Investment benefits per ha 158 110 87 Investment life (years) 12 12 12 Discount rate (%) 12% 12% 12% Assessment: The reported post-completion indicators, although lower in some cases than the appraisal estimates (eg. EIRR and B-C R), are still substantial in relation to the discount rates of 6% to 8% indicated under the Bank's Guidelines for Economic Analysis of Investment Projects. However, the estimates are based on underlying estimates of project results that, in some cases (e.g. crop production and yields) that cannot clearly be attributed to the project. The weaknesses in the project's M&E system (see Section 9) cast some doubt in regard to the extent of the efficiency achievements. Therefore, the project's efficiency is rated Substantial with moderate shortcomings. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome A split evaluation was carried out based on the project results pre-restructuring and post-restructuring following the Additional Financing in June 2015. The results of the split rating are provided in the table below. Page 19 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Pre-Restructuring Post-Restructuring Relevance of PDO N/A Modest Substantial with moderate Efficacy Modest shortcomings Objective 1 Modest Not rated Revised Objective Not rated Substantial 2 Revised Objective Substantial with moderate Not rated 3 shortcomings Revised Objective Not rated Substantial 4 Substantial with Substantial with moderate Efficiency moderate shortcomings shortcomings Moderately Outcome rating Moderately Satisfactory Unsatisfactory Numerical value of 3 4 outcome rating Disbursement US$15.09 million US$41.37 million Share of 27% 73% disbursement Weighted value of 0.81 2.92 outcome rating Final outcome Moderately rating Satisfactory 0.81+2.92=3.73 rounded to 4.00 a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Policy Risks (Moderate): The project provided temporary employment to vulnerable groups, including food- insecure people, in some of the most affected agricultural districts in the country. There is a risk that the benefits may not be sustained unless the GoT continues to address the needs of the agricultural sector, including providing social safety nets. (ICR, para. 88). Page 20 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Administrative Risks (Moderate): Implementation of the IWRM strategy would call for coordination and collaboration among central, regional and local levels of government. There is a risk that adequate resources may not be available in at timely and sufficient manner to ensure effective implementation of the strategy. Institutional Risks (Moderate): Institutions such as the River Basin Organizations (RBOs) and WUAs would have an important role in ensuring success of the IWRM strategy. There is a risk that may not be provided the necessary resources to carry out their work effectively. (ICR para. 86). Legislative risks (Moderate): Implementation of the legislation already put in place will require timely and adequate completion of the necessary by-laws and supporting resolutions at the regional and local levels. There is a risk that delays in putting in place the supplementary regulations would limit the effectiveness of the changes. (ICR para. 85) Financial Risks (Moderate): There is a risk that funding constraints could adversely affect the functioning of various agencies (ALRI, RBOs, RCOs, WUAs) essential for sustaining the benefits initiated under the project. Sustaining of the improvements made through rehabilitation of irrigation and drainage infrastructure would require adequate financial resources for covering operations and maintenance costs. (ICR paras. 86 and 87). 8. Assessment of Bank Performance a. Quality-at-Entry The project was well aligned with GoT and World Bank priorities and strategies for Tajikistan's agricultural sector. The project design benefited from the experience under the the preceding project (Public Employment for Sustainable Agriculture and Water Resource Management). In particular, greater emphasis was laid on rehabilitation of irrigation and drainage infrastructure to increase prospects of sustainability of the benefits. The capacity of the Project Management Unit (PMU) to undertake project implementation activities was assessed and assistance provided to strengthen its capacities in regard to fiduciary aspects - financial management and procurement. One area where there were significant shortcomings was in regard to the design of the Monitoring & Evaluation (M&E) system (see Section 9 below). In particular, while the PDO were framed adequately, many of the outcome indicators included under the Results Framework were more appropriately output or intermediate results indicators. There were no relevant indicators provided to measure the achievement of the principal intended outcome, namely, improved food availability and food access to food-insecure people in poor rural areas supported by the project. Quality-at-Entry Rating Moderately Satisfactory Page 21 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) b. Quality of supervision Project supervision was carried out regularly over the implementation period. A total of 11 implementation support missions were carried out. The supervision teams were adequately staffed, including fiduciary, environmental and safeguards specialists. The project benefited from the task team leader (TTL) being continuously based in Tajikistan. There was only one change in the TTL which provided continuity. International experts were included where judged to be necessary. The supervision team was proactive in seeking timely restructuring of the project and in securing grant financing under the CAEWDP Trust Fund to support capacity building in MEWR and ALRI - the main GoT agencies involved in the project implementation. A notable shortcoming was the formulation of the PDO and related indicators at the first restructuring in June 2015 (deficiencies have been discussed earlier in Section 3 on relevance of objectives). The supervision team was not able to address satisfactorily the significant shortcomings in the project M&E system which persisted over the course of project implementation. These persistent deficiencies in the M&E system led to a downgrading of the project's rating to Moderately Satisfactory in the last ISR (2019) before project closing. The ICR (para. 73) acknowledges that the evaluation of outcomes in the ICR is based on data from the national statistical agency rather than the project's M&E system (ICR para. 73). Further, the ICR reports (para. 80) that procurement performance was affected by low capacity i the PMU which led to substandard bidding documents and evaluation procedures. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The design of the project's M&E system built upon the system that was put in place under the predecessor project (PAMP I). The original PDO (prior to the June 2015 restructuring) were appropriately defined in terms of the intended outcome - improved food availability and food access for food-insecure people in poor rural areas supported by the project. A significant shortcoming was the specification of PDO indicators that could capture the achievement of the intended outcome. Since the outcome impacted food-insecure households/individuals, well-designed and regularly executed beneficiary surveys could have helped in monitoring progress in achievement of the outcome. The output and intermediate results indicators in the Results Framework were relevant and measurable. One exception was in regard to increase in crop production and yields in areas supported by the project. The M&E system did not address the issue of how, and to what extent, the achievements could be attributed to the project interventions. b. M&E Implementation M&E implementation was affected by significant weaknesses. Early during implementation, these included low capacity of M&E staff and significant delay in the operationalization of the Project Page 22 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) Management and Monitoring Information System (PMMIS). This adversely affected progress monitoring and reporting. The ICR (para. 74) reports that M&E data on agricultural production was included for only two out of the seven years of project implementation. Other weaknesses which remained were: (i) progress reporting was not always timely and consistent; (ii) inadequate information in regard to achievement of outcome indicators - importantly, outcome evaluation in the ICR in regard to crop production was based largely on data from the national statistical agency; and (iii) inadequate design and execution of the end-line survey in 2019 the results of which were found to be inconclusive (ICR para. 73). Well-designed and regularly executed beneficiary surveys could have added to better monitoring and evaluation of project progress and impacts. The ICR (para. 73) indicates that one of the reasons that the project closed with a Moderately Satisfactory rating was the deficiencies in the project's M&E system. c. M&E Utilization The project team was constrained to utilize the project's M&E system for monitoring project implementation and progress. The shortcomings indicated above led to the team having to draw upon other sources e.g. the national statistical agency, to supplement the deficiencies in information. Delays in regular updating of the system and incompleteness of data affected the quality of monitoring and evaluation during implementation and the adequacy of information for preparation of the ICR. A useful supplement to the project's M&E system was the Impact Assessment (IA) referred to earlier. The IA provided some relevant information on the project's achievements in regard to providing additional income to households participating in the public works program under the project which improved their access to purchased food. The IA also reports that crop production increased more in project-supported than in non-project-supported areas which indicates an increase in food availability attributable to the project. M&E Quality Rating Modest 10. Other Issues a. Safeguards Environmental and Social: The ICR (paras. 76 to 78) reports that the project was implemented in compliance with applicable World Bank policies. Two safeguard policies were triggered - Environmental Assessment (OP/BP 4.01) and Projects on International Waterways (OP/BP 7.50). The project was assigned Category B - Partial Assessment. A Generic Environmental Management Plan, prepared for the predecessor project, was updated for this project. Site-specific Environmental Management Plans (EMPs) were prepared and disclosed. A Grievance Redress Mechanism (GRM) was established in 2015 under the June 2015 restructuring. 1,039 grievances were recorded of which 45% related to delays in payments and 36% to disputes regarding soil volumes excavated by participants. 96% of the grievances were responded to within the stipulated service standards against the target of 95%. Regarding projects on international Page 23 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) waterways, an exception to notify was granted by the Europe and Central Asia (ECA) Region Vice President in May 2015 (ICR para. 54). The ICR does not report on the ratings in regard to safeguards in the last ISR posted before project closing. b. Fiduciary Compliance Financial Management (FM): The ICR (para. 79) reports that FM arrangements were generally acceptable to the Bank. There were early delays due to weaknesses in the PMMIS as a result of which FM performance was rated Moderately Satisfactory. This improved over time and reporting was timely and acceptable. Annual audit reports were timely and acceptable. In calendar year 2015, there was a qualified opinion from the auditors but the issue was resolved. In 2017, the FM rating was downgraded to Moderately Satisfactory because of blocked funds at TajikSodirotbank (TSB). The project team informed IEG that this was due to a banking crisis in Tajikistan that resulted in serious liquidity problems for some banks, including TSB. As a result, TSB could not make timely and adequate transfers of funds to the PMU. The issue was finally resolved with the intervention of the Ministry of Finance (MoF). The ICR reports, that mainly because of this issue, the FM rating was Moderately Satisfactory in the last ISR posted before project closing. Procurement: The ICR (para. 80) reports that, early during implementation, the rating for procurement was Moderately Satisfactory due to low capacity of the PMU staff. After two years, the rating was upgraded to satisfactory. However, starting 2018, the rating was downgraded again to Moderately Satisfactory and remained so until project closing. The ICR reports that there were significant shortcomings including substandard quality of bidding documents mainly due to weak procurement planning and processing capacity in the PMU. The rating for Procurement was downgraded to MS in 2018 and remained so at closing (ICR para.80). These issues contributed to a downgrading of the DO and IP ratings to MS in the last ISR before project closing (ICR Data Sheet).. c. Unintended impacts (Positive or Negative) There were no significant unintended impacts. d. Other Gender: The project paid special attention to participation of women in the project supported activities. The ICR reports (para. 60) that, in comparison to the predecessor project under which the female participation rate was 11%, the rate increased to 25% under this project. The project assisted women in obtaining valid identity (ID) cards to be able to participate in formal employment. Attention was paid to allocating less demanding work to women. The pay rate for female participants was the same as for male participants. Poverty Reduction and Food-Insecurity: The ICR reports (para. 61) that the project contributed to effective targeting of food-insecure individuals and households to reduce poverty in the project area. It supported the development of a methodology to identify the most vulnerable households and involve them in the public Page 24 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) works (PW) program. Participants received additional temporary income under public works program but also gained potential access to mid- and long-term benefits. The project facilitated official personal registration and documentation processes that enabled people to access social assistance programs. However, the ICR does not provide information to enable an assessment of the adequacy and effectiveness of these activities. Anti-Corruption and Transparency: The project contributed to increasing transparency and accountability, thereby helping increase trust between stakeholders. It supported the development and deployment of Tajikistan's first cashless billing system. This enabled farmers to make direct payments of membership fees and irrigation service fees, based on regular measurement of water volumes received, into the accounts of the servicing WUAs and ALRI. The ICR reports that the ALRI intends to roll out the system to other areas in Tajikistan (ICR para. 62).. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment The ICRR assesses pre- restructuring efficacy as Modest due to insufficient evidence in regard to the achievement of the objectives, and post- restructuring efficacy as Substantial with moderate Outcome Satisfactory Moderately Satisfactory shortcomings due to lack of clarity in regard to attribution of achievements to the project under Sub-objective b. Relevance is also rated Modest given the limited alignment to an outcome based solution to the given development problem. The ICRR assesses Quality-at- Entry as Moderately Satisfactory due to weaknesses in design of appropriate PDO indicators and Supervision as Moderately Bank Performance Satisfactory Moderately Satisfactory Satisfactory due to continuing inability to address weaknesses in the project's M&E system through the implementation period. Quality of M&E Modest Modest Quality of ICR --- Substantial Page 25 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) 12. Lessons The ICR lists a number of useful lessons. The following are highlighted by IEG with some modification to the language: 1. The inclusion of relevant technical assistance in the project design could enable enhancement of farm productivity. Surveys under the project revealed that irrigation related interventions need to be clearly linked to agricultural on-farm support and improvement of the food value chain. The feedback indicated that farmers need to be trained to use water efficiently and to switch to better farming methods. Further gains could be realized from providing support for improving the value chain and providing a linkage to markets. 2. The project demonstrated that public works programs to counter periods of high unemployment and negative effects of economic shocks can prove successful, but cannot serve as a social safety net as some groups may be excluded from the benefits from these programs, like for example, people with disabilities or those unable to undertake heavy labor. The government needs to supplement such programs with other means directed to the target population. 3. A lack of appropriate project level indicators to measure increase in crop yields may lead to difficulties in attributing a potential increase in crop yield to the project. Under this project, yield increases could not be attributed to the project, not only for lack of relevant indicators but also as factors beyond access to irrigation may affect crop yields, such as for example weather and climate conditions and the farmers’ access to farm inputs and technology. 4. Without a flexible design and implementation approach it may proved difficult to effectively deal with emergency situations. This project needed to deal with an emergency situation under the second restructuring, and it then became apparent that a somewhat more flexible design and implementation approaches would have enabled better responsiveness. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is clearly written, concise, candid, and follows Bank guidelines for preparation of ICRs. It provides a clear theory of change and the reporting is outcome-based. The analysis is evidence-based but constrained to some extent by the weaknesses in the project's M&E system. The ICR commendably compensates by drawing upon other sources of information, including the national statistical system, but acknowledges some resulting shortcomings including problems of attribution of the results in some cases (e.g. increased crop yields). The ICR also draws upon beneficiary surveys (to the extent available) to support some of the conclusions. The ICR lists a number of lessons learned which are relevant for projects implemented in similar environments. One area where more information could have been provided is regarding the impacts of the two key items in the Page 26 of 27 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PAMP II (P133327) institutional strengthening component - the National IWRM Strategy and the Kofarnihon River Basin Management Plan. a. Quality of ICR Rating Substantial Page 27 of 27