Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) Report Number: ICRR0022129 1. Project Data Project ID Project Name P122944 RURAL INVSMT (AZRIP) 2 Country Practice Area(Lead) Azerbaijan Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-81890,IBRD-84030 30-Sep-2017 80,000,000.00 Bank Approval Date Closing Date (Actual) 05-Jul-2012 30-Sep-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 30,000,000.00 0.00 Revised Commitment 80,000,000.00 0.00 Actual 80,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Chikako Miwa J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objectives of this Rural Investment Project in Azerbaijan (AzRIP-2) were to improve access to, and use of community-driven rural infrastructure and to expand economic activities for rural households (Loan Agreement dated October 19, 2012, Schedule 1; Loan Agreement for Additional Financing dated October 22, 2014, Schedule 1). Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) For the purposes of assessing the extent to which the project’s objectives were achieved in Section 4 of this ICR Review, the following sub-objectives have been defined. They will be referred to as Objectives 1, 2 and 3 in Section 4. Sub-objective 1: To improve access to community-driven rural infrastructure Sub-objective 2: To improve use of community-driven rural infrastructure Sub-objective 3: To expand economic activities for rural households b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Component 1: Rural Community Infrastructure (Appraisal amount: US$41.61 million. Actual cost: US$122.10 million, which was 293 percent of the approved amount.) Component 1 intended to support three categories of microprojects: (i) infrastructure microprojects in the 11 districts not previously covered by Azerbaijan Rural Investment Project (AzRIP)-1; (ii) infrastructure microprojects in communities that managed well-performing microprojects under AzRIP-1; and (iii) livelihoods microprojects in communities that had performed well under AzRIP-1, to be submitted by Common Interest Groups (CIGs) that were established and provided with technical assistance under Component 2. This component financed activities throughout the project cycle for microprojects, focusing on local intra-community roads, community renovations/land improvement works, potable water systems, and irrigation and drainage systems. It also supported clinics, kindergartens and schools, small-scale rural electrification, as well as commercial investments in poultry incubators, milk collection facilities, and fodder mills. During implementation, the funding and hence the scope of this component (financed by Additional Financing of US$86.47 million (ICR, Table 2, page 12) was expanded by: (i) increasing rural infrastructure investments to two additional districts (Agdam and Fusuli in the Karabagh region), and financing additional community infrastructure microprojects in well-performing communities; (ii) increasing investments in livelihoods activities to cover an additional twenty communities; and (iii) piloting the rehabilitation of inter- community roads, targeting 30-60 communities each. In addition, the average size of infrastructure microprojects funded was scaled up from US$70,000 to US$90,000, considering commodity price increases and inflation; and the ceiling for livelihoods microproject grants was raised from US$120,000 to US$160,000. Moreover, a pilot on Women’s Development and Enterprise Groups (WDEGs) was included in this component in March 2018, to enhance livelihoods activities for rural women who face multiple constraints in accessing the labor market. Component 2: Technical Assistance for Rural Infrastructure (Appraisal amount: US$4.75 million. Actual cost: US$4.75 million.) Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) This component was intended to provide technical assistance to: (1) develop and implement infrastructure microprojects, and (2) pilot activities to support livelihoods in six communities. The former was geared towards strengthening the capabilities of community groups, local authorities, Regional Grant Approval Committees (RGACs), Project Assistance Teams (PATs), and staff in Regional Operations Offices (ROOs) in contracting, procurement, financial management, and participatory monitoring and evaluation (M&E). The pilot livelihoods support services included the mobilization and organization of CIGs, identification of income generation activities, preparation of business plan proposals, legalization of the CIGs’ status, and training of CIG members in business management. Well-performing CIGs were encouraged to submit microproject proposals for financing under Component 1. Technical support services were delivered through cross visits to enable peer-to-peer learning among communities, workshops and seminars, and specific training sessions. During implementation, livelihoods support services were expanded from six to 20 communities, to align with the expansion of scope under the Additional Financing and related restructuring, and support services to the Women’s Development and Enterprise Groups (WDEGs) were expanded. Component 3: Project Management and Results Monitoring (Appraisal amount: US$7.21 million. Actual cost: US$14.06 million, which was 195 percent of the approved amount.) This component aimed to finance project management, including staff costs and overhead, project outreach and the beneficiary feedback mechanism (BFM), support to the RGACs, and regular M&E at the project, district, and community levels. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: At appraisal, the original cost estimate was US$53.60 million (PAD, Table 1, page 7). At project closing, the actual cost was US$140.91 million (ICR, page 2 and Annex 3). Financing: At appraisal, expected financing sources were: US$30 million loan from the World Bank (IBRD), US$20 million from the Borrower, and US$3.60 million from Local Beneficiaries (PAD Data Sheet). At the restructuring in 2014, the Additional Financing (AF) increased the Bank's financing to US$80 million, the Borrower's financing to US$50 million, and the Local Beneficiaries' financing to US$10.07 million. At project closing, the total actual costs were financed with US$80 million loan from the Bank, US$49.39 million from the Borrower, and US$11.52 million from Local Beneficiaries (ICR, page 2). Dates: The project was approved on July 5, 2012 and became effective on December 4, 2012. The Mid- Term Review (MTR) was published on September 30, 2015. The project was closed on September 30, 2019, which was 2 years after the original closing date of September 30, 2017. Restructuring: The project had the following two restructurings: a restructuring in 2014 for Additional Financing (AF) and another restructuring in 2018 to extend the loan closing date. At the AF, which was approved on July 9, 2014, the total planned project costs were increased to US$140.07 million (135 percent of the original planned costs) (ICR, para. 24). At the same time, funds were transferred from Component B and C to Component A, to fulfill the increasing needs for rural infrastructure rehabilitation in the capacitated communities (Ibid). Moreover, at the AF, one PDO indicator was disaggregated by types of microprojects, and four PDO indicators were revised with the higher end targets (ICR, Annex 8, page 56). Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) IEG concurred with the ICR team (para. 47) that "[g]iven that the AF increased the scope of the project without changing the PDO, a split rating of the project is not necessary." 3. Relevance of Objectives Rationale Country Context. Poverty levels were decreased from 49 percent in 2001 to 6 percent in 2012 due to an average of 12.8 percent economic growth from 2000 to 2012. Per capita income increased from US$470 in 1995 to US$4,820 in 2009, driven by the growth of employment due to the oil boom and the public spending, which included a social benefit system. On the other hand, the country faced challenges such as decaying infrastructure and service provision in rural areas and worsening inequality indicated by disparities between regions. Of the total population, 47.1 percent resided in rural areas, where poverty incidence was 8 percent and almost twofold of 4.5 percent in urban. Though the poverty rate was not high, the vulnerable population to shocks and to falling back into poverty was estimated as 65 percent, of which 54 percent of them were rural residents. The increases in oil and gas revenues were expected to slow down within a decade and then decline. Working towards a diversified economy, market-based policies, strengthened institutions, and improved access to and reliability of social services was essential to achieve sustainable and more equitable economic growth. Sector Context. The spatial disparities were not decreased by the economic growth, as rural populations had limited access to infrastructure, services, and economic opportunities. In rural areas, insufficient basic infrastructure and services, including roads, water, energy, health, and education, intimidated people and businesses to reside and invest there, leading to slow poverty reduction. Piped water was accessed by less than 33 percent of households in rural areas, while it was available for over 90 percent of households in the capital city. Supply of gas and electricity was not stable in rural areas, and access to them was limited in remote mountainous areas. Local roads were congested due to the increase in domestic vehicular traffic which grew 10 percent per year since 2003. About 45 percent of main and regional roads and 80 percent of local roads in rural areas were in poor condition. Combined with the poor road conditions, long travel distances from villages to the district market resulted in high transportation costs and lower product prices for the farmers. Saline water intrusion was observed in over 50 percent of all irrigated land because of degraded irrigation and drainage systems. Distance to healthcare facilities limited the rural poor to access those facilities. Access to both preschool and post-secondary facilities was also limited in rural areas. Rate of post-secondary education attainment was below 11 percent in rural areas, compared to 40 percent in the capital Baku. In addition, the poor condition of school facilities in rural areas affected quality of learning at primary and secondary levels (ICR, para.2; PAD, para. 7). Relevance to Bank Assistance Strategies. At appraisal, the objectives of AzRIP-2 were consistent with those of the CPS FY11-14, namely: (i) building a competitive non-oil economy; and (ii) strengthening social and municipal services. The project was intended to contribute directly to Outcome 3 – improved access to rural infrastructure – under the CPS’s Results Area 4 on Improved Municipal and Rural Services. The CPS aimed to generate employment by implementing rural infrastructure projects. AzRIP-2 supported rural communities’ demand-driven investments to improve access to rural infrastructure. At project closing, the project’s objectives remained relevant to the CPF FY16-20. AzRIP-2 supported two objectives of the CPF: objective 1.3 of contributing to improved access to water, sanitation and communal services; and objective 2.4 of supporting enhanced competitiveness of agriculture and rural development. AzRIP-2 supported Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) investing in infrastructure in poor communities, providing them with better access to markets and more productive use of assets. Relevance to Government Strategies. At appraisal, AzRIP-2 was in line with the Government’s strategic vision and development priorities described in the State Program on Socio-Economic Development of Regions for 2009-2013; the State Program for Agriculture for 2008-2015; Azerbaijan 2020: Outlook for the Future; and the State Program on Poverty Reduction and Sustainable Development 2008-2015. AzRIP-2 also ensured to support only those investments that aligned with regional development programs. At project closing, the PDO was aligned with the State Program on Socio-Economic Development of Regions for 2019-2023, as the project aimed to increase balanced growth and employment opportunities, contributing to the diversification of the rural economy emphasized by the latest State Program (ICR, para 28). The project continued to be relevant to Azerbaijan 2020: Outlook for the Future at project closing. Institutional Capacity and Realism. Local executive committees (ExComms) were one kind of local government institutions that represented the various line ministries and provided oversights to all development activities at the district level. In addition, municipal councils were responsible for intra- municipal infrastructure outside the ExComms’ responsibilities, such as rural roads. AzRIP-2 aimed to contribute to strengthening the role of local government institutions by mobilizing local communities to work closely with ExComm authorities and municipalities in the development of infrastructure in underserviced rural communities. Previous Sector Experience. AzRIP-2 was designed to succeed achievements of the preceding rural investment project (AzRIP-1) while expanding its scope to cover new districts. AzRIP-2 also adopted the community driven development (CDD) approach implemented under AzRIP-1, relying on the World Bank’s expertise and experience in supporting the CDD approach all over the world to improve service delivery, local infrastructure, and income generation opportunities. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Sub-objective 1: To improve access to community-driven rural infrastructure Rationale Theory of Change for the project: Technical assistance such as training and financial assistance such as grant provisions would lead to strengthen capacities of: community groups for implementing microprojects, and Regional Grant Approval Committees (RGACs) for screening and approving microprojects. These would support rural communities to develop and implement microprojects to build and rehabilitate social basic Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) infrastructure and to enhance income generation activities. The infrastructure and livelihoods microprojects would lead to improve access to and use of rural infrastructure and to expand economic opportunities for rural households. In the long-term, the improved social capital would translate into financial capital and economic well-being, as well as improve social cohesion at local level and governance and accountability between citizens and the State, contributing to decrease the spatial disparities. Critical assumptions included: (1) Capacities of the community groups and RGACs were strengthened sufficiently and timely for them to develop and implement microprojects within the project duration; (2) Income generation activities supported under the Project would continue to be economically viable after the project closing; and (3) Rehabilitation of social basic infrastructure would support enhancement of income generation activities. Outputs:  884 community roads (totaling 6,198.9 km), 10 inter-community roads (totaling 92.2 km), 180 potable water supply systems, 101 irrigation canals and drainage systems, 46 health clinics, 21 kindergartens, 18 electrical systems, 19 schools, and 234 community facilities were rehabilitated (ICR, para. 32).  1,512 infrastructure microprojects were financed; of which, 97 percent of investments focused on infrastructure (ICR, para. 32).  According to the ICR (page 32), 3.6 million people in rural areas received direct or indirect benefits, exceeding the target of 3.5 million (103 percent of the revised target). Of which, 56 percent was female, exceeding the target of 50 percent (112 percent of the revised target). Outcomes:  Percentage of households who have access to and use of rural infrastructure increased to 33 percent, surpassing the target of 25 percent (132 percent of the revised target). At appraisal, 43 percent of households used certain rural infrastructure items, which increased to 57.2 percent at project closing (ICR, para. 34). The change of 14.2 percentage points represents a 33 percent increase in access to and use of rural infrastructure.  A 53.7 percent reduction in travel time to market, hospital, school, safe water source was achieved, exceeding the target of 48 percent (112 percent of the original target). At appraisal, the average travel time to these destinations was 21.43 minutes, which was reduced to 12.59 minutes at project closing (ICR, para. 32). The reductions in travel time, weighted by the frequency of access to each destination, represented the 53.7 percent reduction. The greatest reductions occurred in travel to preschools (68.7 percent), water supply points (57.97 percent), and secondary schools (55.69 percent). Smaller reductions occurred in traveling to markets (37.55 percent) and the district hospital (24.50 percent). Development of microprojects were led by the communities, which used a Participatory Rural Appraisal process to analyze economic challenges and opportunities in their communities and agree on the investments. These contributed to develop local leadership within the communities, as discussed in Section 10c. On the other hand, the independent technical audit in 2017 suggested a sustainability risk to development outcome, as discussed in Section 7. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) Overall, the access to community-driven rural infrastructure was improved, as evidenced by the increase in percentage of households with access to rural infrastructure and the decrease in travel time to basic rural facilities. The achievement of Objective 1 was therefore rated substantial. Rating Substantial OBJECTIVE 2 Objective Sub-objective 2: To improve use of community-driven rural infrastructure Rationale Outputs:  According to the endline survey (ICR, para. 34), roads rehabilitated under the project were used by 91.9 percent of rural households. Sewage and drainage systems were used by 77 percent of households, schools and kindergartens by 63.8 percent, and potable water supply systems by 63.5 percent.  Traffic volume on rural connector roads reached an average of 321 vehicles per day, exceeding the target of 300 (107 percent of the revised target) (ICR, page 34).  90 percent of subprojects received necessary maintenance operations with community engagement, surpassing the target of 90 percent (100 percent of the revised target) (ICR, page 33). Outcomes:  As was the case for Objective 1, the percentage of households who have access to and use of rural infrastructure increased to 33 percent, surpassing the target of 25 percent (132 percent of the revised target). At appraisal, 43 percent of households used certain rural infrastructure facilities, which increased to 57.2 percent at project closing (ICR, para. 34). The change of 14.2 percentage points represents a 33 percent increase in access to and use of rural infrastructure.  As noted in the ICR (Annex 1, page 30), the percentage of households interviewed by the endline survey were satisfied with the overall quality of basic rural infrastructure increased from the baseline of 39.2 percent to 69.5 percent, not meeting the target of 75 percent (93 percent of the revised target).  The improved use of community-driven rural infrastructure was observed in general; for example, the rehabilitated roads were used by almost all of the rural households, and the rural connector roads were used by an average of 321 vehicles daily compared with a target established at restructuring of 300 per day (ICR, page 34). To summarize, only about two thirds of the households were satisfied with the overall quality of basic rural infrastructure compared with a target of 75 percent. Considering that most of the project investments were made for improvements to infrastructure, the increase in percentage of satisfied households by Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) 30.3 percentage points from the baseline of 39.2 percent was not substantial. Overall, the achievement of Objective 2 was therefore rated modest. Rating Modest OBJECTIVE 3 Objective Sub-objective 3: To expand economic activities for rural households Rationale Outputs:  There were 831 members in Common Interest Groups (CIGs), greatly exceeding the target of 225 members introduced in the 2014 restructuring (369 percent of the target).  25 livelihoods enterprises were financed as pilots to test four types of farm-related activities, i.e. poultry incubators, milk collection facilities, processing of fodder crops, and greenhouses (ICR, para. 36). These activities were identified through a Participatory Rural Appraisal process, analyzing the existing local expertise and gaps in the local supply chains. In addition, 38 Women Development and Enterprise Groups (WDEGs) were formed.  92 percent of livelihood microprojects (i.e. 23 out of the 25 livelihoods enterprises) were ongoing and commercially viable, meeting the target of 90 percent (102 percent of the revised target). Moreover, all of the 110 businesses started by the WDEGs were commercially viable when the project closed.  The establishment of these small-scale enterprises supported capacity strengthening of the communities, as technical experts (e.g. veterinarians, agronomists) working for the enterprises introduced new knowledge and skills to villagers.  There were 327,874 beneficiaries from livelihoods microprojects, exceeding the target of 250,000 beneficiaries (131 percent of the revised target). The total beneficiaries from livelihoods microprojects consisted of 4,648 direct beneficiaries and 323,226 indirect beneficiaries. Outcomes:  A 21.16 percent increase in household income from economic activities, not meeting the target of 23 percent (92 percent of the original target). These data were based on the endline survey conducted by the Government in June 2019. The ICR (para. 62) noted this indicator did not properly capture the intended outcome of “expanded economic activities for rural households”, since income would be generated three to five years after the livelihood pilots. In addition, sustainability risks of livelihoods microprojects were anticipated due to the end of technical assistance at the project closing, as discussed in Section 7. Given the multiple barriers to employment faced by rural women, the piloting of the WDEGs demonstrated a viable model at scale that supported the participation of rural women in economic diversification and growth. The project supported the establishment of 38 WDEGs which consisted of 468 women in 14 districts and developed 110 businesses, which generated income for 1,173 rural residents. Community Business Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) Promoters (CBPs), who were local youth trained through a business skills program and supervised by rural business specialists, supported rural women on-site with business incubation and development. At project closing, two to three new businesses were developed per WDEG, including horticulture, silkworm breeding, beekeeping, animal husbandry, catering, dry fruits processing, cleaning services, and carpet weaving and sewing workshops. WDEGs were also committed to engage in voluntary community services, such as environment clean-up campaigns and care of the disabled and elderly in the community. They coordinated with municipalities on waste collection and water consumption reduction, which empowered their capacities in the community. The WDEGs were networked through the Azerbaijan Rural Women’s Association (ARWA), which continued to support the WDEGs through capacity building and service provision. During the project lifecycle, the percentage of households which met all of their needs increased from 1.84 percent at baseline to 7.74 percent as measured by the endline survey (ICR, para. 39). The percentage of households, which only made ends meet or were in poverty, decreased from 60.7 percent at baseline to 47.3 percent in the endline survey (ICR, para. 39). The ICR team argued that the achievements of the PDO contributed support for households to accumulate assets. For example, a supervision mission observed some villagers who increased the number of livestock, as they were able to secure the potential market for livestock products (ICR, para. 44). The livelihoods microprojects enhanced economic activities of the rural households and increased their incomes to a certain extent. The piloting of WDEGs supported economic and social empowerment of rural women. On the other hand, the livelihoods microprojects would need to address the sustainability risks to continue being economically viable. Overall, based on the evidence available, the achievement of Objective 3 at the project’s close was rated substantial by this review. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The achievement of Objectives 1 was rated substantial, as the access to community-driven rural infrastructure was improved, as the achievements of all the relevant PDO indicators surpassed the targets. The achievement of Objective 2 was rated modest, as the increase in the satisfaction level of the users of the rural infrastructures was not substantial considering the amount of investments. The achievement of Objectives 3 was rated substantial, as the establishment of livelihoods enterprises provided rural households with widened opportunities to engage in economic activities, especially for women. Overall, the efficacy of this project’s achievements was rated substantial but only marginally so because of the low level of household satisfaction with the quality of the additional basic rural infrastructure, and the uncertainty about the extent to which improvements in household livelihoods was attributable to the project's investments. Overall Efficacy Rating Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) Substantial 5. Efficiency Economic Analysis: The PAD demonstrated a positive economic return from the micro-project investments through the cost benefit analysis of a sample of 18 microprojects (water supply, irrigation, health and road projects) for the preceding project (PAD, para. 38). The PAD provided no new economic analysis because it was assumed that as a "repeater" project AzRIP-2 would have "similar positive economic returns and socio- economic benefits" (para 40). This might also have been on the basis that the enterprises to be chosen in a Community Driven Development (CDD) type project could not be anticipated. At project closing, investments financed by the project were covered by an economic analysis in November 2017. The economic analysis targeted: (i) productive infrastructure investments in the rehabilitation of local roads and water supply systems; and (ii) livelihoods investments in commercial milk collection, egg incubation, and fodder mills. The infrastructure microprojects were economically viable. The economic analysis for the infrastructure microprojects was calculated with the estimated lifespan of investments of 15 years (based on the implementation duration of AzRIP and AzRIP-2) and a relatively high 15 percent discount rate. The average road repair investments had an economic rate of return (ERR) of 19 percent with a net present value (NPV) of US$578,797. The average water supply projects had an ERR of 22 percent with an NPV of US$18,990. If potential benefits were to be included to the analysis, the results would improve. The effects of lower future benefits, increased costs, and reduced lifetime of the investment were assessed by the sensitivity analysis, which showed robust results for the infrastructure investment projects, but not for the water supply projects. The average livelihoods microprojects were also economically and financially viable, though there were some variations. The economic analysis for the livelihoods microprojects was calculated with an estimated lifespan of investments of 10 years and a 7 percent discount rate. The poultry incubators had the highest ERR of 31 percent and an NPV of US$128,994. The results would be complemented by the findings from the field research that some communities established new poultry feed sales centers after poultry incubators were supported by the project. The microprojects had an average ERR of 18 percent with an NPV of US$35,321. The fodder mills projects had an average ERR of 19 percent and an NPV of US$39,345. On the positive side, these investments had a crowding-in effect attracting private investments from community members and enterprises. According to the field research, 200 community members reportedly purchased meat and dairy livestock, and seven enterprises started selling fodder products due to higher demand from the local population. At project closing, the actual NPV exceeded the estimated NPV estimated at the AF for the investments in poultry incubators and fodder mills. On the other hand, the investments in milk collection and fodder mills were sensitive to a reduction in the lifetime of the investments. For example, a 50 percent reduction in the life of the investments in milk collection and fodder mills would yield a rate of return below the discount rate and a negative NPV. There was no ERR which consolidated the ERRs of different types of microprojects. Cost Effectiveness Analysis: At project closing, the independent technical audit included a cost effectiveness analysis of the project’s six types of infrastructure investments (electricity, roads, buildings, drainage, potable water systems, and irrigation), though the numerical results of this cost effectiveness analysis were not provided in the ICR (Annex 6A, page 52). Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) Aspects of Design and Implementation that Influenced Efficiency: The Project was designed based on the preceding project and managed by the same PMU with accumulated experience, these facts contributed to accelerate implementation at the early stage and disburse 70 percent of the first loan within two years of approval. On the other hand, the limited human resources in the PMU caused delays in implementation of the livelihoods activities that were expanded considerably as a result of the AF in 2014. The ICR provided no assessment of the efficiency with which Common Interest Groups managed the receival and processing of proposals for the financing of microprojects, as well as no analysis of the cost effectiveness of the social infrastructure program such as schools and health facilities, except for the brief description on the independent technical audit’s cost effectiveness analysis (Annex 6A, page 52). The overall efficiency of the project was rated substantial, as the ex-post economic analysis provided positive results in general, while shortcomings were observed regarding the vulnerability of the partial economic analysis to reasonable changes in the assumptions made. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objective was rated high, due to the project’s alignments with the country and sector contexts and the main strategies of the government and the bank assistance. Efficacy was rated substantial with shortcomings on insufficient evidence for outcome-level achievements. In general, the project contributed to the improvements in access and use of community-driven rural infrastructure and the expansion of economic activities for rural households, with a special attention to women. On the other hand, the achievements of two out of four outcome-level indicators (i.e. the user satisfaction and the income of rural households) were weak in terms of the levels of achievements and the attributability of the achievements to the project. Efficiency was rated substantial, as the ex-post economic analysis for both the infrastructure and livelihoods microprojects showed a positive economic viability. This review rates the overall outcome of the project as Moderately Satisfactory because of the shortcomings in efficacy mentioned above. This rating aligns with the rating by the ICR of the project's overall outcome. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome 1. Risk to the sustainability of infrastructure: According to an independent technical audit (ICR, para. 85), 98 percent of infrastructure financed by the preceding project and this one remained, or are expected to be, in good or fair condition after 10 years. School buildings and health centers were in the best conditions due to the continued maintenance by the ministries of education and health. Potable water systems were in the second-best condition. Infrastructure that required periodic rehabilitation or maintenance, such as local intra- community roads and irrigation systems, were in fair condition. The infrastructure would degrade overtime and require periodical rehabilitation effort. The risk would be mitigated by the community project committees, which would continue coordinating with the local authorities and ensure maintenance of the infrastructure. In addition, the PMU stated that municipalities embedded expected maintenance expenses of intra-village roads to their budgets. When municipalities and communities would be able to request the district authorities, which had jurisdiction over all development activities in districts, to repair the infrastructure. 2. Risk to the sustainability of livelihoods enterprises: After project closing, livelihoods enterprises needed to graduate from technical assistance provided by the PMU, which would pose challenges to the beneficiaries’ abilities to maintain the enterprises by themselves. The risk would lead to narrowing down the economic opportunities expanded by the project. The risk would be mitigated by Azerbaijan Rural Women’s Association (ARWA), which would be expected to continue supporting its member groups including WDEGs through capacity building and service provisions. Moreover, the delay in adoption of the Cooperative Law affected the legal status of enterprises established for livelihoods microprojects. According to the Rural Livelihoods study (ICR, para. 59), the ownership and benefit-sharing arrangements of the enterprises supported by the project were not legally enforceable. After project closing, therefore, there would be no measure to ensure that profits were going to be distributed within the entire community. 8. Assessment of Bank Performance a. Quality-at-Entry At project inception, a diagnostic study was conducted for planning livelihoods support services and analyzing factors that would make the investments effective. The livelihoods support services were designed through extensive consultation with key stakeholders. The PMU for this project was the successor of the PMU for AzRIP-1, enabling the Project to use the accumulated experience and competence for project management. The project refined the targeting strategy of preceding operation by focusing on regions with higher relative incidence of poverty and encouraging smaller communities to group and participate in the project activities. In addition, local authorities, such as representatives of municipalities and Local executive committees (ExComms), were continued to be invited to the Regional Grant Approval Committee (RGAC), to ensure their commitment to the operations and maintenance (O&M) of investments after project closing. Through this, the project aimed for formalizing the handover of infrastructure investments to municipalities. Potential risks on procurement were identified through a Procurement Risk Assessment. Based on the assessment, appropriate risk mitigation measures were Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) developed at project design. The Environmental Management Framework was revised to consider the impacts of the livelihoods support projects and the expanded geographical coverage. Moreover, a gender empowerment strategy was developed, aiming to increase women taking leadership roles in the project. Overall, Quality at Entry was rated by this review as satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision According to the ICR during implementation, the Bank conducted an average of 2-3 missions annually, and 16 supervision missions in total. An extensive training was provided to the PMU to strengthen their capacities, including staff retreat, three study tours to India and Bangladesh, and a week-long training at the Hague Academy of Local Governance. The last one aimed to support the PMU to develop capacities to work with local governments, communities, and the private sector on spatial development, to prepare for the proposed succeeding project. According to the summary of the Borrower’s completion report (ICR, Annex 5A, page 49), the Bank Task Team’s technical support was considered by the Borrower to have been very helpful for the PMU to implement the project and prepare the proposal for the succeeding project. For instance, supervision missions which focused on financial management and procurement identified issues and advised the PMU on corrective measures that were followed through. Though the project had four Task Team Leaders throughout the project cycle, supervision documents did not show any discontinuity of the quality of Bank support. On the other hand, the Bank’s M&E framework had shortcomings on the revised PDO indicator as discussed in Section 9a. as well as the targets that were unachievable considering the size of target population. Moreover, the Bank’s reporting showed some inconsistency in terms of ISR ratings on project progress and the quality of procurement administration. On balance, the Bank’s support to the PMU was sufficient and timely, though there were shortcomings on M&E and reporting that affected the measurement of the achievement of outcomes. On this basis, Quality of Supervision was rated by this review as moderately satisfactory. Quality at Entry was rated satisfactory. Quality of Supervision was rated moderately satisfactory. Overall, Bank Performance was therefore rated moderately satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) 9. M&E Design, Implementation, & Utilization a. M&E Design At appraisal, the project’s monitoring methods relied heavily on physical verification of works, as outlined in the existing M&E system established by the preceding project, while aiming to further improve the use of information management system. The PAD clearly described causal relationships to achieve the PDO. On the other hand, at AF, the PDO indicator on percentage change in income from household economic activity was revised to percentage change in household income from economic activities disaggregated by rural infrastructure and livelihood microprojects. The revised PDO indicator posed two challenges: (i) income changes derived from livelihood activities would not be observable within the project duration; and (ii) income changes disaggregated by their causes would not be measurable without a randomized impact evaluation (IE). The Mid-Term Review pointed out these challenges and recommended that the PMU continue collecting data for the original indicator. The PAD planned to conduct a randomized IE with quantitative and qualitative components and periodical surveys. However, the GoA disagreed with the randomized IE as their targeting criteria did not align with the methodology to establish a control group which would not be able to receive project support. b. M&E Implementation M&E data were collected appropriately for real-time decision making and public access. The data were disaggregated by gender and used to measure impacts of microprojects to the Project, communities, and households. The Project’s M&E had a three-tier structure at the PMU, Regional Operations Offices (ROOs), and community levels, in order to monitor and evaluate project outcomes. The M&E structure and staffing arrangements were considered sufficient by supervision missions. The M&E team monitored indicators, conducted technical audits on financial management and procurement, implemented participatory M&E activities at the community level, compiled beneficiary feedback in active microproject areas, coordinated the IE, and conducted studies on each type of microproject and on project impacts. Public reporting was made available regularly in community meetings. Moreover, the community showed satisfaction with the GIS Map which provided them with project-related information transparently, according to the feedback from community members (ICR, para. 64). The information management system was based on the data in the Excel files submitted weekly from the ROOs to the PMU. The PMU collected and reported data efficiently, as it compiled extensive data for more than 2,000 subprojects under AzRIP-1 and 2 within half a month to develop the GIS Map. The geographical data were then combined with publicly available satellite and crowdsourced data to construct a Vulnerability Index and Map of the country at the district level. The Vulnerability Index Map supported the Project’s targeting strategy to identify the economically vulnerable locations and where microprojects were successfully implemented during the preceding project. c. M&E Utilization The PMU used the ROOs’ weekly monitoring reports and the monthly monitor meetings to verify progress of the microprojects against their milestones and to take corrective actions as needed. During Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) implementation, no significant issues were observed by the PMU, which was confirmed by the reports of the supervision missions. On balance, the M&E design, implementation, and utilization were adequate in general, with the appropriate PDO and sufficient data collection, though there were some minor shortcomings including the challenges posed by the revised PDO indicator. Overall, M&E Quality was rated substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards At appraisal, the Project was classified as Environmental Category B, which required a partial environmental assessment. It triggered two safeguards policies: OP/BP 4.01 on Environmental Assessment, and OP/BP 7.50 on Projects on International Waterways. According to the ICR (para. 69), to comply with the safeguard on Environmental Assessment (OP/BP 4.01), the Environmental Management Framework (EMF) was updated by the Borrower during project implementation after microproject investments were identified. Building upon the EMF of the preceding project, The EMF aimed to address the potential environmental risks of the new aspects that were added to the project, such as investing in livelihoods microprojects, repairing intra-community roads, and expanding geographical coverage. Followed by identification of each eligible microproject investment, environmental impact assessments and environmental management plans were developed. Development of the environmental management plans was supported by the new checklist adopted at the restructuring in 2014. The ICR stated that through the project cycle, the PMU ensured compliance with OP/BP 4.01 by providing beneficiary communities with training and guidance to identify potential environmental risks and develop adequate mitigation measures. Environmental Specialist at the PMU monitored environmental impacts of the microprojects and included findings into reporting to the Bank. Weakness was noted during one supervision mission that the PMU did not monitor the disposal of hazardous waste such as asbestos, the sourcing and licensing of construction materials, and the operation of associated facilities such as the worker’s accommodations. The PMU addressed these issues through appropriate supervision and guidance. Moreover, the PMU worked with the local office of the Ministry of Ecology and Natural Resources to ensure its representatives supervise the collection and disposal of hazardous materials (para. 70). The safeguard on Projects on International Waterways (OP/BP 7.50) was triggered as the project aimed to support water supply and small-scale irrigation systems on rivers flowing into the Caspian Sea and tributaries of the Aras and Kura rivers. The project’s Preparation Team informed riparian states on the geographic locations, the objectives, and the planned activities, and ensured that no adverse impacts were expected on any of the riparian rivers nor on the Caspian Sea. There were no unfavorable responses from any of the riparian states. The project triggered no social safeguards policies. During project preparation, a rapid social assessment was conducted, providing the project with recommendations to support strengthening the leadership of Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) women, target Common Interest Groups (CIGs) for livelihoods support, and engage internally displaced persons to mobilize communities and implement community microprojects. The Beneficiary Feedback Mechanism (BFM) was established and implemented appropriately. Under the oversight of the PMU, the ROOs processed feedback received within their respective areas. After the establishment of the BFM, 2,412 feedback were received from beneficiaries and other stakeholders. Of which, 33 feedback were classified as complaints. Descriptions of complaints and how they were addressed were documented in the quarterly reports (ICR, para. 74). b. Fiduciary Compliance Financial Management: Arrangements on financial management were rated satisfactory in the extensive review at mid-term, as the accounting system, records, and internal control framework for financial operations were adequately established. The review also noted timeliness and quality of annual audit reports and interim financial reports. The project complied with the Loan Agreement, the Bank’s guidelines, and national legislation, as confirmed by supervision missions and independent annual audits. On the other hand, the ICR noted that the implementation of financial management experienced the following minor issues: (i) the PMU used IBRD loan proceeds to temporarily finance the GoA portion of expenditures to cover the delay in the allocation and transfer of counterpart funding to the project; and (ii) the PMU reallocated funds from the savings of Component B and C to Component A, in order to increase the number of microprojects. These practices posed the risk of accounting errors, which was well mitigated. For (i), the reimbursements of accounts were completed on time. For (ii), the independent annual audit for 2017 noted that the financial impact of reallocation of funds between components was negligible. Procurement: At project closing, procurement administration was rated satisfactory. Capacity of the procurement staff was strengthened by training provided under the project. The procurement staff’s competency and familiarity to comply with the Bank’s policies and procedures led to a successful preparation of the Procurement Strategy for Development for the succeeding project. On the negative side, the procurement post-review in 2015 found patterns in the unit price of bids submitted to the project. There was a risk that it would lead to potential collusive practices. Based on the Bank’s recommendations, the PMU revised the Project Operations Manual to introduce procedures to analyze unit prices submitted by bidders. Moreover, the PMU did not obtain the Bank’s prior clearance when it started tendering for new microprojects. As discussed in the above paragraph, the PMU aimed to maximize the number of microprojects by using the savings in Component B and C, and updated the Procurement Plan accordingly. Following the Bank’s recommendation, the updated Procurement Plan was submitted to the Bank for clearance. Furthermore, advance payments of 10 percent to the contractors were not timely made after the contracts were signed, due to the delays in the provision of counterpart funding. The advance payments were completed by 2017 (ICR, para. 78). c. Unintended impacts (Positive or Negative) Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) The project has resulted in the development of local leadership both within the community and at the municipal level. Approximately 416 members of project-related community interest groups were elected members of local municipalities. Several of them continued to use the same modalities of community participation in implementing municipal infrastructure projects. Given the country’s strong centralized form of government, this was a considerable achievement (ICR, para. 57). d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The following two lessons in the ICR stood out as important and relevant to other CDD-type projects and are presented here with some editing. Building strong community ownership and links between communities and local governments ensures that communities maintain infrastructure through active partnership with local authorities. Operations and maintenance of community-driven investments in rural infrastructure are usually inadequate after the project interventions are completed. The project established community infrastructure committees and supported them to strengthen their sense of ownership and relationships with local governments. This contributed to the finding of independent technical audits that 98 percent of infrastructure financed by the preceding and this project remained in good or fair condition after 10 years. Selecting livelihoods investments based on existing local expertise or existing market needs strengthen their commercial viability. During the project implementation, communities identified the livelihoods microprojects based on two key criteria: (i) existing expertise within communities, such as poultry incubation; and (ii) existing market needs due to the lack of supply chain, such as milk collection and fodder processing facilities. The livelihood investments selected based on existing expertise enabled the communities to initiate and sustain activities with minimal technical assistance. The livelihood investments selected based on existing market needs enabled the communities to swiftly begin providing services to other businesses and producers at the local Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RURAL INVSMT (AZRIP) 2 (P122944) level. As a result of successful selection of livelihood investments, 92 percent of community-led enterprises were active and commercially viable at project closing. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a thorough overview of the project with a concisely and well written narrative structured around the project development objective that supports the ratings. In addition, the ICR clearly presented the results chain described in the PAD, though the figure on the Theory of Change (para. 10) could have been improved by including a detailed explanation of intermediate results between outputs and outcomes, as well as critical assumptions relevant to the TOC. The ICR offers a detailed description of the implementation challenges faced by the project, how they were addressed, and of the changes brought by restructurings. Furthermore, the ICR includes a candid discussion of Bank performance. The quality of the evidence and of the analysis is solid. On the other hand, the ICR could have provided more specific explanations on "modest shortcomings in the achievement of some of the project development indicators" (para. 47) which led to the moderately satisfactory rating for outcome, as well as on a rationale on the lack of economic analysis at appraisal.. Notwithstanding these shortcomings, overall, the quality of the ICR is rated by this review as substantial. a. Quality of ICR Rating Substantial Page 18 of 18