ICRR 13217 Report Number : ICRR13217 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/12/2010 PROJ ID : P073973 Appraisal Actual Project Name : Village Investment US$M ): Project Costs (US$M): 19.33 21.85 Project Country : Kyrgyz Republic Loan/ US$M ): Loan /Credit (US$M): 15.10 15.65 Sector Board : ARD Cofinancing (US$M ): US$M): 0.87 1.00 Sector (s): Sub-national government administration (100%) Theme (s): Participation and civic engagement (20% - P) Small and medium enterprise support (20% - P) Rural services and infrastructure (20% - P) Rural non-farm income generation (20% - P) Decentralization (20% - P) L/C Number : CH070 Board Approval Date : 12/16/2003 Partners involved : DFID Closing Date : 08/31/2008 03/01/2008 Evaluator : Panel Reviewer : Group Manager : Group : Hassan Wally John R. Heath IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: "The overarching aim of the project is to contribute to the alleviation of rural poverty . It seeks to achieve this by pursuing three specific objectives : (a) improving governance and capacity at the local level; (b) strengthening the provision of, and access to, essential infrastructure services; and (c) supporting private small-scale group enterprise development" (Project Appraisal Document, p. 2). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (i) Capacity building and empowerment (Estimated Cost at Appraisal US$ 2.83 million, Actual Cost US$ 1.99 million ). Support local capacity building and promote community empowerment and the government's decentralization program. Promote capacity building in social mobilization, strategic planning, micro -project implementation, and good governance . The component focused on three groups : members of participating communities and community based organizations (CBOs), local self-government officials, and partner NGOs . ii) Village investments (Estimated Cost at Appraisal US$ 13. (ii) 18 .15 million ). Support a 13 .36 million, Actual Cost US$ 18. facility and mechanisms for delivering financial and technical support for community -based initiatives around which villagers and local authorities will collaborate in an ongoing process of strategic planning, investment selection and micro-project implementation. Promote a participatory and fully transparent process through which villages will identify and prepare micro-projects that address their needs . iii ) Project management (Estimated Cost at Appraisal US$ 1.64 million, Actual Cost US$ 1.7 million ). Provide (iii) support for the overall management of the project by the Community Development and Investment Agency (ARIS). Funding will be provided to key agency staff, consultants, essential equipment, operating costs, technical assistance, training, office refurbishment and to contract local technical expertise as needed to screen micro -project proposals for their technical and financial soundness and to facilitate and monitor their implementation . In addition, funding will be provided for information dissemination, beneficiary assessments, periodic surveys, participatory M&E activities, a management information system (MIS), project audits, and periodic meetings of the national and oblast level project consultation groups. Also, participating communities will receive support in the initial year from management expenditures. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The project closed about 6 months earlier than expected. It was able to cover more communities than envisaged at appraisal owing to the larger than expected volume of community contributions mobilized after 2005. In all, US$5.8 million of community contributions were raised, double the appraisal estimate (US$2.8 million). 3. Relevance of Objectives & Design: Objectives . (Rating: High). The objectives were consistent with the trend of government policy in the Kyrgyz Republic, which was "one of the first countries of the former Soviet Union to initiate moves towards decentralization " (PAD, p. 5). In 1991 the government changed the law, setting up local administrations with real powers, accountable to central government. Government's decentralization thrust stalled during project implementation (see Section 9 below) but, in the longer term, it remains an important priority for Kyrgyz (and the Bank's) development strategy. The project objectives reflect two priorities discussed by the 2003 CAS which emphasized provision of essential services and strengthening of the governance framework . Also, the objectives are consistent with the current Joint Country Support Strategy (2007-2010) which states that " Fiscal decentralization will be pursued actively, including by giving local communities greater authority over spending priorities, and strengthening their budgets so that poverty reduction can be achieved more effectively on the ground " (JCSS, p.16). Design . (Rating: High). Valid reasons are cited for rejecting alternative designs . "A rural infrastructure project, managed by a central ministry was judged to be too narrowly focused and too detached from locally perceived needs. A Social Investment Fund operation would lack a sufficient employment and income creation dimension " (PAD, p. 20). The choice was made to implement the project directly through grassroots institutions, rather than though a central government ministry, to have communities and community groups plan and implement micro-projects, and to channel the funding directly to these grassroots groups . The establishment of an enabling political and legal environment provided favorable conditions for the project . The government was committed to decentralization and strengthening the role of local governments and community -based organizations. The ICR notes that the project preparation team drew lessons from other community -driven development (CDD) projects and drew on lessons from other donor-funded projects (ICR, p.7). Also, the gradual phase-in of the project allowed sufficient time for building up capacity and commitment . The project preparation process involved a Social and Institutional Assessment (SIA, 2002) which reviewed poverty data, analyzed existing legislation on community -based organizations and reviewed the formal and informal roles of community leaders and institutions (ICR, p. 8). Overall, and given the widespread poverty in rural areas, the design of VIP was appropriate for delivering tangible benefits to the rural poor and for mobilizing local communities . Overall, relevance is rated High. 4. Achievement of Objectives (Efficacy): The overarching aim of the project was to contribute to the alleviation of rural poverty . While attribution to the project is impossible to determine, project sponsored infrastructure and services probably contributed to the fall in the rural poverty rate from 55.5 percent in 2004 to 41.7 percent in 2007 (ICR, p.15). The income generating micro-projects created 4,662 permanent jobs. (i) Improving governance and capacity at the local level . (Rating: High). By project completion the number of village administrative bodies that demonstrated the capacity to prioritize, plan, design, implement, operate and maintain micro-projects reached 475, compared to an original target of 200 (ICR, p. iv). The ICR (p. 15) notes that the project achieved all its targets in social mobilization and capacity building . The project supported local institutions as well as the empowerment of local self-government bodies. As a result of the project activities, 3,700 training courses were conducted with the participation of 75,000 people. Also, 475 Local investment Union Executive Committees (LICs), 1600 Village Investment Union Executive Committees (VICs) and more than 4,500 Micro-Project Groups (MPGs) were elected by villagers and empowered by ARIS through training and learning by doing . The staging of elections and the delivery of a substantial volume of training helped to build local capacity and accountability . ii) Strengthening the provision of, and access to, essential infrastructure services . (Rating: High). With respect to (ii) infrastructure the total number of completed micro -projects was 3,180 compared to the target of 2,000 (ICR, p. iii). Of all the micro-projects financed, 73 percent were for social and economic infrastructure . These sub-projects included: improving access to drinking water, increasing land productivity by improving irrigation and drainage, rehabilitation of schools and kindergarten facilities by installing heating systems, new roofs and floors, and improving access to electricity and telecommunications (ICR, p. 15). The project provided 1,661 villages (87 percent of the country) with improved access to infrastructure (ICR, p. iii). A Beneficiary Impact Assessment (BAI) found that according to 44 percent of the respondents in all target groups, the most significant benefit derived from the micro -projects is the improved access to essential services (ICR, p. 25). iii ) Supporting private small -scale group enterprise development . (Rating: Substantial). Between 2004 and 2007 (iii) the project supported 1,164 income generating micro-projects and created 4,662 workplaces (no target was set at appraisal) (ICR, p. iii). The income generating micro-projects supported rural entrepreneurs through the provision of grants to support small-scale businesses such as bakeries, flour mills and artificial insemination stations . The BAI results show that 70% of the respondents rated the micro -projects as successful (ICR, p. 25). Efficacy is rated High. 5. Efficiency (not applicable to DPLs): The PAD (p. 57) provides an estimated average financial rate of return on micro -projects of 41 percent. The ICR (p. 28) puts the weighted average financial rate of return for a random sample of 185 subprojects at 164 percent. Also, the ICR (p. 29) notes that 30 percent of the analyzed subprojects (57 out of 185) were found to be financially unviable. IEG has some queries about the efficiency analysis in the ICR . Some of these sub-projects (e.g. schools) do not lend themselves to financial rate of return analysis . Also, based on a a spreadsheet supplied to IEG by the project team, it is apparent that 43 of the sample of 185 projects had insufficient data to enable financial rate of return calculation . IEG deems that this is more a reflection on the appropriateness of the type of analysis employed than it is on the efficiency with which project resources were deployed . The following is compelling evidence for efficiency . The project was able to cover more communities than expected but with a less than proportional increase in project costs, reflecting the success in mobilizing beneficiary contributions. The project was initially designed to serve 200 villages, but reached 475 and, as reflected in the efficacy rating, outputs were much higher than expected . Also, the project was completed six months ahead of schedule. Based on these considerations, efficiency is rated substantial. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 41% 83% ICR estimate Yes 164% 83% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Ratings of high relevance, high efficacy and substantial efficiency add up to an outcome rating of highly satisfactory . a. Outcome Rating : Highly Satisfactory 7. Rationale for Risk to Development Outcome Rating: Strong local-level commitment and capacity building provided under the project, plus the satisfactory quality of the works installed increase the prospects for sustainability . A survey of a random sample of 153 subprojects completed in 2005 (15 percent of the total) shows that on a scale of 100 subproject sustainability prospects averaged a score of 83 (ICR, p. 14). Caveats are in order. Ultimately, responsibility for maintaining the public goods that were built (e.g. schools) will rest with local authorities. Various village-level action plans were prepared and these made provision for long-term maintenance. But the ICR notes (p. 19) that the transfer of responsibilities has so far been sporadic . For this reason risk is rated significant . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Quality at Entry . The thorough preparation of the project design and the gradual phase in of the project after securing strong government support and stakeholder commitment contributed to quality at entry . The project team prudently resisted pressure to scale -up too quickly, waiting for completion of lesson -learning from the pilot phase (ICR (p. 19). Quality of Supervision . Supervision intensity was adequate and there was substantial continuity : the project had a single task team leader for five out of six years . The project team developed a close and fruitful working relationship with the implementing agency . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government Performance . Government support for decentralization stalled during implementation . Also there were delays in government counterpart funding . However, the ICR (p. 20) notes that the government remains supportive of VIP activities and has allocated US$ 1.0 million of counterpart funds to the VIP -2 project. Implementing Agency Performance . The ICR (p. 11) notes that the implementing agency (ARIS) handled the various aspects of management competently, including financial, engineering, procurement, staff training and personnel management. ARIS developed an excellent working relationship with the Bank team . The requests by other donors that ARIS manage their projects reflects its success as an implementing agency . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Highly Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design . The PAD (Annex 1) identifies key performance indicators and reflects monitorable outcomes and impacts in key areas of the project. The PAD proposed 20 indicators which according to the ICR (p. 9) seem "too many to provide meaningful assessment or guidance ." Some indicators were inadequate, failing to reflect the outcome of the project activities. Information supplementary to the ICR indicates that sound provision was made for assessing village-level changes in the course of the project . Implementation &Utilization . In each village baseline information was collected, assembled and stored in the ARIS M&E database. This baseline data was compared with data collected subsequently at regular intervals, allowing for the assessment of changes . Every six months poverty data were collected and added to the database of the National Statistics Commission. Also, an impact assessment was carried out in 2007 which included villages not yet included in the project with villages that had already participated for several years . The ICR (p. 13) notes that the M&E system played a critical role in project management . The management information system developed by the project was adopted by various government structures as well as by other donor funded projects . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards . During the project appraisal the only safeguard deemed applicable was an Environmental Assessment (OP 4.01). The ICR (p. 13) reports that "mitigating measures and monitoring activities were performed as prescribed and the safeguard policy has been satisfactorily complied with throughout the project ." Fiduciary . The ICR (p. 13) highlights the satisfactory financial and procurement management throughout the project life and reports no fiduciary issues . Unintended outcomes . The ICR (p.18) discusses a number of unintended positive outcomes . For example, community contributions were double the expected amount (US$ 5.8 million compared to US$ 2.9 million), allowing more investment in infrastructure improvement . Also, the enthusiasm and local support for the project led to the expansion of the project activities to cover 100 percent of the country. The project activities generated high interest on behalf of other donors which brought significant amounts of cofinancing to VIP -1&2. Finally, the project activities increased the availability of specialists skilled in community mobilization and micro -projects. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Highly Satisfactory Highly Satisfactory Risk to Development Moderate Significant Assumption of maintenance Outcome : responsibilities by local government has so far been sporadic. Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Building partly on the lessons in the ICR, five lessons are emphasized : Sound project design creates a strong foundation for future success . The VIP project preparation team successfully put together a project design that benefited from research work, previous CDD lessons and a pilot phase that preceded the main project . The design insured the commitment of the government and different stakeholders and successfully adopted mechanisms compatible with country conditions . Sound institutional building entails avoiding nepotism and favoritism in personnel selection . The creation and success of ARIS as a management body for the project activities clearly demonstrated that personnel selection is critical in building a credible and professional organization that can successfully manage a project . Communities can mobilize significant contributions . The VIP activities appealed to community ownership and provided a voice for the local population to decide what subprojects best matched their needs . This approach translated into higher contribution from communities - almost double the expected amount . Appropriate baselines and M&E are critical for gauging success and progress . Projects aimed at capacity enhancement and community participation should explicitly and systematically monitor and evaluate the extent to which these are achieved using both quantitative and qualitative measures and appropriate baselines. Sustainability considerations should be thoroughly addressed from the beginning of the project . The biggest challenge for projects of this nature is to ensure funding for operation and maintenance (O&M) of infrastructure works. The public good aspects of O&M need to be fully funded by the government, with transfers from central to local authorities where appropriate . 14. Assessment Recommended? Yes No Why? To assess the impact and the sustainability of the project activities and provide more information on the costs and benefits of CDD projects such as this . 15. Comments on Quality of ICR: The ICR is clear and well written. It provides thorough coverage of project achievements and a candid report on the shortcomings.The ICR gives a comprehensive account of the beneficiary assessment and stakeholder workshop organized as part of the completion process . However, the sections on risk to development outcome and monitoring and evaluation could have been more comprehensive and the analysis of efficiency could have been more rigorous . a.Quality of ICR Rating : Satisfactory