ICRR 13595 Report Number : ICRR13595 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/19/2011 PROJ ID : P074872 Appraisal Actual Project Name : Community US$M ): Project Costs (US$M): 69.80 58.11 Development And Livelihood Improvement "gemi Diriya" Project Country : Sri Lanka Loan/ US$M ): Loan /Credit (US$M): 51.00 52.17 Sector Board : US$M): Cofinancing (US$M ): Sector (s): Other social services (30%) General agriculture fishing and forestry sector (25%) Water supply (20%) General industry and trade sector (15%) Sub-national government administration (10%) Theme (s): Rural services and infrastructure (29% - P) Participation and civic engagement (29% - P) Rural non-farm income generation (28% - P) Gender (14% - S) L/C Number : CH078 Board Approval Date : 03/30/2004 Partners involved : Closing Date : 03/31/2009 03/31/2010 Evaluator : Panel Reviewer : Group Manager : Group : Ridley Nelson John R. Eriksson IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: The overall development objective of the program was to enable the rural poor to improve their livelihoods and quality of life. The objective of Phase 1 was to target poor communities in the Uva and Southern provinces to improve their livelihood and quality of life . It would enable them to build accountable and self -governing local institutions and manage sustainable investments by achieving the following sub -objectives: (i) devolving decision-making power and resources to community organizations; (ii) strengthening selected local governments that demonstrate responsiveness and accountability to rural communities; and, (iii) working with federations of Village Organizations (VOs), private sector, and Non-Governmental Organizations on economic empowerment to increase size and diversity of livelihood . The objective as stated in the Development Grant Agreement was the same with some minor editorial differences of no substantive consequence . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project was an APL and there were five components in the original design . Component A. Village Development . (Appraisal US$49.50 million, Actual US$40.78 million). This was the largest component and included the development and strengthening of village organizations and the funding of community sub-projects. The latter sub-component supported capacity building, social and community infrastructure, livelihoods activities and an incentive fund for well performing villages . Component B. Institutional Strengthening . (Appraisal US$5.70 million, Actual US$2.70 million). This supported capacity building for local and national agencies and support organizations . The component was to introduce social and public accountability mechanisms and to include support for policy makers to develop policies, rules, systems and institutional arrangements that would enable the government to transfer funds and technical support to communities on a demand-driven basis. Component C. Innovation Seed Fund . (Appraisal US$1.30 million, Actual US$0.05 million). This supported the piloting of innovative ideas that needed experimentation before possible scaling up, including agricultural processing, value addition and information technology . Component D. Project Management . (Appraisal US$2.50 million, Actual US$4.70 million). This supported overall coordination and management of the project at national provincial and divisional levels . Component E. Village Self -Help Learning . (Appraisal US$1.00 million, Actual US$1.00 million). This was to complete the implementation of the then incomplete precursor pilot project, the Village Self -help Learning Initiative Pilot. At the time of the MTR it was decided that since all the main activities under Component E had been completed, the pilot would be mainstreamed into the first component . In March 2005 the equivalent of US $8.29 million was reallocated to a new project component, Part Z, which was added to fund a Tsunami Response . It was agreed at the time that these funds would need to be covered by the APL Phase 2 funds. Thus Phase 2 includes US$15 million partly to compensate for this reallocation but also to offset shortfalls due to the addition of the Gama Neguma Program (see below). In July 2008, a Component F was added to support the piloting of closer cooperation and possible integration of the Gemi Dirya program with the Gama Neguma Program* * Note: Gemi Dirya (meaning: "strength of a village") was the CDD program that the project supported . The Gama Neguma Program was the on-going government program with which the project was to gradually integrate through the added Component F. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost . At closing Total Project Costs were 83% of planned. Financing . After the MTR, US$4.5 million was reallocated to add to the original US$ 2.7 million project management while reducing funds to the Village Development Component . This was in response to unanticipated revisions in the government salary structure, although, in the end, only US$ 4.7 million was actually spent. The reduction in the Village Development Component was compensated for by having some sub -projects shifted to Phase 2. Borrower Contribution . Borrower financing at US$5.30 million was less than the US$11.00 million estimated at appraisal due to the priority of government funding for the tsunami and a shifting of some of the government funding directly to the Gama Neguma funding. Dates . Project closing was extended by one year to ensure continuity with the anticipated Phase 2.. 3. Relevance of Objectives & Design: Relevance of Objectives . Rated Substantial . The ICR offers somewhat limited discussion on the relevance of objectives and design . However, it says that the project objectives continue to have high strategic relevance and are well aligned with IDA's country assistance strategy. This appears to be true since the strategy focuses on growth and equity as well as the consolidation of peace. The ICR (p. 13) argues that the project was relevant in strengthening, transparency, efficiency and good governance through the connections with the wider Gama Neguma program, the IDA Reawakening Project in 2008, and the phase II Samurdhi program . Relevance of Design . Rated Substantial . The design drew (ICR p. 21) on the lessons of the earlier Village Self -Help Learning Initiative, scaling up and applying the well-developed procedures that were then further developed . The design appropriately focused on policies and investments aimed at the enhancement of livelihoods at the grassroots level and the empowerment of poorer communities to exploit opportunities for growth . The connection of the design with the existing grassroots government Gama Neguma program through the Gemi Diriya Foundation (GDF) was a sound design and an appropriate program association to ensure a local level participatory approach . Through this connection the project was relevant to the wider aims of government . The lending instrument of an APL, with two further phases planned, was a very appropriate instrument choice that supported the sustainability of such a longer -term CDD type of intervention. 4. Achievement of Objectives (Efficacy): At the Outcome level: The extent to which the project targeted poor communities in the Uva and Southern provinces, improved their livelihood and quality of life. Rated High . Overall, the project exhibited a high level of achievement as is shown by the extent to which the lessons are being transferred to other countries . The ICR attributes (p. 38) the achievements largely to the implementation capacity of the public but autonomous Gemi Diriya Foundation (GDF) established at the outset through Bank advice and support. According to discussion with the project team, this offered a valuable measure of independence while not removing the foundation from the parent line ministry . The foundation was highly successful although not without inevitable teething problems. Some 87% of district and divisional GDF teams received positive scores through the Community Score Card system. On targeting, the ICR reports that the project operated in districts with the highest poverty headcount outside the North and Eastern conflict provinces . The Unmet Basic Needs Index was used to develop lists for selection, a method of targeting poorer villages that was found to be relatively successful by the 2010 Impact Assessment Study (IAS). The incidence of political influence was found to have been low . The actual number of villages where the project was being implemented was 1,034 against the target of 1,000. The number of households participating of 184,000 was somewhat above the target of 150,000. (Once all infrastructure subprojects have been completed these numbers will rise somewhat.) Approximately 100,000 households have access to the Livelihoods Fund loans some 10,000 poorest of the poor received one -time grants. (There is some overlap in these beneficiary numbers .) Numbers benefiting from loans are increasing as funds revolve . Although no income increment targets were set, the IAS found a 41% increase in income for poor project households taking loans over the control area incomes . Poor households overall, including those who were not just loan beneficiary households, experienced income increases over the controls of 27%. These findings were confirmed through triangulation using financial farm enterprise models during the ICR mission and also through beneficiary workshops and interviews. In the youth program, nearly 50% of youths who had taken loans found wage employment with 80% of them repaying their loans. The ICR team estimated that earnings were in the range of Rp 13 to 19,000 per month, enough to support a family. It was estimated that this reduced youth unemployment by 4.2% in the project area compared to a reduction of 2.2% nationally. The Community Professional model to assist communities with the provision of professional skills proved more successful than initially anticipated . The project allocated resources directly to communities to be used by them to access technical service providers, some from within the community some from outside, supported by some project training. The ICR reports that overall this worked and community infrastructure subproject quality has therefore been of a generally high standard with assistance also from GDF staff . Under the Village Self-help Learning Initiative, 47 villages fulfilled their Village Development Plans with these subprojects completed and being operated and maintained, a large increase over the original target of 75% of the originally planned 12 villages under this particular initiative . Approximately 40% of Village Organizations under the program were functioning as registered business enterprises . In 2008, an interconnectivity program was added to the project to test a model to implement larger scale inter -village roads providing connectivity at local and regional level . This was a more ambitious proposal and there was initially some skepticism by communities that they would be given real responsibility for such large amounts of money compared with the earlier smaller intra-community subprojects. Moreover local officials were also initially skeptical about the capacity of communities to handle such a level of responsibility and funding . However, the ICR found that this increased community confidence in directly engaging local officials . Interviews suggested that stakeholders benefited from the reduced travel time enabled by such investments and lower costs and losses especially for tea, fruit and vegetables, these being products where better road surfaces and rapid collection have a high payoff in terms of quality and therefore price . There was reported evidence that land prices were increasing from this enhanced transport inter-connectivity. As might be expected, in this interconnectivity program it was more difficult to get local contributions than it was for the smaller village subprojects since some people were living closer to these connecting roads than others . There was some delay in communicating to communities the amount of contribution expected. At the output level there were the following sub -objectives that supported the overall objective above : The extent to which the project devolved decision -making power and resources to community organizations . Rated High Communities formed their own organizations . Some became registered companies for profit, generally those anticipating significant income earning activities . Others became not-for-profit companies, generally those focused more on social services. Beyond differences in tax advantages, the practical difference between these two forms of organization seems not usually to have been very great since profits retained in companies formed for profit were, in any case, reinvested for the benefit of the community . Decision-making power on how to use resources provided to communities were entirely in the hands of those communities although, as noted above, they sought technical and business advice from Community Professionals and others and they had to comply with the criteria for resource transfer. The community M&E system, with computerized records at community level that could be aggregated up to federation and government levels contributed considerably to the development of community decision -making power and the management of resources . There was substantial strengthening of community capacity, 1,473 Community Professionals were trained of whom 73% were women and 30% were poor. A majority of communities had a range of areas of expertise and were supporting a number of project communities as well as their own Village Organizations . The average CP earnings were estimated to be Rps 26,000 per person annually. Approximately 10% of Village development funds were earmarked for capacity building . The NPR noted that the volume of training may have caused some loss of enthusiasm due to the time, travel and expenses required . The project included a competitive grant Innovation Seed Fund to pilot innovative ideas with potential for scaling up . Activities included community radio, environmental proposals, cultural groups, and a web portal, that enabled access to market information and the display of products to attract business . Further technology systems with internet connectivity were set up in over 450 villages. The extent to which the project strengthened selected local governments that demonstrated responsiveness and accountability to rural communities. Rated Substantial In discussion, the Team Leader noted that this objective had been given somewhat lower priority for this first phase because it was believed that it was better to focus on building community capacity first in order to articulate the demands on the local government organizations whose strength would need to grow to meet those demands . Nevertheless, this objective was by no means neglected . The Pradeshiya Sabha was the most important and lowest level of elected local government administrative entity to be supported . The ICR documents a range of outputs including extensive training programs for local and national staff and local political leaders, visits to other countries, for example to the Society for the Elimination of Rural Poverty in Andhra Pradesh in India for local and national staff, political leaders, and VO members. There were also visits to CDD-type programs in Philippines, Brazil, Bangladesh and Indonesia. The project team considered that the training had enhanced community skills in both technical and financial areas. The performance of the project suggests considerably increased interaction between local governments and village communities . The comprehensive computerized M&E system developed, alongside more traditional methods such as the use of village notice boards displaying investment plans and proposed actions, also ensured community pressure on the local governments to adhere to their responsibilities and thus to be accountable . As found in other countries, elected members in local government often overlapped with elected people on village committees . The extent to which the project worked with federations of Village Organizations (VOs), private sector, and Non-Governmental Organizations on economic empowerment to increase size and diversity of livelihoods . Rated Substantial . About 48% of VOs were in federations (ICR p. 28). Some 55 Support Organizations were contracted to assist in the basic mobilization, community planning facilitation and basic organization . There were some problems with the quality of staff recruited by these organizations and considerable staff turnover . The project assisted with training for organizational management and other areas . An independent evaluation of the performance of these organizations in 2009 found that about 70% of VO members expressed satisfaction with the work carried out by these organizations but they did note weaknesses in the depth and facilitation of support . 5. Efficiency (not applicable to DPLs): Rated High . The overall ERR was estimated by the ICR at 30.5%. (No ERR was done ex ante.) This included the costs of mobilization and management . The evidence for the analysis came from the project database, the impact assessment, data from the infrastructure unit of GDF, comparisons with other programs, secondary data sources, the ICR field missions findings, and supervision reports . All project costs were included . In looking at cereal crop benefits, income impacts, efficiency impacts and equity impacts were analyzed . The ICR notes that the assumptions were conservative, for example, limiting livelihoods income generation at the levels found at the closing of the project when in fact they are expected to continue to grow, and valuing infrastructure benefits only in terms of cost savings . It is expected that the project institutions and processes established will lead to the number and size of activities expanding and diversifying in future . The ICR (p. 49) offers an instructive analysis of a VO case of resource allocation "convergence", demonstrating the complementarity between the infrastructure and livelihood investments . In this VO, the first priority was given to the extension of irrigation water to an additional 40 acres. This was then followed by a connecting road to the main road benefiting 80 acres. A variable user fee, depending on irrigation water used, was collected for the maintenance of the infrastructure. In addition to these investments, some two thirds of the Village Savings and Credit Organization funds were directed towards high -value vegetables benefiting 120 farmers. The estimated level of savings on infrastructure costs compared to the equivalent government constructed works was 36%. For government, in financial terms, there were also savings due to the 30% contribution in cash and labor from communities. Local government officials estimated that the quality of completed works was high and likely to give an approximately 25% longer lifespan than equivalent government works . It is expected that this, together with the improved beneficiary incomes, will position communities to be better able to contribute the O&M funds needed . Although there were no targets, the ICR reports that community led investments in skill development achieved cost efficient job placement for unemployed youths with an average cost per job created of only Rs .12,500, only a little over US$100, and, with 58% of skill development loans already repaid, leading on to further job generation . Although there does not appear to be clear quantitative evidence on leakage, the ICR argues that the strict adherence to rules, accountability, community decision -making, social inclusion and community supervision functions reduced leakage and waste and offered further savings in costs . Finally, not noted in the ICR under the heading Efficiency but discussed elsewhere and noted in discussion by the project team, the impact of the project's demonstration effect both within and outside Sri Lanka, although difficult to quantify, should be factored into both this national level efficiency analysis and, notionally, into a larger global one . The project has had some influence on a number of other programs . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate Yes 30.5% 75% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: This project exhibits substantial relevance of objectives and design and high efficacy and efficiency . This review gives the highest weighting to the overall outcome objective and to the first output objective since, although there is some overlap between the two, these focus on the broader intent to empower communities, to target the poor and to enhance livelihoods. The project was innovative and well monitored and had significant unintended benefits in terms of demonstrating an innovative CDD type of project elsewhere in Sri Lanka and to other countries . While there is some looming political uncertainty about risk and sustainability, there are no obvious alternative project designs or implementation strategies that could have reduced this uncertainty . Moreover, the success of the project gives some reason to be hopeful that threats to its continued success as it moves into the next phase will not eventuate . It has been an exceptional learning experience for the Bank and the borrower . a. Outcome Rating : Highly Satisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR, very frankly, rates risk high . On balance, this review concurs but there are positive factors that may suggest this rating is somewhat too negative . A rating of high risk may seem surprising for such a successful project but recently the program has been transferred to the Ministry of Economic Development and there may be political pressures to accelerate the integration into the regular state apparatus too fast and with insufficient care . There are potential turf battles that could disrupt the highly promising progress up to this point . This is notwithstanding borrower assurances that there will be no change in the "scope, components, and modality of implementation " of phase 2. Other issues are that the program is inevitably becoming more complex as it moves towards higher level linkages and this will test relationships between federations, savings and credit organizations, community learning centers, and the public agencies. More positively however, the Bank itself has proven in the program so far to be nimble and innovative in adapting the project as it progressed . This gives hope that the Bank and borrower can work together to develop a viable future institutional pathway provided the Bank can continue to apply the needed political skills in shepherding this transition. a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: Quality at Entry . Overall, this has been an exceptional achievement . On Quality at Entry, the ICR rates it satisfactory . The key creation of the public foundation was instigated by the Bank and has been a key to success . The safeguards put in place and the risk assessment were satisfactory . The design drew from past experience both outside and within Sri Lanka. Quality of Supervision . The Bank applied valuable skills to supervision . The ICR reports regular supervision including workshops and cross-country visits. Missions appear to have been well staffed with appropriate skills and to have been pro-active in addressing issues . There is evidence of strong adaptation as the program progressed and the creative use of experience from elsewhere and the sharing of the project experience . The ICR notes that supervision could have given more attention to some aspects of M&E, tracking the emerging PDO outcome levels. There were some complaints that consultants imposed heavy data burdens on the borrower and did not always disseminate findings. at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government Performance . The borrower exhibited commitment to the Community Driven Development (CDD) concept. It supported the Gemi Diriya Development Foundation and accepted its autonomy . It went further (ICR p. 22) by introducing some of the approach in other public programs . It has fully supported the development of the phase 2. There were some delays in early recruitment. According to the ICR, the Board of GDF did not inject enough on policy, focussing more on administrative issues . Implementing Agency Performance . The ICR finds the Implementing Agency had only minor shortcomings . There were some issues with clarity of functions and staff roles and in capacity but these were rectified .There were some weaknesses in M&E but there were challenges, in particular difficulties with the streamlining of information flows across agencies was to be expected in such a project . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Design . Design included participatory monitoring, an MIS system, a baseline, an end of project impact assessment, six-monthly independent reviews of processes and procedures, and mechanisms to capture process and learning aspects. The ICR notes that greater emphasis on livelihoods, rapid appraisal and communication of impacts could have facilitated more learning. The design included surveying of control communities enabling some evidence on attribution. Implementation . The ICR finds that the main elements of the monitoring, learning and evaluation system were implemented . However, there were some weaknesses . The baseline was done at the MTR stage, delayed partly due to contracting problems, howevert a "proper survey" was produced. As volume of data increased initially staff were over -stretched with a paper and spreadsheet system . This then shifted towards a more sophisticated MIS system but the process of design of this (done during the implementation phase ) took longer than expected . The inputting of data was a large task which tended initially to overshadow the output of analyzed data, a common problem in new M&E systems . Trained community youths were inputting data at village level but even by the time of the ICR there were no specific analyzed outputs from the new system at village level although the data itself seems to have been accessible . The monitoring of livelihoods changes and qualitative factors proved difficult . However, less formally, according to the project team, CPs did provide some useful process feedback to peers in other villages . Utilization . The M&E was utilized at a number of levels . Village level participatory monitoring has been useful within communities with community capacity to monitor expenditures and achievements, sufficient apparently (according to the TL) to enable notices of project activities to be posted on community notice boards . The findings were an important input into the ICR. a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards . No major safeguard issues were anticipated . An Environmental and Social Management Framework was put in place with screening, checklists and capacity building . Some issues were identified with lack of drainage and erosion . Only about 60% of Environmental Management Plans were being implemented . There was some lack of capacity at field level. CPs were later mobilized and trained to better address environmental issues . There was no involuntary land acquisition. Most land was provided by government at both higher and lower levels and by temples . About a quarter was donated by private donors, mostly large landowners . No complaints were found. However, there was a need for a better land acquisition database . It had been found that OP 4.10 would not be triggered due to few tribal groups and none were found in the Phase 1 area. However, the project did adapt manuals for Tamil and other groups in plantation areas but these were not indigenous groups under the OP . Fiduciary . The ICR found that financial management and procurement had been satisfactory . VOs who were companies had to do annual audits and most did. A Governance and Accountability Action Plan review (GAAP) in 2008 found that "good governance principles have been infused in (the program's) core values and embedded in project policies and procedures". Unintended Impacts . The project had a number of positive unintended impacts : land prices rose in some areas, there were shortened routes to markets, and there was greater production of certified paddy seed from the villages . The project also contributed to the peace process by extending the project approach to conflict areas . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Highly Satisfactory Borrower comments in the ICR ask why this project was not rated highly satisfactory (or excellent in their parlance). This was a valid question. This review finds Relevance to be substantial and Efficacy and Efficiency to be high. There were very few weaknesses for such a challenging project and to counteract those, the project evolved and innovated as it progressed and has provided a valuable learning experience for both the borrower and the Bank. Given some recent political shifts there is a high risk for the next phase , which the ICR has been open about. IEG agrees with the borrower assessment. Risk to Development High High Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Exemplary NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR offers a number of lessons of which the following four, with editorial and some substantive adaptation, are the most important: 1. Continuity of community support up to a defined point of graduation that includes mechanisms for involvement of the poor and the marginalized, is important for longer term sustainability of participatory community development processes. 2. The use of experienced community leaders to transfer knowledge to other communities is very effective for scaling up. 3. Synergy between productivity, livelihoods and infrastructure investments, carefully designed to complement each other, can offer high community development benefits . 4. A relatively autonomous implementing agency allowing greater flexibility than a line agency may be very valuable for testing new participatory models but the roadmap of its evolution towards sustainable institutionalization, i.e. folding it back into a government institution, needs to be planned in advance and then later, if necessary, adapted as the political and economic environment evolves . 14. Assessment Recommended? Yes No Why? The apparent success of this program may offer significant lessons for wider Bank work in CDD types of projects. However, the transition to main-streaming in the institutional sense still lies ahead . This may offer further lessons. Any IEG PPAR would need to be done with perhaps two years delay to pick up this transition . 15. Comments on Quality of ICR: While it is often easier to write an ICR on a good project, this ICR is exceptional in its coverage, analysis, and openness. It offers valuable lessons and a strong efficiency analysis including a useful and rare discussion of investment convergence (complementarity of investments). The only small weakness is the somewhat brief Relevance section although some of the evidence for that section is covered later under Quality at Entry . a.Quality of ICR Rating : Exemplary