Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) Report Number : ICRR0021545 1. Project Data Project ID Project Name P115247 GH-Social Opportunities Project (FY10) Country Practice Area(Lead) Additional Financing Ghana Social Protection & Labor P146923 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-47350,IDA-54660 30-Jun-2016 132,636,652.39 Bank Approval Date Closing Date (Actual) 20-May-2010 31-May-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 88,600,000.00 0.00 Revised Commitment 137,914,078.58 0.00 Actual 132,636,652.39 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Judyth L. Twigg Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD, p. viii) and the Financing Agreement of September 8, 2010 (p. 5) the objectives of the project were “to improve targeting in social protection spending, increase access to conditional cash transfers nationwide, increase access to employment and cash-earning opportunities for the rural poor during the agricultural off-season, and improve economic and social infrastructure in target districts.” Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) As part of a May 2014 restructuring, the objectives were revised: “to improve targeting of social protection programs and provide income support to poor households through Livelihood Empowerment Against Poverty (LEAP) Grants and Labor-Intensive Public Works (LIPW) infrastructure in targeted districts.” Outcome targets were all revised upward. The project’s original first three objectives were similar to counterpart elements of the revised objectives and are assessed together in Section 4. The project’s fourth original objective is assessed separately. It should be noted that the scope of the first two original objectives was nationwide and the third and fourth original objectives focused on target districts, while all revised objectives focused on target districts. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 27-May-2014 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The original project included five components: Component 1: Rationalize National Social Protection Policy (appraisal estimate US$2.5 million, actual US$9.79 million): This component was to finance technical assistance, studies, training, and Secretariat services to support the government in redirecting social protection expenditures to the most effective areas and reduce those in less effective activities. Capacity was to be built within the Ministry of Finance and Economic Planning (MoFEP) and the Ministry of Local Government and Rural Development (MLGRD). At the May 2014 restructuring, this component’s name was revised to Social Protection Policy and Systems Strengthening. The component received additional financing of US$ 9 million to implement two sub-components: i) Social Protection Policy Formulation: the sub-component was to strengthen the coordination and implementation of social protection programs through the preparation of a national social protection policy, strategy, and implementation action plan; ii) Social Protection Systems Strengthening: The sub-component was to support the strengthening of social protection systems through the establishment of the Ghana National Household Registry (GNHR) to improve the efficiency and effectiveness of social protection interventions. Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) Component 2: Labor Intensive Public Works (LIPW) (appraisal estimate US$56 million, actual US$66.44 million): This component was to provide targeted rural poor households with access to employment and income-earning opportunities through rehabilitation and maintenance of public or community infrastructure in northern Ghana to respond to external shocks and the agricultural off-season from November to March/April. The aim was to maximize local employment while rehabilitating productive infrastructure assets through establishing a LIPW-based social protection scalable instrument, which was to provide quick-response mechanisms against external shocks, such as floods or droughts, during a crisis. The payment modality for LIPWs was to be cash. The scope of works eligible for LIPWs were to be defined based on labor content and scope to generate significant local employment. At the May 2014 restructuring, the component’s name was revised to LIPW Implementation and Capacity Building. The component received additional financing of US$19.2 million to support the scale- up of LIPW to reach approximately 173,000 additional beneficiary households. Several modifications of the component, including to manage oversubscription, improve targeting, and enhance delivery and monitoring of payments, were introduced. Also, system delivery and accountability were strengthened through the introduction of a management information system (MIS) and electronic payments system. Capacity building activities for LIPW were moved from component 5 to this component. Component 3: Livelihood Empowerment Against Poverty program (LEAP) (appraisal estimate US$20 million, actual US$32.21 million): This component was to support the full rollout of the pilot phase of the LEAP program by strengthening its management and administration, providing technical assistance to improve targeting, providing cash transfers to beneficiaries under LEAP, and providing incentives to ensure that the government’s annual budget allocations were sufficient through a results-based financing instrument with disbursement-linked indicators (DLIs). At the restructuring in May 2014, this component’s name was revised to LEAP Implementation and Capacity Building. The component received additional financing in the amount of US$15.7 million to provide LEAP grants to an additional 250,000 beneficiary households. The scale-up included deepening coverage in the existing districts with higher poverty incidence. In addition, this component was to strengthen the program’s operations at the national, regional, and district levels and improve systems delivery through the introduction of MIS and e-payment systems (which were previously under component 5). Component 4: Capacity Building (appraisal estimate US$4.1 million, actual US$6.64 million): This component was to build capacity at the national and local levels to implement the National Social Protection Strategy (NSPS) in selected project districts, with the view of enabling gradual scaling up and targeting at the national level. Several capacity-building activities were to be supported: i) establishing a LIPW-supportive policy and institutional framework; ii) building capacity to support LIPW implementation; iii) building capacity to support implementation of LEAP; iv) building district and regional capacity; and v) other activities and pilot programs that were to become necessary during implementation to support the overall objective of strengthening safety nets and providing safety ladders, including building capacity for disaster risk management. This component was dropped when the project was restructured in May 2014. Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) Component 5: Project Management and Coordination (appraisal estimate US$6.5 million, actual US$18.72 million): This component was to finance project management and coordination; relevant technical assistance; project monitoring and evaluation; consultancy services; communication and training; equipment and vehicles; and incremental operating costs. In addition, the component was to include the cost of annual impact evaluation surveys, including a baseline study and an end-of-project comprehensive impact evaluation study. At the May 2014 restructuring, this component became component 4, as the original component 4 was dropped. The component received additional financing in the amount of US$6.1 million. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$89.1 million. In May 2014, the project received Additional Financing in the amount of US$50 million. Actual cost was US$133.8 million. Financing: The project was to be financed by a IDA credit of US$88.6 million, an additional credit of US$50 million, and Borrower contribution in the amount of US$500,000. Borrower Contribution: The Borrower was to contribute US$500,000 through in-kind community contribution, which were exceeded by the government and local communities. Dates: The project was restructured three times: • On September 24, 2014 the project was restructured to reallocate funds between disbursement categories to roll out the National Targeting System and strengthen support to districts under LIPW. • On May 27, 2014 the project received Additional Financing to scale up the existing interventions under LIPW and LEAP, modify the PDO, and revise the Results Framework and project activities. The use of DLIs was discontinued because the system as established -- to provide financial incentives to the unified treasury account to ensure that the government would allocate sufficient budget for LEAP to meet its targets -- did not function. In addition, the newly created Ministry of Gender, Children, and Social Protection (MoGCSP) was added as implementing agency. The project’s closing date was extended from June 30, 2016 to June 30, 2017. • On June 20, 2017 the project’s closing date was extended from June 30,2017 to May 31, 2018 to allow for the completion of data collection for the Ghana National Household Registry (GNHR) and for LIPW sub-projects. 3. Relevance of Objectives Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) Rationale Despite Ghana’s remarkable progress in decreasing its national poverty levels from 52 percent in 1992 to 29 percent in 2006, at appraisal the country had been experiencing an accelerating geographically-based socioeconomic gap. According to the PAD (p. 1), while the three northern regions made up about 18 percent of the total population of Ghana, their national share of the poor was about 40 percent at the time of project appraisal. Over one-quarter of all 38 districts had poverty levels above 80 percent, and these districts were located predominantly in the Northern Region, Upper East Region, and Upper West Region, with pockets in the Coastal and Forest Zones of the Central Region. There were many reasons for under-development in these areas, including being landlocked, poorly developed transport structure, a pre-disposition to droughts and floods, and soil and land degradation. These factors had a negative impact on agricultural production and those people depending on it. In 2007, the government of Ghana put in place a comprehensive NSPS that represented its vision of creating an all-inclusive and socially-empowered society through the provision of sustainable mechanisms for the protection of persons living in situations of extreme poverty and related vulnerability and exclusion. The original and revised objectives of the project supported this strategy. Also, the original and revised objectives were in line with the country’s Growth and Poverty Reduction Strategy, which aimed for Ghana to achieve middle-income status and the Millennium Development Goals, in addition to ensuring that the benefits of growth were shared across the population. The original and revised objectives were in line with the Bank’s Country Partnership Framework (2013-2018) that was in place when the project closed, and which focused under pillar III on protecting the poor and vulnerable. The revision of the objectives did not make them less relevant, but streamlined them and their links to the main components of the project. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve targeting in social protection spending (original objective) / Improve targeting of social protection programs (revised objective) Rationale The project’s theory of change linked the conducting of a rationalization study, the approval of a social protection policy by the cabinet, and the focus on inclusion of beneficiaries from the poorest 20 percent (for Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) LEAP) and the poorest 30 percent (for LIPW) of the population with improving targeting in social protection spending/social protection programs. Outputs • The project financed the development of a rationalization study to review expenditures on social protection with a view of redirecting parts of national resources toward the poor and vulnerable. The study suggested scaling up the LEAP program and identified 11 social protection programs (out of 44 programs that were included in the study) for the government to increase coverage to the poor and vulnerable, with an estimated cost of about 2.1 percent of the Gross Domestic Product (GDP). Outcomes • In May 2014, the percentage of social protection spending allocated to programs targeted to the poor was 30.35%, surpassing the target of 6%. This indicator lacked a baseline. This indicator was dropped during the restructuring, as data were difficult to collect. • LEAP and Ghana Health Insurance indigent exemption expenditures in the government's overall package of pro-poor expenditures, as defined by the MoFEP, was 0.73% in May 2014, not achieving the target of 3%. The baseline for LEAP was 0.19%. The health insurance exemption part of the indicator lacked a baseline. This indicator was dropped at the restructuring. According to the ICR (p. 17), the target value was achieved before the project closed, but no specific information was provided. • The project’s impact evaluation showed that between 2014 and 2018, the percentage of LEAP beneficiaries who were from the poorest 20 percent of the population increased from 19 percent in 2014 to 46.3 percent in 2018, not quite achieving the target of 50 percent. • According to the project’s impact evaluation, the percentage of LIPW beneficiaries who were from the poorest 30 percent of the population increased from 39 percent in 2014 to 86.9 percent in 2018, surpassing the target of 50 percent. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Increase access to conditional cash transfers nationwide (original objective) / Provide income support to poor households through LEAP grants (revised objective) Rationale Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) The project’s theory of changed linked the identification of districts to participate in LEAP programs, the conducting of sensitization activities and enrollment of LEAP beneficiaries, the administration of proxy means testing to identify LEAP beneficiaries, and building capacity to institutionalize LEAP with increasing access to conditional cash transfers nationwide and provision of income support to poor households through LEAP grants. Outputs • The project supported the expansion of a government-funded LEAP pilot. Beneficiaries were sensitized to send children between the age of 5 and 15 years to school and to ensure regular clinical visits and full immunizations for all children in beneficiary households. However, according to the ICR (p. 18), the project did not implement penalties if the beneficiaries did not comply. • A social protection policy was drafted by the MoGCSP and submitted to the cabinet, achieving the objective of doing so. Outcomes • 944,000 people directly benefited from the cash transfers, exceeding the original target of 80,000 and revised target of 200,000 beneficiaries. 55% of the LEAP beneficiaries were female, surpassing the target of 25% but not meeting the revised target of 70%. • An impact evaluation conducted in 2013 found that LEAP increased school enrollment among secondary school age children and reduced grade repetition among both primary and secondary school age children, but no specific data were provided in the ICR. Absenteeism among primary school age children was reduced by 10 percentage points. Preventive care among girls between the age of zero and five was increased by 15.7 percentage points. • 78 percent of beneficiaries were satisfied with the LEAP program, surpassing the target of 70 percent. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Increase access to employment and cash-earning opportunities for the rural poor during the agricultural off- Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) season (original objective) / Provide income support to poor households through LIPW infrastructure in targeted districts (revised objective) Rationale The project’s theory of change linked sensitizing and enrolling beneficiaries to participate in LIPW, as well as the training of district assembly monitoring teams, with increasing access to employment and cash-earning opportunities for the rural poor during the agricultural off-season and providing income support to poor households through LIPW infrastructure in targeted districts. Outputs • The project provided rural poor households with access to employment and income-earning opportunities through the LIPW program, especially during the agricultural off-season between November and March/April and in response to external shocks. 902 LIPW sub-projects were completed in 1125 communities. Among these, 250 climate initiatives were completed, with 2,268 hectares planted with fruit trees and multi-purpose woodlots, almost achieving the target of 2,360 hectares. 263 small earth dams and dug-outs were rehabilitated, not achieving the target of 300. 1,340 kilometers of road were rehabilitated, surpassing the target of 700 kilometers. • 60 maintenance programs were established for completed sub-projects, achieving the target of 60 programs. • The number of private contractors technically capable of implementing and supervising LIPW increased from 74 contractors in 2014 to 284 contractors in 2018, surpassing the target of 125 contractors. The percentage of completed sub-projects inspected annually by communities under the supervision of a district technical specialist increased from 50 percent in 2010 to 87.4 percent in 2018, almost achieving the target of 90 percent. • 35 percent of payments to workers under LIPW program were made within 20 days after the attendance at the work sites, not achieving the target of 80 percent. The average time lapse between submission of interim certificates by LIPW contractors and payments by designated accounts decreased from 30 days in 2013 to 24 days in 2018, not achieving the target of 20 days. Outcomes • 13.6 million person days of unskilled work disaggregated by district were achieved, surpassing the original target of 1.5 million and the revised target of 9.5 million. One person-day of work referred to LIPW tasks expected to be performed in 6 hours. In 2016, an independent analysis of the LIPW found that over the 2012-2015 period, LIPW generated a total of 2.93 million person-days of employment, of which feeder and access roads sub-projects contributed about 36 percent, climate change mitigation activities Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) contributed 14 percent, and small earth dams/dugouts contributed about 50 percent of the employment created. A 2017 analytical study found that the LIPW program increased the youth employment rate by 34.2 percent, 28.6 percent, and 32.8 percent among beneficiary households through the feeder road, climate change, and small earth dam/dugout sub-projects, respectively. • 167,000 persons directly benefited from the LIPW program, exceeding the original target of 16,800 but not quite achieving the revised target of 190,000. Of these, 61% were female, exceeding the original target of 30% and the revised target of 50%. • The number of person days of unskilled labor was 13.6 million, surpassing the original target of 1.5 million days and meeting the revised target of 13.6 million days. The share of total unskilled worker earnings increased from 40 percent of LIPW cost in 2010 to 51 percent in 2018, not achieving the target of 62 percent. The average annual earnings of unskilled workers in LIPWs was 560 cedi, surpassing the original target of 150 cedi and the revised target of 200 cedi. Targets were adjusted annually for inflation. • 81.3 percent of beneficiaries were satisfied with the LIPW program, surpassing the target of 70 percent. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 4 Objective Improve economic and social infrastructure in target districts (original objective) Rationale The project’s theory of change linked rehabilitating works for small earth dams and dug outs, planting trees and maintaining them in woodlots, and hiring people for maintenance work with improving economic and social infrastructure in target districts. Outputs are listed under Objective 3 above. Outcomes As noted under Objective 3, 902 LIPW sub-projects were completed in 1125 communities. Among these, 250 climate initiatives were completed, with 2,268 hectares planted with fruit trees and multi-purpose woodlots, almost achieving the target of 2,360 hectares. 263 small earth dams and dug-outs were rehabilitated, not achieving the target of 300. 1,340 kilometers of road were rehabilitated, surpassing the target of 700 kilometers. The ICR did not provide information on access to or use of this improved infrastructure, but access/utilization were not part of the objective. Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale All four sets of objectives (original and revised) were substantially achieved. Overall efficacy is rated Substantial under both the original and revised objectives. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic Efficiency The PAD did not include a traditional economic analysis. The PAD (p. 24) stated that no overall rate of return for the project was calculated because specific LIPW investments were to be determined by targeted districts only during the course of implementation. Instead, the economic analysis included the following: (i) a brief review of past assessments of the benefits of cash transfer and LIPW programs; (ii) a rough estimate of potential employment and income impacts for those participating in LEAP and LIPW; (iii) an ex-ante assessment of the return to investment of public works based on indicative models of road and dam rehabilitation; and (iv) an initial assessment of cost-effectiveness of LEAP and LIPW components. The PAD stated that analysis at the appraisal stage confirmed the general viability of project investments and the strong potential for high levels of return to LIPW investments, particularly in feeder road rehabilitation. The impact of both the programs on improving consumption of the poor appeared to be good. The ICR (p. 24) cited an independent cost-effectiveness analysis for the LIPW finding that the economic rate of return (ERR) for typical feeder road rehabilitation was approximately 26 percent. The benefit-cost analysis of typical dam rehabilitation estimated a base case ERR of 16 percent. Both ERRs indicate that the investments were worthwhile, though the ICR did not provide exact information on project costs by type of works, making it impossible to determine what percentage of total costs is represented by these ERRs. Operational Efficiency Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) According to the ICR (p. 23), the project’s administrative costs included service charges for LIPW wages and cash transfers, research and consultancies, communication and awareness campaigns, and salaries of project staff and other related expenses. The ICR stated that the administrative costs, which were 16 percent of the total, were higher than those of similar projects in other countries (the global standard being 10 percent). The higher administrative costs were due to limited capacity at the district assemblies, resulting in delays and requiring the support of the project’s national coordinating unit and regional coordinating offices which hired consultants to support implementation. The administrative costs for LEAP declined from 24 percent at the start of project implementation to 12 percent after the mid-term project restructuring, against a target of 11 percent. Also, according to the ICR (p. 28), the project experienced infrequent payments of LEAP grants between 2010 and 2013, producing trust issues between the beneficiaries and the government. During project implementation, the government in Ghana changed twice, resulting in staff changes and therefore implementation delays. The project’s closing date was extended twice after it received additional financing, with a total extension of 23 months. Importantly, however, the project was able to use the extension period to reach a larger number of beneficiaries, also leveraging financing from another donor (about US$30 million in parallel funding annually from the United Kingdom Department for International Development, DfID, for cash grants) and about US$ 20 million annually from the government. Although there were moderate implementation inefficiencies, the positive rate of return for some of the LIPW projects and large gains made during the additional financing period produce an overall efficiency rating of Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) The project's objectives were highly relevant to country conditions, Bank strategy, and government strategy. Achievement of each of the original and revised objectives was Substantial. Efficiency was Substantial, given positive rates of return for some of the public works projects and strong gains during the additional financing period. These ratings are indicative of only minor shortcomings under both the original and revised objectives, leading to an overall Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome According to the ICR (p. 32), the current government continues to be committed to the social development agenda and has requested more support from the Bank for the two main programs of the project, LEAP and LIPW. The project was anchored in national, regional, and district structures as well as in local communities. Capacity was developed and is likely to be sustained in areas such as targeting, management, coordination, and policy oversight for social protection programs within national ministries. Furthermore, regional and district assembly administrative structures received support with technical planning and supervision, implementation, and oversight of LEAP and LIPW activities. However, there are several risks to the sustainability of the project outcomes. First, social protection systems are relatively costly, and strong, continuous political will is necessary to maintain them. Second, funding of these systems might be affected by changing government priorities, resulting in the deterioration of established systems. 8. Assessment of Bank Performance a. Quality-at-Entry According to the ICR (p. 31), the project built on experience from previous operations in the country, such as the Community-Based Rural Development Project (US$100.55 million, 2004-2011), the Bank’s global knowledge on the implementation of conditional cash transfers and public works within social protection frameworks, and analytical studies. For example, the project design was informed by a study on improving targeting of social protection programs. Throughout project implementation, the project conducted additional studies such as two on proxy means tests in 2014 and 2016, and a Social Protection Expenditure Review in 2016, to inform implementation. According to the Task Team Leader (March 12, 2019), the Bank team was adequately staffed with all the requisite expertise including financial management, procurement, social and environment safeguards, and M&E The PAD (p. 21) identified several risks to implementation. However, all identified risks were rated low or moderate, and mitigation activities were adequate. Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) Quality-at-Entry Rating Satisfactory b. Quality of supervision According to the ICR (p. 31), the Bank’s supervision was proactive, and the Bank team cooperated effectively with the government and other operations. The Bank team addressed implementation bottlenecks promptly, including issues involving the use of contractors versus unskilled labor, development of sub-project menus, completion of policy and operational reviews on social protection, and recognition of changes in local conditions. The ICR further stated that the Bank provided expertise for MIS implementation support when requested by the government team. During political transitions, the Bank provided additional support as necessary. When the project was negatively affected by an unstable macroeconomic environment between 2010 and 2013, the Bank cooperated with DfID to mitigate these effects. The project was flexibly restructured three times, and additional financing was processed in 2014 to expand its scope. The ICR (p. 30) stated that due to the Bank’s proactive approach, the project's M&E design was adjusted through “learning by doing,” resulting in the project’s M&E being rolled out into a sector-wide M&E framework. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s original objectives were clearly specified, and the theory of change connecting outputs to intended outcomes was adequate. Most indicators included a baseline and were sufficiently specific and measurable. The project was to conduct a detailed impact evaluation to track project contributions to improved household income, impacts on local economies, and improvements in service delivery. The impact evaluation was to include a baseline survey (year 1) and two follow-up surveys (years 3 and 5). Overall responsibility for M&E was to sit with the MLGRD, in collaboration with the Ministry of Social Welfare for the LEAP component. Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) b. M&E Implementation According to the ICR (p. 29), data collection was well structured at the community, district, regional and national level. The project developed data collection templates that were completed on a regular basis. Four completed impact evaluations for LEAP and LIPW (two each) provided useful information on implementation bottlenecks. The ICR (p. 29) stated that the 2012 and 2013 LEAP impact evaluations showed that late payments to beneficiaries reduced the impact of the program, resulting in the government improving financing sources to ensure timely payments. The project designed a MIS for delivery of payments to LIPW beneficiaries; linkages to banks and other commercial players resulted in increased transparency and a transition from manual to electronic payment systems, reducing the amount of time needed to process beneficiary payments. When the project was restructured in May 2014, the PDO was modified to be better aligned with the two main project components. Indicators in the Results Framework were adapted to better measure the PDO statement, and targets were increased to reflect the expansion of the scope of project activities. c. M&E Utilization According to the ICR (p. 30), the project produced quarterly M&E reports to inform on the project’s implementation progress and allow for improvement. Project managers at all different implementation levels used M&E data to provide feedback to lower levels. Also, project M&E was extensively used for management purposes and informed decision making. For example, M&E data supported the team's request for additional financing when project monitoring showed that implementation took place at a faster pace than originally expected. The development of the MIS and electronic payment system allowed for a close to real-time monitoring approach and more transparency and accountability with regard to beneficiary payments. According to the ICR (p. 30), learning from the project’s M&E implementation was translated into development and implementation of a sector-wide M&E framework. The ICR (p. 30) stated that project M&E tools are still being used even after project closing. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) The project was classified as environmental assessment category B and triggered the Bank’s safeguard policies Environmental Assessment (OP 4.01), Involuntary Resettlement (OP 4.12), Safety of Dams (OP 4.37), and International Waterways (OP 7.50). The ICR (p. 30) stated that the project disclosed an Environmental and Social Management Framework and Resettlement Policy Framework and included these frameworks in the project implementation manuals. According to the PAD (p. xi), a dam safety plan was developed during project preparation and was included in the project implementation manuals as well. According to the Bank team (March 12, 2019), the project complied with the necessary safeguards policy requirements by screening all sub-project activities, conducting broad stakeholder consultations especially with community beneficiaries, preparing and implementing an environmental and social management plan for earth dams, and developing and implementing an environmental and social audit for the project. The project conducted a safeguards audit, and the report showed no incidence of involuntary resettlement. Most of the rehabilitation (feeder roads, small earth dams, and dugouts) was built on existing facilities and did not cause any displacement. By project design, works were undertaken during agricultural off-seasons, which minimized impacts on crops. Land for planting under the climate change activities was arranged on a voluntary land donation basis. b. Fiduciary Compliance Financial Management According to the ICR (p. 31), the project’s financial management built on a solid foundation which created through previous Bank projects implemented in Ghana. The project complied with Ghana’s and the Bank’s financial management provisions. According to the Bank team (March 12, 2019), the project’s audited financial statements were submitted on time either on or before the due dates of June 30. The most recent audit (for the year ending December 2017)was submitted to the Bank on June 30, 2018. Throughout implementation, auditors issued an unqualified audit opinion on the financial statement for both implementing agencies. The project fully complied with financial management covenants, and acceptable audit reports were submitted. However, the reports by the MoGCSP experienced some delays in meeting quarterly submission dates, but invariably the reports were always eventually submitted. Procurement According to the Bank team (March 12, 2019), the Bank’s procurement guidelines were always followed. The project did not experience any significant implementation delays due to procurement-related issues. The Bank always provided timely support to resolve procurement difficulties. c. Unintended impacts (Positive or Negative) Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons The ICR (p. 32-33) provided useful lessons, adapted here by IEG: • Mainstreaming social protection into national and decentralized structures and identifying roles and responsibilities is critical to ensure sustainability. While this project made efforts to mainstream social protection into national and decentralized structures, it lacked a clear transition plan in terms of roles and responsibilities for supervision of activities for the decentralized government to fully take over the program when the project closed. • Impact evaluations and studies that are conducted by local academic institutions in cooperation with local governments are more likely to be implemented. Under this project, the LEAP and LIPW impact evaluations were conducted by the Institute of Social and Statistical Research at the University of Ghana, allowing for analysis and dissemination of results in a locally comprehensible manner that resulted in policy- making in near real time. • Introducing strong management tools impacts outcomes and trust among beneficiaries. This project designed a MIS for the delivery of payments to LIPW beneficiaries that linked beneficiaries to banks and other commercial players and allowed for a transition from manual to electronic systems for the LEAP and LIPW programs. 13. Assessment Recommended? No Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Social Opportunities Project (FY10) (P115247) 14. Comments on Quality of ICR The ICR provided a strong, candid overview of project preparation and implementation and was outcome- driven. It was internally consistent and concise. It brought data outside the formal results framework to bear on the assessment of outcomes. Furthermore, the ICR provided useful lessons learned. However, the ICR did not include a traditional economic analysis, and it provided limited information on financial management, procurement, and safeguard compliance and little information on quality at entry. The ICR did not rate each objective individually, instead providing overall efficacy ratings before and after restructuring (which does not follow established guidelines for a split rating). a. Quality of ICR Rating Substantial Page 17 of 17