Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) Report Number: ICRR0021746 1. Project Data Project ID Project Name P111545 KE-Cash Transfer for OVC (FY09) Country Practice Area(Lead) Kenya Social Protection & Jobs L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-45530,IDA-53120,TF-97272 31-Dec-2013 133,591,676.82 Bank Approval Date Closing Date (Actual) 31-Mar-2009 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 50,000,000.00 79,284,446.00 Revised Commitment 133,449,909.40 74,979,310.35 Actual 133,591,676.82 74,979,310.35 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Judyth L. Twigg Joy Behrens IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD, p.ii) and the Financing Agreement of May 8, 2009 (p. 6), the objective of the project was “to increase social safety net access for extremely poor orphans and vulnerable children (OVC) households, through an effective and efficient expansion of the Program." Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) When the project received Additional Financing (AF) in October 2013, the PDO was revised: "to increase social safety net access for extremely poor OVC households and to build the capacity of the government to more effectively deliver the National Safety Net Program (NSNP).” Since the AF scaled up capacity building support to cover the entire NSNP, beyond the CT-OVC, and the objectives were revised accordingly, a split rating will be conducted. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 31-Oct-2013 c. Will a split evaluation be undertaken? Yes d. Components Component 1: Policy Development and Institutional Strengthening (appraisal estimate US$10.0 million, AF US$10.0 million, actual US$16.5 million). This component consisted of several sub- components. Sub-component 1.1: Strengthening the capacity of the Ministry of Gender, Children, and Social Development to coordinate social protection (SP) interventions. This sub-component was to finance the development of a policy framework to better coordinate existing SP programs and support the overall planning and coordination of SP interventions, mainly through technical assistance (TA) and training of key staff. Sub-component 1.2: Strengthening the capacity of the Department of Children Services to more efficiently manage the program. This sub-component was to finance support for operational activities, TA for procurement, financial management, an update of the operational manual, training materials, and hardware. It was also to provide support for strengthening linkages with complementary programs to: (i) strengthen the supply side of the program, such as links with schools and health centers, and also coordination with other human development promotion programs (for example, school bursary schemes, HIV/AIDS awareness programs, and health fee waiver programs); (ii) promote income generating activities and skills development of OVC caregivers, by linking with programs such as the Kenya Women’s Finance Trust; and (iii) promote skills development among youth, by linking with youth polytechnics and programs such as the Youth Enterprise Development Fund. Sub-component 1.3: Governance and accountability. This sub-component was to finance: (i) provision of information to beneficiary households and the population as a whole regarding the program and its benefits; (ii) implementation of a communication strategy, including the design of the program website; (iii) information, communication, and education campaigns; (iv) program leaflets, posters, and other promotional information and materials; (v) raising awareness of the importance of immunization, nutrition, and infectious Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) diseases including HIV/AIDS, and other knowledge to improve the lives of OVC households; and (vi) enhanced oversight and accountability systems to improve program implementation to ensure that beneficiaries as well as non-beneficiaries had access to the appeals and complaints mechanism. This sub- component also was to finance spot checks and citizens' report cards to ensure and measure beneficiary satisfaction. Sub-component 1.4: Management information, monitoring and evaluation. This sub-component was to finance: (i) refinements to the Management Information System (MIS); (ii) a Financial Management Module linking with payment providers; (iii) technical support for operational and impact evaluations to assess impact and operational efficiency of the management of the program; (iv) financial strengthening processes; and (v) studies and a pilot to test alternative payment mechanisms. Through the AF, the scope of component 1 was expanded to strengthen the government’s capacity to support the entire NSNP. The NSNP, supported by the Bank through a US$ 300 million Program for Results (PforR) Credit (2013-2020), aimed to establish effective support for poor and vulnerable households through the four main CT programs in the country: the CT-OVC program, the Older Persons Cash Transfer, the Persons with Severe Disability Cash Transfer, and the Hunger and Safety Net Program. The NSNP needed further investment in institutional and system strengthening to ensure quality of the results achieved under the program. Component 2: Program implementation in selected sub-counties (appraisal estimate US$40.0 million, actual US$37.9 million): This component consisted of two sub-components. Sub-component 2.1: Cash benefits for OVC households in selected districts: This sub-component was to finance cash transfers of KSh 1,500 per month per household in 25 of the 40 government-funded districts and up to 56,000 households. Districts were targeted for inclusion in the program according to the number of extreme poor OVC households (PAD, p. 35). The program was to reach seven districts in the first year, covering approximately 20,000 households. The program was to expand to an additional six districts each year, reaching 12,000 additional households in each of the second, third, and fourth years, totaling 56,000 households by the end of FY12. At the AF, the cash transfer to beneficiary households was increased from KSh 1,500 per month per household to KSh 2,000. Sub-component 2.2: Transaction costs for CTs: This sub-component was to finance transaction costs related to cash transfers. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$50 million. Actual total cost was US$541.5 million. Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) Financing: The project was financed by a US$50 million credit from the Bank, of which US$44.9 million was disbursed, and AF of US$10 million, of which US$9.4 million was disbursed. UNICEF contributed US$26.4 million of a planned US$12 million, and the British Department for International Development (DfID) contributed US$75 million of a planned US$34 million, through a Multi-Donor Trust Fund toward TA and direct support for the CT-OVC and, after the AF, toward scaling up support for the entire NSNP. Borrower Contribution: The Borrower contributed US$381.46 million, significantly more than the planned US$30 million, reflecting sharply increased government support for the CT-OVC. Dates: The project was restructured three times:  On October 31, 2013, the project received AF as well as funding from DfID to allow for the expansion of support to the entire NSNP. The project's objectives, first component (as described above), and results framework were revised accordingly, and the closing date was extended from December 31, 2013 to December 31, 2015.  On November 7, 2016, the project was restructured to: (i) extend the closing date to October 31, 2018; (ii) modify some of the indicators linked to the Disbursement Linked Indicators (DLIs) under the parallel PforR to ensure alignment, better measure key indicators, and revise some targets; and (iii) amend institutional arrangements based on organizational changes in the Ministry of Labor and Social Protection.  On October 30, 2018, the project was restructured to extend the closing date to December 31, 2018, to allow for sufficient time for the government to complete outstanding procurement activities and conclude the analysis of beneficiary surveys and impact evaluations. 3. Relevance of Objectives Rationale In 2009, almost half of Kenya’s population lived in poverty, and nearly 20 percent lived in extreme poverty. Reasons for households falling into poverty included shocks such as food price inflation, droughts, floods, and illness and death caused by HIV/AIDS. A rapid assessment conducted in 2004 found that 1.8 million children were orphans and, of these, nearly two-thirds had lost their parents due to HIV/AIDS. Nearly one- quarter of OVC lived in extreme poverty, and only 85 percent attended school (compared to 93 percent of other children). In 2004, the government implemented a pre-pilot CT-OVC program, financed by UNICEF and the Swedish International Development Agency, that provided cash transfers to 500 poor households with OVC in three districts. In 2006, this pilot was expanded to 40 additional districts. In 2008, with the support of the World Bank, the government expanded the program further and aimed to reach about half of all OVC living in extreme poverty through a nationwide program covering at least 100,000 households by 2012. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) The objective of the project supported the government’s National Social Protection Strategy, under development at the time of project preparation but never formally adopted, which aimed to harmonize SP interventions while meeting the immediate needs of the poorest and most vulnerable. A new government SP strategy was being prepared when the project closed. The objectives of the original project and the AF were in line with the Bank’s Country Assistance Strategy at appraisal (2004-2008), which aimed to invest in people, reduce vulnerability, and strengthen community; they also reflected the broad outlines of the Bank’s most recent Country Partnership Strategy (CPS, FY14-FY18), which prioritizes “promoting protection and potential to ensure all groups share in advancing prosperity and helping the vulnerable to develop their potential." However, it is noteworthy that the FY14-FY18 CPS explicitly states (p. 37) that "orphans and street children" were likely to receive different/less focus moving forward, reflecting "difficult trade-offs" and "competing pressures" as the Bank practices selectivity at an operational level. Relevance of the project's objectives is therefore rated Substantial, as the objectives were supportive of country conditions and government strategy at appraisal, but remained less relevant to Bank strategy at closing. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Increase social safety net access for extremely poor OVC households Rationale The project’s theory of change linked the provision of cash benefits to OVC households with increasing social safety net access for the extremely poor. Furthermore, the project linked support for financial management and procurement, the development of a single registry and a harmonized targeting tool, and conducting of an impact evaluation and beneficiary surveys with strengthened governance and capacity for program management, ultimately enhancing access for beneficiaries. Outputs:  350,000 OVC beneficiary households were covered by the CT-OVC program in 2018, surpassing the original target of 50,000 and almost meeting the revised target of 365,000. This represents about 17 percent of all OVCs in Kenya.  82,530 children were immunized, surpassing the target of 18,000 children.  The households of 177,507 OVC aged 0-17 received free basic external support in caring for the child. Regular awareness sessions for beneficiaries on immunization, nutrition, and HIV/AIDS prevention were conducted. Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545)  109 officers were trained on various aspects of the program including designing and implementing a social transfer program, integrating information systems, M&E, and project management. The trainings targeted officers in headquarters and the field.  55 computers, 10 photocopiers, 25 printers, 13 scanners, and assorted accessories were procured. 20 motor vehicles and 20 motorcycles were purchased to enhance field operations.  According to the ICR (p. 24), during implementation an agreement was reached with the government to test the introduction of penalties to determine their effect on fulfillment of co-responsibilities. The pilot was implemented for approximately 18 months but encountered several challenges related to capacity such as data collection, accurate implementation of penalties, and timely payments. It was decided to discontinue the pilot and put resources instead into other areas of the program. Outcomes:  The percentage of OVC in beneficiary households aged 6–17 years who have completed or are enrolled in basic education increased from 78 percent in 2009 to 95 percent in 2018, surpassing the original target of 90 percent and meeting the revised target of 95 percent.  The percentage of OVC living in beneficiary households 0–17 years of age holding birth certificates increased from 19 percent in 2009 to 67 percent in 2018, surpassing the original target of 50 percent and the revised target of 60 percent.  The percentage of OVC in beneficiary households age 0–5 years who were fully immunized increased from 65.5 percent in 2009 to 80 percent in 2018, meeting the original target of 80 percent and the revised target of 70 percent. Rating Substantial OBJECTIVE 2 Objective The original project did not contain a second objective. Rationale The original project did not contain a second objective. Rating Not Rated/Not Applicable OBJECTIVE 2 REVISION 1 Revised Objective Build the capacity of the government to more effectively deliver the NSNP Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) Revised Rationale The capacity of the government to deliver the NSNP was to be enhanced by strengthening program systems to ensure good governance; harmonizing CT programs to improve sector coherence; and capacity building for expanding CT programs to promote more comprehensive and equitable coverage. Outputs:  A Social Protection Secretariat was established. An SP Policy Framework was adopted in June 2011 and was being updated as of the time of the ICR, but the target of developing a new SP strategy was not achieved.  TA was provided to develop alternative payment mechanisms for beneficiaries.  Trainings were implemented related to M&E, MIS, enrollment, targeting, and beneficiary outreach. 100 percent of NSNP progress reports produced by the MIS met agreed reporting standards, meeting the target.  An electronic payment mechanism with two-factor authentication was introduced to ensure that CTs became more secure, accessible, and predictable.  The project introduced biometric enrollment and identification of beneficiaries, improved targeting mechanisms, and consolidated financial management processes. A grievance and case management system was established to enable beneficiaries and others to lodge grievances or provide updates on household data. Outcomes:  The percentage of NSNP beneficiaries who conformed to the program targeting criteria increased from 30 percent in 2009 to 82 percent in 2018, exceeding the original target of 50 percent and almost achieving the revised target of 85 percent.  The percentage of NSNP beneficiaries for whom payments were made electronically through payment services providers and used two-factor authentication increased from 40 percent in 2013 to 100 percent in 2018, surpassing the original target of 75 percent and the revised target of 90 percent.  The percentage of NSNP beneficiaries who were able to name two means of making a complaint increased from 15 percent in 2013 to 30 percent in 2018, not achieving the original/revised target of 47 percent.  The percentage of NSNP beneficiaries who knew the program's objectives and entitlements increased from 37 percent in 2013 to 89 percent in 2018, surpassing the original/revised target of 60 percent.  13 percent of NSNP payments were disbursed to payment service providers on time in 2018, not achieving the target of 70 percent, and marking a decline from the 2009 baseline of 26 percent. According to the ICR (p. 17), between 2014 and 2017 NSNP payments were mostly made on time, and in 2017 the government achieved a DLI under the PforR related to timeliness of payments. A 2017 Program Implementation and Beneficiary Satisfaction (PIBS) survey confirmed that 77 percent of NSNP beneficiaries reported that they received payments on time. However, in 2018, the NSNP experienced payment delays due to the introduction of an Older Person Cash Transfer, which was given priority by the government, and the rollout of a new payment mechanism giving beneficiaries Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) access to a bank account and a choice of payment provider. The 2018 drop in performance is therefore likely to be temporary.  The distance traveled by beneficiaries to the nearest pay point declined, with 61 percent of beneficiaries reporting that they traveled more than 6 kilometers in 2016 compared to 15 percent in 2019. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy is rated Substantial, based on achievements in coverage of cash transfers to OVC beneficiary households and fulfillment of co-responsibilities, as well as improvements in delivery of the NSNP. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency Neither the PAD nor the project paper for the AF included a traditional economic analysis. Instead, they highlighted the strong economic rationale for investing in the CT-OVC program. Kenya has approximately 2 million OVCs who represent 10.8 percent of all children and 4.4 percent of the total population. 12.5 percent of OVC live in extreme poverty, compared to 10.2 percent of all children. The PAD and project paper also cited global experience with CTs and their track record of demonstrated success in reducing poverty and boosting human capital development. Operational Efficiency According to the ICR (p. 21), the project was able to reduce the administrative costs for implementing the NSNP from 30 percent in FY07-08 (before project implementation) to 2 percent during the last year of implementation. This is significantly lower than the average administrative costs for social safety net program spending in Africa, which were estimated at 17 percent in 2018. The ICR (p. 19) reported that the project achieved significant economies of scale by integrating a systems approach across the NSNP, and also by using existing country financial management systems. Effective Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) collaboration among development partners facilitated efficient support for all the elements of the NSNP and timely response to government requests for TA (ICR, p. 34). The project conducted several reviews of the CT- OVC MIS and payroll systems, identifying irregularities in beneficiary enrollment and payments such as payment of beneficiaries without names and ID numbers. Facilitating the correction of these errors was a positive marker of efficient use of project resources. There were minor implementation inefficiencies. The project experienced several implementation delays due to procurement and financial management-related issues, resulting in some vehicles not being delivered on time and cancellation of funds (see Section 10b); however, most of these issues were resolved, with support from the Bank and other partners, well before project closing. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the original and revised objectives was Substantial, responding effectively to country conditions and government strategy, but not meeting the selectivity terms of the most recent Bank country strategy. Overall efficacy is rated Substantial, based on achievements in coverage of cash transfers to OVC beneficiary households and fulfillment of co-responsibilities, as well as improvements in delivery of the NSNP. Project efficiency is also rated Substantial. These ratings indicate only minor shortcomings in the project's preparation and implementation, resulting in an Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome According to the ICR (p. 31), the government continues to be committed to maintaining and expanding the NSNP. The project's November 2017 Aide Memoire reported that in September 2017, the government had Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) taken over the financing of 40,000 CT-OVC households and 11,000 additional households under the HSNP, all previously financed by DfID. This was seen as a major achievement, with 100 percent of the project- financed CTs now financed by the government. The project will be followed by a new Bank operation, the Kenya Social and Economic Inclusion Project (US$250 million, 2018-2023), aiming to strengthen delivery systems for enhanced access to social and economic inclusion services and shock-responsive safety nets for poor and vulnerable households. Also, the government’s capacity to formulate and implement policies in various institutions has been strengthened. For example, a framework to combat error, fraud, and corruption as well as a detailed financial management action plan were developed. Furthermore, extensive trainings in M&E, MIS, procurement, financial management, and awareness sessions for beneficiaries in areas such as immunization, nutrition, and HIV/AIDS were provided, and the capacity and knowledge generated through these trainings appears likely to be sustained. While the timeliness of CTs to beneficiaries has improved (assuming the 2018 downturn in timeliness is temporary, as anticipated) and a new payment system was rolled out, it is still critical that the National Treasury continues to ensure timely transfers of funds, and that the government continues to allocate necessary resources toward program administration. 8. Assessment of Bank Performance a. Quality-at-Entry According to the ICR (p. 30), the Bank worked closely with the government and other development partners during project preparation. The design of the project was based on analytical and operational work conducted by the Bank in Kenya, Asia, and Latin America. The Bank team identified relevant risks at appraisal, such as an inadequate governance environment including fiduciary risks, challenging technical and design issues, limited implementation capacity, and challenges with development partner coordination/harmonization. According to the Bank team (October 10, 2019), the Bank mitigated these risks by making substantial investments into systems to strengthen the technical design and fiduciary aspects of the project. These included strengthening the MIS of the program and establishing the Single Registry; strengthening fiduciary systems, including development, implementation, and monitoring of financial management and procurement action plans; introducing the two-factor authentication electronic payment mechanism; and strengthening the grievance and case management system. In response to the implementation capacity risk, considerable support was provided for training of program officers, and also for engagement of temporary staff for certain intense periods of expansion as well as the development of a consolidation strategy and its implementation. External reviews, such as a project implementation and beneficiary satisfaction survey, were introduced to provide important guidance to program management; concrete actions were taken in program implementation as a result. The project’s M&E was designed adequately (see Section 9a). Quality-at-Entry Rating Satisfactory Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) b. Quality of supervision The ICR (p. 30) stated that the field-based Bank team had an adequate skill mix and provided regular project implementation supervision. Throughout implementation, the Bank team conducted 20 support missions and several short technical missions, and according to the ICR (p. 31), aides-memoire were of good quality. According to the ICR (p. 25), the Bank closely monitored project implementation jointly with other development partners. The project received AF, and the objectives were revised accordingly, in response to the evolution of the government's strategy for the NSNP. The ICR (p. 31) stated that even though the project had four different Task Team Leaders throughout implementation, continuity of supervision was ensured through regular communication with the government. The Bank team worked closely with the government and addressed implementation bottlenecks quickly as they arose. It addressed procurement and financial management-related implementation delays by providing technical assistance to counterparts. For example, the Bank supported the government to develop a time-bound financial management action plan, and additional procurement staff were hired in a timely manner when procurement-related challenges arose (see Section 10b). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s theory of change and the linkages between key activities/outputs and intended outcomes were reflected well in the results framework. The objectives of the original project and the AF were clearly specified. The outcome indicators encompassed all elements of the objectives statement, and most indicators were appropriately specific and measurable. Some of the indicators measured more than one aspect, such as the outcome indicator “additional number of beneficiary households receiving timely and predictable cash transfers (as defined in the operations manual)." Also, the ICR (p. 24) noted that some of the indicators did not have real baseline values, but instead only estimates. These estimates resulted in challenges during implementation when the real baseline values turned out to be higher or lower than originally expected, and targets had to be modified. According to the ICR (p. 26), institutional arrangements for M&E were adequate to collect data on a regular basis and allow for analysis to inform program management. Initially, the Central Program Unit and later the Social Assistance Unit (SAU) were responsible for the project’s M&E activities. These activities included regular operational monitoring reports as well as an impact evaluation and two rounds of the PIBS survey, in 2015/2016 and 2017/2018. Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) b. M&E Implementation When the project received AF, the results framework was adapted to reflect the government’s plans for the NSNP. According to the ICR (p. 26), M&E implementation was monitored continuously, and the government and the Bank provided technical assistance to address any challenges. Due to lengthy procurement processes, the impact evaluation and the PIBS surveys were delayed, resulting in the baseline for the impact evaluation not being completed until 2015 and the impact evaluation itself, as well as the final round of the PIBS survey, only being completed just before project closure. However, according to the ICR (p. 26), the PIBS survey provided independent and rigorous data for monitoring the performance of the NSNP and the satisfaction of beneficiaries of the CT program. According to the Bank team (October 10, 2019), M&E implementation was monitored systematically through continuous implementation support, as well as collaboration and dialogue with the government. The government brought on board additional M&E staff when necessary, and the Bank provided additional support to resolve some of the challenges faced related to M&E design and capacity. This support included ensuring adequate attention to M&E activities on all aspects of project implementation, continuous engagement with external M&E firms to ensure relevance of activities to program implementers, and compliance with respective terms of reference. As an example, project communication was found to be lacking in both rounds of the PIBS. The SAU responded to the findings through the development and implementation of a thorough beneficiary communication strategy leveraging a range of stakeholders and instruments to improve beneficiary outreach. c. M&E Utilization According to the ICR (p. 27), the project’s M&E data and analysis were used to inform project management and decision making. The ICR (p. 27) stated that during the last two years of the project, external operational monitoring of the NSNP was conducted on a regular basis. Each report focused on pre-identified key areas such as program harmonization, payments, targeting, MIS, and other areas. The information obtained through this monitoring allowed for modifications in program implementation. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The original project was classified as category C and did not trigger any of the Bank’s safeguard policies. The AF triggered the Bank’s safeguard policy OP/BP 4.10 (indigenous people). According to the ICR (p. 28), the project conducted an environmental and social assessment which found that CT programs sometimes cause tensions among households who can participate in the program and those who cannot. The assessment also identified the need to strengthen the participation and voice of poor and vulnerable groups in the NSNP. In response, the government prepared a Vulnerable and Marginalized Group Framework. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) The ICR (p. 28) stated that in July 2016 the project conducted a Social Safeguards Review for the NSNP that found no evidence for minority groups systematically being excluded. According to the ICR, the project’s safeguard rating was Moderately Satisfactory or Satisfactory throughout project implementation. b. Fiduciary Compliance Financial Management According to the ICR (p. 29), the project used the country’s financial management system, which resulted in its strengthening. Planning and budgeting followed the government’s procedures. Budget implementation was monitored by the government’s Integrated Financial Management Information System on a monthly basis. In addition, bi-monthly beneficiary payment cycle reports and reconciliations and quarterly interim financial reports were issued. The ICR (p. 29) stated that several project-conducted reviews of the CT-OVC MIS and payroll identified irregularities in the beneficiary enrollment and payment system, including payment of beneficiaries without names and ID numbers. The Bank and SAU addressed these issues by developing and implementing an action plan. Also, a Financial Management Policies and Procedures Manual was developed to introduce more comprehensive internal control procedures. According to the ICR (p. 30), the Office of the Auditor General expressed a qualified opinion for the year ending June 30, 2018. The audit found several shortcomings: (i) non-financial assets procured during the year and lack of a complete fixed asset registry on asset management; (ii) funds transferred from a closed IDA counterpart project account and some assets procured in its lifetime not disclosed; (iii) failure to provide CT reconciliation reports by the Payment Service Providers (PSPs); (iv) continuation of payments for some deceased beneficiaries, beneficiaries that were over 18 years, or households that did not have an OVC ; (v) delays by PSPs in submitting reports on transfers to beneficiaries; and (vi) overpayment of bank commissions to Kenya Commercial Bank. The Bank addressed these shortcomings by developing an action plan that was 80 percent implemented by the time the ICR was written. Procurement According to the ICR (p. 29), the project followed the Bank’s guidelines, and procurement plans were updated at least annually. The project’s procurement performance was rated Satisfactory throughout most of the implementation period, with only a short period in 2010/2011 when it was rated Moderately Satisfactory. The project experienced delays due to insufficient funds included in the government’s budget or supplementary budget. The ICR (p. 29) stated that, for example, in September 2018 several procurement packages for vehicles were still pending because the National Treasury had not given the authority to the State Department for Social Protection (SDSP) to incur expenditures. Once the SDSP received this approval, the project's closing date was extended to December 31, 2018 to complete the Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545) procurement of these vehicles; however, since there was limited time left, not all vehicles could be delivered, resulting in a refund of the Bank in the amount of US$76,644. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR (pp. 32-35) included several valuable lessons, adapted here by IEG:  Adequate targeting mechanisms and regular recertification are critical for a conditional cash transfer program in order to ensure that the money goes where it should. In this project, there were reports that households continued the program despite the OVC turning 18 years, or that households claimed benefits even though there was no OVC living in the household. In order to address these issues, the government has rolled out a new payment mechanism that requires proof of life and IDs for registration and is updating data inputs into the targeting tool as well as the recertification strategy.  Finding a sustainable, streamlined, and systematic process for ensuring timely payments to beneficiaries is critical for the success of a program. In 2016, this project conducted an analysis of the steps required to complete a payment cycle, finding that 24 internal steps had to be taken over a total of 90 days. The government addressed this lengthy process through streamlining and was able to decrease the timeline to 54 days, resulting in the government meeting the performance standard for timely payments for three consecutive payment cycles. However, this performance was not sustainable when the National Treasury did not prioritize SDSP disbursements. Therefore, in order to increase the likelihood of a timely process, the National Treasury agreed to include all CTs on the recurrent budget instead of the development budget. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review KE-Cash Transfer for OVC (FY09) (P111545)  Allowing sufficient time in the project’s design for the introduction of new delivery systems is important to ensure benefits in the long run. In 2015, the project introduced a new two-factor authentication payment mechanism, resulting in the disruption of project implementation due to challenges with collecting biometric data. However, evidence from field visits also indicated that beneficiaries have benefited from choosing providers in much closer proximity than before, reducing the cost and time to access the pay point. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a strong overview of project preparation and implementation and included valuable lessons learned. The ICR was internally consistent and outcome-driven. Its account of the rationale and processes underlying the project's Additional Financing and restructuring was straightforward. The M&E framework and implementation, as well as challenges and solutions in defining baselines for some indicators, were well explained. However, the contributions of development partners were not outlined clearly, and the downgrading of support for OVCs in the most recent Country Partnership Strategy was not acknowledged. Overall, the ICR’s quality is Substantial. a. Quality of ICR Rating Substantial Page 15 of 15