Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Report Number: ICRR0022373 1. Project Data Project ID Project Name P130735 BF-Youth Employment & Skills Development Country Practice Area(Lead) Burkina Faso Social Protection & Jobs L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-52560 15-Dec-2018 45,006,522.69 Bank Approval Date Closing Date (Actual) 23-May-2013 15-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 50,000,000.00 0.00 Revised Commitment 48,170,357.12 0.00 Actual 45,006,522.69 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Salim J. Habayeb Joy Maria Behrens IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (page 5) and the Project Appraisal Document (PAD, page 9), the project objectives were as follows: To increase access to temporary employment and skills development opportunities for out-of-school youth. Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) At the time of the June 2017 restructuring, targets for key project outcome indicators were revised downwards due to implementation delays and overly ambitious targets. Therefore, a split rating is applied. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components Component 1: Labor Intensive Public Works (Appraisal: US$ 25.0 million; Actual: US$ 25.8 million):This component aimed to provide immediate employment for youth with little or no education through labor- intensive public works (LIPW). The targeted beneficiaries were youth (16-35 years old) in priority urban, peri-urban, and rural areas. Participation of women was ensured through selecting activities supportive of women's employment and reserving a percentage of the LIPW jobs specifically for women. Activities included: sensitization and communication campaigns, wages, tools and small equipment, training, and non-salary costs of rural investments. Component 2: Skills Development (Appraisal: US$ 20.0 million; Actual: US$ 14.0 million): This component aimed to improve youth employability by offering youth with different skill levels their first training experience. Activities included: design and equipping of two training centers; provision of dual training program, on-demand training program, apprenticeship program, and entrepreneurship program. Component 3: Institutional Capacity Strengthening and Project Management (Appraisal: US$ 5.0 million; Actual: US$ 5.2 million): This component aimed to strengthen the capacity of public and private sector institutions to engage in policy dialogue on youth employment, improve the knowledge base, and support the project coordination unit (PCU), including M&E functions. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost  The project cost at appraisal was US$ 50.0 million. The amount was revised to US$ 48.17 million, of which US$ 45.01 million disbursed. Financing  The project was financed entirely by an IDA grant of US$50.0 million. Borrower contribution Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735)  There was no expected Borrower contribution. Dates  June 2017: The project was restructured to revise key outcome indicator targets downwards (number of beneficiaries, number of man-days of labor created, % youth who complete training programs, % trained youth who are employed or self-employed), given implementation delays and overly ambitious targets. Funds were reallocated among disbursement categories and sub- components - including the cancelling of heavy equipment for the building/public works training center, due to implementation delays, and additional allocation to entrepreneurship program to provide start-up grants. Cost projections were also revised downward due to the downward revision of targets and a devaluation of the FCFA against the US Dollar.  November 2018: The project closing date was extended from December 2018 to December 2019, to allow for completion of activities. 3. Relevance of Objectives Rationale At the time of project appraisal, Burkina Faso was a low-income country, among the poorest in the world, with an undiversified economic structure reliant on agriculture. Prospects for economic growth and stability remain highly vulnerable to external shocks related to weather, fuel and food crises, fluctuating commodity prices, and global financial crises. A large proportion of the population is young (59% under the age of 20) and the labor force is made up of predominantly young people living in rural areas (80% of population is rural). Youth unemployment was high (84% of unemployed are aged 15-34) due to formalization of the economy and limited capacity of potential employees to fill positions with adequately trained workers. These conditions contributed to social unrest that took place in the spring of 2011, signaling the need for immediate job creation in a fragile context. The project objectives were highly relevant to government strategy. The government's poverty reduction strategy at the time of appraisal (2012-2016) recognized the role of job creation, productivity, and income growth to improve living standards of the poor. The special Program for Job Creation for Youth and Women (2012-14), developed in response to the 2011 unrest, included aims to promote access to employment for out-of-school youth through labor-intensive public works, internships, and income-generating activities. Similarly, the Bank Country Strategy at the time of appraisal (2013-16) identified minimizing economic vulnerability and promoting growth by increasing productivity, and promoting shared growth through effective social service delivery, in particular in education and training, as key objectives. The Bank's current Country Partnership Framework (FY2018-23) identified investing in human capital and social protection systems as one of three Focus Areas, with support to inclusive high quality education and skills development as an objective (measured by the number of out of school youth enrolled in skills development programs). Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To increase access to temporary employment for out-of-school youth (original targets) Rationale The theory of change for this objective was overall sound, as the setting up of labor-intensive public works (LIPW) projects, including outreach, equipment, wages, and training activities, was expected to create temporary employment for the target population. The target beneficiaries were youth (defined as individuals 16 to 35 years of age) not in school or in a training program (whether formal or informal) and without a regular source of income. The project covered Ouagadougou and Bobo-Dioulasso, which were the regions with largest concentration of unemployed/ underemployed youth, as well as two or three regional capitals with high growth potential including both urban and rural areas. 31,000 youth were expected to enroll in the program, of which 16,000 in urban areas and 15,000 in rural areas. There were 43,703 direct beneficiaries (original target: 46,800; revised target: 34,200) for both project components, of which 57.0% were female (target: 30.0%). Outputs  Creation of labor-intensive public works (LIPW) projects. In urban areas, the projects consisted mainly of cleaning public areas. In rural areas, the projects consisted of rehabilitating or constructing productive infrastructure such as: o 133.3 km of rural roads (original target: 240.0 km; revised target: 120.0 km). o 60.0 km of urban roads (target: 60.0 km). o 124.8 hectares of community woodlots (original target: 2,400 hectares; revised target: 355.0 hectares) o 107.2 hectares or reclaimed floodable areas (target: 200 hectares) o 41 large diameter wells (target: 37) o 12 dug-outs/ponds o 29 vegetable plots  Provision of supplemental mechanisms for the urban LIPW projects, including training (social behavior, entrepreneurship, literacy) and savings programs which aimed to increase nutrition levels, human capital, financial capital, and financial inclusion. In addition to the project beneficiaries who participated in these supplemental activities, the government was able to provide the above to an additional 9,718 beneficiaries in social behavior training, 9,411 in income generation training, and 1,953 in technical training. Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735)  Rehabilitation of three Job Counseling Youth Centers to assist participants in identifying permanent employment or employment enhancing opportunities at the end of their LIPW employment. However, at the time of ICR preparation, the centers were not yet operational.  Technical assistance to the government in developing LIPW programs, through contribution to the national LIPW registry and modeling of programs of elements including training, savings programs, mobile childcare, and safeguards supervision. Of note, the project developed mobile childcare pilots to support childcare arrangements for women participating in the LIPW programs. As reported in the ICR (page 27), in addition to the positive effects on children, the mobile childcare sites allowed pregnant women and women with breastfed children to participate.  The planned impact evaluation was not conducted as planned, although the Bank team worked with the European Space Agency to use satellite data to assess the quality and quantity of LIPW infrastructure. Outcomes  2,277,092 man-days of labor were created in the labor-intensive public works (LIPW) projects, falling short of the original target of 3,900,000. Of these, 2,098,200 were through urban LIPWs and 178,892 in rural LIPWs. A total of US$ 5.4 million were paid out in wages.  29,332 youth were employed through LIPW, falling short of the original target of 31,000. Of these, 64.0% were female (target: 30.0%), 13,988 were employed in urban LIPWs and 15,344 were employed in rural LIPWs.  The urban LIPW beneficiaries were employed for 150 days on average, as planned. Of the urban LIPW participants surveyed (as reported in ICR, page 16), 88 percent said that their self-esteem and confidence on the labor market increased due to their participation in the LIPW. Among the illiterate beneficiaries participating in the literacy training, 71 percent said they were able to read and write after the training. 78 percent opened a bank account compared to only 20 percent having a bank account before the LIPW. 65 percent said they were able to save some amount of their income during the LIPW. However, positive effects from LIPW, with regards to decline in unemployment and increase in income "seem to be mainly associated with savings accumulated through the LIPW" and less with the training (ICR, page 17).  The rural LIPW beneficiaries were employed for 12 days on average, compared to the 100 days planned. According to the ICR (page 18), delays in the start of the implementation of the rural LIPW, mainly due to the political transition, led to time pressure and the need to work during the agricultural season, which, combined with additional difficulties to find labor, forced the implementing NGO to hire more beneficiaries but for fewer days. Achievement is rated Modest due to significant shortfall in the number of man-days of labor created, although the number who participated in LIPW projects nearly achieved the original target. The shortcomings were in rural areas, as the rural LIPW participants were only employed for 12 days on average, compared to 100 days planned. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective To increase access to temporary employment for out-of-school youth (revised targets) Revised Rationale See outputs reported above. Outcomes  2,277,092 man-days of labor were created in the LIPW projects, falling short of the revised target of 3,172,194.  29,332 youth were employed through LIP, surpassing the revised target of 20,500. Achievement is rated Substantial due to evidence of increased access to temporary employment, with moderate shortcomings in achieving the target for the number of man-days of labor created. Revised Rating Substantial OBJECTIVE 2 Objective To increase access to skills development opportunities for out-of-school youth (original targets) Rationale The theory of change for this objective was overall sound, as the development of training programs and systems, including design studies, facilities and equipment, training of trainers, curriculum, outreach, contacts with private sector, and allowances, was expected to increase access to opportunities for skills development for the target population (see above on targeting). Outputs  Development of four youth skills training programs, including in dual training, on-demand training, apprenticeships, and entrepreneurship  Design studies for two vocational training centers to provide dual training (building/public works and tourism/hotel/restaurant). (Note: the government financed the actual construction of the two centers). However, only 48 youth were enrolled at the time of project closing (original target: 1,800; Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) revised target: 700). This major shortfall was due to significant delays in the construction of both centers (due to litigation related to procurement process) and inability to recruit participants using other premises.  Development of on-demand training curricula, based on direct demand from the private sector (including the Chamber of Commerce), introduction of dual training approach, training of trainers (1,199 master craftsmen, 68 master trainers, and 50 young entrepreneurship trainers), and translation of training modules into local languages.  Provision of start-up grants for selected participants in the entrepreneurship program.  Beneficiary surveys were not conducted as planned, although a tracer study was conducted by the Ministry of Youth. Outcomes  The proportion of enrolled youth who completed the training programs was 95.0%, nearly achieving the original target of 100.0%. Of these, 43% were female. The breakdown by training type was as follows: vocational training: 100%; on-demand training: 96%: apprenticeships: 84%; entrepreneurship training: 100%.  15,061 youth enrolled in the four types of training programs, nearly achieving the original target of 15,800 and surpassing the revised target of 13,700. Of these 43.0% were female (target: 30.0%).  The proportion of youth who are employed or self-employed one year after completion of training was 60.0%, falling short of the original target of 100%.  1,239 youth were awarded with business plan prizes, achieving the target of 1,200.  The tracer study showed changes in the composition of employment: a reduction of those working as family aides (-32 percent), and an increase in the number of employers, notably due to a change from self-employed to employers (+26 percent) in the entrepreneurship training program. According to the ICR, both are signs of improvements in the quality of employment.  On average, income rose by 38 percent for surveyed beneficiaries of the training programs. The tracer study showed that over 90 percent of the participants perceived that the training had a positive impact on their employability and self-esteem. It also showed that the participants were very satisfied with the quality of the training (97 percent on average), that it increased their skills and that it corresponded to market needs (both over 90 percent). Achievement is rated Modest due to significant shortcomings in enrolled youth who completed training and youth who are employed one year after completion. As noted in the ICR (page 21), the original target of 100% employed after training was "unrealistic, even in less difficult environments." Rating Modest OBJECTIVE 2 REVISION 1 Revised Objective To increase access to skills development opportunities for out-of-school youth (revised targets) Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Revised Rationale See outputs reported above. Outcomes  The proportion of enrolled youth who completed the training programs was 95.0%, surpassing the revised target of 80.0%.  The proportion of youth who are employed or self-employed one year after completion of training was 60.0%, achieving the revised target: 60%. Achievement is rated Substantial due to meeting or targets for youth who completed training and who are employed one year after training. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy under the original targets is rated Modest, due to significant shortcomings in achieving the target for the number of man-days of labor created (measuring the objective to increase access to temporary employment for out-of-school youth and in achieving the target for the proportion of enrolled youth who complete training and who are employed one year after completion (measuring the objective to increase access to skills development opportunities for out of school youth). Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Overall Efficacy under the revised targets is rated Substantial. While there were still shortcomings in achievement of the first objective to increase access to temporary employment for out-of-school youth, the targets for the second objective to increase access to skills development opportunities were achieved (as measured by the proportion of enrolled youth who complete training and who are employed one year after completion. Overall Efficacy Revision 1 Rating Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Substantial 5. Efficiency At appraisal, the economic analysis covered both LIPW activities (Component 1) and the training activities (Component 2). Benefits were estimated according to increased earnings associated with the higher training/education levels, with costs drawn from direct project cost projections. The net present value was estimated at US$ 25.2 million (revised to US$16.12 million at restructuring) and the internal rate of return was 15% (revised to 12% at restructuring). At completion (ICR, Annex 4), the figures used in the economic analysis were updated as follows: Project benefits were still the increased earnings due to higher training/education levels, but the ICR noted that the benefits derived from community use of the rural infrastructure (aside from the wages paid out) were not included in the analysis due to difficulty in calculation, although the ICR referenced the ex-post economic and financial evaluation of subprojects financed under similar LIPW projects in other Bank operations, which found that high rates of return and positive benefit-cost ratios for such projects. Project costs comprised the actual amount disbursed, noting there was no opportunity cost associated with program participation for Component 1 as the project targeted unskilled and unemployed youth in urban areas and in off-season agricultural rural areas. The net present value was updated as US$ 9.2 million and internal rate of return was 10%, both lower than expected at project appraisal and restructuring. Also, the rural LIPW paid out a much lower amount on wages but actual costs were more than appraised. Additionally, the ICR cited a recent analysis of the costs of 35 economic inclusion projects around the world, which indicated that this project's costs for delivery of interventions and staffing were slightly above the mean (16% vs. 14%). The grants paid out for the business plans (US$ 3,639 per plan) were also significantly higher than the average grant size for other jobs and livelihood programs analyzed (US$ 416), although the ICR noted that the higher grant amount was deliberately chosen to allow beneficiaries to significantly increase productivity and move from self-employment to entrepreneurship (including through access to commercial credit). With regards to implementation efficiency, the ICR noted several aspects of project design that increased complexity and thus likely decreased efficiency of implementation: different types of LIPW projects in urban vs rural areas, four different training systems, and use of multiple implementing agencies (contributing to lengthy decision chains and additional procurement challenges, transaction costs, and implementation delays). Private sector engagement may have increased efficiency due to synergies but these gains were likely offset by the implementation delays. Administrative costs for project implementation were less than 10% of project costs but there were delays in hiring for the project implementation unit, followed by high turnover in staff and delays in procurement and approvals. Lastly, there were political transitions that contributed to a one year project extension. Efficiency Rating Modest Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Outcome under original targets - Moderately Unsatisfactory Relevance is rated High due to strong alignment with country conditions, government and Bank strategies. Efficacy is rated Modest due to significant shortcomings in achieving the target for the number of man-days of labor created (measuring the objective to increase access to temporary employment for out-of- school youth and in achieving the target for the proportion of enrolled youth who complete training and who are employed one year after completion (measuring the objective to increase access to skills development opportunities for out of school youth). Efficiency is rated Modest due to lower than appraised net present value and internal of return, as well as significant implementation delays. Outcome under revised targets - Moderately Satisfactory Relevance is rated High due to strong alignment with country conditions, government and Bank strategies. Efficacy is rated Substantial as the targets for the second objective to increase access to skills development opportunities were achieved (as measured by the proportion of enrolled youth who complete training and who are employed one year after completion), although there were still moderate shortcomings in achievement of the first objective to increase access to temporary employment for out-of-school youth. Efficiency is rated Modest due to lower than appraised net present value and internal of return, as well as significant implementation delays. According to harmonized OPCS/IEG guidelines on overall ratings for restructured projects, the overall outcome is weighted 34% for the period prior to restructuring (US$ 15.29 million disbursed of US$ 45.01 million) and 66% for the period after restructuring (US$ 29.75 million disbursed of US$45.01 million). Under the original targets, the outcome is rated Moderately Unsatisfactory with a weight value of 1.02 (3 x 0.34); and, under the revised targets, the outcome is rated Moderately Satisfactory with a weight value of 2.64 (4 x 0.66). These weighted values add up to 3.66 (rounded to 4), corresponding to an overall outcome rating of Moderately Satisfactory, indicative of moderate shortcomings in the project’s preparation, implementation, and achievement. a. Outcome Rating Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Moderately Satisfactory 7. Risk to Development Outcome Although youth employment remains a key stated priority of the government, the impacts of COVID and the continued fragility in the security situation will likely affect sustainability of project activities by the government and the prospects for job creation in the economy. Although ownership of LIPW infrastructure is likely high due to the participatory nature of the selection process, maintenance of such infrastructure is highly uncertain, relying completely on voluntary engagement of community members. Also, the training centers are not yet operational with some critical elements yet to be finalized (acquisition of heavy equipment and development of business plans). At the same time, the project contributed to strengthened institutional capacity. As reported in the ICR (pages 27-28), the implementation arrangements for the LIPW program strengthened the relevant unit in the Ministry of Infrastructure and Transportation as well as at the commune level. Also, institutional capacity to deliver training programs increased through developing curricula and training materials, adapting existing modules to reach youth with lower education levels, and providing training to trainers. Lastly, knowledge exchange was supported through study tours to other countries. 8. Assessment of Bank Performance a. Quality-at-Entry The project design drew upon findings and analysis from the 2012 Bank report "Burkina Faso: Employment and Skills Development," which concluded that the skill level of the workforce was too low to support diversification and competitiveness of the economy and that the national skills development system was weak. The project design built upon the country's significant experience (and effectiveness) in using LIPW programs to address income vulnerability, while also introducing training approaches to increase the likelihood of longer-term gains (i.e. higher-earning employment). The training programs explicitly included aspects of preparing youth with little or no schooling for self-employment, given the lack of formal sector jobs in the country. However, there were some shortcomings in other elements of quality-at-entry, including design complexity (urban vs. rural programs, four different training programs), implementation arrangements, M&E design and arrangements (see Section 9), and assessment of potential risks related to implementation (government leadership and ownership, implementation capacity) and the fragile environment. The PAD (page 19) recognized the substantial risk due to delays in decision making and implementation due to involvement of several Ministries, and low implementation capacity. However, the project team suggested that the engagement of the private sector (specifically the Chamber of Commerce as a lead institution) was a bold and pathbreaking step that entailed risk, yet was intentionally pursued with the goal of increasing commitment and sustainability beyond the project period. Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank team provided intensive supervision support in response to the multitude of implementation challenges that were largely outside the control of the Bank. These included limited convening power of the Ministry of Youth (ICR, page 30), security challenges, political transitions (the onset of a transitional government after project launch), and counterpart staffing issues. The project coordination unit (PCU) was not fully staffed until the end of 2015, including the key role of the training specialist, who was not hired until June 2017, three years into project implementation. In addition, there was high turnover in the PCU, with four coordinators in six years, and extended periods without staff in essential functions including safeguards, M&E, and internal audit. The ICR noted one reason for the high staff turnover was a lack of motivation due to salaries: after salaries had been increased for six months without prior non-objection of the World Bank, they were lowered to the initial level which led to departures. The project team noted that despite challenges in counterpart commitment and performance, the team was effective in supporting the client in improving its programs and in making an action plan to ensure continuity beyond project period, including building strong commitment from the Ministries of Youth and Finance by project closing, as reflected by demand for a follow up project. Although the challenges of overly complex design and implementing arrangements were recognized at project restructuring, reducing the complexity (by dropping subcomponents and implementing agencies) would not have proven feasible given the political economy issues. Instead, the project was restructured by decreasing key outcome targets across all project objectives, and the Bank supervision team conducted frequent phone calls with the PCU, which together with in-person meetings where needed in between missions, helped to speed up implementation of the project. M&E shortcomings were partially resolved during the project period (see Section 9). However, there were shortcomings in safeguards compliance and fiduciary performance (see Section 10). Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results chain was overall clear although the M&E framework had some shortcomings. Indicators were not identified for some key project elements (outcomes for the supplemental mechanisms) and targets for some key indicators were overly ambitious (% youth trained who are employed one year after completion of training, % enrolled who complete the training). Monitoring arrangements were somewhat complex as Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) they relied on regular and timely submission of monitoring data from multiple implementing agencies, without use of an IT system to collect and organize the data. Evaluative studies including an impact evaluation for Component 1 and beneficiary surveys for Component 2 were planned (although not subsequently carried out). b. M&E Implementation Some shortcomings in the M&E design were addressed during the project period, such as revising outcome targets downward in line with more realistic expectations and adding indicators to more fully capture achievements. Security challenges led to difficulties in monitoring, although the Bank team made efforts to address the difficulties by hiring a consultant to verify the results of the apprenticeship sub- component and partnering with the European Space Agency (ESA) to develop maps to assess the infrastructure built or rehabilitated through the rural LIPW. The planned evaluative studies were not carried out as originally designed but instead, the project commissioned a final evaluation of the project (conducted by the Statistical Department in the Ministry of Youth) which was turned into a tracer study with support from the World Bank, in order to cover existing data gaps. An impact evaluation of the mobile childcare pilots, financed by a Trust Fund, was also initiated (though not yet completed). However, M&E implementation remained overall inadequate. As noted in the ICR (page 32), with the exception of the Maison de l'Enterprise, the implementing entities showed deficiencies in terms of M&E or did not have the required budget for supervision. The M&E specialist position in the PCU was vacant from April 2018 to June 2019 and no IT software was used to systematically register and check the data reported by the implementing agencies, leading to the need to correct results from one ISR to the next. An international M&E firm was not hired as planned to support the PCU. c. M&E Utilization According to the ICR (page 32), the M&E results were used to inform project restructuring and to inform decisions on changes of implementing agencies. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as an Environmental Category "B" project due to potential impact of LIPW projects, such as loss of vegetation and increase in solid waste. Safeguards policies on Environmental Assessment (OP/BP 4.01), Natural Habitats (OP/BP 4.04), and Involuntary Resettlement (OP/BP 4.12) were triggered. An Environmental and Social Management Framework and a Resettlement Policy Framework were prepared. However, the social safeguards specialist of the PCU was only hired six months Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) before project closure. No social assessment was done and no Resettlement Action Plans (RAP) were put in place for the roads and reclaiming floodable areas. The following challenges regarding safeguards arose during project implementation (as reported in the ICR, page 33): A change of the location of one of the vocational training centers rendered the environmental and social impact assessment obsolete, and construction started without a new assessment; the terms of reference for the construction of the second center were not validated by the National Office for Environmental Evaluations; social and environmental clauses were not included in the tender documents for the vocational centers; and the Youth Center in Manga was reconstructed instead of rehabilitated as agreed, but the safeguards framework was not adapted accordingly. Also, two of the key implementing agencies (PTR-HIMO and Helvetas) did not have an environmental or social safeguards expert contracted during the project period, therefore, the infrastructure constructed or rehabilitated through LIPW was not subject to environmental and social screening. Although an environmental and social safeguards audit was conducted before the project closing in order to address some of the shortcomings in safeguards performance, follow-up on observations from the Bank and the National Office for Environmental Evaluations was not finalized before project closure. According to the ICR, the latter agency did, however, eventually confirm that the project conformed with environmental safeguards. b. Fiduciary Compliance Financial management: Financial management was rated as satisfactory at the beginning of the project but downgraded to unsatisfactory from May 2018 onwards. According to the project team, the downgrades were due to a change in the financial specialist staffing in the project unit, which led to decline in performance. Ineligible expenses of 44 million FCFA for salary increases were identified and subsequently reimbursed. Project audits were completed but the ICR did not state whether there were any qualifications. Procurement: Procurement was rated moderately unsatisfactory throughout most of project implementation. The government did not authorize the project to use Bank procurement procedures until 2015, which led to initial delays. There were persistent delays in the execution of procurement activities, whether for contracts awarded, or those already executed. One reason was that the position of the procurement specialist was vacant for long periods of time during project implementation and previous experience with Bank projects was not taken into account when hiring a replacement, despite Bank guidance. c. Unintended impacts (Positive or Negative) None noted. Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons Lessons drawn from the ICR, adapted by IEG (pages. 37-38):  Complexity of project design can hinder fully satisfactory achievement of a PDO that was set at the appropriate ambition level (albeit with subsequent revision of targets). In the case of this project, the PDO took into account the novelty of the objective and the low capacity environment; however, the implementation arrangements were complex, including a multitude of public and private sector implementing agencies (with resultant long decision chains) and varying capacity levels, and multiple types of training programs.  In a labor market characterized by sufficient availability but low quality of employment, interventions addressing the quality of jobs and related monitoring can capture important labor market impacts. In the case of this project, interventions were designed to lift youth into more productive, better paying jobs, not necessarily employment per se. The original results framework did not include indicators to measure these impacts but a project evaluation was subsequently included to provide evidence of achievement along the lines of quality of employment.  Job-training in and of itself is insufficient if the PDO includes insertion into the labor market. In the case of this project, measuring supply side achievements (i.e. the number of youth trained or the proportion of youth who completed the training) does not reflect successful integration into the labor market. Additional interventions that match supply with the existing labor demand, provide access to capital for self-employment opportunities, and enable beneficiaries to increase savings can be effective. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BF-Youth Employment & Skills Development (P130735) Please Explain To verify longer-term outcomes of integration into the labor market, and to learn more lessons on designing employment/skills development projects as labor market interventions rather than as social safety net interventions. 14. Comments on Quality of ICR The ICR was clear and candid, with a strong focus on development impact. The theory of change was adequately illustrated, but could have drawn out more on the longer-term goals intended (i.e. the project was not intended to be solely a social safety net project). The evidence was of good quality, overcoming some shortcomings in the results framework and M&E design. For example, the analysis of efficacy discussed impacts beyond what the results framework measured i.e. the quality of improvement. Of particular note, the lessons were specific and well-crafted, and highly informative for other operations of this nature. The ICR was overall consistent with guidelines, although it did not provide sufficient information on fiduciary compliance (subsequently provided by the project team), and the main report was somewhat lengthy, reaching 39 pages. a. Quality of ICR Rating Substantial Page 16 of 16