ICRR 12404 Report Number : ICRR12404 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 05/26/2006 PROJ ID :P070204 Appraisal Actual Project Name :Labor Force Development Project Costs 6.20 5.80 Project US$M ) (US$M) Country :Benin Loan/ US$M ) Loan /Credit (US$M) 5.00 5.08 Sector (s):Board: ): ED - Vocational US$M ) Cofinancing (US$M) 0.00 0.00 training (100%) L/C Number :C3397 FY ) Board Approval (FY) 00 Partners involved : Closing Date 06/30/2004 06/30/2005 Evaluator : Panel Reviewer : Division Manager : Division : Manisha J. Modi Kris Hallberg Alain A. Barbu IEGSG 2. Project Objectives and Components a. Objectives The project development objective (PDO) was to assist in the start-up and pilot testing of a new government program to: (a) increase the availability, quality, and cost -effectiveness of labor force training, targeting the informal sector and women, and (b) strengthen public sector capacities to monitor and evaluate labor force training needs and private sector capacities to improve its training capacity . The PDO was not formally revised; however, the quantitative targets were revised downward at the mid -term review. b. Components (or Key Conditions in the case of Adjustment Loans ): 1) Training Grants (estimated: US$3.40 million; actual: US$3.44 million) to support the promotion, identification, evaluation, design, and supervision of subprojects providing training programs for the modern, informal, and agricultural sectors and facilitating labor force entry . Subcomponents included training in the informal, agricultural, and modern sectors, and training to facilitate labor force entry . 2) Monitoring and Evaluation (estimated: US $0.7million; actual: US$0.26): development and the implementation of a monitoring and evaluation (M&E) system. Subcomponents included monitoring and evaluation of training program impact, preparation of an annual report, analyzing and consolidating all relevant information on Benin ’s labor markets and employment, and supporting studies to identify types of skills that are critical to high -growth sectors. The project was originally designed so that this component would mainly finance activities carried out by the Observatoire de l'Emploi (OE), while activities carried out by FODEFCA (Funds for the Development of Vocational Training and Apprenticeship) were budgeted under the institutional development component . 3) Institutional development (estimated: US$2.10 million; actual: US$2.20) to strengthen the organizational, managerial, and strategic capacities of agencies implementing the project . c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The World Bank financed the project as a Learning and Innovation Loan (LIL). The total cost of the project was estimated at US$ 6.20 million—US$ 0.70 million from the government, US$ 0.50 million from beneficiaries, and US$ 5 million from the International Development Association (IDA). The project was implemented between March 2001 and June 2005, with a one-year extension. At project closing, 93% of IDA funds had been disbursed, and the government contribution totaled US $ 0.81 million, exceeding the government's initial commitment . At the beginning of the project, beneficiaries were contributing 15 percent of the training cost; this participation was reduced to 10 percent at the midterm reivew. Reallocations of the credit were approved across various components were approved in March 2003 and November 2004, to cover operating and consultant costs . 3. Relevance of Objectives & Design : The project objectives are relevant and in conformity with the national human resources enhancement policy as well as the Bank’s CAS. In addition, the project drew relevant lessons from other experiences carried out in Benin and in countries with similar socioeconomic contexts, and the project ’s three components are well aligned with the PDO . However, the QAG supervision assessment and the project's midterm review both concluded that the objectives were too ambitious for a LIL. The project was initially designed as an investment loan (SIL), and later converted to a LIL without adequate re-design. In addition, for at least one training subcomponent (training to facilitate labor force entry), there was a disconnect between the objective of training for job creation and the targeting of training to people who were already employed. Finally, although the PAD acknowledged that other employment observatories, particularly in francophone Africa, generally failed to produce useful types and quality of data on labor markets and training, the project still proceeded to place most of the project's M&E "eggs" in the OE's "basket." Given the poor track record of the similar institutions, a different partner should have been chosen, or special measures taken to address OE's weaknesses . This led to weak monitoring and evaluation in the project, but also to the failure to develop local capacity to carry out M&E . 4. Achievement of Objectives (Efficacy) : Increasing the availability, quality, and cost -effectiveness of labor training (particularly for women in the informal sector): Substantial . Generally, output targets were met, quality of training improved, and the targeted population was served. Numbers trained met or exceeded targets in the informal nonagricultural and agricultural sectors, and there was evidence of skills utilization and earnings /productivity impacts of training . However, training to facilitate labor force entry and for the modern sector fell somewhat short of quantitative targets (partly due to lack of demand). Increasing the capacity of the public sector to monitor and evaluate labor force training needs, and the capacity of the private sector to deliver training : Modest . Two impact studies were completed, but with a different design than originally planned. The OE, which was to play a central role in monitoring /evaluation and policy research failed to implement two out of three sub-components. The OE did produce an annual report on labor markets, employment and training. However, it is not clear whether an M&E system was actually put in place, as opposed to producing one-off studies. A database was eventually created, but lack of an M&E framework early on precluded setting a baseline against which project performance could be assessed . FODEFCA's capacity was substantially developed, but OE's results were largely disappointing . Training provider capacity was strengthened and exceeded targets for private sector training providers . A trainer qualification process was developed, which improved both the training application process as well as the quality of training . 5. Efficiency : No economic rate of return analysis was done . Efficiency was to be assessed by assessing cost -effectiveness, piloting progressive cost-sharing approaches, analyzing labor market demand for type /amount of training, and evaluating the economic impact of training provided on beneficiaries . Studies showed that training costs were commensurate with beneficiaries' ability to pay, and in fact beneficiaries contributed a far larger share of training costs than anticipated at the outset . Impact studies showed growth in skills, productivity, and employment . However, in-depth labor market studies were never implemented, and there is some ambiguity about whether the project provided the most cost -effective amount and type of training needed in the labor market . 6. M&E Design, Implementation, & Utilization: Performance indicators in the PAD were imprecise and inadequate to assess the PDO . The monitoring and evaluation system did not come online until after the midterm review . In retrospect, the OE was a poor choice for a partner to implement most of the monitoring and evaluation activities of the project . 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): The major risk identified in the PAD was that FODEFCA may not win the support of its stakeholders, but the project implementation plan did not allocate adequate time on promoting and marketing project activities among potential training providers and trainees . 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory [The ICR's current four-point scale does not allow a "moderately sat." rating]. Although many output targets were met or exceeded, monitoring and evaluation and institutional development constituted two out of three project components which were somewhat less successful (see below). Institutional Dev .: Substantial Modest The project aimed to develop capacity of two main institutions: FODEFCA, and the OE. While FODEFCA's capacity was substantially developed, the project failed to develop capacity in OE. OE even failed to meet output objectives for which it was responsible; ultimately, impact studies were carried out by consulting firms, and data for training needs in selected areas was eventually obtained elsewhere . Sustainability : Likely Likely Bank Performance : Satisfactory Unsatisfactory There were major shortcomings in Bank performance, including project design flaws, delays in project effectiveness, poorly-specified or nonexistent performance indicators, and supervision weaknesses (particularly during the first half of the project). Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: * The effectiveness and sustainability of the training fund mechanism depends in large part on its autonomy as well as an adequate and secure funding source . * Ensuring availability of other inputs such micro -finance and market accessibility is essential to improve the impact of training on beneficiaries, especially in the informal sector . * Adequate evaluation work and suitable communication strategies are necessary to ensure equitable distribution of grants, as well as to ensure effective implementation of a skills development mechanism . 10. Assessment Recommended? Yes No 11. Comments on Quality of ICR: Overall the ICR is satisfactory. However, it could have provided more evidence to support a number of its conclusions. In addition, there appears to be a slight discrepancy in the project costs presented in Annex 2; the project cost by component table shows a latest estimate of US$ 3.34 million for training, while the project financing by component table shows a total of US$ 3.4 million. If the latter table is correct, the total actual project cost would be US$5.90 million, and not US$5.80 million.