Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004379 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD 8269-AG ON A LOAN IN THE AMOUNT OF US$10 MILLION TO ANTIGUA AND BARBUDA FOR A PUBLIC AND SOCIAL SECTOR TRANSFORMATION PROJECT (PSST) ( P126791 ) March 28, 2018 [This ICRR replaces the version published in the Board Operations System on April 3, 2018. Ratings in datasheet have been changed to match the ratings in the main ICR text] Social Protection and Labor Global Practice Latin America and the Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective November 20, 2017) Currency Unit = Eastern Caribbean Dollar (EC$) EC$2.70 = US$1 FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS ABSTEP Antigua and Barbuda Skills Training and Empowerment Programme ABWREC Antigua and Barbuda Waste Recycling Corporation ALMP Active Labor Market Program CDF Comprehensive Debt Framework CMU Country Management Unit E-Waste Electronic Waste Management GARDC Gilbert Agricultural Rural Development Center GRM Grievance Redress Mechanism GOAB Government of Antigua and Barbuda HRM Human Resource Management ICR Implementation Completion and Results Report KPIL Key Project Implementation Leader LD Labor Department LDTU Labor Department Technical Unit LO Local Organization MEFEPA Ministry of Finance, the Economy, Public Administration, Public Broadcasting and Information M&E Monitoring and Evaluation MOU Memorandum of Understanding MTR Midterm Review MTSDP Medium-term Strategic Development Plan NBMIS National Beneficiaries Management Information Systems NEST National Economic and Social Transformation Plan NWEP New Work Experience Program OECS Organisation of Eastern Caribbean States OSEC One-Stop Employment Center PAD Project Appraisal Document PDO Project Development Objective PMU Project Management Unit PPA Project Preparation Advance PSC Public Service Commission PSMS Public Service Management System PSST Public and Social Sector Transformation Project RPS Regional Partnership Strategy SSB Social Security Board TP Training Program TEP Temporary Employment Program UPP United Progressive Party Regional Vice President: Jorge Familiar Country Director: Tahseen Sayed Senior Global Practice Director: Michal J. Rutkowski Practice Manager: Pablo Gottret Task Team Leader(s): Karla J. McEvoy, Juan Martin Moreno ICR Main Contributor: Maria R. Puech Fernandez TABLE OF CONTENTS DATA SHEET .......................................................................................................................................1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES .................................................................5 A. CONTEXT AT APPRAISAL ................................................................................................................... 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ............................................. 8 II. OUTCOME ................................................................................................................................9 A. RELEVANCE OF PDOs ........................................................................................................................ 9 B. ACHIEVEMENT OF PDOs (EFFICACY) ............................................................................................... 10 C. EFFICIENCY ...................................................................................................................................... 12 D. JUSTIFICATION OF OVERALL OUTCOME RATING ........................................................................... 13 E. OTHER OUTCOMES AND IMPACTS (IF ANY).................................................................................... 13 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ...........................................14 A. KEY FACTORS DURING PREPARATION ............................................................................................ 14 B. KEY FACTORS DURING IMPLEMENTATION ..................................................................................... 15 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ............. 16 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................................................... 16 B. ENVIRONMENTAL SAFEGUARD, SOCIAL SAFEGUARD, AND FIDUCIARY COMPLIANCE .................. 17 C. BANK PERFORMANCE ..................................................................................................................... 18 D. RISK TO DEVELOPMENT OUTCOME ............................................................................................... 20 V. LESSONS LEARNED AND RECOMMENDATIONS.........................................................................20 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................................22 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................................33 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................................35 ANNEX 4. EFFICIENCY ANALYSIS .......................................................................................................36 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ................. 37 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................................48 ANNEX 7. THE TEMPORARY EMPLOYMENT PROGRAM ......................................................................49 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P126791 Public and Social Sector Transformation Project (PSST) Country Financing Instrument Antigua and Barbuda Specific Investment Loan Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Antigua and Barbuda Project Management Unit, Ministry of Labour Project Development Objective (PDO) Original PDO The objective of the Project is to assist the Borrower in: (i) strengthening its capacity in managing public policies and the public service; (ii) improving the efficiency of social protection spending through an integrated monitoring and targeting system; and (iii) improving the income and employability of the vulnerable population through temporary employment and training programs. Page 1 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) FINANCtNG Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing . 10,000,000 2,564,254 2,564,254 IBRD-82690 Total 10,000,000 2,564,254 2,564,254 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 10,000,000 2,564,254 2,564,254 KEY OATES Approval Effectiveness MTR Review Original Closing Actual Closing 05-Jun-2013 12-Sep-2013 . 10-Apr-2016 31-Dec-2018 30-Sep-2017 RESTRUCTURtNG AND/OR ADOtTrONAL FINANCING - ·- - - - Date(s) ~mourt! Di~~~r~~d tus?~l Key Re~ision~ _ 28-Sep-2017 2.90 Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories KEY RATfNGS Outcome Bank Performance M&E Quality .Highly Unsatisfactory U nsatisfactorv Modest RATlNGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements {US$M) 01 22-Nov-2013 Satisfactory Satisfactory .57 Page 2 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) 02 21-Jun-2014 Moderately Satisfactory Moderately Satisfactory 2.19 Moderately 03 23-Dec-2014 Moderately Unsatisfactory 2.19 Unsatisfactory 04 29-Jun-2015 Unsatisfactory Unsatisfactory 2.19 05 22-Dec-2015 Unsatisfactory Unsatisfactory 2.19 06 31-May-2016 Unsatisfactory Unsatisfactory 2.51 07 19-Dec-2016 Unsatisfactory Unsatisfactory 2.68 08 19-Jun-2017 Unsatisfactory Unsatisfactory 2.93 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 47 Central Government (Central Agencies) 12 Other Public Administration 35 Social Protection 53 Social Protection 41 Public Administration - Social Protection 12 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 40 Public Administration 40 Administrative and Civil Service Reform 40 Social Development and Protection 39 Social Protection 39 Social Safety Nets 25 Social protection delivery systems 14 Page 3 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Human Development and Gender 22 Labor Market Policy and Programs 22 Labor Market Institutions 11 Active Labor Market Programs 11 Private Sector Development 100 Jobs 100 ADM STAFF Role At Approval At ICR Regional Vice President: Hasan A. Tuluy Jorge Familiar Calderon Country Director: Francoise Clottes Tahseen Sayed Khan Senior Global Practice Director: Keith E. Hansen Michal J. Rutkowski Practice Manager: Mansoora Rashid Pablo Gottret Karla J. McEvoy, Juan Martin Task Team Leader(s): Juan Martin Moreno Moreno ICR Contributing Author: Maria R. Puech Fernandez Page 4 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Antigua and Barbuda is an upper-middle-income country, with better economic indicators than some of its Organisation of Eastern Caribbean States (OECS) neighbors. 1 With a population of 89,620 (2011), it had a poverty headcount of 18.3 percent at project appraisal, compared to poverty rates above 28.8 percent for most other OECS countries of similar size. 2 Between 2009 and 2011, however, the country experienced its worst recession in decades, following the global financial crisis. This crisis was exacerbated by the collapse of the Stanford Group of Companies in 2009, which left over 1,200 workers unemployed (approximately 3 percent of the workforce). This was followed by the collapse of one of the island’s largest domestic banks, Antigua and Barbuda Investment Bank, in 2011. The severe economic and financial crisis triggered a rise in unemployment, 3 which, coupled with Antigua and Barbuda’s budgetary constraints (due in part to a public sector wage bill that accounted for 40 percent of recurrent expenditures and represented 9 percent of gross domestic product in 2008–2009), had left the country with limited choices for recovering from the crisis. While the Government implemented some successful fiscal reforms in 2010 that stabilized the macro fiscal situation, as of 2012, economic activity remained sluggish and below potential. 2. To address the situation, the Government of Antigua and Barbuda (GOAB) embarked on a National Economic and Transformation Plan (NEST), under which public and social sector reforms were key components. The NEST objectives were to: (a) relieve budget pressure caused by the size and nature of the public sector; 4 (b) establish well-targeted social protection programs, capable of protecting the economically disadvantaged and vulnerable populations, especially those affected by the crisis; and (c) develop a well-functioning social safety net targeted to the poor. 3. Given the severity of the crisis, for the first time since independence in 1981, the GOAB requested World Bank assistance. The Government and the World Bank opted for a multisector investment loan that would carry out public sector and labor market reforms. 1 Antigua and Barbuda is one of the six countries that conform to the OECS. It is one of the two IBRD countries in the group. The other four are IDA countries. 2 Antigua and Barbuda became independent from the United Kingdom in 1981. The Labour Party had consecutively held the Government until 2004, when the United Progressive Party (UPP) won the elections for the first time. The UPP was in office until 2014, when the Labour Party won national elections and brought a new set of priorities for the country. 3 In 2011, in the absence of an official rate, a proxy unemployment rate was calculated using the latest figures from the Social Security Board (SSB). The proxy rate estimated unemployment at 10 percent, four times higher than in 2008. 4 According to the Project Appraisal Document (PAD), rigid public service establishment rules prevented expedient hiring, thus creating incentives to bypass the regulation and grow public employment outside the established public service. This resulted in two employment streams: the permanent public service, governed by the Constitution and other regulations and known as the ‘established civil service’, and the ‘non-established civil service’, formed through appointments authorized by the Cabinet over the last three decades. Page 5 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) 4. The Antigua and Barbuda Public and Social Sector Transformation Project (PSST) was aligned with NEST objectives, 5 as it focused on improving public expenditure efficiency, while building active labor market programs (ALMPs) and improving service delivery. The project was also consistent with the World Bank’s Regional Partnership Strategy (RPS) for the OECS for 2010–2014, as it centered on the RPS’s priorities of protecting and improving human capital, enhancing competitiveness and stimulating growth, and strengthening institutional and organizational capacity. Theory of Change (Results Chain) Table 1. Theory of Change Components Activities Outputs Outcomes 1 Build effective • Technical assistance to • Establishment of a • Enhanced Government institutions for strategic develop skills with the policy unit under the capacity to improve management of public sector Cabinet Secretary quality decision making government policies • Training of civil servants • Development of a • Improved planning and • Workshops and regulatory framework coordination to better consultations for policy coordination allocate Government • Development of websites and strategic planning resources, aligned with policies 2. Modernize Human • Preparation of an initial • Data incorporated in • Integrated public Resource Management public employee register the civil service service employment (HRM) • Technical assistance to management regime design a single public Government database. • Effective PSMS service structure • Staff schedule with • Comprehensive public • Technical assistance to profiles service data develop a Public Service • Plan for staff Management System optimization (PSMS) • Analysis of capacity- building needs • Training plan to meet these needs • Policy and legislation for PSMS defined 3. Improve the • Technical assistance to • Creation of a unified • Enhanced capacity to efficiency of social design the National targeting system and a design, implement, and protection spending Beneficiaries single registry for monitor social policies Management beneficiaries and interventions Information System • Census of potential (NBMIS) beneficiaries • Proxy means test designed • M&E tools designed 4.Support Active Labor • Temporary Employment • Qualifying beneficiaries • Increased employability Market Programs Program (TEP) trained and income of (ALMP) • Training Program (TP) • One-stop employment vulnerable population 5 The NEST Program had four focus areas: (a) the Fiscal Consolidation Program, (b) the Economic Action Plan, (c) the Social Transformation Plan, and (d) the Financial Sector Stability Program. Page 6 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) • Provision of equipment center (OSEC) • LD capacity to monitor to the Labor Department strengthened and evaluate (LD) employment programs strengthened Project Development Objectives (PDOs) 5. The objective of the Project was to assist the Borrower in: (i) strengthening its capacity in managing public policies and the public service; (ii) improving the efficiency of social protection spending through an integrated monitoring and targeting system; and (iii) improving the income and employability of the vulnerable population through temporary employment and training programs. Key Expected Outcomes and Outcome Indicators Table 2. Key Outcomes and Indicators Outcome Outcome Indicators Strengthening capacity in • Percentage of policies approved by the Cabinet, aligned with the managing public policies and the medium-term Government priorities and reflected in the Government’s public service medium-term Strategic Development Plan (MTSDP) (projected at least 70 percent) • Percentage of career public servants covered by the new Public Service Act establishing a single employment regime (projected 100 percent) • Percentage of ministries and departments that have capacity to manage their human resources within the delegated authority to hire, administer professional development and disciplinary action, and manage performance and apply incentives (projected 80 percent) Improving the efficiency of social • Percentage of poor Antiguans and Barbudans identified by National protection spending through an Beneficiaries Management Information Systems (NBMIS) and enrolled integrated monitoring and in a social protection program (projected 50 percent) targeting system • Percentage of beneficiaries in the Single Registry of Beneficiaries identified by the NBMIS as non-eligible (projected 10 percent) Improving the efficiency of social • The Temporary Employment Program (TEP) beneficiaries’ labor income protection spending through an is at least 40% higher than the one for non-enrolled eligible integrated monitoring and beneficiaries over the six month period of program implementation targeting system • The Training Program (TP) beneficiaries’ placement rate is at least 40% higher than the one for non-enrolled elegible beneficiaries three months after the end of the internship phase Components 6. Component 1: Build effective institutions for strategic management of government policies (US$1.062 million IBRD). This component aimed to improve the borrower’s decision-making capacity, through coordinated planning to facilitate alignment of resources with Government policy priorities. Support included creation of a policy unit within the Cabinet and within line ministries, development of guidelines for these units, establishment of a regulatory framework for policy coordination and strategic planning, and training in policy making for line ministries. Page 7 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) 7. Component 2: Modernize Human Resource Management (HRM) (US$2.465 million IBRD). This component aimed to modernize the borrower’s civil service by establishing an integrated public employment regime and developing an effective PSMS. Activities included developing a public employee register, carrying out organizational and functional reviews of government ministries, developing a new remuneration and benefits policy, and designing a new PSMS, among other things. 8. Component 3: Improve the efficiency of social protection spending (US$1.207 million IBRD). This component was to support the Ministry of Health, Social Transformation, and Consumer Affairs in its design and implementation of a unified targeting system and a single registry for beneficiaries of social protection programs. 9. Component 4: Support Active Labor Market Programs (ALMP) (US$4.10 million IBRD). This component was designed to support the Ministry of National Security and Labor in the (a) establishment of a TEP and a TP and (b) strengthening of its employment and counselling service, its capacity to operate the competence certification system, and its capacity to coordinate, monitor, and evaluate ALMPs. 10. Component 5: Project Management (US$1.141 million IBRD). This component financed the staff of a Project Management Unit (PMU), established within the Ministry of Finance (MEFEPA), to manage the fiduciary and administrative aspects of the PSST, including the contracting of annual audits. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 11. The PDO and the outcome targets were not revised. 12. Despite a lengthy dialogue on revising PDOs and activities, which started shortly after the new Government took office in June 2014, the project was never restructured during implementation. Once the Government requested the cancellation of the operation in July 2017, a restructuring was carried out, but only to cancel the undisbursed loan amount, reallocate funds between expenditure categories, and change the closing date to reflect the cancellation. Revised PDO Indicators 13. The PDO indicators were not modified. Revised Components 14. Components were not modified. As previously explained, shortly after taking office in June 2014, the current administration expressed its wish to restructure the project to decrease the loan amount and focus resources on an energy project. During the ensuing discussion to restructure the project accordingly, the Government changed its request to include the cancellation of Components 2 and 3, an expansion of Component 1, and the maintenance of Components 4 and 5. During the dialogue to redesign the project based on this request, the Government informed the World Bank that, in addition to cancelling Components 2 and 3, it wanted to partially cancel Component 1, expand Component 4, and keep Component 5. The last restructuring request included all of this but added the complete cancellation of Page 8 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Component 1. Hence, the final restructuring paper, dated August 2016, only included Components 4 and 5. This restructuring was never processed, and in July 2017, the Government sent a formal request to cancel the loan entirely. Other Changes 15. As noted earlier, the project was cancelled at the Government’s request. Rationale for Changes and Their Implication on the Original Theory of Change 16. The theory of change remained the same throughout the life of the project, as the project was not restructured until its closure. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: Modest 17. The PDOs at appraisal were in line with the priorities set in the RPS covering FY2010–2014, and they continue to be relevant under the current RPS for 2015–2019. Discussed by the World Bank Board in November 2014, the latest RPS focuses on creating the conditions for sustainable and inclusive growth in Antigua and Barbuda, Dominica, Grenada, St Kitts and Nevis, St Lucia, and St Vincent and the Grenadines. 6 Anchored in the Comprehensive Debt Framework (CDF), the RPS focuses on three areas of engagement: (a) competitiveness, (b) public sector modernization, and (c) resilience. The RPS is being extended by one year, to 2020, although no lending is planned. 7 18. Given this, the PDOs remain relevant today. However, the project itself lost relevance as the vehicle with which to achieve them. This was due to the new administration’s desire to address public sector transformation outside of the project 8 and to focus loan proceeds more on employment and jobs, but being tepid on the type of employment supported by the project. In addition to complexities in trying to align project components with new Government priorities, starting in October 2015, a key project legal covenant was in non-compliance status, namely having an adequately staffed Project Management Unit (PMU). Therefore, the PDO relevance is rated Modest. 6 The CDF acknowledges the multifaceted nature of the OECS challenges and aims at helping governments in the design of country-specific solutions to their high indebtedness. The CDF focuses on the interrelated areas of improvements in competitiveness, reduction in sovereign debt levels, fiscal adjustments to ensure macro sustainability, and enhanced resilience to shocks are interrelated aspects. OECS governments have embraced the CDF and used it in forming their own reform strategies and priorities. 7 Instead, the World Bank intends to stay engaged through dialogue and analytical work. 8 The newly elected Government wanted to address public sector inefficiencies through initiatives such as e-government and improving productivity. Page 9 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome Rating: Negligible PDO1: Strengthening its capacity in managing public policies and the public service Rating: Negligible 19. This PDO was not achieved, and no targets were met, as there were limited activities carried out in this area. Only a consultancy on the Regulatory Framework for Policy Coordination and Strategic planning was carried out between December 2013 and June 2015 to develop an approach to strengthening policy management. The following activities were completed: • A set of new and revised guidelines and templates to improve the efficiency of Cabinet meetings • A presentation of the new procedures and tools to the Cabinet Ministers and Ministers of State, the Cabinet Secretary and the staff of the Cabinet Secretariat and Permanent Secretaries, and other senior staff • Initial training of a select group of policy analysts • A draft Cabinet Manual outlining Cabinet procedures, templates, and the roles and responsibilitites of all participants in the new system • Presentation of the draft manual to key Government officials • Training the staff of the Cabinet Secretariat and key ministry staff on the content of the manual and on the new cabinet process • Drafts of policy documents prepared following the templates, prepared by Government staff with a consultant’s guidance • Final Cabinet Manual that incorporated feedback from key Government figures 20. Regarding the modernization of the public service, there was no achievement of this dimension of the PDO, although a number of activities were completed, as follows: Page 10 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) • A consultancy was hired to support the Government as a Policy Adviser on the public service reform. The Policy Adviser completed (a) a draft policy paper on the topic of public service modernization for Cabinet of Ministers consideration; (b) presentations of the paper to key stakeholders; (c) a guidance note on the coverage of the Public Service Law, identifying the issues to regulate and to support the implementation of the policy option to be chosen by the Government; and (d) a final Policy Paper that included an annex with the stucture of the issues to regulate, a guide to the stakeholder comments, how these comments were incorporated, and recommendations from these consultations. • A consultancy on Functional Reviews of key Government departments: the Budget Office, the Treasury Department, the Strategic Development Unit in the MEFEPA; the LD, and the Department of Social Policy, Research, and Planning. This consultancy included the following activities: (a) a consultation with key stakeholders on the process and implementation plan; (b) a report with defined porposals on changes in tasks functions, organizational structure, capacity building, and professional development; (c) an analysis on the strengths and weaknesses by department reviewed; and (d) a final report with an action plan and a change management strategy in line with the Government needs. 21. The Government did not pursue any of the recommendations made in the public sector modernization paper nor in the functional reviews. PDO 2: Improving the efficiency of social protection spending through an integrated monitoring and targeting system Rating: Negligible 22. The PDO was not achieved. None of the PDO indicators were met, and there was no progress in the development and implementation of an integrated monitoring and targeting system. PDO 3: Improving the income and employability of the vulnerable population through temporary employment and training programs Rating: Modest 23. This PDO was partially achieved owing to the implementation of the TEP. The TEP sought to help less experienced unemployed individuals find temporary employment and then enter or reenter the labor market. The TEP provided training on basic vocational and life skills and a job experience. Eligible beneficiaries were low-income individuals between ages 17 and 50 years, who were not registered as an active employee or self-employed in the records of the SSB. 24. By project completion, the TEP had provided temporary employment to a total of 136 individuals divided into three cohorts. A tracking survey of the first cohort of beneficiaries showed that participation in the TEP had a causal impact on placement/employment and on income levels of beneficiaries (see Efficiency section). Of the 132, a total of 98 completed the six-month subprojects program. As shown in the TEP evaluations, a significant group of beneficiaries (56 percent in the first cohort, 70 percent in the second, and 63 percent in the third) felt that the knowledge and information gained from the TEP met their expectations. Around 56 percent of the first cohort, 40 percent of the second, and 66 percent of the third felt that the knowledge gained from the program would be beneficial to their job search. Page 11 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) 25. The LD, the PMU, and the local community organizations effectively coordinated to address the challenges the TEP faced. Most of these had to do with managing behavioral problems by a small number of participants in the first two cohorts. The inclusion of the life skills training in the TEP sought to identify and mitigate/alleviate psycho-social issues facing participants. As the evaluations revealed, participants reported a boost to their self-esteem through their participation in the life skills training, as well as through the participation in math and English remedial classes. 26. Preparations to launch the TP began in August 2016; however, it was never implemented because by that time, the Government was not in compliance with the legal covenant of adequately staffing the PMU, and the World Bank would not approve the restructuring without this compliance. Because of this bottleneck, in February 2017, the Government informed that it was considering cancelling the loan, which put the launch of new activities (that is, the TP), on hold. 27. Despite the TEP’s achievements, it had limited impact on achieving the component’s objective. This was due to (a) the less than envisaged number of beneficiaries who participated in the three cohorts (136 versus a target of 300) and (b) the fact that the TP, which would have complemented the TEP, was not implemented. 28. The project did not achieve its first two objectives. It partially achieved its third objective, as a result of the implementation of the TEP. Therefore, the overall rating for Efficacy is Negligible. C. EFFICIENCY Assessment of Efficiency and Rating 29. The efficiency rating is Negligible. The PAD did not include a cost-benefit analysis. It did highlight that the economic rationale of project activities, geared to improving the income and employability of the population, was related to the improvement of the ability of the unemployed to reenter the labor market, to avert temporary poverty, and enhance the welfare of the poor. 30. Given this, the Implementation Completion and Results Report (ICR) does not carry out an economic analysis, but it does summarize the results of the tracking survey of the first cohort of the TEP. This survey indicated that, at the end of the first year of participation in the TEP, beneficiaries experienced a higher placement/employment rate (3 percent) and a higher average salary rate (16 percent higher) than the control group. The tracking survey also showed that female beneficiaries who completed the TEP reported incomes 81 percent higher than those in the control group.9 Additional details on this survey can be found in annex 4. 31. The overall project cost was much lower than anticipated, given that many activities were never implemented. Component 5 costs were a higher percentage of loan proceeds than expected, due to (a) the lack of implementation of Components 1, 2, and 3 and (b) the GOAB decision not to absorb the PMU 9 The male counterparts, however, reported incomes 16 percent lower than that of the males in the control group. The survey indicated that this disparity in gender impact was inferred to be related to conditions of the general economy, since both male and female beneficiaries completed the same TEP program. Page 12 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) into the civil service, maintaining the PMU as a separate unit financed by the project. While the relative cost of the PMU is high for World Bank project standards, it is important to note that the unit managed two World Bank grants, 10 unrelated to the PSST, which were not envisaged in the PAD. D. JUSTIFICATION OF OVERALL OUTCOME RATING 32. The overall outcome of the project is rated Highly Unsatisfactory, as there were severe shortcomings in its efficacy and efficiency: • The PDO relevance is rated Modest, given the change in priorities by the incoming Government and the failure to revise the PDO and the project outcomes through a restructuring. • The efficacy of the project is considered Negligible (negligible regarding the public sector reform outcomes, Components 1, 2, and 3, and modest regarding the labor market outcomes, Component 4). • The efficiency of the project is rated Negligible; the rating for Components 1, 2, and 3 is negligible, as the resources used under these components did not translate into any outcomes. The rating for the labor component is rated modest, as some progress toward indicator targets was made, although implementation delays hampered that progress, and eventually the project was cancelled. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 33. As previously noted, the project financed evaluations of the TEP cohorts to provide feedback to improve the design and implementation of the program. Because of these evaluations, the TEP design was changed to make the program more accessible to women. For example, after the first round of training, the program began providing child care stipends so that women with children could participate in the TEP. In addition, just before the Government sending its request for cancellation, the World Bank team had worked with the Labor Department Technical Unit (LDTU) and PMU to create a Grievance Redress Mechanism (GRM), as well as Codes of Conduct to be included in the agreements signed by local organizations (LOs). The purpose was to create a more secure work environment for all beneficiaries, particularly for women. However, due to the loan’s cancellation, the development of these was never completed. Institutional Strengthening 34. The project contributed to some strengthening of the institutional capacity related to labor activities. The project supported strengthening capacity at the OSEC. Seven people were hired as civil servants to work at the OSEC to support the implementation of the TEP. These staff interviewed beneficiaries, registered them in the system, and provided job counseling, among other tasks. The project 10 The two grants were Supporting Economic Management in the Caribbean (SEMCAR) and Roadmap. Page 13 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) also financed the equipping of the OSEC throughout the life of the project. The Government relocated the OSEC to a more central and suitable location in 2017, as foreseen in the PAD. 35. The LDTU supervision capacity for labor programs was strengthened as well during the life of the project. The PSST contributed to the establishment of a supervision mechanism for the TEP that was well implemented. Close supervision of the TEP allowed for the identification of issues with beneficiaries and LOs and for a quick response to implementation problems. While supervision itself was carried out by a consultant, the supervision framework has remained even after project completion, and it is likely to be applied for an entrepreneurship program to be financed by the Caribbean Development Bank, in support of rehabilitation efforts after Hurricane Irma. Mobilizing Private Sector Financing 36. Not applicable. Poverty Reduction and Shared Prosperity 37. Not applicable. Other Unintended Outcomes and Impacts 38. Not applicable. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 39. Given that Antigua and Barbuda was a new borrower, preparation took some time, but not an undue amount. Preparation began in May 2011 with the dissemination of the Social Protection Assessment for Antigua and Barbuda. A Concept Note meeting was held in October 2011, and a Project Preparation Advance (PPA) was prepared and approved by April 2012. The PPA, amounting to US$0.8 million, was used to ‘prepare for preparation’, as it were. Because Antigua and Barbuda had not borrowed from the World Bank, the PPA was used to carry out financial management and procurement assessments, to gather information to design Components 1 and 2 and to establish a PMU, with well described staff profiles and an Operational Manual. 40. Solid analysis of the civil service and social safety net situation in the country. The project’s technical issues were well assessed and researched. Several studies of the public sector had been carried out under a World Bank Institutional Development Fund grant, illustrating the need for civil service reform, aimed at integrating the large cohort of so called non-established public employees under a single legal framework governing public service. A World Bank-financed Social Protection Assessment, completed in 2011, identified duplication and fragmentation of Antigua and Barbuda’s social programs and made recommendations for improving the country’s safety net. The project design relied heavily on these findings and recommendations. Page 14 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) 41. Strong Government commitment at the time of preparation. The project had strong champions in the MEFEPA and the Ministry of Public Administration during preparation. This was important in having the Government understand both how the reforms would be carried out and the implications of the changes. Most importantly, it was critical in building ownership at the top level of the Government and subsequent approval of the loan, which was the first lending operation between Antigua and Barbuda and the World Bank. 42. Insufficient involvement of key stakeholders. While there was strong commitment at top levels of the Government, there was not such a strong commitment at other levels of the civil service and society. The World Bank team worked with the Council of Permanent Secretaries that was formed to participate in discussing the reform options. The World Bank met a group of relevant high- and mid-level civil service professionals several times during preparation to discuss the objectives and activities of the project. Despite these exchanges, there was not enough ownership of the proposed reforms to successfully carry out project activities. Neither were representatives from opposition parties sufficiently consulted. Considering that the project included politically sensitive reforms and it was close to the 2014 elections, there was not enough direct involvement of the opposition in the project preparation conversations. B. KEY FACTORS DURING IMPLEMENTATION 43. The change in government changed the dialogue and dynamics: a new government was elected in June 2014, shortly after Project effectiveness in late 2013. The new administration had different policy priorities compared with the previous administration. This resulted in delays in implementation and the decision to restructure the project. While working on the parameters of the restructuring, the Bank and counterpart teams focused on getting the TEP up and running, and helping the new administration to identify some small-scale pilot projects in public sector transformation. The activities under the public sector pilot projects were ultimately discontinued given the Government’s increasing focus on employment. While more of the original project activities were being dropped, the GOAB was reluctant to cancel the project. The Bank team therefore concentrated on working with the Government on an acceptable proposal to restructure the Project, focusing on labour and employment. 44. Difficulties maintaining a Government focal point to lead Project implementation. The new administration replaced the original focal point with another appointee. However, this position became less influential, after June 2014, given the Government’s new priorities, and this affected progress in the public sector components. The labour component made more progress, both because it was politically more palatable and because implementation depended on the Ministry of Labour, which had a good technical team dedicated to the task. The PMU also facilitated the implementation of the labour component. 45. Resistance to the public-sector reforms supported by the Project. The public-sector work envisaged a substantial institutional reform, proposing the redistribution of authority in governance of the civil service from the Public Service Commission (PSC) to a new entity responsible for civil service management. This was a difficult recommendation, as the Commissioner of the Public Service is one of the highest public offices in civil service management. Concerns were also expressed by some non- Page 15 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) established civil servants, who were not bound by the civil service rules and obligations. 11 Also, although the diagnostic recommending changes to the cabinet agenda, central coordination and strategic planning was completed, the new administration did not pursue these recommendations. 46. The change of administration resulted in staffing issues of the PMU. At Project launch, the PMU was fully staffed with consultants. Staff profiles were well defined, and an operational manual was prepared. The idea in the Project design was that government would eventually absorb PMU staff into the civil service. However, the new administration did not pursue this approach. In October 2015 the PMU coordinator resigned and was not replaced. In addition, the PMU’s Procurement Specialist became Acting PMU Coordinator, which resulted in a conflict of interest (that is, the PMU Coordinator signed off on procurement decisions). The decision not to absorb the PMU (Component 5) resulted in the component’s costs as a percentage of total Loan resources being much higher than anticipated. 47. Limited information sharing between government agencies. The Social Security Board (SSB) was responsible for identifying, based on its records, individuals eligible for the TEP, and for providing that list to the LDTU. However, the SSB was reluctant to share that information until a Memorandum of Understanding was signed between the LDTU and SSB. The MOU required beneficiaries to authorize the sharing of their information, which facilitated data exchange between the two agencies --but which also took time, given the nature of the requirement. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 48. The PAD included a description of the M&E arrangements and a Results Framework with indicators to be monitored for all components and PDOs. The Cabinet Office and the Establishment Department would monitor the indicators for Components 1 and 2, with support from the PMU and the Public Sector Transformation Unit. The system to monitor Components 3 and 4 was not fully in place; instruments were to be created under the project to do so. The project did foresee the development of a monitoring module within the NBMIS and within the LD. Additionally, it planned to implement the tracking survey for the TEP to monitor the placement rate and income level of beneficiaries of the employment programs. 49. The M&E design was ambitious and there were some weaknesses: the indicators for public sector transformation were difficult to measure and not well defined (that is, outcome indicator 3); it trusted the M&E process for two components to tools that still had to be created, which is not best practice, and ultimately, the tools were never created. 11 This group represents approximately 4,000 public employees, which could not be easily transferred to the civil service due to salary differences and in some cases inability to meet the civil service professional requirements. Page 16 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) M&E Implementation 50. The M&E arrangements were not executed as planned. The PMU was responsible for consolidating monitoring information from the technical units and preparing progress reports to share with the World Bank. The PMU carried out these responsibilities to the best of its abilities; nevertheless, the effectiveness of this task was undermined by several factors: (a) the PMU was understaffed for a substantial part of implementation, as explained in section III.B, and (b) the PMU was never absorbed into the Government, which affected its ability to coordinate data sharing. 51. There was hardly any implementation for Components 1 and 2, 12 thus little activity to monitor. No activities were carried out under Component 3. While Component 4 was affected by the very slow implementation, three of the anticipated four rounds of training were carried out, and evaluations for the three cohorts were completed. These evaluations captured the assessment of beneficiaries and the main implementation challenges for each of the cohorts and provided recommendations for improving the TEP’s design. The quality of the evaluation reports was adequate and improved over time by including an analysis of the data gathered by the LDTU and additional information on subprojects. As previously noted, a tracer study of the first cohort was carried out as well. It is worth highlighting that the borrower would have carried out a second tracer study if the project had not been cancelled. M&E Utilization 52. There was hardly any data gathered on the project performance. Thus, there were no data to be used to inform project management and decision making, except for the TEP. Justification of Overall Rating of Quality of M&E 53. The overall quality of M&E is Modest, since other than the TEP, no other M&E activities took place. B. ENVIRONMENTAL SAFEGUARD, SOCIAL SAFEGUARD, AND FIDUCIARY COMPLIANCE 54. During the April 2016 Midterm Review (MTR), the TEP team indicated that there was a strong demand for certain kinds of subprojects that were not eligible under the original TEP design. These mostly included projects such as rehabilitation of walls in protected areas and eradication of an invasive species on the island. The World Bank team agreed that these activities would provide good work experience for beneficiaries, were in demand by the public, and would benefit the country environmentally and socially. Therefore, with assistance from the World Bank team, the Government prepared a new environmental and social framework to trigger additional World Bank safeguards (cultural heritage and pesticides) under the PSST. The idea was that the new Environmental and Social Framework, which was consulted and approved by August 2016, would become part of the project restructuring. 55. Financial management. The overall financial management for the project is considered Moderately Satisfactory. There were no compliance issues during implementation, nor substantial observations in any of the audit reports during the life of the project; however, there was consistently late reporting with errors and incomplete information in the reports. The World Bank provided substantial 12 The initial activities were carried out under these components in the first months of the project and during the PPA. Page 17 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) capacity building and training to the PMU staff and regularly accompanied the PMU during implementation and particularly at closing. 56. Procurement. Being the first World Bank lending operation for Antigua and Barbuda, it is perhaps not surprising that there were challenges and delays in procurement. As was the case for financial management, the procurement activities were affected by understaffing issues. (the Procurement Specialist worked as the PMU Coordinator for the project for an extended period, performing two functions). Despite the World Bank providing training to the PMU to strengthen capacity and reinforce understanding of the procurement process, there were challenges around communication between the PMU and the World Bank on several activities: on hiring the staff for the PMU, on the revision and update of the procurement plan, on the revision of the Operational Manual for the TEP, and for the launching of the TP, for example. C. BANK PERFORMANCE Quality at Entry Rating: Unsatisfactory Strengths 57. The World Bank public and social protection sector teams worked in close coordination, jointly carried out the preparation missions, and completed a sound analysis of the country’s sector context. The World Bank team was strong technically and operationally, and it used technical assessments to design the project and secured a PPA to assist preparation. Weaknesses 58. Although the project was well prepared and technically sound, it tackled two sectors in a single operation, with no lack of complexity and abundance of sector issues. The project was ambitious given the challenging nature of the public sector reforms. In this context, additional support to the Government in building a broader national consensus around the reforms would have been beneficial. Given the World Bank experience in other countries, it could have included activities—stakeholder consultations 13 in various forms and information and education campaigns—in the project to help build that consensus. While the project included an in-depth analysis of the risks around the project and the risk ratings were mostly realistic, the World Bank team could have better assessed the political economy of Government officials around the transformation of the public sector, in the context of impending national election. 59. The M&E framework included several public sector transformation indicators that were subjective and hard to measure. 13 The World Bank consulted regularly with the Council of Permanent Secretaries, technical stakeholders and beneficiaries, members of Parliament, and trade unions. The team did not meet with the leader of the opposition party before the elections. The World Bank underestimated the political economy of the reforms. Page 18 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Quality of Supervision Rating: Moderately Unsatisfactory 60. Though the World Bank carried out regular missions and virtual meetings between 2014 and 2016, it could have done more to restructure the project so that the PDOs could be achieved. Political sensitivities surrounding cancellation of the project also delayed the Government’s ultimate decision to cancel the loan. The Bank team believed it could arrive at a solid restructuring proposal, and given that this was the first operation in Antigua and Barbuda, efforts were focused on continuing with a restructured project. Strengths 61. The World Bank provided training on procurement and financial management and carried out a continuous dialogue with the borrower to support implementation. Though there were misunderstandings within the borrower teams regarding procurement matters, the World Bank was always available to clarify issues and provide advice. The World Bank was responsible and proactive about changing the TEP design when it was agreed that doing so would improve the program’s effectiveness. The addition of child care stipends described earlier and the triggering of additional environmental safeguards as part of the proposed restructuring are examples of this. The World Bank also responded quickly to the need for development of a GRM and Codes of Conduct when the TEP team requested advice on handling conflicts between beneficiaries and LOs. Weaknesses 62. Though the Bank made continuous efforts to support the government in restructuring the project, an agreement was not reached on a way forward that would achieve Government objectives while adhering to Bank policy. While the government performance was challenging (lack of commitment to the PDOs, changes in priorities and strategies of the new administration, non-compliance with the legal covenant), the Bank could have done more to resolve the paralysis/impasse affecting the implementation and restructuring of the Project, particularly since this was the first lending operation in the country and the only vehicle for Bank dialogue. 63. Though support for procurement and financial management was provided, this was the first investment project for Antigua and Barbuda, and the lack of experience in working with the Bank was a factor that weighed heavily on project performance. The Bank could have provided stronger implementation support, which would have been appropriate in the context of a first-time borrower, a multi-sector project, and a project as the main vehicle for dialogue with the Government. 64. Poor management of the country dialogue. In a context of low Government capacity and scarce interest in the Project as designed, the Bank could have played a more active role during the period after the Mid-Term Review in April 2016, particularly during negotiations for the restructuring parameters. Page 19 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Justification of Overall Rating of Bank Performance 65. The overall World Bank performance is rated Unsatisfactory. While the World Bank carried out a solid technical project preparation, the overall rating is due to the following: • Missing the potential change in the political will to carry out the project, given that elections ended up happening very soon after effectiveness • Lack of sustained high-level interface between the borrower and the World Bank. As this was the first lending operation in Antigua and Barbuda and the PSST was expected to tackle politically tricky reforms, more dialogue should have taken place between the GOAB and the World Bank. • Weak M&E framework and its implementation • Inadequate efforts to build a national consensus around public sector reforms D. RISK TO DEVELOPMENT OUTCOME 66. Two of the three dimensions of the PDO were not achieved, and there has not been a follow-up dialogue with the Government to pursue them under another World Bank-supported operation, or with national budget resources. While improving the income and employability of the vulnerable population through temporary employment and training programs continues to be a priority for the GOAB, the Government has not continued supporting the TEP since the project’s cancellation. The Government’s own national employment program remains the main tool to provide support to unemployed citizens. None of the design features of the TEP have been incorporated into the national employment program as of the writing of this ICR. V. LESSONS LEARNED AND RECOMMENDATIONS Lessons for the World Bank 67. Importance of designing ‘vanilla’ operations when engaging with a first-time borrower with limited institutional capacity. The World Bank must carefully assess the breath and complexity of a project when starting a lending relationship with a first-time borrower. It is not advisable to engage in a multisector investment operation for sectors that involve complex reform issues as the first project for a first-time borrower. In the case of Antigua and Barbuda, the lack of experience in IBRD operations was combined with limited institutional capacity to tackle diverse and complex sector reforms, and this led to the poor outcome of the operation. 68. ‘The chain is as strong as its weakest link’. Change management at lower levels of government could have been stronger. During project preparation, much attention was paid to garnering the support of top leaders for public sector transformation. These individuals might have been ready for change, but significant groups under them were not. For example, Antigua and Barbuda has an established civil service as well as nonestablished public employees, and the potential for resentment between the two in the face Page 20 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) of reforms that imply reducing the number of nonestablished public employees and payroll reform should have been expected. The degree of resistance for change was underestimated, especially in the face of pending elections and a potential change in political leadership. 69. Instead of aiming for large overall, public sector reforms, in a context such as Antigua and Barbuda’s, a project should opt for a focus on technical aspects, tailoring decisions to the borrower context. Instead of aiming for a full-fledged public sector reform, it would have been more effective to pilot practical and concrete reforms. It is also important to factor in the sequencing and pacing of development and implementation. In the Caribbean, as is the case in similar-size countries, teams are relatively small and they are almost always affected by changes in government. It is important to start in small, logical steps—pilots, for example—to give stakeholders time to adjust and adapt. Without this sequencing and pacing, reforms might not be successful. 70. Importance of building deep, national consensus around reforms. When envisaging sector reforms, the country should ensure that significant consensus building is carried out during preparation. Consensus-building activities should reach out to different stakeholders in a country to maximize understanding and support during implementation. Communication and reach-out activities by the Government during preparation are crucial to explain to stakeholders the objectives and extent of sensitive sectoral reforms. 71. Working with a new borrower requires a different kind of engagement. When a country does not have experience in investment operations, there is a need to provide early and sustained support during implementation. This was Antigua and Barbuda’s first investment project with the World Bank, and while the latter provided solid support in terms of financial, procurement, and safeguards, more hand- holding was necessary to build momentum and support the borrower leadership in early implementation. Also, small countries have usually less in-country World Bank presence, and this needs to be overcome by intense supervision. . Page 21 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of policies approved by the Percentage 0.00 70.00 0.00 Cabinet aligned with the medium-term Government 30-Jun-2013 31-Dec-2018 30-Sep-2017 priorities and reflected in the Government’s Medium-Term Strategic Development Plan (MTSDP) Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of career public servants Percentage 45.00 100.00 45.00 covered by the new Public Service Act establishing a 30-Jun-2013 31-Dec-2018 30-Sep-2017 single employment regime Page 22 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Comments (achievements against targets): Not Achieved. By Project completion the number of civil servants covered by the Public Service Act remained at 45 percent. The other 55 percent of civil servants continued to be outside of these regulations, regime. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of ministries and depts that Text Public Service 80% of ministries and Public Service have the capacity to manage Commission has departments manage Commission has their HR w/in the delegated centralized authority HR within the centralized authority authority to hire, admin. over hiring, firing and delegated authority. over hiring, firing and professional develop. and disciplinary actions. Continued in disciplinary actions. disciplinary action, and 'comment' section. manage perform. and apply incentives 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Not achieved. Comments: Original Target: All public sector entities submit annual reports on civil service management to the PSC. Annual performance appraisal of civil servants is introduced in 100 percent of ministries and departments. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of poor Antiguans Percentage 0.00 50.00 0.00 identified by National Beneficiaries Management 30-Jun-2013 31-Dec-2018 30-Sep-2017 Information System (NBMIS) Page 23 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) and enrolled in a social protection program. Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of beneficiaries in the Percentage 0.00 10.00 0.00 Single Registry of Beneficiaries identified with 30-Jun-2013 31-Dec-2018 30-Sep-2017 the NBMIS as non-eligible. Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion The Temporary Employment Percentage 0.00 40.00 16.00 Program (TEP) beneficiaries’ labor income is at least 40% 30-Jun-2013 31-Dec-2018 30-Sep-2017 higher than the one for non enrolled eligible beneficiaries over the six month period of program implementation. Comments (achievements against targets): Partially achieved (40 percent achieved). Page 24 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion The Training Program (TP) Percentage 0.00 40.00 0.00 beneficiaries’ placement rate is at least 40% higher than 30-Jun-2013 31-Dec-2018 30-Sep-2017 the one for non-enrolled eligible beneficiaries three months after the end of the internship phase. Comments (achievements against targets): Not achieved. A.2 Intermediate Results Indicators Component: Component 4: Support Active Labor Market Programs (ALMP). Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Non-established public Text 55% of public servants 100% of non- 55% of public servants servants integrated under a are not governed by established posts are are not governed by common regime by Cabinet the civil service law. eliminated. the civil service law. The public service Page 25 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) database lists all public employees with appropriate legal status, defined in law. 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Staffing levels are aligned Text Employment records All ministries and Employment records with core tasks exist for government agencies exist for approximately 85% of completed staff lists. approximately 85% of public employees of public employees of which 55% are non- which 55% are non- established. established. 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New decentralized personnel Text Centralized HR HRM network Centralized HR management system management in the institutionalized; management in the established. established civil established civil service and fully service and fully Central Civil Service Page 26 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) decentralized, partially Authority monitors decentralized, regulated HR civil service partially regulated HR management for non- compliance; management for non- established established employees. employees. 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Legal Framework for Policy Text Policy development Policy approval Policy development Making and Coordination and coordination process, including and coordination established process is ad hoc prior review, is process is ad hoc institutionalized. 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of public servants in Number 0.00 15.00 0.00 line ministries capable to develop policy proposals in 30-Jun-2013 31-Dec-2018 30-Sep-2017 line with new standards. Page 27 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of beneficiaries Percentage 0.00 70.00 0.00 of the TP who graduate from the program 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): The TP program was not implemented before Project completion, due to delays in implementation. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of beneficiaries Percentage 0.00 70.00 71.00 of the TEP subprojects who complete the subproject 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Fully achieved (101.5 percent of target). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of TEP Percentage 0.00 60.00 100.00 subprojects that satisfactorily completed their proposed 30-Jun-2013 31-Dec-2018 30-Sep-2017 targets Comments (achievements against targets): Surpassed (166 percent of target). Page 28 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Memorandums of Number 0.00 5.00 0.00 Understanding signed with the implementing agencies 30-Jun-2013 31-Dec-2018 30-Sep-2017 of SP program Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of databases of SP Number 0.00 15.00 0.00 Programs transferred to the Single Beneficiary Registry 30-Jun-2013 31-Dec-2018 30-Sep-2017 Comments (achievements against targets): Not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Beneficiaries of Labor Market Number 0.00 300.00 130.00 programs (number) 30-Jun-2013 31-Dec-2018 30-Sep-2017 Beneficiaries of Labor Number 0.00 150.00 39.00 Market programs - Female (number) 30-Jun-2013 31-Dec-2018 30-Sep-2017 Page 29 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Comments (achievements against targets): Partially achieved (43 percent of target). B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Strengthening its capacity in managing public policies and the public service 1. % of policies approved by the Cabinet aligned with the medium- term Government priorities and reflected in the Government’s Medium-Term Strategic Development Plan (MTSDP) 2. % of career public servants covered by the new Public Service Act Outcome Indicators establishing a single employment regime 3. % of ministries and departments that have the capacity to manage their HR with the delegated authority to hire, administer professional development and disciplinary action, and manage performance and apply incentives 1. Non-established public servants integrated under a common regime by Cabinet 2.Staffing levels are aligned with core tasks Intermediate Results Indicators 3.New decentralized personnel management system established 4. Legal Framework for Policy Making and Coordination established 5. Number of public servants in line ministries capable to develop policy proposals in line with new standards. Consultancy on ‘Regulatory Framework for Policy Coordination and Strategic Planning’: Key Outputs by Component • A set of new and revised guidelines and templates to improve (linked to the achievement of the Objective/Outcome 1) the efficiency of Cabinet meetings • A presentation of the new procedures and tools to the Cabinet Ministers and Ministers of State, the Cabinet Page 30 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Secretary and the staff of the Cabinet Secretariat and Permanent Secretaries and other senior staff • Initial training of a select group of policy analysts • A draft manual Cabinet Manual outlining Cabinet procedures, templetes and the roles and responsibilitites of all participants in the new system • Presentation of the draft manual to key government officials • Training the staff of the Cabinet Secretariat and key ministry staff on the content of the manual and on the new cabinet process • Drafts of policy documents prepared following the templates, prepared by government staff with consultant’s guidance • Final Cabinet Manual that incorporated feedback from key Government figures • A draft policy paper on the topic of public sector modernization and consideration for Cabinet of Ministers consideration • A guidance note on the coverage of the Public Service Law • A final Policy Paper Objective/Outcome 2: Improving the efficiency of social protection spending through 1an integrated monitoring and targeting system 1. % of beneficiaries in the Single Registry of Beneficiaries identified with the NBMIS as non-eligible Outcome Indicators 2. % of poor Antiguans identified by National Beneficiaries Management Information System (NBMIS) and enrolled in a social protection program 1. Number of databases of SP Programs transferred to the Single Beneficiary Registry Intermediate Results Indicators 2. Number of Memorandums of Understanding signed with the implementing agencies of SP program Page 31 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Key Outputs by Component n.a. (linked to the achievement of the Objective/Outcome 2) Objective/Outcome 3: Improving the income and employability of the vulnerable population through temporary employment and training programs 1. The Temporary Employment Program (TEP) beneficiaries’ labor income is at least 40% higher than the one for non enrolled eligible beneficiaries over the six month period of program implementation. Outcome Indicators 2. The Training Program (TP) beneficiaries’ placement rate is at least 40% higher than the one for non-enrolled eligible beneficiaries three months after the end of the internship phase. 1.Percentage of beneficiaries of the TP who graduate from the program Percentage of beneficiaries of the TEP subprojects who complete the subproject Intermediate Results Indicators 2.Percentage of TEP subprojects that satisfactorily completed their proposed targets 3.Beneficiaries of Labor Market programs (number) 4.Beneficiaries of Labor Market programs - Female (number) 1.Labor Department OSEC equipped, refurbished, and staffed Key Outputs by Component 2.Three cohorts of TEP carried out (linked to the achievement of the Objective/Outcome 3) 3.Evaluations of each of the three cohorts of TEP 4.Tracking survey for the first TEP cohort Page 32 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Kathy Lalazarian Consultant Svetlana I. Proskurovska Senior Public Sector Specialist Fanny Weiner Senior Public Sector Management Specialist Gaston Blanco Representative Juan Martin Moreno Task Team Leader Asha Williams Social Protection Specialist Supervision/ICR Karla J. McEvoy, Juan Martin Moreno Task Team Leader(s) Daniela V. Felcman Public Sector Specialist Svetlana I. Proskurovska Senior Public Sector Specialist May Olalia Senior Public Sector Specialist Sonia Cristina Rodrigues Da Fonseca Procurement Specialist(s) Eduardo Franca De Souza Financial Management Specialist Cristina Elizabeth Coirolo Social Safeguards Specialist Patricia O. Orna Team Member M. Yaa Pokua Afriyie Oppong Safeguards Adviser Michael J. Darr Environmental Safeguards Specialist Page 33 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY11 1.290 13,975.43 FY12 37.741 204,968.22 FY13 49.133 221,372.63 Total 88.16 440,316.28 Supervision/ICR FY14 26.390 145,250.41 FY15 23.928 110,368.67 FY16 18.187 95,584.42 FY17 25.950 113,266.30 FY18 19.572 98,826.05 Total 114.03 563,295.85 Page 34 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 3. PROJECT COST BY COMPONENT Components Amount at Approval (US$, Actual at Project Percentage of Approval millions) Closing (US$, (US$, millions) millions) Component 1. Build 1.06 0.44 41.5 effective institutions for strategic management of government policies Component 2. Modernize 2.47 0.40 16.0 Human Resource Management (HRM) Component 3. Improve the 1.21 0.00 0.0 efficiency of social protection spending Component 4. Support 4.10 0.88 21.4 Active Labor Market Programs (ALMP) Component 5. Project 1.14 1.25 110.0 Management Total 9.98 2.99 23.00 Page 35 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 4. EFFICIENCY ANALYSIS 1. The tracking survey covered information on a control group and an experimental group. The control group included those persons who qualified as potential beneficiaries for the TEP but did not complete the enrollment (meaning that they qualified but did not enroll, attended orientation but did not enroll, or enrolled but did not complete the program). This group included 26 people. The experimental group in the labor tracking survey was defined as the group of beneficiaries who completed the TEP. 2. The PMU Coordinator and TEP Supervisor collected information between October and December 2015. The key findings of the analysis indicate that (a) 53 percent of the experimental group found work versus 50 percent of the control group, (b) there was no advantage observed for men as far as placement rates are concerned and women who completed the TEP showed 8 percent higher placement rate over their counterparts in the control group, and (c) the salary range for the employed control group is EC$325– 700, while that of the experimental group is EC$300–900. 3. The tracking survey also showed that at the end of the first year of participation in the TEP, there was an increase in the placement/employment rate for the beneficiaries (3 percent) and an increase in the income levels of the TEP beneficiaries (16 percent). Female beneficiaries who completed the TEP reported income around 80 percent higher than those in the control group. Their male counterparts, however, reported income that was 16 percent lower than that of those in the control group. The survey concluded that the disparity in gender impact was likely due to the conditions in the overall economy, since male and female beneficiaries completed the same content of the TEP. Page 36 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS The following text is the Borrower’s contribution to the ICR of the Project. The World Bank sent its draft ICR to Government for review, but Government did not provide comments. Borrower’s Implementation Completion Results and Report (ICR) is attached. Page 37 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 38 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 39 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 40 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 41 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 42 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 43 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 44 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 45 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 46 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Page 47 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) • PAD • Legal Agreement • Aide Memoires 2011–2017 • Statement of Mission Objectives and Back-to-Office Reports • Evaluation of Antigua and Barbuda Skills Training and Empowerment Program: Cohort 1. LDTU. August 2015 • Evaluation of Antigua and Barbuda Skills Training and Empowerment Program: Cohort 2. LDTU. June 2016 • Evaluation of Antigua and Barbuda Skills Training and Empowerment Program: Report and Evaluative Comments on Cohort 3. LDTU. August 2017 • Tracking Survey for measuring the Employment History of Beneficiaries in the TEP and a Control Group: Analysis and Findings. January 2016 • Functional Reviews of Five Government Departments in Antigua and Barbuda. Oxford Policy Management. June 2014 - Draft V2 • Government of Antigua and Barbuda. Implementation Completion and Results Report for PSST. February 2018 Page 48 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) ANNEX 7. THE TEMPORARY EMPLOYMENT PROGRAM 1. The main objective of the TEP was ‘to help the less experienced and unemployed find temporary employment’. It sought ‘to prepare individuals to reenter the labor market by providing them with opportunities to work on small projects managed by LOs for a period of six months’. The TEP will provide training to beneficiaries in basic vocational skills, provide job experience that increases the capacity of beneficiaries to interact as a team and learn and practice life skills, and pay beneficiaries for their participation with a direct cash transfer. 2. The following specifics were stressed in the project design: • Subprojects should be community based and managed by identified community-based organizations, service organizations, nongovernmental organizations, Government departments, and Statutory Boards—labelled as LOs. • Beneficiaries are those unemployed persons between ages 17 and 50 years who have been assigned to work on the subproject. They are expected to gain practical hands-on work experience; basic life, literacy, and employment readiness skills; training in the basic vocational skills specific to the subproject to which they are assigned, and skills in team interaction and practical life skills. 
 Summary of Performance • By the end of July 2017, a total of 136 beneficiaries participated in subprojects for the duration of three cohorts with 98 of those completing the program.
 • Preparatory exercises were well executed with LOs interacting fully with prospective beneficiaries before commencement. 
 • There were behavioral problems among beneficiaries in all cohorts. The TEP module caters to the already less-privileged individuals who come to the program with limitations in education and social skills and who are unemployed. Supervisors complained of the loss of valuable time when they were obliged to intervene and settle disputes. 
 • Professional counselling was sought for serious cases. A GRM to deal with matters relating to unacceptable behavior was planned and initial design finalized, although it was not completed due to the project’s cancellation. • The inclusion of life skills training in a program of this nature sought to identify and/or alleviate psychosocial issues facing participants. Beneficiaries commented positively on the boost to their self-esteem through participation in the sessions and on the benefits of the two remedial subjects, mathematics and English. 
 • All parties benefitted from the open lines of communication between beneficiaries and the Antigua and Barbuda Skills Training and Empowerment Programme (ABSTEP Supervisor and Page 49 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) between the Supervisors and the LDTU. There was a prompt response to issues requiring disciplinary action. 
The increased number of visits to subproject sites proved beneficial to LOs, beneficiaries, and the LDTU. Most attendance sheets were submitted on time along with the LO monthly reports. 
 • LOs, particularly those that accommodated beneficiaries from the suspended subproject, could have benefitted from financial support for transportation, since the transfers placed some beneficiaries at a disadvantage, being forced to take two buses to arrive at the subproject site. This would certainly explain the complaint from LOs about lateness and irregular attendance of some beneficiaries. 
 • The LDTU was successful in its efforts to persuade the Ministry of Labor to provide transportation to all beneficiaries to the life skills training location. It afforded all participants the opportunity of a change in environment one day per week, as well as the enjoyment of working in a space conducive to reflection and sharing. 
 First Round of the TEP 3. The first cohort in the TEP module was executed in Antigua and Barbuda from January 12 to July 10, 2015, on three subproject sites with a complement of 44 beneficiaries, of whom 34 completed the program. 4. There were difficulties in finding LOs to accommodate a minimum of 10/maximum of 15 beneficiaries and only 3 of an expected 10–15 qualified for the program. All 3 LOs completed the delivery of the TEP over the required six-month program. 5. Most beneficiaries (56 percent) felt that the knowledge and information gained in the program met their expectations; 35 percent were less positive and 9 percent were very disappointed. About 4 percent of beneficiaries felt that the knowledge and information gained in the program would not be useful in their job search. About 56 percent were positive and 40 percent, somewhat unsure. 6. The main implementation challenges of the TEP were defined as the poor behavioral attitudes of a minority of beneficiaries that led to program disruption; wage and payment issues; and communication problems between the PMU, LDTU, and LOs. High levels of team work and cooperation established between the TEP Supervisor, the Program Coordinator, and the Key Project Implementation Leader (KPIL) contributed to the successful implementation of the TEP. 7. The evaluation report carried out after the first round suggested the following recommendations: • Revision of the specifics in the subproject design relating to the qualifying number of beneficiaries, the duration of temporary employment, and the level of fees, with a view to decreasing the number of beneficiaries that a subproject can accommodate, lessening the duration of temporary employment, and increasing the level of fees. • Collaboration between the LDTU and PMU to ensure timely and accurate processing of financial instruments and transfers, especially those that negatively affect beneficiaries. Page 50 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) • The LDTU must be prepared to make early interventions when and where necessary to minimize distractions caused by poor behaviors and attitudes of beneficiaries as reported by LOs. • Address issues affecting the beneficiaries’ attendance regarding transportation and child care. Regarding transportation, making an allocation in the budget of the TEP to support transportation to the subproject sites. Two options were suggested: (a) The LDTU could enter into a contractual arrangement with the Transport Board to provide transportation to beneficiaries and (b) beneficiaries would be provided with payments in addition to their biweekly stipend to assist with public transportation. The report suggested considering supplement payments to child care providers where necessary. • Broadening the content of the orientation exercises of both the LTDU and LOs to more closely align the expectations of beneficiaries with those of the subproject. Second Round of the TEP 8. For the second cohort of the TEP module, activities at all subproject sites commenced on October 5, 2015, and concluded on April 8, 2016. A total of 40 beneficiaries participated in subprojects organized by three LOs: • Gilbert Agricultural Rural Development Centre (GARDC) - 15 
 • Antigua and Barbuda Waste Recycling Corporation (ABWREC) - 10 
 • Grays Farm Fundamental Baptist Church: Electronic Waste Management (E-Waste) - 15 
 9. A total of 26 individuals or 65 percent of enrollees completed the program. The LDTU identified several challenges confronting LOs that demonstrated an interest and willingness to be part of TEP. These challenges were identified after they had participated in preparatory workshops and orientation sessions and prepared subproject proposals: • Lack of proper accommodation for beneficiaries (bathroom facilities and administrative offices) • Inability to provide ongoing supervision daily for a six-month period 
 • Insufficient project funds to supplement the stipend for a full-time supervisor or provide 
 additional services, for example, transportation 
 • No additional donor funds for general subproject activities. 
Before the implementation of the second cohort, the LDTU examined the recommendations included in reports on the first cohort submitted by the ABSTEP Supervisor, the Life Skills Provider, the Program Coordinator, and LO Supervisors and the feedback from beneficiaries themselves. 10. During the period under review, all parties were in full compliance with the following Page 51 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) recommendations: 
 • LO monthly reports to include information on beneficiary performance 
 • Attendance sheets submitted on time • Quick alerts of cases of indiscipline which would warrant LDTU action 
 • Timely and accurate processing by the LDTU and PMU of financial instruments and transfers to LOs and beneficiaries • Greater frequency of site visits by the LDTU 11. The LDTU also embraced and implemented the recommendation for a broadening of ‘the content of orientation exercises to allow for closer alignment of the expectations of beneficiaries with those of the subproject.’ Additional information was provided during the orientation and an extended period for questions and answers allowed on the nature of the subprojects to which beneficiaries were assigned. 12. Attention was also paid to two of the main issues affecting attendance in the first cohort: (a) Transportation. A decision was taken to forego the proposal to increase the biweekly stipend to assist with transportation fees. However, transportation was provided to beneficiaries of the ABWREC and E-Waste to allow for their participation in the life skills course. It was noted that only a very small number of beneficiaries encountered problems with transportation and this was not frequent. (b) Child care. A decision was taken to provide supplementary payments to beneficiaries with child care challenges. The TEP intake form was revised to include information indicating circumstances—including child care issues—which could present barriers to attendance on subproject sites. 13. Of the 40 beneficiaries who started the subprojects and enrolled in the life skills training, 26 completed the exercises. Four persons received termination letters, eight found full-time employment, and two left voluntarily without finding alternative employment. 14. Beneficiaries were asked to comment on (a) the knowledge and information gained from the program and (b) the usefulness of the information in their job search. About 70 percent felt that the knowledge and information gained from the program met their expectations; the remaining 30 percent were less positive. About 40 percent felt that the knowledge gained in the program would be beneficial to their job search. About 15 percent replied ‘mostly’, 40 percent replied ‘somewhat’, and 5 percent replied ‘not at all’. 
The decision to act promptly in response to LO reports of negative behavior resulted in the termination of four beneficiaries before the end of the six-month period. While the result was a decrease in the number of participants, this intervention helped foster an environment that was more conducive to learning for those who remained. 
 15. The positive team work and cooperation established between the TEP Supervisor, the Program Page 52 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) Coordinator, and the KPIL during the first cohort was further strengthened. 16. The following recommendations were made: • Further review of the subproject design, leading to a reduction in numbers (from the minimum of 10 to a minimum of five) considering the ongoing challenges to LOs associated with accommodation of larger numbers • Formalize proposals for ‘payment to child care providers where necessary’, for inclusion in the revised Operational Manual • A review of the unit costs to be paid to LOs to allow for an increase in remuneration to supervisors of LO subprojects Third Round of the TEP 17. Based on the feedback from participants in previous rounds, some changes were additionally introduced to the Operational Manual in December 2016. These included the following: • Provision of child care benefit to eligible beneficiaries 
 • New work day schedules in two modules. One of practical work (four days weekly/six hours daily) and life skills training (with five hours for one day) • A stipend of EC$25.00 per day for the life skills day along with the established national minimum wage for the four working days. Beneficiaries would now receive EC$410.00 every fortnight, approximately EC$30.00 more than the former EC$379.28. 
This increase made the TEP stipend approximately 50 percent of that of the ‘New Work Experience Program’ (NWEP). Notwithstanding the shorter work week or the benefit of the life skills experience of the ABSTEP, unemployed persons seeking placement opted for the NWEP as first choice. 
 • Amendments to the environmental framework to allow for the limited use of pesticides and work on historical sites 
 18. The third cohort in the TEP module was executed in Antigua from November 24, 2016, to May 19, 2017, and from January 24 to July 19, 2017. The first set of activities commenced on Monday November 24, 2016, at the Victory Centre—the Revitalization of the Victoria Park, Botanical Gardens, a collaborative effort between the Victory Centre and the Environment Division within the Ministry of Health and the Environment with 15 beneficiaries. 
 19. Four subprojects came on stream on January 24, 2017, during this cohort accommodating an additional 37 beneficiaries:
the Freetown Peace Park, executed by the Freetown Community Group, the ABWREC, the Reservoir Restoration Project, executed by the Barnes Hill Community Development Organization, and E-Waste. Page 53 of 54 The World Bank Public and Social Sector Transformation Project (PSST) (P126791) 20. A total of 38 individuals or 73 percent of enrollees completed the program. The GARDC was awarded the consultancy for the delivery of the life skills training. Additional funds for the hiring of a professional counsellor were not approved. 
 21. End of cohort surveys of beneficiaries and LO debriefings offered the following feedback: (a) Subprojects benefitted greatly from the work of the beneficiaries. 
 (b) Better facilities made for greater beneficiary commitment to the process. 
 (c) LO Supervisors were generally satisfied that the subprojects allowed for transfer of skills to beneficiaries. 
 (d) Most beneficiaries expressed appreciation for the information provided throughout the life skills training. 
 (e) Most beneficiaries who completed questionnaires (19 individuals or 63 percent) felt that the knowledge and information gained in the program met their expectations, while (11 individuals or 37 percent) were less positive. None of the beneficiaries expressed disappointment.
 (f) About 66 percent of beneficiaries thought that the knowledge and information gained in the program would be useful in their job search. Another 25 percent were positive, with 10 percent unconvinced or somewhat unsure. 
 Page 54 of 54