Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004761 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF-11447 ON A GRANT IN THE AMOUNT OF US$150 MILLION TO THE Islamic Republic of Afghanistan FOR THE AFGHANISTAN CAPACITY BUILDING FOR RESULTS FACILITY (CBR) July 25, 2019 Governance Global Practice South Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective July 25, 2019) Currency Unit = Afghani (AFN) AFN 80.80 = US$1 FISCAL YEAR July 1 – June 30 Regional Vice President: Hartwig Schafer Country Director: Henry Kerali Senior Global Practice Director: Edward Olowo-Okere Practice Manager: Ismaila B. Ceesay Task Team Leader(s): Yousif Mubarak Elmahdi, Atiqullah Ahmadzai ICR Main Contributor: Kirk David Schmidt; Pragya Shrestha ABBREVIATIONS AND ACRONYMS ACSI Afghanistan Civil Service Institute AEP Afghan Expatriates Program AFMIS Afghanistan Financial Management Information System ARTF Afghanistan Reconstruction Trust Fund CAA Civil Aviation Authority CBR Capacity Building for Results Facility CBRP Capacity Building for Results Program CPF Country Partnership Framework CSMD Civil Service Management Directorate DFID Department for International Development EPAP Emergency Public Assistance Project EU European Union FPIP Fiscal Performance Improvement Plan FSP Fiscal Performance Improvement Support Project GDP Gross Domestic Product GoIRA Government of the Islamic Republic of Afghanistan HR Human Resource HRMIS Human Resource Management Information System IARCSC Independent Administrative Reform and Civil Service Commission IDA International Development Association IDLG Independent Directorate of Local Government IEG Independent Experts Group IFR Interim Financial Report IP DPG Incentive Program Development Policy Grant ISR Implementation Status and Results Report LEP Lateral Entry Program LMAs Line Ministries’ Agencies LOTFA Law and Order Trust Fund for Afghanistan M&E Monitoring and Evaluation MAIL Ministry of Agriculture, Irrigation, and Livestock MCP Management Capacity Program MMD Mid-level Management Development Program MoCIT Ministry of Communications and Information Technology MoF Ministry of Finance MoFA Ministry of Foreign Affairs MoHE Ministry of Higher Education MoI Ministry of Interior MoIC Ministry of Industry and Commerce MoLSA Ministry of Labor and Social Affairs MoMP Ministry of Mines and Petroleum MoRR Ministry of Refugees and Repatriation MRRD Ministry of Rural Rehabilitation and Development MTR Mid Term Review NGO Non-Governmental Organization NPA National Procurement Authority NTA National Technical Assistant P&G Pay and Grading PAD Project Appraisal Document PDO Project Development Objective PIU Project Implementation Unit PRR Priority Reform and Restructuring PSM Public Sector Management PSU Project Support Unit SAO Supreme Audit Office SC Steering Committee SRBC State and Resilience Building Contract TA Technical Assistance TAGHIR Tackling Afghanistan’s Government HRM and Institutional Reforms ToR Terms of Reference UNDP United Nations Development Program WDR World Development Report TABLE OF CONTENTS DATA SHEET ............................................................................................................................ 1 EXECUTIVE SUMMARY ........................................................................................................... 5 PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ............................................................. 7 A. CONTEXT AT APPRAISAL .........................................................................................................7 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 11 II. OUTCOME ....................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................ 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 15 C. EFFICIENCY ........................................................................................................................... 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 22 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 23 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME .................................. 27 A. KEY FACTORS DURING PREPARATION ................................................................................... 27 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 29 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .... 31 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 31 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 33 C. BANK PERFORMANCE ........................................................................................................... 34 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 36 V. LESSONS AND RECOMMENDATIONS ................................................................................ 37 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 41 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 47 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 49 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 50 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 52 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 54 ANNEX 7. SPLIT RATING CALCULATION ................................................................................ 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P123845 Afghanistan Capacity Building for Results Facility (CBR) Country Financing Instrument Afghanistan Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Independent Administrative Reform and Civil Service Ministry of Finance Commission Project Development Objective (PDO) Original PDO The project's development objective is to assist the government in improving the capacity and performance of select line ministriesin carrying out their mandates and delivering services to the Afghan people. This will be achieved through the implementation of specific capacity and institution building programs, which include systematic monitoring of and reporting on results. Revised PDO To improve the capacity and performance of priority line ministries and independent agencies in selected reform areas. Page 1 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 100,000,000 100,000,000 72,110,449 TF-11447 Total 100,000,000 100,000,000 72,110,449 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 100,000,000 100,000,000 72,110,449 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 11-Dec-2011 21-Jan-2012 20-Jan-2014 31-Dec-2017 31-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 25-Apr-2016 45.80 Change in Implementing Agency Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Financing Plan Change in Institutional Arrangements Change in Implementation Schedule Other Change(s) 20-Jun-2017 54.51 Change in Implementing Agency Change in Loan Closing Date(s) Change in Institutional Arrangements 08-Jun-2018 62.09 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Satisfactory Modest Page 2 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) Moderately 01 10-Jul-2012 Moderately Unsatisfactory 25.00 Unsatisfactory 02 31-Dec-2012 Moderately Satisfactory Moderately Satisfactory 25.08 03 12-Mar-2013 Moderately Satisfactory Moderately Satisfactory 25.55 04 26-Jun-2013 Moderately Satisfactory Moderately Satisfactory 25.55 Moderately 05 10-Dec-2013 Moderately Satisfactory 29.21 Unsatisfactory Moderately 06 22-Mar-2014 Moderately Satisfactory 32.83 Unsatisfactory Moderately 07 08-Aug-2014 Moderately Satisfactory 35.28 Unsatisfactory Moderately 08 26-Jan-2015 Moderately Unsatisfactory 38.33 Unsatisfactory Moderately 09 14-Apr-2015 Moderately Unsatisfactory 38.90 Unsatisfactory Moderately 10 12-Oct-2015 Moderately Unsatisfactory 42.36 Unsatisfactory Moderately 11 24-Mar-2016 Moderately Unsatisfactory 45.24 Unsatisfactory 12 12-Sep-2016 Moderately Satisfactory Moderately Satisfactory 48.49 13 13-Mar-2017 Moderately Satisfactory Moderately Satisfactory 54.51 14 24-Oct-2017 Moderately Satisfactory Moderately Satisfactory 54.51 15 18-Sep-2018 Moderately Satisfactory Moderately Satisfactory 77.01 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 100 Other Public Administration 100 Page 3 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 100 Public Administration 100 Administrative and Civil Service Reform 100 ADM STAFF Role At Approval At ICR Regional Vice President: Isabel M. Guerrero Hartwig Schafer Country Director: Nicholas J. Krafft Henry G. R. Kerali Director: Nicholas Paul Manning Zoubida Kherous Allaoua Practice Manager: Antonius Verheijen Ismaila B. Ceesay Gretchen Melissa Biery, Satyendra Yousif Mubarak Elmahdi, Task Team Leader(s): Prasad Atiqullah Ahmadzai ICR Contributing Author: Kirk David Schmidt Page 4 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Executive Summary The Capacity Building for Results (CBR) Facility was designed at a difficult period in Afghanistan’s reconstruction transition. The project was conceived in response to a context of very low state capacity that was being compensated through heavy donor financing of non-civil service staff known as a ‘parallel (or second) civil service’. Donor-funded technical assistance proved to be integral to the stabilization of state capacity, helping to address short-term capacity gaps and support urgent development objectives. However, declining aid volumes (dominated by security priorities) increasingly placed the sustainability of donor-funded programs and by extension, technical assistance at risk. At the time, donor expenditure on technical assistance was estimated to have reached US$1 billion per year with parallel civil service salaries generally between 6-16 times higher than comparable grades in the core civil service and international consultancy fees charged in Afghanistan amongst the highest in the world. CBR aimed to strengthen state capacity by providing a facility for the integration of high caliber Afghans into the core civil service. This would be achieved through offering remuneration that was considerably higher than uncompetitive regular civil servant pay but also substantially lower than unsustainable parallel civil servant salaries. Donor and development partners who widely advocated for and endorsed the CBR approach, would work in tandem with the project by harmonizing remuneration (at CBR levels) and phasing out their technical assistance within Government entities supported by CBR. Project human resource capacity injection would be strategically prioritized by aligning support to higher level Government objectives. The process would be demand driven, requiring interested line ministries to apply for project support and to devise medium-term reform plans that outlined line ministry level results (aligned to higher level Government objectives) to be targeted through project human resource capacity injection. The level of project support devoted to line ministries would follow a tiered system that classified line ministries based on progress in implementing reform plans. With human resource capacity injection of a targeted 2,400 positions largely focused on higher grade civil servants in ‘key agent of change’ positions, the project would increase the scale of capacity building through a complementary focus on in-service training targeted across all civil service grade levels. Ultimately, CBR fell short of its original objectives, and accordingly the project’s outcome has been rated Moderately Unsatisfactory by this Implementation Completion and Results Report (ICR). The project was highly relevant to the development challenges faced in Afghanistan at the time. Its inception benefited from a broad coalition of support across Government and donor partners, including at the highest levels. Project design was a culmination of an over one-year process that directly involved key partners and was grounded in political economy analysis that correctly identified key risks and appropriate mitigation measures. In hindsight however, it is clear the magnitude of those risks outweighed the project’s ambition. In particular, three key interrelated assumptions did not materialize, severely impeding implementation during the first four years of the project when just two ministries reached the highest tier and only 128 positions were recruited. First, line ministry commitment to self-reform was minimal with many ministers not particularly concerned with structural changes; wanting staff for implementing tasks, not for pushing reforms. Meanwhile, higher-level Government steering was absent or quickly dissipated and as a result there was little pressure from the top to implement change. Second, donors neither harmonized technical assistance pay nor reduced financing for this parallel form of support. This contributed to limiting line ministry demand for CBR as technical assistance was easier to access. It also Page 5 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) constrained both the quality and quantity of civil servant recruitment by the project which could not compete with the higher pay offered to technical assistants. Third, dual implementation of the project by the Independent Administrative Reform and Civil Service Commission (IARCSC) and Ministry of Finance (MoF), borne out of contestation and low respective capacities, promoted disruptive competition as opposed to complementarity. Dual implementation also affected coordination with respective project components implemented in silos from one another, reducing reform linkages. This disconnect hampered the targeting of project supported recruitment as well as civil service training with the latter also constrained by capacity deficits within the IARCSC’s training arm. Project performance significantly improved following project restructuring in 2016, however, by then CBR’s intended objectives were narrowed compared to original design. Restructuring simplified line ministry access to the program, doing away with the tier approach and instead focusing support towards 13 ministries determined based on contribution to key Government priorities. Demand for CBR was also considerably enhanced by renewed higher-level commitment following a change in Government, and the appointment of a new leadership team within a strengthened IARCSC which assumed sole implementation responsibility for the project. Meanwhile, improved donor commitment to the CBR approach was reflected in their broad compliance to a harmonized technical assistance pay scale and guideline that CBR helped to develop in tandem to project restructuring. These combined factors allowed the project to recruit a total of 1,053 civil servants against a reduced target of 1,500 positions. Many of these project appointees have made important contributions to service delivery outcomes, especially at the sub- national level where 44 percent of recruitment was made. Unfortunately, however, the project did not include performance metrics (within the 22 ministry reform plans that were finally developed) to help quantify appointees’ contributions to line ministry reform processes. Meanwhile, capacity was by and large spread too thin as loosened line ministry entry requirements meant that human resource capacity injection became allocated well beyond the project’s 13 priority ministries. Scale of capacity development could not be significantly enhanced through civil servant training although CBR has helped to lay the foundation for IARCSC’s training arm to make future advances in this area. The project has provided a foundation for future public administration reforms in Afghanistan to build upon, but its risk to development outcome remains high. Human resource capacity injection provided by CBR is not beset with the same accountability and policy ownership challenges of parallel technical assistance. It does however share the same fiscal sustainability challenges, albeit at a lesser magnitude. Based on lessons from CBR, the Tackling Afghanistan’s Government HRM and Institutional Reforms (TAGHIR) project has been designed to provide an orderly transition to core Government budget of ‘projectized’ human resource capacity injection. TAGHIR is being implemented as part of broader administrative reforms related to public sector functional reviews, personnel and payroll management, pay structure and policy, and professional cadre development; all of which are expected to contribute to improved wage bill management. Ultimately, however, this will necessarily be a long-term journey that will require consistency across a number of years, an attribute not found in the Afghan State, especially with the talk of a peace deal with the Taliban. Page 6 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. In 2011, Afghanistan had just emerged from several decades of war and thus was undertaking a reconstruction effort to build an effective state. The country remained fragile, and security was an obstacle in the reconstruction process. Afghanistan’s poverty and social indicators remained among the lowest in the world, and the pace of implementation of reconstruction programs fell short of citizens’ expectations. According to 2010 key development indicators, the GDP per capita growth was 0.6 percent and the poverty headcount ratio was 36 percent. Only 48 percent had access to improved water source and 37 percent to improved sanitation facilities. Revenue projections remained dismal and state capacity low. The Government of the Islamic Republic of Afghanistan (GoIRA)’s Budget Statement (2012) prioritized revenue mobilization but prospects remained uncertain in the short to medium term. Even with mining revenues of about 17.5 percent of GDP, rising security and non-security expenses were still twice the size of domestic revenues.1 2. A key priority of the GoIRA was to build the country’s public administration capacity. Afghanistan and the international community embarked upon a transition strategy for the Government to take greater responsibility for security, reconstruction, and development. As part of this transition, development partners expected GoIRA to assume responsibility for managing its public administration system, while Government committed to improve budget execution, revenue collection and to strengthen public financial management (PFM) systems to enable better management and oversight of anticipated greater amounts of donor aid being channeled through the Afghan budget. 3. This transition was complicated by a shortage of suitably qualified Afghan professionals willing to join the public sector. This was largely due to the prevalence, at the time, of direct donor financing of large numbers of national (and international) staff (as consultants) in key functional and advisory positions. This growing trend resulted in the emergence of a ‘parallel (or second) civil service’ made up of largely ad hoc, project based, and donor driven national technical assistance (NTA). While integral to the stabilization of state capacity, by helping to address short-term capacity gaps and support urgent development objectives, this form of technical assistance was costly. It was also largely ungoverned and created many perverse incentives. Critically, it constrained efforts to build core public administration capacity, luring talented professionals away from core civil service positions. 4. The CBR Facility was developed within this complex background; with the project intended to provide a more institutionally and fiscally sustainable alternative. The original objective of the project was to assist the government in improving the capacity and performance of select line ministries in carrying out their mandates and delivering services to the Afghan people. This would be done through provision of better and more sustainable staffing to build public administration capacity and implement donor-funded projects. In doing so, CBR followed from (and reflected lessons) of various reform programs, such as the Afghan Expatriates Program and the Lateral Entry program (known as AEP and LEP) and the Management Capacity Program (MCP). While achieving a few notable improvements, these programs had 1 World Bank/ GoA (2014). Transition in Afghanistan: Looking Beyond 2014. World Bank. Page 7 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) limited scope, addressing only a small number of civil service functions. As a result, capacity of most line ministries remained weak, and many continued to rely heavily on NTAs. Project Rationale 5. World Bank support to Afghanistan’s public administration reform agenda commenced through the Emergency Public Administration Projects (EPAP I and II). These projects helped with the development of the Government’s public administration reform program, including support for the establishment of the IARCSC with a mandate to both lead civil service reforms and manage the human resource functions of the government. World Bank support later evolved in response to a context in which the GoIRA attempted to address severe capacity constraint by injecting short-term capacity into line ministries and agencies, largely to implement donor funded projects. This approach, while addressing critical short-term capacity gaps, resulted in the creation of largely fragmented, uncoordinated and costly parallel management structures and systems in line ministries, thereby adversely impacting civil service reform progress. 6. To address the problem of the so called ‘parallel (or second) civil service’, several initiatives were introduced starting with the Priority Reform and Restructuring (PRR) process. Under the PRR, selected departments would be provided with civil servant staffing on higher pay rates. In exchange, such departments would agree to be restructured. This form of human resource capacity injection was continued and expanded through several iterations of programming including the AEP, LEP, and MCP. These projects aimed to level the playing field among the international agencies, Non-Government Organizations (NGOs), and the Government as competing employers for scarce Afghan professionals. However, these programs were limited in both scale; able only to address the challenges of low capacity in few instances with the impact of low capacity continuing to be demonstrated most clearly in the low execution rates of the development budget across Government. 7. Reflecting the lessons from predecessor projects, CBR was designed to support – over a longer time frame – a government-led and context specific approach to capacity and institution building that complemented existing ministry-specific service delivery and other sectoral reform programs. The central premise of CBR’s design was that institution building and civil service reforms required consensus at the highest levels. Accordingly, project-supported capacity building was intended to be demand-driven, results focused, and fully owned by the Government. Under CBR, ministries would lead their own reform process guided by a prioritized medium-term capacity and institution building framework. Ministries would also define results and monitoring arrangements, implementation responsibilities and timetable, and resources required to implement the framework. The initial project duration of six years was based on the understanding that institutional reforms required a longer time horizon to mature. 8. CBR supported the implementation of the GoIRA’s public administration reform commitments across the whole of Government. However, this support was pre-conditioned with project resource allocation designed to follow a competitive three-tier system (Tier 1 being the highest), wherein a higher level of resources would be provided to ministries willing to undertake challenging but appropriate reform programs. This was partly due to limited resources which meant that equal support could not be provided to all parts of Government, but also reflected the need to create buy in and mitigate resistance to reforms. Page 8 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Theory of Change (Results Chain) 9. The Project Appraisal Document (PAD) did not include a detailed results chain as this was not a requirement of World Bank procedures at the time of project preparation. In 2016, during the first of three project restructurings, a results chain (Fig. 1) was developed. The Theory of Change (ToC) for the restructured project assumed that if higher caliber personnel, including existing NTAs, could be integrated as core civil servant staff in selected departments, they would drive policy development and reform implementation whilst enhancing accountability and ownership in those ministries. Although the results chain provided a clear path from intervention to impact, and properly identified risks, it may have underestimated the likelihood of risk materialization and impact on project implementation. Figure 1. Results Chain Page 9 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Project Development Objectives (PDOs) 10. The original PDO was to “assist the government in improving the capacity and performance of select line ministries in carrying out their mandates and delivering services to the Afghan people.” Table 1: Key Expected Outcomes and Outcome Indicators of the Original Project Indicator Name Baseline End of Project Target Development budget execution rates in 30% Highest tier ministries’ development budget execution highest tier ministries rates tend to rise over the course of the project, reaching at least 60 percent on average for highest tier ministries by project closing. Business process improvements in 0.0 Number and percentage of targeted business process highest tier ministries (as outlined in the improvements achieved in highest tier ministries tend ministry’ s performance plan) to rise over the course of the project. Service delivery improvements in the 0.0 Number and percentage of targeted service delivery highest tier ministries (as outlined in the improvements achieved in the highest tier ministries ministry’ s performance plan) tend to rise over the course of the project. Strategic staffing in place across 70 Number of senior management group positions filled. government Components 11. The project comprised four components. 12. Component 1: Technical Assistance Facility for Preparation and Implementation of Line Ministry Capacity Building Programs. This component was designed to support ministries to prepare and implement Capacity Building for Results Programs (CBRPs). The objective was to attract and retain qualified staff in key positions by identifying a ministry’s specific needs and capacity and by helping assess the Tashkeel and ministerial needs for creating professional cadres, common function and senior management groups that shall receive more competitive salaries. 13. Component 2: Building Human Resources. Activities under this component were designed to focus on (a) enabling selected ministries to implement approved CBRPs and (b) supporting ministries to implement critical system-wide civil service reforms. It comprised two subcomponents. (a) Subcomponent 2.1: Support to IARCSC for Civil Service Reforms. The project aimed to provide targeted assistance to the IARCSC for the implementation of critical elements of the Government’s public administration reform agenda. (b) Subcomponent 2.2: Appointment of Senior Management, Common Function and Professional Cadres. This subcomponent aimed to fund the CBR ministries through the MoF to support the additional cost of the senior management group, the common function group, the professional cadres, and management intern salaries. (i) Professional cadres. The objective was to provide TA to ministries to help them determine the scope and nature of the professional groups to best meet their mandate. Page 10 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) (ii) Common function groups. Ministries in the highest tier were eligible to receive a cadre of common function staff for activities which cut across all ministries, such as financial management, procurement, audit, and HR management. The objective was to standardize common pay bands (and steps within bands) across ministries as the common function staff were expected to move between ministries for career progression. (iii) Senior management group. The project aimed to create this group based on the managerial staffing requirements as proposed in a ministry’s CBRP—a complement of critical managerial staff would be appointed through open competition to key managerial positions in Kabul and the provinces for a period of up to five years. 14. Component 3: Civil Service Training. The Government of the Islamic Republic of Afghanistan (GoIRA), through the support of the Korean International Cooperation Agency, U.S. Agency for International Development, and other donors, established the Afghanistan Civil Service Institute (ACSI). Between 2005 and 2011, the GoIRA provided a range of training for civil servants at the center and provincial levels. This agency aimed to provide harmonized and targeted training programs for civil servants and develop partnerships with national and international training institutes. Building upon the lessons learned, this component was designed to support the development of civil service capacity through institutionalized training delivered through the ACSI as part of career enhancement programs, subject to regular monitoring and oversight. 15. Component 4: Project Management, Monitoring and Evaluation. Initially, the project established a CBR Project Support Unit (PSU) in the MoF and a CBR PSU in the IARCSC. The MoF CBR PSU was responsible for project management and administration (for example, financial management and procurement), assistance in the assessment of proposals, and project monitoring and evaluation (M&E) against the Results Framework and oversight. The IARCSC CBR PSU was responsible for implementation of Components 2 and 3, including the CBR civil service components (for example, institutional development and public administration reform functions). The MoF CBR PSU managed funding for services on behalf of the participating line ministries and was also responsible for the contracting of relevant firms which would provide TA and associated logistical requirements. In addition, the MoF CBR PSU served as a Secretariat to the Steering Committee while the IARCSC CBR PSU coordinated the advisory groups. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 16. The project was restructured three times; in 2016, 2017, and 2018. The first project restructuring in April 2016 was a Level 1 restructuring that amended the PDO. The original PDO targeted improved delivery of services; the assumption being that through improved capacity, line ministries and independent agencies (LMAs) with service delivery functions would improve performance in this area. However, this proved both difficult to measure and attribute to CBR. The PDO was therefore revised to move emphasis from service delivery improvements to improved capacity, helping to enhance relevance and realism and to improve measurability and attribution. 17. As a result, restructuring aimed to increase the likelihood of achieving the PDO. This necessitated removal of key identified project implementation bottlenecks together with re-aligning of Page 11 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) project activities with the priorities of the then new Government. Project restructuring additionally targeted more efficient utilization of project resources. Tangibly, this involved a relative narrowing of project scope (reflected in revised PDO indicators and outcome targets), including a reduction in recruitment ceiling from 2,400 to 1,500 positions; to be re-focused on 13 priority LMAs. Component 3 on “Civil Service Training” was also dropped, with training activities instead mainstreamed across other project activities, to support stronger linkage to reform results. Accordingly, a partial cancellation of the original committed amount was processed as part of restructuring, reducing total project financing to US$150 million. The original grant amount (first tranche release2) was signed for US$100 million out of the total approved ARTF financing commitment of US$350 million. Project restructuring did not change the original grant amount of US$100 million but relinquished US$200 million of the remaining US$250 million ARTF financing commitment for the project. Table 2: Revised PDO Indicators Original Indicator3 Revised Indicator4 Comments Priority line ministries and Development budget execution rates for Target represents a weighted average. 6 of 13 independent agencies development highest tier ministries tend to rise over the priority LMAs were already achieving or exceeding budget execution improves to at least course of the project the target at the time of restructuring 60% Priority line ministries and Target represents the number of priority LMAs— Number of targeted business process independent agencies that have reduced from 25 to 13 to reflect the more focused improvements achieved in highest tier implemented at least 2 business approach adopted under restructuring. The number ministries tends to rise over the course of process simplifications planned in of business process improvements required of each the project their CBRP Civil service positions LMA was reduced from 4 to 2 to reflect remaining recruited by CBR implementation period. Number of Senior Management Group Civil service positions recruited by CBR Indicator was revised to reflect gender component. (SMG) positions filled filled by women Targeted recruitment of 450 woman through CBR Civil service positions recruited by CBR Civil service positions recruited by CBR Target was reduced from 2,400 to 1,500. Number and percent of highest tier ministries achieving progress on at least Indicator was dropped one service targeted in the CBR programs (dropped) Revised Components 18. Component 1: Technical Assistance Facility in Support of Civil Service Reform. The component initially in the original project focused on selecting the highest and middle tier ministries, but the eligibility 2 The ARTF financing modality uses tranches to release funds on a need basis so as not to tie up funds. 3 As of original project approval (January 21, 2012). 4 As of first project restructuring approval (April 22, 2016). Page 12 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) criteria proved overly complex, thereby limiting the line ministry access to the CBR. As this led to a lack of ministry-level commitment, it delayed implementation. Restructuring of this component entailed streamlining participating ministries’ access to the CBR and providing TA to line ministries for development and implementation of CBRPs. The restructuring allowed project to support CBRP recruitment in priority LMAs. Access to CBR was also simplified by eliminating the requirement to apply. Similarly, the CBRP template was simplified and new criteria for prioritizing ministries developed. Restructuring also abolished the Advisory Group and streamlined CBRP approval process. 19. Component 2. Building Human Resources. Restructuring of this component involved the transfer of responsibility for recruitment of Grades 1 and 2—previously under the IARCSC—to LMAs. However, the change in this oversight arrangement envisioned the following risks: (a) quality of recruitments, (b) adequacy of complaints’ handling, and (c) timeliness of meeting pre-agreed targets. 20. The restructured project introduced the following risk mitigation measures: IARCSC developed clear procedures to guide the line ministries and independent agencies to carry out their recruitment responsibility; HR firm would advise the IARCSC on quality assurance for LMAs’ recruitments; IARCSC approval of contracts, performance management of recruited civil servants to be conducted by the IARCSC, and strong monitoring by the World Bank Task Team. 21. Restructuring entailed changes to CBR-specific recruitment measures and arrangements, as follows: Support to IARCSC for civil service reforms was retained largely due to satisfactory performance of this activity. On the other hand, the restructuring introduced recruitment and training of 30 common function positions (Grades 1–4 of approved Tashkeel) and a similar number of non-uniform civilian personnel. It also introduced support to ministries not part of CBR, but that were deemed ready for reform; a management internship program consulting services to assist with recruitment and recruitment of managerial staff contracted under the management capacity program. 22. The Component on Civil Service Training was dropped to mainstream capacity building. Restructuring mainstreamed LMA training into their CBRPs under Component 1, while training in support of HR-related outcomes, led by IARCSC, ACSI or the HR Firm, including delivery of the MMD Program, was planned under Component 2. Component 4: Project Management, Monitoring and Evaluation. Restructuring helped improve coordination and clarify the roles of the MoF, the IARCSC, and their individual PSUs. It also attempted to correct deficiencies related to a lack of high-level government leadership through reformation and reactivation of the Project Steering Committee (SC) which would now report monthly to the President monthly, and quarterly to Cabinet. To accelerate civil servant recruitment, recruitment responsibility was transferred from the IARCSC directly to the respective LMA, it put in place a grievance redressal mechanism to inject transparency and accountability in the recruitment process and hired an M&E firm to improve the monitoring and reporting of results. Other Changes 23. Following the restructuring in 2016, CBR was restructured twice more; in June 2017 and finally in June 2018; both Level 2 restructurings. The second project restructuring in 2017 consolidated project implementation arrangements by vesting full control of the project in a single entity, the IARCSC, for better management and focus. With a new IARCSC Chairman and Board of Commissioners having been Page 13 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) appointed by the President of GoIRA in March 2017, there was significant trust in the IARCSC’s new leadership’s ability to rebuild the image of the Commission and reinstate public confidence in it as an independent entity. With this renewed trust in the Commission, capacities and accountability lines for civil service management functions were consolidated within the strengthened and reconstituted IARCSC. As a result of this institutional reconfiguration, responsibility for CBR now fully and solely aligned with the IARCSC’s mandate, resolving previous disputes with MoF. A no-cost extension of the project’s original closing date by six months, from December 31, 2017 to June 30, 2018, was also processed as part of this restructuring. This extension intended to facilitate sufficient time for the transition of the project into IARCSC, and for the IARCSC to plan the next phase of support. 24. The final project restructuring in June 2018 was solely to provide an additional six month no- cost extension of the project’s closing date, from June 30, 2018 to December 31, 2018. The extension was required because of a delay in preparation of the follow-on phase of support through the TAGHIR Project. This final no cost extension was therefore required to preserve gains by ensuring a continuity of support, especially the timely payment of civil servant salaries, until TAGHIR became effective. Rationale for Changes and Their Implication on the Original Theory of Change 25. Change in Government in 2014 provided renewed momentum for CBR. Many of the issues tackled by the first project restructuring in 2016 had long been recognized, however, political economy factors were not conducive to an earlier restructuring. The Mid-Term Review (MTR) held in February 2014, identified several corrective measures, but ultimately, the outgoing Government did not possess the political will for the reforms targeted by CBR, manifesting in implementation bottlenecks at the line ministry level. While a disputed election and a complicated and drawn out political transition consumed the new Government’s focus throughout most of 2014 and the first half of 2015, further delaying the implementation of corrective measures, the change in Government provided renewed momentum for CBR. Following a series of meetings between the Bank, IARCSC, and MoF with the new President of the GoIRA, Presidential Order 852 on (reform of) CBR was issued on July 29, 2015. This was followed by a formal government restructuring request proposing six key changes to the project design, submitted to the Bank in November 2015. 26. Initiated in November 2015, the restructuring process was itself drawn out. The process was complicated by prolonged dispute over consolidation of project implementation responsibility within the MoF (at the President’s instruction). Ultimately, it was decided to maintain the dual implementation partner arrangement but with clearer demarcation of responsibilities and division of labor amongst MoF and IARCSC. 27. Project restructuring developed a ToC for the project; this had previously been missing. Meanwhile, if successful, the measures introduced as part of restructuring, including simplified LMA access to the project and streamlined recruitment procedures, enhanced the likelihood of achievement of the project’s ToC; premised around the benefits of integrating core civil servant staff in selected departments. Page 14 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 28. The relevance of PDO is Substantial. CBR, through human resource capacity injection, based on a demand-driven approach emphasizing the establishment of transparent and robust eligibility criteria (intended to allow LMAs to demonstrate ownership over reforms), aimed to build the capacity, performance and accountability of LMAs. In doing so, the project aligns with Pillar 1 of the World Bank Group Afghanistan Country Partnership Framework (CPF) 2017-20215 – Building Strong and Accountable Institutions. The overall objective of this pillar is to contribute to the government’s long-term goals of state building and self-reliance6 and tackling the underlying drivers of fragility by strengthening the institutional and regulatory framework for service delivery, planning, and fiscal and public financial management (PFM). CBR is closely aligned within this pillar, namely: improved public financial management and fiscal self-reliance; improved performance of key Line Ministries and Independent Agencies (LMAs) and municipalities; and improved service delivery through enhanced citizen engagement. B. ACHIEVEMENT OF PDOs (EFFICACY) Rating Before Project Restructuring: The efficacy before restructuring is rated Negligible. Rating After Project Restructuring: The efficacy after restructuring is rated Modest. Assessment of Achievement of Each Objective/Outcome 29. The first project restructuring revised the PDO, indicators, and associated outcome targets. Separate efficacy ratings have been assigned and weighted in proportion to the actual disbursements made before and after the 2016 restructuring. Based on the two ratings, as shown in Annex 7, the overall efficacy of the project is rated “Modest”. A description of the CBR’s achievements, before and after the first project restructuring in 2016, is provided below. Table 3: Original and Revised Indicator and Outcome Targets Original Indicator Target Description Development budget execution rates for highest tier ministries tend to rise 60% Revised below over the course of the project Number of targeted business process improvements achieved in the 6 Revised below highest tier ministries tends to rise over the course of the project Number of Senior Management Group (SMG) positions filled 750 Revised below to reflect gender component Number and percentage of highest tier ministries achieving progress on at N/A Deleted in restructuring least 1 service targeted in their CBR proposals Revised Indicator (2016 Restructuring) Target Description 5 FY2017–20, Country Partnership Framework (Report No. 108727-AF, Oct 2, 2016). 6 “RealizingSelf-Reliance: Commitments to Reforms and Renewed Partnership,” Islamic Republic of Afghanistan, 2014. Page 15 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Priority line ministries and independent agencies development budget Retained from original but revised wording to eliminate 60% execution improves to at least 60% reference to Tier system Priority ministries and independent agencies that have implemented at Reduced number of business process improvements to 2 least 2 business process simplifications planned in their CBRP Civil service 13 for each LMA and reduced number of priority LMAs to 13 positions recruited by CBR Total positions recruited by CBR 1500 Reduced number of positions to 1500 from 2400 New Indicator to align with the overall civil service Civil service positions recruited by CBR filled by women 450 benchmark Before Restructuring 30. Original PDO: “To assist the government in improving the capacity and performance of select line ministries in carrying out their mandates and delivering services to the Afghan people”. (Rating: Negligible) 31. CBR had difficulty gaining momentum and meeting its overall recruitment targets ahead of the first project restructuring. Slow progress during the first four years of implementation was partly due to the complex LMA entry process, and nascent project implementation capability across both IARCSC and MoF. Applying to receive CBR support entailed various criteria for LMAs to fulfill ahead of accessing project resources, including recruitment. LMAs were required to submit an application, and once approved, to develop a comprehensive CBRP for consideration (and tier classification) by the Project Steering Committee, following recommendation of the Advisory Group supported by an Independent Expert Group (IEG). This process, intended to ensure strategic allocation of resources and sustained LMA buy-in, ultimately led to reduced demand for the project. In cases where demand existed, constrained by a lack of upfront project resourcing, LMAs often lacked sufficient capacity to prepare CBRPs. These factors delayed project implementation during the period 2012-2016. 32. Early project implementation was also constrained by IARCSC’s limited capacity to process initiated recruitment packages. This was partly due to work load, with IARCSC at the time responsible for carrying out all recruitment at SMG level (the primary focus of the project). It was also due to deficiencies in recruitment capacity and systems, with most recruitment packages failing quality verification by the Bank (and later HR firm) during this period. This was the case both for higher (Grades 1-2) and mid-level (Grades 3-5) grade recruitment files processed by IARCSC and LMAs, respectively. The political transition that took place starting in early 2014 further delayed CBR recruitment of key positions into the civil service. As part of transition arrangements, all civil servant recruitment was formally suspended for one year, through a September 2014 presidential decree. Once the suspension was lifted, other recruitment problems continued to persist. These involved: (i) political commitment: lack of sustained line ministry commitment – accentuated by the turnover of ministers – resulting in approved CBR positions not being taken up; (ii) process: lack of clarity and contestation over responsibility for recruitment between line ministries and IARCSC; (iii) capacity: limited capacity of line ministries, and lack of resources (commissioners) in IARCSC to oversee large-scale recruitment; and (iv) quality of recruitment: recruitment processes were largely generic rather than being competency based. 33. As a result, by the time of the first project restructuring, CBR had only processed 128 civil service positions out of a final target of 2,400. This poor outcome was the driving force for a presidential decree (issued on September 6, 2015) that moved responsibility for Grades 1 and 2 civil servant recruitments from IARCSC to hiring LMAs (with oversight of IARCSC). At the time, the IARCSC was close to finalizing around 600 recruitment packages which LMAs decided to re-start. Page 16 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) 34. During this period, CBR LMAs made some progress towards proxy performance targets, but it is difficult to attribute gains to CBR. Ultimately, the tier system and requirements to progress from the lowest to the highest level ended up being restrictive, with only two LMAs reaching the highest tier by restructuring. Amongst those, only the Ministry of Agriculture, Irrigation, and Livestock (MAIL) had shown progress on an identified service delivery target within its CBRP. CBR LMA development budget execution performance was relatively better with 6 of the 13 LMAs that would later become priority LMAs having met or exceeded the project target of 60 percent by restructuring. However, with the limited number of CBR civil servants in place and development budget execution rising across most LMAs – overall achievement across Government for 2015 stood at 54 percent – factors other than CBR support likely played an important role. After Restructuring 35. PDO: “To improve the capacity and performance of priority line ministries and independent agencies in selected reform areas”. (Rating: Modest) 36. Restructuring accelerated implementation by improving LMA access to CBR. The restructuring attempted to focus CBR recruitment across 13 priority LMAs, out of which 11 developed approved CBRPs. Eleven (11) non-priority LMAs also developed streamlined CBRPs. In terms of LMA performance on CBRP targets, 14 CBR LMAs, including 9 priority LMAs, met or exceeded the minimum target for business process simplification, as did the Ministry of Interior (MoI) with the direct support of IARCSC. Going into the final year of project implementation, development budget execution had reached an average of 74.6 percent, with all but 2 of the 13 LMAs and three central institutions – IARCSC, MoF, and the Administrative Office of the President (AOP) – prioritized by the project having met or exceeded the 60 percent CBR target. The two exceptions were the Ministry of Communications and Information Technology (MoCIT) and the Ministry of Mines and Petroleum (MoMP). 37. While it is difficult to attribute post-restructuring LMA performance advances solely to CBR (as was the case before restructuring), these did coincide with substantially improved CBR recruitment progress. In the 32 months after its first project restructuring, CBR recruited 925 new positions, compared to 128 positions in the preceding 52 months. However, total contracted positions which reached 1,053, including 462 (44 percent) at the subnational level, fell short of the end target of 1,500 positions. The project also fell short on achieving its target of recruiting at least 450 women (30 percent of the project’s overall recruitment target), added as part of restructuring to align with civil service benchmarks. At closing, the project had only recruited 76 women (7.2 percent). While this was well below the civil service average (26.8 percent), it closely matched female recruitment rates (7.3 percent) within Grades 1 and 2 that CBR prioritizes, reflecting reduced availability of qualified candidates as well as more pronounced recruitment vulnerabilities at these higher-grade levels. CBR did invest extensively in improving recruitment transparency whilst also attempting several innovative measures to increase female recruitment but many of these were ultimately frustrated by limitations in the original Civil Servants Law. Additionally, while prioritization of LMAs was intended to concentrate recruits within a smaller number of LMAs, to enhance scale of capacity building, prioritization did not materialize in practice. Non-priority LMAs requested and received waivers to continue to utilize CBR. As a result, the 1,053 positions contracted by CBR were spread thinly both across Grades 1–5 of Tashkeel and across 47 LMAs. Page 17 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) 38. The harmonization of NTA pay was a key intangible not in place during early implementation and from which CBR considerably benefited post-restructuring. CBR design factored project support to P&G reform implementation. Building on efforts of its predecessor projects, CBR contributed to application of the eight grade P&G structure across 23 LMAs. Reflecting lessons from early project implementation and building on newly converging Government and donor incentives derived from a shifting external aid environment, CBR later also led the effort to regulate and harmonize the pay of donor funded NTAs. The NTA Scale and Guideline, which CBR strongly advocated for and was tasked to develop, was approved by the GoIRA Cabinet and launched by the MoF Budget Department in March 2016. By equalizing the pay of NTAs with that of CBR appointees, this reform has reduced a longstanding constraint in the path towards mainstreaming capacity into the core civil service. Tangibly, it helped the project to attract high caliber recruits that were previously out of reach, contributing to improvements in both recruitment numbers and quality in the post-restructuring period. 39. Through NTA harmonization, CBR has made broader contributions to the GoIRA’s self-reliance agenda. In addition to supporting institutional sustainability, this effort has directly contributed to GoIRA’s fiscal sustainability agenda. Lower NTA salaries have helped to better balance between capital and operational expenditure, especially on the development budget. Assisting LMAs and donors to correctly implement the NTA Guideline has also had the indirect benefit of availing the Bank and CBR with opportunities for enhanced coordination on capacity development. In several cases, these conversations have led to donors agreeing to gradually phase out NTA financing in the assurance that relevant human resource capacity injection would be availed to those ministries by CBR. A key example is the MoF whose departments of Treasury, Budget, Customs and Revenue were previously almost exclusively run by TA, primarily international firms. CBR has also supported NTA rationalization in the Ministry of Refugees and Repatriations (MoRR), the Ministry of Economy (MoEc) and the Ministry of Labor and Social Affairs (MoLSA), and on a relatively less substantial basis due to continued high NTA thresholds, the Independent Directorate for Local Governance (IDLG), and the Ministry for Rural Rehabilitation and Development (MRRD). CBR and the Bank also worked closely with the UNDP-implemented Law and Order Trust Fund for Afghanistan (LOTFA) program to help guide replication of the CBR model in the MoI. Justification of Overall Efficacy Rating 40. The achievement of PDOs is Modest, given minimal progress towards the original objective and improvements after restructuring. Overall, progress toward achievement of PDOs was slow over the first 3–4 years of implementation—due to issues with project design, capacity, and political economy factors. By limiting CBR’s scope and revising the PDO and indicators as part of the first restructuring in 2016, the project was able to make improved progress towards its objectives. This progress provided both the foundation and necessary momentum for the development of the follow-on TAGHIR Project. C. EFFICIENCY Assessment of Efficiency and Rating Rating: Modest 41. The PAD provided some qualitative analysis of the investments planned under CBR. However, cost benefit analysis was complicated by difficulty in quantifying the value of capacity and institution building projects. The nature of CBR’s activities does not lend itself to standard economic or financial Page 18 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) analyses to determine value for money. As a result, the efficiency assessment of the project is based mostly on implementation efficiency – for instance, time overruns, operating costs etc., and the efficacy of human resource capacity injection. Annex 4 provides more information on the efficiency analysis conducted for CBR. Based on this analysis, efficiency is rated Modest. 42. The fiscal scenarios that informed CBR were drawn from the joint World Bank/GoIRA Transition in Afghanistan: Looking Beyond 2014 report. This report projected a financing gap of around 25 percent of GDP in 2021-22 given a set of expenditure and domestic revenue assumptions. As part of project design, the civil service wage bill (including CBR) was costed against these growth and revenue projections, taking account of security, operations and maintenance (O&M) and other recurrent expenditures created through the external budget and that – working on a scenario of a gradual decline in aid – were expected to transfer progressively to the core government budget by 2021. Underlying this was the need – within an extraordinary country setting characterized by a shortage of suitably qualified Afghan professionals on the supply side and competing demands by international agencies, donors and NGOs operating in Afghanistan – for improved Government capacity to be able to absorb, manage and effectively utilize increasing amounts of donor funding provided through the Afghan budget, rather than funded and implemented externally. 43. Based on this analysis, the key cost-benefit conclusion was that human resource capacity injection– the project’s largest expenditure outlay – represented an urgent development priority. While this would increase the Government’s overall wage bill, the increase was considered manageable. This was due to the modest scale of project-financed human resource capacity injection – up to 2,400 positions. The targeting of these positions was clearly defined and limited to SMGs in Grades 1 and 2, common function positions across four defined technical areas, and to specific non-managerial technical positions in mid-level grades (Grades 3-5) only in Tier 1 ministries. This was intended to help the project target key ‘agent of change’ functions but to also contain the risk of pressure to increase the wage bill across Government. It was also assumed that donors would continue to support the GoIRA’s wage bill throughout the lifetime of the project and probably for several years beyond. 44. When viewed in the country context, the costs associated with maintaining the status quo – at the time of project design – far exceeded the fiscal costs of the project. Project human resource capacity injection was considered to represent the least-cost approach (in the absence of the ability of regular P&G to attract and retain higher caliber staff in key positions) to developing capacity in selected line ministries, relative to the prevailing heavy reliance on international and local consultants for project/program management. CBR monthly salaries – ranging from 25,000-320,000 AFN (equivalent to around US$300- US$4,000 based on today’s rates) – represented a portion of the cost donors were spending on technical assistance, estimated to cost into the billions at the time of project design. Ahead of adoption of the harmonized NTA Scale, some donor-funded NTA salaries could reach up to US$20,000 per month. Today, the NTA approach consisting of roughly 20,000 on-budget NTAs is estimated to cost US$212 million7 per year i.e. representing more than a quarter of the annual cost of the entire civilian core civil service (US$800 million) of over 400,000 employees. CBR was therefore additionally expected to create considerable net savings to the GoIRA’s largely donor-funded development budget, should it succeed in contributing to better management and rationalization of NTA numbers (and salaries). Given that the higher-paid civil servants positions financed under the project are part of the core civil service structure, this form of human resource capacity injection avoided – and to an extent redressed – the accountability, policy 7 Afghanistan Financial Management Information System (AFMIS) data, 2017. Page 19 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ownership and institutional sustainability challenges associated with NTAs and other parallel technical assistance. 45. A summary of the original approved cost and actual expenditure is presented in table 2. The project was extended by one year from the original six years to nearly seven years; the first restructuring included an extension, eliminated one of the components (civil service training), narrowed the scope of the program, and reduced the ARTF financing commitment from US$350 million8 to US$150 million. When the project ended, final expenditures had reached at US$78 million (78 percent) not including the remaining ARTF commitment of US$50 million that was not requested. The time extension and disconnect between planned and actual expenditures suggests a level of disconnect between planning and implementation. Table 4: Actual vs Expected Expenditures of the CBR Program Planned vs Actual Disbursements - Annual in millions US$ Total 2012 2013 2014 2015 2016 2017 2018 Original Amount 5 30 65 77 85 88 0 350 Revised Amount (2016) 29 4 6 5 30 45 31 150 Actuals9 4 4 6 7 31 6 20 78 46. CBR recruitment has made noted contributions across the service delivery spectrum. CBR brought in highly qualified people to core government positions as Tashkeel. During project design and implementation, the type of qualified staff that CBR was targeting were not being employed as Tashkeel. Instead they were employed directly with donors or as external advisors. ICR interviews held in Kabul10 provided anecdotal evidence on the contributions of CBR recruits, many of whom instilled new ideas and a strong level of leadership that had been missing within their LMAs. The project recruited 27 provincial director positions in the Ministry of Education (MoE) and 22 in the Ministry of Public Health (MoPH), intended to support education service delivery performance and health provision oversight improvements at the sub-national level. These positions were previously politically appointed, and their professionalization has enhanced transparency and accountability of functions with CBR being perceived to have succeeded in “breaking monopolies (of corruption) at the provincial level”. The project introduced a younger and more qualified generation of hospital directors, enhancing the professionalism of hospitals’ management. 47. Nearly half of CBR recruits are key Director General and Director positions responsible for key government policies and programs. Many of these appointees have been valuable counterparts to IDA and ARTF financed projects. For instance, the current Director General of the Civil Service Management Directorate (CSMD) has been a key protagonist of GoIRA’s current civil service professionalization efforts. He has been key to CBR implementation and played a leading role in the design of the successor TAGHIR 8 The original amount approved for CBR was US$350 million, but only US$100 million was programmed (signed in financing agreement). The remaining amount was to be programmed as needed. 9 A US$25 million project advance was disbursed in year one of the project but remained unspent until after project restructuring. This has been reflected in the table which reduces actual expenditure for 2012 from US$29 million to US$4 million and increases actual expenditure for 2016 from US$6 million to US$31 million. 10 Interviews held in Kabul with IARCSC, MoF, MoE, the World Bank Management and GP teams. Page 20 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) project. Many of the activities supported by TAGHIR, including enhancements to personnel (HR) and payroll management – as well as the administrative reforms complementary to achievement of its PDO, are within his direct remit which by nature (but also due to his proactivity) are central engagements across LMAs and donor and development partners. Another CBR appointee, the Director General for Policy at MoE is known to be a key modernization champion, leveraging donor and development community partnerships to support enhancements in the quality of education services and student learning, as well as broader governance and accountability reforms in that ministry. 48. Despite the cited examples of positive impact of CBR human resource capacity injection, project support was invariably spread too thin. By reducing the number of priority ministries in line with key agreed criteria11, the first project restructuring in 2016 attempted to improve the focus and cost- benefit of investments in human resource capacity injection. By limiting the spread horizontally, across 13 priority LMAs, each LMA would have received an average of 115 new positions; thereby maximizing scale of impact towards targeted CBRP results. In practice, this prioritization did not hold due to continuous higher-level political pressure (exerted upon the Project Steering Committee). While flexibility in recruitment allocation allowed the project to meet many important ad hoc needs across Government, it ultimately contributed to a thin layer of capacity being spread across 47 separate LMAs (and across senior and middle-level management, and technical grades). For example, in the Ministry of Higher Education (MoHE), a very large ministry with a total workforce of 12,073, CBR only provided 10 staff (around 0.08 percent). There are many other smaller LMAs in which CBR has recruited less than five (often just one) positions. Figure 1: Percent of Total Tashkeel as CBR Percent of Total Tashkeel as CBR Ministry of Rural Rehabilitation and Development Ministry of Economy Ministry of Urban Development and Housing Ministry of Refugees and Repatriations Ministry of Agriculture, Irrigation and Livestock Ministry of Communication and IT Ministry of Energy and Water Ministry of Public Health Ministry of Labor, Social Affairs & Martyrs, Disabled Ministry of Finance All other LMA's (36 Total) Ministry of Education 88% 90% 92% 94% 96% 98% 100% Total Tashkeel Contracted CBR Positions 49. Ultimately, project restructuring, through a narrowing of CBR’s scope, did have a positive impact (relative to original design) on the overall value for money of the project. Restructuring involved a decrease in project funding; partly resultant from dropping of the dedicated “Civil Service Training” Component (Component 3). Instead, training activities were mainstreamed across relevant components. 11LMAs were prioritized based on: (a) their contribution to service delivery; (b) their potential for revenue generation, and employment generation; (c) their share of the development budget; and (d) whether they had a high NTA ratio. Page 21 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) LMA training was integrated into their CBRPs under Component 1 “Technical Assistance Facility in Support of Civil Service Reform”, while training in support of HR-related outcomes, led by IARCSC, ACSI or the HR Firm, including delivery of the MMD Program, was planned and channeled under Component 2 “Building Human Resources”. This rearrangement facilitated stronger linkage between LMA training and results, by ensuring that training needs were upfront identified in relation to achievement of CBRP outcomes. This ensured that project financing of training was more strategic (demand-driven) and better planned and coordinated, helping also to circumvent issues related to ACSI’s often limited capacity and comparative advantage to deliver sectoral training programs; often better availed through existing in-house LMA training institutes or targeted outsourcing to specialized training bodies. 50. The reduction of the CBR recruitment target from 2,400 to 1,500 positions was the other key aspect of the restructuring that drove down the overall cost of the project . This rationalization was somewhat necessitated by the slow recruitment progress and relative short time remaining till project closure, at the time of restructuring. It more so reflected a renewed emphasis on fiscal sustainability owing to an external aid environment that had markedly altered from the time of original project design when a continuation of donor funding support of the government wage bill had been anticipated to hold throughout and potentially beyond the project lifetime. Determination of the new recruitment ceiling of 1,500 was based on a weighting by GoIRA and the Bank of realistic and affordable capacity enhancement requirements. Dialogue around fiscal sustainability and the subsequent course correction initiated by the restructuring helped to create a strong foundation and positive momentum for the successor TAGHIR project. TAGHIR which maintained the cumulative recruitment ceiling of 1,500 positions, is by-and-large a transitionary financing facility. Grounded in a financing partnership and strategy with Government, the project is intended to ensure an orderly transition (‘de-projectization’) – into core Government budget – of ‘legacy’ appointees recruited through CBR (that GoIRA deems as priority positions to avoid a disruption in Government operations), and for new TAGHIR appointees (in areas in which GoIRA requires immediate capacity ‘surge’). Building on CBR lessons, TAGHIR has formalized in its financing agreement upfront agreed mechanisms that limit, in line with key government objectives, the spread both vertically (eligible grades) and horizontally (number of LMAs) of new and ‘legacy’ appointees transitionally financed under the project; whilst still provided scope for ‘controlled’ flexibility to respond to emerging or unforeseen capacity needs. 51. As the transition, applied through TAGHIR, remains ongoing, the success of GoIRA retention of CBR recruits on core Government budget cannot yet be determined. Turnover is a risk – that GoIRA, in preparing TAGHIR, has been willing to take – particularly for those ‘legacy’ CBR positions that have not been deemed by GoIRA as a priority for retention through TAGHIR. Accordingly, those position-holders whilst legally remaining as permanent Tashkeel civil servants, have had their remuneration reduced to regular base P&G pay corresponding to their grade level. For those staff, the continued but reduced availability of NTA positions with significantly higher pay, presents an obvious alternative, especially ahead of maturation of recently initiated civil service pay and professionalization (cadre development) reforms. To date, there has been no evidence of unusually high (if any) turnover amongst this cohort. D. JUSTIFICATION OF OVERALL OUTCOME RATING Rating: Moderately Unsatisfactory 52. The overall outcome is rated Moderately Unsatisfactory. The rating is based on a Substantial rating for the Relevance of the PDO (at closing); a Negligible rating for Efficacy before restructuring; a Page 22 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Modest rating for Efficacy after restructuring; and a modest rating for Efficiency. Using the ICR guidelines for a split rating, the overall outcome rating is Moderately Unsatisfactory. Annex 7 of this ICR provides the calculation for the split rating and overall outcome. Table 5: Ratings Relevance of Efficacy Efficiency Overall Outcome Split Ratings Development Assessment Assessment Ratings Objectives Before Restructuring Negligible Unsatisfactory Substantial Modest Moderately After Restructuring Modest Unsatisfactory Moderately Overall Outcome Rating Unsatisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 53. CBR promoted equal opportunity and gender balance. The project required LMAs to include a gender strategy and monitoring mechanism in the CBRPs. Further, 90 women were selected across the two batches of MMD training. The trainings focused on capacity building in the areas of financial management, change management, organizational behavior, and general management among other areas to build capacity for participants to move to another grade. At least two women have been promoted since completing MMD. The 2016 restructuring added a specific gender PDO indicator targeting the recruitment of 450 women through CBR or 30 percent of all recruits. The project only managed to recruit 76 women (7.2 percent) out of 1,053 contracted positions. Although lower availability12 of candidates is a factor (26.8 percent of females participate in the labor force), there is strong anecdotal evidence of qualified female candidates being excluded from heavily male-dominated processes due to discrimination. To mitigate these traditional gender barriers, the original Civil Servants Law of 2008 (revised in March 2018) introduced a clause barring gender-based preference in any form, although this clause inadvertently prohibited CBR from reserving specific position for women only. CBR could only provide an additional 5 points on written test scores, whilst also refraining from re-advertising positions held by women if not agreed to by the incumbent. Institutional Strengthening 54. CBR has made important contributions across multiple administrative reform streams. Importantly, throughout its lifetime, it has invested heavily in the capacity and systems of the IARCSC, enhancing its ability as driver to Afghanistan’s public administration reform agenda. Critically, CBR has helped to better equip and structure the Commission’s organizational and institutional setting to respond to its expanded mandate. Renewed trust in the Commission among the political leadership has led to its increased its role and responsibilities in recent years—beginning with its own reorganization. Today, civil service management functions previously shared with MoF and AOP are consolidated within the IARCSC. 12According to the Afghanistan Living Conditions Survey (ALCS) of 2016-17, 4.8 percent of women aged 18-22 are enrolled in tertiary education, as compared to 14.9 percent of males. Page 23 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) The IARCSC’s responsibilities now encompass all of (amongst others): establishment control, recruitment oversight, career and performance management, training, and NTA monitoring. CBR support, guided by a Bank led functional review of the IARCSC, has been key to helping the Commission assume this mandate. 55. CBR has made some important contributions to institutional strengthening across the public sector. CBR has made important recruitments in the health and education sector, particularly at the sub- national level. In the Independent Land Authority (ARAZI)13, CBR appointees have helped to advance modernizations in land administration, and land acquisition and clearance for mega projects. CBR filled a critical gap in civil aviation with high caliber recruits into the Civil Aviation Authority (CAA) taking over responsibility of Kabul airport management and air control from a costly private contractor. Within IARCSC, CBR has been at the forefront of civil service management, which is led by a CBR appointee, helping to conceptualize and roll-out reforms across civil service pay14, merit-based recruitment procedures and practices15, and performance appraisal systems. Critically, IARCSC has recently accomplished revision of the Civil Servants Law (and development of supporting secondary legislation) which had long been an impediment to these improvements. 56. CBR has laid the foundation for ACSI – IARCSC’s training arm – to make future advances in the quality and scale of training across the public sector. CBR provided capacity support to ACSI to carry out two batches of Mid-level Management Development Program (MMD), including resourcing, provision of trainers and training of trainers etc. The MMD was the equivalent of a two-year work experience that allowed graduates to qualify for accelerated promotion to mid-level manager positions across the civil service. The curriculum was mostly drawn from ACSI and focused on: Financial Management, Change Management, Conflict Management, Presentation Skills, Procurement, Communications Management, and Policy and Strategy. The MMD trainings consisted of Grade 3 and 4 civil servants and at least 7 of the participants including two women were promoted to Grade 2 upon completion of the program. Furthermore, 25 MMD participants were promoted from Grade 4 to Grade 3. These promotions were achieved through competitive process, providing a strong precedent towards stability in the civil service. 57. The MMD program formalized a more transparent and competitive process in selection of participants and delivery of trainings. From the first batch, ACSI selected 100 participants from a pool of 631 applicants through a test that the IARCSC Board of Commissioners supervised. The second batch comprised 100 participants (including 50 women) from a pool of 860 applicants. The selection was based on test scores and ensured a good representation from many LMAs. The participants provided weekly feedback on the trainings through the IARCSC M&E Department and evaluated trainers monthly— evaluations were used as a basis to promote trainers. Initially, the program struggled with appointing dedicated MMD trainers with the requisite skills, but this came through eventually with the recruitment of dedicated trainers. A Training Needs Assessment carried out ahead of the second phase, better informed areas of training and selection of prospective students. As highlighted by IARCSC within Annex 5, training programs require a clear strategy and clarity on the expected outcome. 58. ACSI had various capacity constraints but overall, it managed to deliver a successful training program. Ahead of recruitment of additional trainers, there were capacity gaps in matching the level of training—including training materials—with participants across various levels. A mentoring program was 13 Now merged into the Ministry of Urban Development, Housing and Land (MUDHL). 14 This includes P&G and recent enhancements to this through the newly approved Civil Servants Pay Policy. 15 Such reforms include measures to promote increased recruitment of women. Page 24 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) part of MMD but could not be delivered due to lack of capacity. During the first phase, the program also suffered from the lack of coordination with the two PSUs, which also contributed to delays in procurement of equipment such as laptops, which caused inconvenience to the participants. Many participants expressed that there was a lack of coordination between the ACSI and LMAs—for instance, being marked as absent in their respective LMAs, unapproved overtime and being treated unfairly by other LMA colleagues who were not part of the training program. The HR firm evaluated the program to be successful. The M&E department also assessed impact on the competency levels for every subject, which showed an increase by 35.3 percent for all subjects during the first phase. Improving the scale of such career progression outcomes can eventually help ACSI segue into specializing in capacity building throughout the civil service as the primary training institute. It already has the basic resources in the form of 16 training managers and some Directors. 59. CBR has contributed to efficiency improvements in key common function areas through recruitment of HR, Finance & Accounting, and Procurement Director positions across Government. HR Directors have been key interlocutors of CBR recruitment within line ministries and have carried responsibility for ‘socializing’ and overseeing implementation of many of the administrative reforms (around civil servant recruitment, pay, performance management, gender inclusion, professionalization etc.) that have been initiated by IARCSC in tandem to CBR, and later, the design of TAGHIR. Common function directors are also the focal points and champions of business process simplification supported under CBR, many of these being related to ministry backbone and shared services, especially Finance & Accounting. In Procurement, CBR has driven the centralized batch recruitment (utilizing computer-based testing and e-recruitment systems), and training and on-boarding (through the National Procurement Authority) of 697 procurement professionals. These are intended to later make up the procurement cadre – the first professionalization pilot, alongside Customs – for which a dedicated governing regulation was approved by the GoIRA Cabinet in April 2019. In the interim, CBR financed the salaries of the 15 personnel recruited at Procurement Director (Grade 2) level and the 88 recruits at Sub-Director level (Grade 3). 60. CBR appointed Director Generals and Directors’ contribution to ministry outcomes has often been recognized at the highest level of government. In several cases, they have been promoted to Deputy Minister and even Minister level in the case of MAIL. The CBR appointed Director General for Extension Policy and Agriculture Research also became a Deputy Minister in the same ministry. CBR alumni have become Deputy Ministers in MoF, MoLSA, the Ministry of Industry and Commerce (MoIC), and MoMP. The current Advisor to the Executive Director representing Afghanistan at the World Bank Board of Director was formerly a CBR appointed Director General for Budget in MoF. In fact, the CBR MoF PSU Director left his position to become Deputy Minister for Revenue and Customs in MoF. Mobilizing Private Sector Financing 61. As Afghanistan transitions from donor dependency to self-reliance, the efficient use of public resources is essential. By supporting more sustainable capacity building within the core civil service coupled with efforts to rationalize the size and cost of the parallel civil service, CBR has helped the GoIRA to use scarce resources more effectively on its most significant expenditure – salaries and wages. This approach can be considered as a key enabler for the broader reform agenda supporting mobilizing finance for development (MFD). Although MFD is in an early phase in the Afghan context, surveys of Afghanistan’s Chambers of Commerce and businesses consistently indicate that improved public services are required to attract new capital investments. Staffing in key ministries, tax agencies, and regulatory bodies is fundamental to an improved business climate. The CBR approach has made important contributions in Page 25 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) this regard, enhancing the Government’s ability to attract and retain top talent in key positions, including within ministries such as the MoF, MoCI, MoEc and IDLG that are key enablers of private sector led growth. Poverty Reduction and Shared Prosperity 62. CBR has contributed to service delivery outcomes through its support to LMA capacity. Although difficult to measure in quantitative terms, there is substantial anecdotal efforts on for instance, the impact of project support to education and health service provision performance and oversight at the sub- national level. Although not directly targeting poverty reduction, P&G reform which CBR helped to implement, introduced greater salary decompression. This has led to generally higher salaries for most civil servants, albeit salaries have since significantly declined in real terms. The recent Civil Service Pay Policy does include more direct poverty targeting through a salary supplement for lower grade non- professional civil servant positions. This is derived from recognition that although the median public sector pay is higher than some non-governmental sectors, 39.8 percent16 of households headed by a salaried public sector worker fall below the poverty line. Other Unintended Outcomes and Impacts 63. CBR provided a testing ground for many recent innovations to civil servant recruitment processes, contributing key lessons to the new Civil Servants Law and its secondary regulations. The project has adopted new computerized testing processes—an attempt to move towards more transparent competency-based recruitment that IARCSC is gradually introducing across the public sector. These processes are grounded in an e-recruitment system that was developed as part of CBR. The centralized recruitment of procurement staff (utilizing e-recruitment systems) that CBR supported was amongst the first to be carried out through batch hiring. Many new measures to improve female recruitment, enabled through amendment of the Civil Servants Law, were conceptualized in response to frustrations over a continued lack of progress within CBR in this area. These measures include female-only positions, quota systems, and several related to awareness and trust building. 64. CBR played a critical role in pay reform. The project built on the work of its predecessor projects, helping to complete P&G reform (2012-2014) in 23 ministries. This intensive administrative reform involved the re-advertising and filling of over 200,000 civil service positions as part of gradual replacement of the former 12-grade scheme. Today, over 90 percent of non-security civil servant salaries are structured according to the eight grade P&G. Overall, the P&G system introduced form and structure, contributing to stability in public employment and the stability of institutions. In LMAs where P&G has been completed, functional units are better defined and operationalized whilst staff in these LMAs have more structured career paths. Payroll processing and pensions administration have also improved in these agencies. Finally, the reform has contributed to building capacity in some specialist areas such as teaching. 65. CBR was at the forefront of government efforts to rationalize and regulate the pay of NTAs. The NTA Scale (March 2016) applies to all donor-funded NTA and provides a harmonized, consistent, fair and transparent pay framework at CBR levels. The scale removed the financial incentive for qualified Afghans to apply to an NTA position compared to a CBR civil servant position, which is critical for sustainability. The NTA scale also strengthened the ability of government to hire national contractors through the national budget, which has enhanced ownership and accountability. 16 2016-17 Afghanistan Living Conditions Survey (ALCS) Page 26 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) 66. These advances have provided a strong foundation for a next generation of civil servant pay and professionalization. CBR provided important lessons that were reflected in the Civil Service Pay Policy. This new policy maintains P&G as the single harmonized salary spine whilst introducing competency-based pay increases to supplement base P&G pay. In this sense, the new policy is designed to provide momentum as well as the incentive structure for civil service professionalization efforts, primarily the establishment of professional cadre groups. CBR has supported batch recruitment and onboarding for the first cadre to be established in procurement. As cadre development efforts mature, these groups are expected to absorb relevant staffing from within the current cohort of NTA and CBR/TAGHIR appointees, supporting continuity of this built-up capacity. High performing civil servants that are becoming harder to retain on P&G, as demonstrated by ‘recycling’17 within CBR recruitment, would also be eligible for entry. The new policy is to be financed through discontinuation of the current multitude of parallel pay scales and ad hoc allowances, simultaneously addressing issues around misuse of public funds and a lack of ‘equal pay for equal work’. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 67. There were several factors that constrained the reform environment during preparation. These were mainly: low capacity, large project scope, project complexity, executive interference and weak coordination. CBR was built on the principle that public administration capacity building must be demand driven, led by ministries, and focused on deliverable results. The project recognized weak capacity in line ministries as a key challenge to service delivery and low development budget execution rates across Government. For this reason, the project was designed to address capacity constraints at the central, provincial, and district levels. The team also recognized that political factors would affect capacity building efforts and would therefore require long-term partnership between the Government and donors. It prepared a six-year facility that aimed to complement existing ministry-specific service delivery and other sectoral reform programs. 68. LMAs would lead their own reform process under CBR. Participating LMAs had to demonstrate ownership and commitment through formulation of a prioritized capacity building program with concrete and measurable results. The project required LMAs to develop a prioritized medium-term capacity and institution building framework18. An Advisory Group19 and IEG were formed to assist LMAs in preparation of CBRPs. However, the Advisory Group turned out to be too high level to provide technical support. LMAs did not have adequate capacity nor the technical support to develop key aspects of their proposals. Restructuring helped streamline the process by removing the Advisory Group and simplifying the CBRP template for the priority LMAs that had not yet prepared a CBRP. 17‘Recycling’ refers to the practice of reappointment of incumbent or another current staff member from the same department. This runs contrary to the objective of CBR, which aims to re-advertise (on higher pay) key civil service positions deemed to need capacity upgrade. 18 Defining expected results and monitoring arrangements, implementation responsibilities and timetable, results monitoring arrangements, and resources required to implement the framework within the specific political economy of the context 19 Comprising of technical experts from MoF, IARCSC, Office of Administrative Affairs (Tashkeelat Commission) Page 27 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) 69. The tier system was designed to support broad reforms for all LMAs. The requirements to achieve high and middle tier status to receive increased project support slowed down implementation. All ministries were to be supported in completing essential reforms, such as the implementation of P&G. Support for Tier 1 and Tier 2 was to be provided once the ministry designed a reform plan, and continued support depended upon the ministry meeting the targets it established for its reform plans. To advance to the highest tier, an LMA had to submit a comprehensive reform proposal and to advance to the middle tier LMAs needed to demonstrate a commitment and ability to develop a full CBRP. Many LMAs initially ignored CBR and sought support through equally competent and readily available NTA advisors. 70. The project designed a dual implementation structure with MoF and IARCSC to cater for mandate overlaps and compensate for relative capacity deficiencies. As explained by IARCSC within Annex 5, it had the mandate to lead administrative reforms but did not enjoy the required level of trust and capacity to implement such a large program alone at the time of project design. As a result, MoF was given overall leadership—including leadership of the Steering Committee—and primary responsibility for CBRPs. This reflected MoF’s central convening role including its provision of policy reform oversight across Government. The ministry was also better capacitated relative to IARCSC although it could not dedicate the necessary attention to the project at leadership level, largely delegating this responsibility to its Budget Department which itself was overstretched. The Budget Department’s relevance lay in its primarily responsibility for expenditure management, including leading establishment setting and control and for issuing NTA allotments. IARCSC, on the other hand, held responsibility for the HR and training components that fell within its civil service management mandate. While this distribution of responsibilities made sense in theory, in practice, rather than complementing respective mandates and capacities, the two entities engaged in competition, which negatively impacted on project implementation, coordination, reporting and M&E. 71. The project anticipated several political economy risks in the operating environment. The main risks were: lack of central agency cooperation; biased recruitment and selection of senior management group staff; unethical behavior in the civil service; politicization of CBR project funding allocations; misuse of project funds, inter-cadre competition; weak service delivery accountability; limited impacts of technical assistance; and pay setting risks. The project anticipated that the patrimonial nature of the state could impede the process of building a merit and skill-based civil service. The project proposed hiring an HR firm to address the first two risks and to enhance compliance with recruitment policies, thereby, ensuring transparency and accountability. There were significant delays in hiring the HR firm until 2014 and the Bank dedicated its supervision time monitoring recruitment compliance in the interim. 72. The overall risk to achieving the PDO was assessed to be High. The project recognized directives from the AOP to build capacity within Government and review of salary structures of nationals working on donor funded projects. Donors were engaged during preparation to help reinforce the need for strong government commitment towards this project. Further, the project anticipated that donors would commit to rationalizing NTAs. On the contrary, donors continued to support NTAs through their individual projects. CBR also identified the challenge of wage bill sustainability due to heavy reliance on NTAs and aimed to harmonize NTA pay. The project, however, did not prepare an exit strategy with the assumption that the wage bill will continue to receive external financing. While the project was intended to improve capacity within LMAs, it failed—through its complex project design—to factor in existing capacity constraints. Page 28 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) B. KEY FACTORS DURING IMPLEMENTATION Before Restructuring 73. The risk to project implementation was assessed to be High. The key risks during project implementation were deteriorating security, changes in leadership of selected ministries, slower than anticipated implementation of reforms, weak political oversight and commitment, inadequate ownership, resistance from ministries not directly benefiting from the project, and poor donor coordination. Other political economy risks such as the increased potential for rent seeking through recruitment practices were also high. CBR included a small high-level Steering Committee comprised of the Finance Minister and IARCSC Chairman to ensure effective decision-making. 74. LMA access to CBR was overly complex, which delayed implementation. CBR required ministries to submit elaborate reform plans and demonstrate reform progress to graduate to higher levels of support. CBRPs were to ensure strategic utilization of project resources with the tier system. In practice the paper-work and conditionality kept many ministries away or caused them to disengage mid-track. In the few cases of strong line ministry commitment, without sufficient upfront resourcing, there was little or no reform progress. Interview panels and the Advisory Group responsible for pre-approval of proposals—did not convene on time—while the grievance redressal often took a long time and occasionally derailed the recruitment process. Ahead of the first project restructuring in 2016, CBRPs were only approved for eight ministries. 75. The project was to follow a demand-driven approach, but the level of demand from LMAs was low. The project diluted the scope by targeting too many ministries. The approval process was complex and lengthy, which resulted in delays with recruitment, and hence project implementation. LMAs did not have the capacity to develop such a thorough CBRP—which was a prerequisite to initiate recruitment. Moreover, many CBR ministries were being led by acting ministers during the initial implementation phase. Initial design misread the level of political commitment. For example, the tier approach was intended to instill ownership, but this required higher level commitment to help see it through. For these reasons, the reform lacked ownership and buy-in from the ministries because there were other avenues of acquiring high caliber staffing (as NTAs) in a more streamlined manner. As a result, there were too few recruits from the CBR in LMAs. 76. There was no clear lead between the MoF and the IARCSC. Having in place two separate PSUs— at the MoF and the IARCSC—did not facilitate a smooth implementation for the CBR recruitment process. As a result of suboptimal coordination between the two entities, project components were often implemented in silos from one another, diluting linkage of recruitment and training with CBRP reforms. The IARCSC PSU initially did not have sufficient capacity. The PSU in MoF was well managed and capacitated, however, it did not have a dedicated focus on CBR at the responsible line department level or at the level of the Steering Committee. The split leadership arrangement between the MoF and the IARCSC came with many limitations and without any clear benefit. While there were not many initiated recruitments of civil servants, many faced quality issues with Terms of Reference (TORs), inconsistent application of shortlisting criteria, and salary scale application. Ahead of procurement of the HR Firm, the Bank was responsible for providing ex-ante quality verification which invariably led to rejections due to non-compliance. In cases where selected candidates did meet minimum requirements, they had often not been the most qualified applicant. Bank staff recount being approached by many applicants who despite Page 29 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) meeting criteria had not been shortlisted or were rejected. This coincided with a period during which accusations of lobbying, bribery and other corrupt practices were rampant. After Restructuring 77. The project was restructured for the second time in June 2017 to amend the governance structure. It fully transferred the implementation responsibility to the IARCSC and consolidated the two PSUs into one Project Implementation Unit (PIU). This PIU was anchored in the IARCSC—to collaborate with all relevant departments of the IARCSC. It also comprised existing staff from the discontinued MoF and the IARCSC PSUs to sustain capacity. As planned, the PIU was later integrated within the IARCSC’s core Taskheel. Following restructuring, the IARCSC, under the oversight of its Board of Commissioners, was responsible for daily management and administration (including fiduciary) of the project, evaluation of project activities, leading recruitment, appeals, pay, performance management, and training of civil servants. IARCSC served as Secretariat to the High Council for Reforms and Good Governance which replaced the Project Steering Committee. This is chaired by the President of the GoIRA and includes the MoF. 78. Restructuring and renewed political support accelerated implementation but undermined transparency. Before restructuring, a presidential decree halted recruitment for nearly a year. The IARCSC was not part of the Cabinet and, at the time, was facing issues of leadership. Because of lagging recruitment targets, another presidential decree20 was issued in September 2015, this time transferring responsibility for recruitment of Grades 1 and 2 positions from IARCSC to individual ministries. The decree applied to all civil servant recruitment at these grade levels, including within CBR. LMAs were already responsible for recruitment of Grades 3 and below. Restructuring consequently built-in quality assurance measures and streamlined recruitment procedures, capping the timeline at 50 days. Although, this increased the pace of recruitment in the medium-term, it was a setback for CBR recruitment in the short- term. During that period, LMAs abandoned around 600 recruitment packages that were close to completion by IARCSC. Giving the responsibility to LMAs also meant increased scope for ministerial interference and there were now more issues of political interference and non-compliance with recruitment procedures – the HR Firm reported a rejection rate of 31 percent during this period. Targeting was a further concern, with 42 percent of contracted positions during this period classified as ‘recycled’ i.e. reappointment of incumbent or another current staff member from the same department. This runs contrary to the objective of CBR, which aims to re-advertise (on higher pay) key civil service positions deemed to need capacity upgrade. 79. As part of the first restructuring in April 2016, CBR attempted to sharpen its focus on 13 priority LMAs. The focus on these LMAs did not hold in practice. The Steering Committee approved recruitment allocations for almost all LMAs and as a result, this diluted the level of concentration within the 13 LMAs as well as across all 47 LMAs that were supported. The positions requested by the LMAs were not always strategic as they were ad hoc with no robust process. CBRPs did not require the designation of positions that were to be recruited—the M&E firm that was responsible for addressing this was unable to do so. In some cases, the selection of positions was based on removing staff that were unfavored or conversely to increase the salaries of existing staff. Further recruitment of highly skilled people was not consistent across all LMAs. Both recruitment quality and targeting considerably improved following consolidation of 20 Presidential Decree Number 82 on Recruitment issued on September 6, 2015. Page 30 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) the project within the IARCSC under its new leadership in 2017, with fewer rejections by the HR firm and the largest number of contracts being processed from this point up to project closing. 80. The project aimed to bring cohesiveness in donor support. The intent was to resolve the fragmented public sector system with the assumption that the project would prevent donors from funding the civil service in parallel. However, there was no mechanism in place to align donor support and donors continued to finance the NTAs through their individual projects. This began to improve with the introduction of the NTA Scale which CBR developed in tandem with the first restructuring in early 2016. In ‘socializing’ the NTA scale and guidelines, CBR also worked directly with individual LMAs and donors on synergizing NTAs with CBR. Several donors began scaling back their funding of NTAs in LMAs that were supported by CBR. In some cases, CBR support was directed towards replacing NTAs—for instance—in conjunction with the European Union (EU) and Department for International Development (DfID) in MoF and MoRR. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 81. M&E responsibility was split between the MoF and IARCSC PSUs, mirroring project implementation arrangements. The MoF was responsible for supporting LMAs in developing CBRPs and the IARCSC was given responsibility for components 2 and 3 and would support CBRP results related to civil service reform and HR management. The two units were required to jointly produce a quarterly assessment of project implementation and a semi-annual progress report detailing LMA progress. This arrangement proved problematic due to the lack of cooperation and coordination (detailed in other parts of this report) between the MoF and IARCSC. 82. The initial Results Framework was ineffective due to an ambitious reform agenda. Three out of the four PDO output indicators were targeted towards achievements of highest tier LMAs, pre-supposing the LMA’s ability and willingness to advance through the tier system. Meanwhile, there were no metrics to measure progress towards tier advancement. Additionally, the PDO indicators on business process simplification and improvement in budget execution rate were difficult to attribute to the project alone and there were no built-in metrics also within annual performance appraisal of CBR appointees to measure appointees’ contribution to these aspects. Furthermore, the development of results frameworks by LMAs for their CBRPs was a new process that required support of the MoF and IARCSC PSUs. 83. The PDO indicators clearly identified the mechanism for data collection and monitoring. The procedures identified were either through the utilization of MoF Afghanistan Financial Management Information System (AFMIS) or through coordination and utilization with the IARCSC and its PSU. However, the overall results framework did not provide a set format or process for measuring LMA progress towards the CBRPs even though that would be the focus of the semi-annual report jointly provided by MoF and IARCSC. 84. The post restructuring results framework was better aligned with measuring achievement towards the revised PDO. The removal of the tier system in favor of a list of priority LMAs provided a clear understanding of which LMAs the indicators were applied to. Additionally, the revised indicators Page 31 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) were much more explicit in measuring both the capacity injection and service delivery components of CBR following restructuring. These indicators were in relation to development budget execution and business process simplification, which were proxy indicators for performance. CBR’s contribution was through capacity improvements within LMAs, however, the project did not include metrics within annual performance appraisal to measure the contribution of CBR appointees to these and other key LMA results. It is also difficult to quantify CBR’s contribution on budget execution. Following the initial restructuring, the project adopted a monthly monitoring strategy to measure progress against the indicators—a practice that it continued until the end of the project. The HR firm also produced both monthly and quarterly monitoring reports that provided quantitative and qualitative progress on recruitment compliance. M&E Implementation 85. The CBR PSUs were unable to report timely on the project results framework indicators. Both M&E units were originally understaffed and needed additional capacity to help LMAs in development of their results frameworks and the monitoring and development of the CBR results framework. As a result, after almost two years of CBR implementation, very little reporting or data collection on M&E had been completed. Project restructuring entailed monitoring the results monthly in addition to monthly and quarterly progress reports from the HR firm. The first restructuring improved coordination between MoF and IARCSC by clearly delineating responsibilities and fostering coordination mechanisms. 86. A specialized M&E firm supported the LMAs to develop and monitor their CBRPs. CBR hired the firm in August 2016 to: streamline CBRPs; highlight reform orientation; make the results indicators more focused, measurable and attributable; and to better link and upfront specify requested project inputs including staffing, TA and training. In few cases where priority LMAs did not yet have CBRPs, the firm was to help them prepare these in the new streamlined format. The firm was supposed to support priority LMAs to prepare CBRPs in the new streamlined format. GoIRA and the Bank raised questions over the quality of the M&E firm’s support. None of the CBRPs were revised although the firm claimed to have developed a number of these. The original contract of the firm was for one year due to delays in procurement. As the firm was costly and could not deliver on its objectives, value for money was deemed very low—and hence the contract was not extended when the project closing date was extended. Despite questions over quality, the firm filled a needed gap that the Government was unable to fill during CBR’s implementation. 87. Restructuring introduced a new reporting template that allowed for regular monthly reporting. The revised indicators, along with increased support for the M&E units allowed for regular monthly reporting on CBR progress. In addition to the reporting templates, monthly CBR implementation briefings, and a CBR newsletter were produced and widely distributed. M&E Utilization 88. Project support missions and reviews were conducted on a regular basis and identified the problems with M&E and the overall project indicators by early 2014. Although the benefits of restructuring and a closing date extension was raised during the February 2014 MTR, it took an additional two years before a project restructuring could be finalized. There were multiple factors that contributed to this delay including the drawn-out Presidential elections and political infighting between the MoF and IARCSC over who has the mandate and capacity to effectively lead implementation of CBR after restructuring. Page 32 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) 89. Regular reporting, after restructuring bolstered project management. LMAs, the World Bank, donors, and other implementing partners received monthly progress reports on CBR. These provided the level of detail the Government and task team needed to rightly identify the bottlenecks and appropriately respond. Justification of Overall Rating of Quality of M&E 90. The overall quality of M&E is Modest. The initial M&E design and implementation provided little support to the project. The PSUs had limited capacity to support the LMAs, establish baselines on the indicators and report in a timely manner. After restructuring in 2016, the framework was redesigned and better suited to measuring progress towards the revised PDO. Additionally, extra support provided to the PSU allowed for consistent and timely monthly reporting on progress. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 91. The environmental and social safeguards compliance. The project did not entail activities that involved significant social risks or dealt directly with the population at large. The project financed consulting services, goods, training, and workshops. The project did not undertake any civil works or activities that could have negative environmental implications. The project was classified as Category C. Fiduciary Compliance 92. The project maintained agreed financial management arrangements for the period of implementation except the internal audit of the project where the World Bank did not receive any report during 2016 – 2019. This problem was not specific to CBR and is common across the World Bank portfolio due to capacity constraints within the Internal Audit Department (IAD) of MoF, responsible for carrying out internal audit of World Bank/ARTF projects. Financial Management Performance of the Project 93. The financial management performance rating fluctuated between Moderately Satisfactory and Moderately Unsatisfactory throughout the life of the project. The financial management rating at project closure was Moderately Unsatisfactory because the World Bank identified certain ineligible expenditures although these stemmed from ex-post HR (recruitment compliance) audits and not fiduciary deficiencies. The Bank required that all disbursements made in respect of the civil servant contracts determined as non-compliant be refunded and that no further salary payments be made from the project for those positions. The review for the third-quarter of 2016 determined 4 of 17 sample packages as non-compliant with minimum CBR quality standards, which required a refund of US$111,440. The review for the fourth- quarter of 2018 determined 13 of 46 sample packages to be non-compliant with minimum CBR quality standards, which required a refund of US$364,230 that continues to remain outstanding. The other reasons were absence of complete record of fixed assets, no physical verification of fixed assets, weak contract management, overall low budget utilization, and not using the project-developed financial management database until the project closure despite a significant investment in time and resources. The firm that was recruited could not deliver on time and required multiple contract extensions. Eventually, it provided the database by project closure—as a result, the project could not adopt it. Page 33 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) 94. The World Bank has received all the financial management reports on time except the internal audit reports of the project, which the World Bank has not received since 2016. There are no overdue interim unaudited financial reports (IUFRs) or external audit reports with respect to the CBR project; The ineligible expenditures were also settled. Moreover, all the fixed assets purchased from the grant proceeds were transferred to the IARCSC before the end of June 2019. 95. Table 3 has the expenditure details by component as of December 21, 2018, based on IUFRs. The figures may slightly change once the project submits its final withdrawal application. The World Bank’s Disbursement Unit has allowed additional time for IARCSC to do so beyond the project’s disbursement grace period which ended on June 30, 2019. This has not yet commenced due to a request from MoF to delay this process until it can complete outstanding refund of US$364,230 in civil servant salaries declared ineligible by World Bank ex-post HR (recruitment compliance) audit carried out in the fourth-quarter of 2018. Table 3. Expenditure by Component Project Component Cumulative Expenditure (US$) Component 1 6,233,176 Component 2 53,427,919 Component 3 563,350 Component 4 9,774,600 Remaining Bank Balance 7,120,000 Total Project Expenditure 77,119,045 C. BANK PERFORMANCE Quality at Entry 96. The overall quality of World Bank performance at entry was Moderately Satisfactory. Although the project considered challenges in Afghanistan’s public sector and lessons learned from predecessor projects, it was not well grounded in the realities of implementing public sector management projects in the country. The project did a thorough analysis of risks and identified adequate mitigation measures. However, those mitigation measures relied on assumptions that eventually did not materialize—for instance on political commitment, implementation complementarity and donor coordination and commitment on NTA rationalization and salary harmonization amongst others. The project comprised an experienced and committed task team to provide technical support to the project, which was critical. The project team focused on the gaps and opportunities for strengthening public sector management to reach efficiency and transparency targets. While designing the project, the World Bank considered the adequacy of project design and many relevant aspects—including strategic relevance, technical, financial, economic, and fiduciary aspects. Quality of Supervision 97. The quality of supervision was Satisfactory. The World Bank’s team included the task team leaders and financial management and procurement specialists who worked closely with Bank counterparts, particularly with the IARCSC and relevant line ministries. The project had three TTLs for the entirety of the seven-year implementation period. The World Bank conducted regular supervision Page 34 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) missions, however, leveraging an in-country Co-TTL and a Dubai-based TTL, the task team adopted a continuous and intensive implementation support model that did not rely on formal missions. There were 14 ISRs that the task team prepared during the project implementation period, however, the team did not prepare Aide Memoires regularly. 98. The task team was prompt in responding to the Government’s requests. The World Bank’s technical and fiduciary teams provided regular support to meet the PDO and also restructured the project to streamline project design and align the program with the Government’s strategic priorities. The original M&E framework resulted in limited initial reporting on the project, however, through continued engagement with Government counterparts, the TTLs were aware of many of the issues early on. Even though restructuring was not immediately feasible due to the 2014 elections, the TTLs and counterparts were able to start the restructuring design allowing CBR to continue to move forward once the restructuring was approved. Project restructuring helped consolidate implementation responsibility within the IARCSC and amalgamate the PSUs in the MoF and the IARCSC into one PIU. The team also extended the project closing date to align with the Government’s request. 99. The task team provided intensive hands-on support to project implementation. The Bank provided ex-ante quality verification of recruitments ahead of procurement of the HR Firm. Following project restructuring, the team continued to be diligent in trying to uphold the integrity of recruitment processes, including when IARCSC (by its own admission), was unable to resist some of the political pressure. In addition to carrying out periodic ex-post review of recruitment files, the task team directly engaged IARCSC and the HR Firm on ongoing problematic recruitment cases, and on introducing measures to strengthen quality assurance mechanisms but also streamline processes, and to increase female recruitment; producing various Issues Note for IARCSC. In efforts to strengthen CBR’s reform linkage, the task team provided quality review of CBRPs, bringing in relevant GP colleagues to add sector and ministry knowledge and nuance to reviews. The task team played an instrumental role in improving project M&E and reporting, designing the monthly CBR monitoring tables and preparing and disseminating these on behalf of MoF and IARCSC in initial months. The team has carried out direct outreach, coordination and provided ‘just-in-time’ hands-on implementation support to Government and donors on the CBR/NTA approach and helped to connect MoF and IARCSC to these partners. It produced many dissemination briefs including CBR responsibility flow charts, FAQs on the CBR approach and on NTA harmonization, and several related policy notes intended to inform these agendas. On wage bill management – intersections between pay, professionalization and rationalizing parallel structures more broadly – the Bank team has long been a champion and primarily through the entry point of CBR, the team has been a key proponent of recent related policy reforms (Civil Servants Law, Pay Policy etc.) with which the successor TAGHIR Project has been interlinked. The task team was entrusted to carry out a functional review of the reconstituted IARCSC. CBR has supported implementation of its recommendations which have guided preparation of IARCSC’s new 5-year Strategic Plan. Justification of Overall Rating of Bank Performance 100. The overall World Bank performance was Moderately Satisfactory, given the quality at entry (Moderately Satisfactory) and supervision (Satisfactory). Page 35 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) D. RISK TO DEVELOPMENT OUTCOME 101. CBR did not fully achieve the development outcome it endeavored, and hence the Moderately Unsatisfactory rating. The project had limited traction in influencing LMA-level reforms. The project provided some important staffing upgrades and there are many examples evidencing where CBR has substantially contributed to improved ministry outcomes capacity improvements. Ultimately however, capacity enhancement has been uneven, due to inconsistencies in recruitment quality and transparency and deficiencies in the targeting of CBR positions. Capacity injection was also spread too thinly to have tangible scale of impact within the LMAs that the project supported. While CBR does not entail the accountability and policy ownership issues of NTA, externally financed ‘projectized’ capacity support is also afflicted by sustainability issues. The adoption of a harmonized NTA scale was an important step driven by the project, however, NTA pay is still costly and continues to be financed outside of Government payroll. In a context of declining external aid, this risks loss of long built-up capacity. Unlike other contracted and project-financed staff in PIUs, large turnover of CBR appointees would have a serious disruptive impact across the Government. As already noted, almost half of the currently contracted CBR posts are of key Director General and Director level positions across the Government, and such staff frequently lead on policy development, management and technical execution issues. 102. CBR’s risk to development outcome is therefore high. The country’s difficult fiscal situation and overall bleak macroeconomic outlook restricts Government’s ability to transition CBR positions to core Government budget at this time. This is more so the case given that continued insecurity necessitates that the largest share of Government expenditure be channeled towards the security sector. Political uncertainties posed by ongoing peace talks are only likely to further exacerbate challenges. A potential peace deal with the Taliban whilst expected to support rationalized security costs in the medium-term, would entail new expenditure priorities, including increased development expenditure directed towards current Taliban-held areas. Reintegration of Taliban forces into society, including providing them with jobs, is another peacebuilding priority. One possibility currently being discussed involves absorbing large numbers of ex-combatants into the civil service corps. This poses a difficult trade-off. It may enhance prospects for stability, but this would come at the cost of fiscal sustainability and longstanding efforts to elevate state capacity; also rolling back recent hard-won gains in merit-based recruitment processes. 103. In response to this context, the Bank has proactively utilized the transition from CBR to the follow-on TAGHIR project (and supporting analytical engagements) to help Government determine a path to a more fiscally sustainable and meritocratic civil service. TAGHIR has been designed to help Government gradually transition away from CBR’s projectized approach to full budget funding of key civil service positions. The project is intended to provide an orderly transition (‘de-projectization’) for those ‘legacy’ appointees supported by CBR that the Government deems as priority positions to avoid a disruption in government operations. To anchor this transition, an upfront financing strategy and partnership has been agreed in line with fiscal space considerations and underlying assumptions regarding LMAs to be prioritized. 104. TAGHIR’s financing strategy is grounded in the new Civil Service Pay Policy. The policy aims to maintain the P&G structure as the single harmonized salary spine to classify all civil servant positions whilst introducing a variety of competency-based pay increases to supplement base P&G pay. In turn, parallel pay structures such as the Super Skills and NTA Scale (and ad hoc allowances) will be gradually phased out in sequence with the introduction of these competency-based pay increases. Through the introduction of cadres, the policy will cater for the gradual transition of current CBR/TAGHIR and Super Page 36 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Skills civil servants and help to attract qualified outside entrants into the core civil service, including current NTA staff. Factoring in fiscal space considerations, the pay increases will not yield pay as high as current NTA pay levels but will ensure that net pay for qualifying civil servants is considerably higher than base P&G which has largely failed to attract high caliber staffing into the core civil service. The policy is designed to be cost neutral; generating savings (e.g. from harmonizing allowances, discontinuing ad hoc allowances, phasing out NTAs etc.) from which the additional costs to the wage bill are to be financed. This is intended to maintain wage bill expenditure at around 5 percent of GDP. 105. Ultimately, however, this will necessarily be a long-term journey. TAGHIR is an important transition mechanism that will provide momentum for improved wage bill management. However, it is only a three-year operation and cannot be expected to secure sustainability. This will require consistency across a number of years, an attribute not found in the Afghan State, especially with the talk of a peace deal with the Taliban. Presidential elections planned in September 2019, and the potential for a drawn- out post-election transition, present a near-term risk of disruption. Mitigation is provided by the fact that reforms are derived from the Afghanistan National Peace and Development Framework (ANPDF) and as such these are firmly Government owned and driven; with legal and administrative prerequisites already completed. Meanwhile, existing policy-based instruments, including the IDA/ARTF Incentive Program Development Policy Grant (IP DPG) and European Union (EU) State and Resilience Building Contract (SRBC) have been utilized to provide an incentive structure for these reforms. That said, in the event that forthcoming elections lead to a change of Government or a peace deal with the Taliban, reversal in political will to implement agreed reforms becomes an increasing risk. V. LESSONS AND RECOMMENDATIONS Lessons for public sector reform and institution building in fragile states: 106. Projects in fragile states must anticipate a difficult authorizing environment and provide adequate flexibility to maneuver. The project was heavily impacted by political economy factors, especially in initial years during which high level and sustained government leadership was not sufficient for a national program of CBR’s magnitude. Before restructuring, a presidential decree halted recruitment for nearly a year. Moreover, there were additional issues with lack of sustained line ministry commitment – accentuated by the turnover of ministers – resulting in approved CBR positions not being taken up. CBR was restructured thrice in the project lifetime. 107. Leadership plays a key role in instigating the reform process. After the change in the Presidency—strong support for the CBR by the President’s Office contributed significantly to expedited recruitment and accelerated implementation. The country’s priority was to hire competent and highly skilled officials to implement public policy. The appointment of the current IARCSC chairman in 2017 enhanced collaboration with LMAs, improved IARCSC’s performance and strengthened project management. This period following new leadership was the most productive in meeting recruitment targets and quality. The project processed an additional 725 recruitments—a 121 percent improvement within a 21-month period. 108. Projects must consider long term (and evolving) nature of this kind of engagement. Strengthening institutions requires a long-term strategy. Initially, CBR was designed to address immediate capacity needs to support Afghanistan’s transition. However, while some quick capacity enhancements can be secured when well targeted and politically owned, institutional strengthening is an iterative and Page 37 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) sequential process. In this sense, CBR should not be considered in isolation but rather as part of a broad series of incremental interventions which in fragile contexts, need to be adaptive and flexible to fluid circumstances. The project built upon some of the gains of predecessor projects such as the MCP and similarly, has provided key lessons for the follow-on TAGHIR Project. 109. A ‘de-projectized’ approach is required to strengthen sustainability. CBR and its predecessor projects fully recognized the sustainability challenges associated with external financing of human resource capacity injection. However, given the context at the time of their design, these projects prioritized support to immediate priorities in the government development agenda critical to short-term security and political transitions, the need for which outweighed longer-term sustainability factors. In doing so, these projects implicitly envisaged continued external financing beyond completion. While this materialized in the past, today, the overall aid environment has significantly shifted. Going forward, there is a need to transition away from external financing of capacity injection. This transition is necessary not only to address the unsustainable competition for public sector skills, but also to build ownership and accountability within key LMAs. 110. Upfront and sustained Government and donor commitment is critical to reform success. A critical assumption in CBR’s design was that donors would synergize their capacity building support to LMAs with CBR. This was an important prerequisite towards ensuring the utility of the CBR approach and critical to enhancing both the fiscal and institutional sustainability of capacity building reforms. In practice, this assumption did not hold, despite Government and donor partners prioritization of the CBR approach and the extensive joint work carried out with partners during project design. LMAs continued to request the easier option of NTAs and donors continued to finance this in parallel. This contributed to lower outcomes during early project implementation but improved as incentives began to align due to a shifting aid environment (providing the platform for a harmonized NTA Scale and for some scale-back in NTA numbers). There remain important steps on this agenda of supporting a more fiscally sustainable meritocratic civil service corps for Afghanistan. Government has made foundational strides through initiating the next generation of pay and professionalization reforms. The success of these reforms will strongly depend on early and sustained mutual commitment to implementation. Lessons for project design in fragile states: 111. Projects should factor in demand and realism while designing reforms. CBR results demonstrated that a whole of government focus turned out to be unrealistic. The Tier system tried to focus specific types of support, but it was not effective. The idea to intensify support to priority LMAs as they worked their way up a tier system resulted in some LMAs either lacking the demand, interest or ability to work their way up the tiers to access the support. Even after restructuring, the program was unable to concentrate enough positions to have a measurable impact. CBR recruits became spread across 47 LMAs in senior and mid-level management and technical grades, but this was not adequate to strengthen LMA’s capacity or to create significant ownership. A main reason for low ownership from LMAs emanates from lower concentration of the CBR appointees in the LMAs. Moving forward a focus on recruiting more positions within a select number of priority LMAs would have a better chance of creating more ownership and demonstrating tangible improvements. 112. Reforms in fragile states need to be narrowly targeted, measurable, and minimize risk. Initially, CBR was extremely complex for the level of capacity Afghanistan had at the time. CBR supported LMAs in producing broad reform plans that included significant structural changes. These plans proved to be over- Page 38 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ambitious and most were not implemented—LMA participation and commitment to reforms was inconsistent. Additionally, CBR had too many risk factors that delayed progress. CBR identified the risks and proposed mitigation measures, but the risks were too numerous for the mitigation measures to overcome them. Following restructuring, CBR became more narrowly focused on strategic interventions that supported delivery of key government priorities where buy-in and ownership was clear. Additionally, a lack of built-in performance benchmarks constrained the project’s ability to hold LMAs accountable for realizing their commitments to reforms. This further impaired the project’s ability to assess LMAs’ contribution towards broader public administration reforms as well as to LMA-specific reforms captured in CBRPs. Linked to this, the project did not include metrics within annual performance appraisal to measure the contribution of CBR appointees to key LMA results. In addition to narrowing scope of focus across ministries, the impact of capacity building can be enhanced through intra-ministry prioritization. Targeting capacity improvements towards desired ‘pockets of effectiveness’ or key functional areas aligned with higher level government or ministerial objectives would additionally serve the purpose of strengthening the reform orientation of support and improving measurability of capacity support impacts on Government outcomes. Lessons for project implementation in fragile states: 113. Sequencing matters to implementation trajectory. The success of public administration reforms is subject to the authorizing but equally the enabling regulatory environment. Civil service and administrative reforms are also intrinsically prone to the prevalent ecosystem of political economy. Owing to the various oversight actors with often overlapping functions and the top-down application of reforms, public administration reforms are highly susceptible to contestation. The capacity of responsible public sector entities to conceptualize and lead implementation of reforms is another constraint, especially in fragile settings. CBR implementation, especially in earlier years, suffered from the absence of this enabling environment. For instance, issues over recruitment quality derived largely from underdeveloped systems that were susceptible to manipulation; meanwhile transfer of recruitment responsibility to LMAs took place before requisite HR capacity within these entities could be built. Innovative measures to increase female recruitment could not be implemented until constraining clauses were removed from the Civil Servants Law. NTA rationalization efforts were staggered by lack of regulation, harmonization and coordination across financing partners. Training was impeded by the ACSI’s own need for capacitation. These examples highlight that on top of a longer time horizon for reforms to mature, sequencing of implementation is important. Reforms should be incrementally focused; helping to gradually build momentum. In turn, incentive structures, such as those provided by policy-based financing instruments, can provide useful vehicles to further coordinate built-up momentum. 114. Strong implementing partners with adequate authority to execute largely determines implementation success. The dual implementation structure with MoF and IARCSC affected implementation. This was mainly due to: lack of dedicated support at MoF; limited capacity accompanied with an initial lack of leadership at IARCSC; and lack of coordination between the two PSUs. 115. Third-party verification is critical in ensuring the transparency and quality of recruitment, especially in fragile settings. As noted by IARCSC within Annex 5, in such settings third-party agencies can provide a valuable shield against political intervention by carrying out robust and independent ex-ante quality assurance of recruitment processes. In tandem to meeting this capacity gap, these agencies could be channeled to help build up longer-term core Government systems and capacity for transparency and recruitment verification. Although the IARCSC hired an HR firm through CBR, the firm adopted a largely Page 39 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) check-list (non-robust) approach to monitoring compliance while time-based contracting of the firm and a one-year hiring freeze reduced the HR firm’s value for money. The HR firm was also often sidetracked from its main quality assurance role as it continually sought an expanded role beyond its core terms of reference, including helping to design training and performance appraisal strategies and systems. . Page 40 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve the capacity and performance of priority line ministries and independent agencies Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Priority line ministries and Percentage 30.00 60.00 74.60 independent agencies development budget 21-Jan-2012 21-Jan-2012 31-Dec-2018 execution improves to at least 60% Comments (achievements against targets): The indicator was fully achieved (exceeded). Priority LMAs reached an average budget execution rate of 74.6 percent, exceeding the CBR target of 60 percent. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proposed: Priority line Number 0.00 13.00 9.00 Page 41 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ministries and independent 21-Jan-2012 22-Apr-2016 31-Dec-2018 agencies that have implemented at least 2 business process simplifications planned in their CBRP Comments (achievements against targets): The indicator was partially achieved. In total, 14 CBR LMAs implemented at least 2 business process simplifications planned in their CBRP, including 9 of 13 priority LMAs. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Civil service positions Number 0.00 2400.00 1500.00 1053.00 recruited by CBR 21-Jan-2012 21-Jan-2012 22-Apr-2016 31-Dec-2018 Civil service positions Number 0.00 450.00 76.00 recruited by CBR filled by women 21-Jan-2012 22-Apr-2016 31-Dec-2018 Comments (achievements against targets): The indicator was partially achieved. In total, 1,053 civil service positions were recruited by CBR, against an overall target of 1,500 positions. 76 (7.2 percent) of the 1,053 positions recruited were held by women, against an overall target of 450 (30 percent). Page 42 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) A.2 Intermediate Results Indicators Component: Technical Assistance Facility in Support of Civil Service Reform Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Priority line ministries and Number 0.00 13.00 11.00 independent agencies which have CBRPs 21-Jan-2012 22-Apr-2016 31-Dec-2018 Comments (achievements against targets): The indicator was partially achieved. In total, 22 CBR LMAs met or exceeded this target, including 11 of 13 priority LMAs i.e. 11 non-priority LMAs also developed streamlined CBRPs. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of ministries which Number 8.00 23.00 22.00 23.00 have completed pay and grading. 21-Jan-2012 22-Apr-2016 22-Apr-2019 31-Dec-2018 Comments (achievements against targets): The indicator was fully achieved (exceeded). 23 LMAs completed pay and grading with CBR support, against an overall target of 22. Page 43 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Component: Building Human Resources Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Civil servants trained through Number 0.00 200.00 200.00 Mid-level Management Development Program 21-Jan-2012 22-Apr-2016 31-Dec-2018 (MMD) Comments (achievements against targets): The indicator was fully achieved. 200 civil servants were trained through the Mid-level Management Development Program (MMD), including 90 women. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Grievance redress Yes/No N Y Y mechanism implemented 21-Jan-2012 22-Apr-2016 31-Dec-2018 Comments (achievements against targets): The indicator has been fully achieved. The project established and maintained a functioning grievance redress mechanism. Page 44 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Civil servants in priority Percentage 0.00 20.00 15.00 43.87 ministries recruited for provinces 21-Jan-2012 21-Jan-2012 22-Apr-2016 31-Dec-2018 Comments (achievements against targets): The indicator was fully achieved (exceeded). 462 (43.87 percent) of the 1,053 civil servant positions recruited by CBR were for provinces, against an overall target of 15 percent. Page 45 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) A. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: To improve the capacity and performance of priority line ministries and independent agencies in selected reform areas. 1. Priority line ministries and independent agencies development budget execution improves to at least 60% 2. Priority line ministries and independent agencies that have implemented at least Outcome Indicators 2 business process simplifications planned in their CBRP Civil service positions recruited by CBR 3. Civil service positions recruited by CBR filled by women 1. Number of priority line ministries and independent agencies which have CBRPs 2. Number of civil servants trained through Mid-level Management Development Program (MMD) Intermediate Results Indicators 3. Grievance redress mechanism implemented 4. Civil servants in priority ministries recruited for provinces 5. Number of ministries which have completed pay and grading 1. Improved accountability of civil service Key Outputs by Component 2. Improved public spending via budget execution and procurement (linked to the achievement of the Objective/Outcome 1) 3. Elimination/reduction of parallel structures Objective/Outcome 2: Not Applicable Outcome Indicators Not Applicable Intermediate Results Indicators Not Applicable Key Outputs by Component Not Applicable (linked to the achievement of the Objective/Outcome 2) Page 46 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Yousif Mubarak Elmahdi, Atiqullah Ahmadzai Task Team Leader(s) Aimal Sherzad Procurement Specialist(s) Akbar Ali Mohammadi Financial Management Specialist Shankar Narayanan Social Specialist Rahimullah Wardak Team Member Mohammad Arif Rasuli Environmental Specialist Mohammad Asif Qurishi Team Member Syed Waseem Abbas Kazmi Team Member Pragya Shrestha Team Member Ahmad Rafi Otofat Team Member Bernard James Haven Team Member Kirk David Schmidt Team Member A. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY11 10.627 100,901.85 FY12 4.150 74,778.84 Total 14.78 175,680.69 Supervision/ICR Page 47 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) FY12 24.700 166,035.84 FY13 72.746 458,000.08 FY14 91.697 450,266.76 FY15 77.657 357,209.38 FY16 41.626 151,689.93 FY17 44.760 186,147.11 FY18 29.495 116,703.15 FY19 20.433 101,586.88 Total 403.11 1,987,639.13 Page 48 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Closing Percentage of Components (US$, millions) (US$, millions) Approval 1. Technical Assistance Facility in 30 6.2 21 Support of Civil Service Reform 2. Building Human Resources 280 53.4 19 3. Civil Service Training 20 0.6 3 4. Project Management, Monitoring 20 9.8 49 and Evaluation 5. Remaining Bank Balance 0 7.0 N/A Total 350 77.00 22 Page 49 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 4. EFFICIENCY ANALYSIS 1. The nature of CBR’s activities does not lend itself to standard economic or financial analyses to determine value for money. Therefore, the efficiency analysis is based on implementation efficiency. Implementation efficiency involves actual versus planned costs, the extent to which key activities were implemented on time, the extent of staff retention, efficacy of human resource capacity injection, the existence and amount of procurement issues and delays, and cost overruns, among others. 2. The fiscal scenarios that informed CBR were drawn from the joint World Bank/GoIRA Transition in Afghanistan: Looking Beyond 2014 report. This report projected a financing gap of around 25 percent of GDP in 2021-22 given a set of expenditure and domestic revenue assumptions. As part of project design, the civil service wage bill (including CBR) was costed against these growth and revenue projections, taking account of security, operations and maintenance (O&M) and other recurrent expenditures created through the external budget and that – working on a scenario of a gradual decline in aid – were expected to transfer progressively to the core government budget by 2021. Underlying this was the need – within an extraordinary country setting characterized by a shortage of suitably qualified Afghan professionals on the supply side and competing demands by international agencies, donors and NGOs operating in Afghanistan – for improved Government capacity to be able to absorb, manage and effectively utilize increasing amounts of donor funding provided through the Afghan budget, rather than funded and implemented externally. 116. Based on this analysis, the key cost-benefit conclusion was that human resource capacity injection– the project’s largest expenditure outlay – represented an urgent development priority. While this would increase the Government’s overall wage bill, the increase was considered manageable. This was due to the modest scale of project-financed human resource capacity injection – up to 2,400 positions. The targeting of these positions was clearly defined and limited to SMGs in Grades 1 and 2, common function positions across four defined technical areas, and to specific non-managerial technical positions in mid-level grades (Grades 3-5) only in Tier 1 ministries. This was intended to help the project target key ‘agent of change’ functions but to also contain the risk of pressure to increase the wage bill across Government. It was also assumed that donors would continue to support the GoIRA’s wage bill throughout the lifetime of the project and probably for several years beyond. 117. When viewed in the country context, the costs associated with maintaining the status quo – at the time of project design – far exceeded the fiscal costs of the project. Project human resource capacity injection was considered to represent the least-cost approach (in the absence of the ability of regular P&G to attract and retain higher caliber staff in key positions) to developing capacity in selected line ministries, relative to the prevailing heavy reliance on international and local consultants for project/program management. CBR monthly salaries – ranging from 25,000-320,000 AFN (equivalent to around US$300- US$4,000 based on today’s rates) – represented a portion of the cost donors were spending on technical assistance, estimated to cost into the billions at the time of project design. Ahead of adoption of the harmonized NTA Scale, some donor-funded NTA salaries could reach up to US$20,000 per month. Today, the NTA approach consisting of roughly 20,000 on-budget NTAs is estimated to cost US$212 million21 per year i.e. representing more than a quarter of the annual cost of the entire civilian core civil service (US$800 million) of over 400,000 employees. CBR was therefore additionally expected to create considerable net 21 Afghanistan Financial Management Information System (AFMIS) data, 2017. Page 50 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) savings to the GoIRA’s largely donor-funded development budget, should it succeed in contributing to better management and rationalization of NTA numbers (and salaries). Given that the higher-paid civil servants positions financed under the project are part of the core civil service structure, this form of human resource capacity injection avoided – and to an extent redressed – the accountability, policy ownership and institutional sustainability challenges associated with NTAs and other parallel technical assistance. 118. Project support was invariably spread too thin. By reducing the number of priority ministries in line with key agreed criteria22, the first project restructuring in 2016 attempted to improve the focus and cost-benefit of investments in human resource capacity injection. By limiting the spread horizontally, across 13 priority LMAs, each LMA would have received an average of 115 new positions; thereby maximizing scale of impact towards targeted CBRP results. In practice, this prioritization did not hold due to continuous higher-level political pressure (exerted upon the Project Steering Committee). While flexibility in recruitment allocation allowed the project to meet many important ad hoc needs across Government, it ultimately contributed to a thin layer of capacity being spread across 47 separate LMAs (and across senior and middle-level management, and technical grades). For example, in the Ministry of Higher Education (MoHE), a very large ministry with a total workforce of 12,073, CBR only provided 10 staff (around 0.08 percent). There are many other smaller LMAs in which CBR has recruited less than five (often just one) positions. 119. Ultimately, project restructuring, through a narrowing of CBR’s scope, did have a positive impact (relative to original design) on the overall value for money of the project. Restructuring entailed a reduction in project funding; partly from dropping the dedicated “Civil Service Training” Component (Component 3). Instead, training activities were mainstreamed across relevant components. LMA training was integrated into their CBRPs under Component 1 “Technical Assistance Facility in Support of Civil Service Reform”, while training in support of HR-related outcomes, led by IARCSC, ACSI or the HR Firm, including delivery of the MMD Program, was planned and channeled under Component 2 “Building Human Resources”. This rearrangement facilitated stronger linkage between LMA training and results, by ensuring that training needs were upfront identified in relation to achievement of CBRP outcomes. 120. The reduction of the CBR recruitment target from 2,400 to 1,500 positions was the other key aspect of the restructuring that drove down the overall cost of the project . This rationalization was somewhat necessitated by the slow recruitment progress and relative short time remaining till project closure, at the time of restructuring. It more so reflected a renewed emphasis on fiscal sustainability owing to an external aid environment that had markedly altered from the time of original project design when a continuation of donor funding support of the government wage bill had been anticipated to hold throughout and potentially beyond the project lifetime. Determination of the new recruitment ceiling of 1,500 was based on a weighting by GoIRA and the Bank of realistic and affordable capacity enhancement requirements. 22LMAs were prioritized based on: (a) their contribution to service delivery; (b) their potential for revenue generation, and employment generation; (c) their share of the development budget; and (d) whether they had a high NTA ratio. Page 51 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Thank you very much for giving us the opportunity to comment on your draft Implementation Completion and Results Report for CBR and share with you our side of the story. It is important to document a few key lessons and facts about the design and implementation challenges of the Capacity Building for Results (CBR) Facility in your Implementation Completion Report. As its name indicates, CBR was designed as a facility rather than a typical project. Going back in time, it should be acknowledged the key issues CBR intended to tackle and the context in which it was going to be implemented: huge reliance on a costly and unstainable technical assistance at a time that the international community was drafting plans for security and economic transition. 2010/2011 were the years when the international assistance reached its peak and it was predicted that it will gradually but substantially decline over the coming years. A large part of this aid was technical assistance including capacity building and institutional reforms of the state institutions. Some reports suggest that the capacity building and institutional reform projects in 2010 had surpassed the one billion United States Dollar figure. In such a context the design of a facility like CBR, where donors could provide pool funding to support the capacity building and institutional reforms through a single-window rather than having overlaps with each other with uncoordinated redundancies, makes perfect sense. Some may question the rationale behind why having both the Independent Administrative Reform and Civil Service Commission (IARCSC) and the Ministry of Finance (MoF) as joint implementing partners. It is worth noting that while IARCSC had the mandate to lead administrative reforms, it did not enjoy the same level of trust back then amongst its international partners and clients and did not have the capacity to implement such a big program alone. It was therefore decided to make MoF as an equal implementing partner responsible for financial management and oversight of the project. This arrangement was changed when the Government succeeded in appointing a new team to lead IARCSC and the civil service reforms’ agenda. While the overall architecture of the CBR design was sound, the ambitious targets set under the Facility and a large allocation for training show that both the Government and the Bank may have back then overestimated the absorptive capacity in the Government, accepted the donors' commitment in good faith that they will phase out their bilateral capacity building and reforms related projects and programs; and underestimated the political economy of the merit-based recruitment in a country that was emerging from decades of long conflict and still suffering from an active insurgency that was spreading terror all around and governed under complex political consensus. That is probably why it took off quite slowly and could not disburse much in its initial years of implementation. Some of the tracking data show that it took between 6 to 18 months to recruit a civil servant under CBR. Such a lengthy recruitment process had made it harder to bring on board the targeted number into the civil service and to create a substantial mass of CBR appointees within the system that one could expect to have a useful impact. The data shows that after the new leadership took over and the Project was restructured, this recruitment process was substantially shortened and by the end of the Project, the overall recruitment reached over 1,250 from around 170 baseline. We agree that three assumptions that were made during the design did not fully hold. The assumption that there will be substantial demand in the line ministries for ministry-level structural reforms and Page 52 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) therefore going with a demand-driven approach; the assumption that the bilateral donors and multilateral agencies will predominantly phase-out their capacity building and institutional reforms projects and therefore having a large project and single window of support; and the assumption that a detailed and comprehensive ministry level reform proposal will enable all stakeholders to align their support and resources around it, therefore, having a very complicated process for ministry-level CBR proposals. It is also worth noting that prior to the first restructuring of CBR a very complex tiered approach and complicated line ministries’ proposals approval process had limited the entry of line ministries to CBR and after the restructuring a more flexible approach of entry to the program led CBR to lose the focus and ended up having CBR appointees in over 43 entities. The Mid-level Management Development (MMD) program a 14 months long training financed by CBR where two batches of 100 civil servants at Grade 3 and 4 in each batch, half of them women, were targeted showed good results and potential for expansion. This also indicates that training programs may only work if there is a clear strategy and clarity on the expected outcome. In absence of an overall strategy and clarity on the division of responsibilities within the Government entities on who can and may deliver what type of capacity building programs; a training program for overall civil service may not work. It is also worth noting that the independent firm that was hired for the project was costly and did not add much value in terms of quality assurance. However, they did play a good buffer role and helped the IARCSC to resist political pressure and political interventions from external political actors. We learnt from this experience that if a firm is getting hired for such a quality assurance role, it should focus mainly on recruitment quality rather than having expanded role such as designing training strategy or establishing performance management systems and so forth and so on. The new leadership of the IARCSC proved in practice that if there is strong leadership, a genuine commitment for thought-through reforms and ownership in Government projects like CBR can be a very useful vehicle for injecting capacity into the civil service in very turbulent and difficult times. However, it requires the same level of effort and commitment to have a clear strategy for an exit as programs of this nature should be transitional and targeted. Page 53 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) Emergency Project Paper, Capacity Building for Results Facility, December 6, 2011 Project Appraisal Document Tackling Afghanistan’s Government HRM And Institutional Reforms, November 27, 2018 Restructuring Paper, Capacity Building for Results Facility, April 22, 2016 Restructuring Paper, Capacity Building for Results Facility, June 20, 2017 Restructuring Paper, Capacity Building for Results Facility, June 8, 2018 Aide-Memoires, Capacity Building for Results Facility, December, 2012 – September, 2018 Implementation Status and Results Reports, Capacity Building for Results Facility, June, 2012 – September, 2018 FY2017–20, the World Bank Group Afghanistan Country Partnership Framework (Report No. 108727-AF, October 2, 2016) The Afghanistan Independent Administrative Reform and Civil Service Commission (IARCSC) Functional review, December 5, 2017 Page 54 of 55 The World Bank Afghanistan Capacity Building for Results Facility (CBR) (P123845) ANNEX 7. SPLIT RATING CALCULATION Before After Restructuring Restructuring Relevance of Objective (at closing) Substantial Efficacy (PDO) Negligible Modest Efficiency (at closing) Modest Moderately 1 Outcome Ratings (ICR Guidelines, Appendix H) Unsatisfactory Unsatisfactory 2 Numerical Value of the outcome ratings* 2 3 3 Disbursement^ $21 Million $57 Million 4 Share of Disbursement 27% 73% 5 Weighted value of outcome rating (Row 2 x Row 4) 1.18 1.23 Moderately Unsatisfactory 6 Final Outcome Rating (1.54 + 2.19= 2.73 rounding to 3) *Highly Unsatisfactory (1); Unsatisfactory (2); Moderately Unsatisfactory (3); Moderately Satisfactory (4); Satisfactory (5); Highly Satisfactory (6) ^25 million was initially disbursed in year one of the project but was unspent by GoIRA until after project restructuring. Page 55 of 55