Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004724 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-51620, TF-13592, TF-99878, TF-A5193) ON A CREDIT IN THE AMOUNT OF SDR 397.8 MILLION (US$600 MILLION EQUIVALENT) ON A RECIPIENT EXECUTED TRUST FUND IN THE AMOUNT OF US$616 MILLION ON BANK-EXECUTED TRUST FUNDS IN THE AMOUNT OF US$32 MILLION TO THE FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA FOR THE PROMOTING BASIC SERVICES PHASE III (PBS3) PROJECT June 13, 2019 Social Protection & Jobs Global Practice Africa Region CURRENCY EQUIVALENTS Exchange Rate Effective Apr 24, 2019 Currency Unit = Ethiopian Birr 28.65 Ethiopia Birr = US$1 US$1.38 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Hafez M. H. Ghanem Country Director: Carolyn Turk Senior Global Practice Director: Michal J. Rutkowski Practice Manager: Robert S. Chase Task Team Leader(s): Louise Victoria Monchuk, Yoseph Abdissa Deressa ICR Main Contributor: Julia Van Domelen ABBREVIATIONS AND ACRONYMS AfDB African Development Bank BoFED BoFED Bureau of Finance and Economic Development (Regional) COPCD Channel One Programs Coordinating Directorate CPS Country Partnership strategy CSA Central Statistical Agency CSO Civil Society Organizations Das Development agents (local extension workers) DFID UK Department for International Development DHS Demographic and Health Surveys DP Development Partners DO Development objective EDQAF Ethiopia Data Quality Assessment Framework EMCP Expenditure Management and Control Program ESMF Environmental and Social Management Framework EU European Union FM Financial Management FTA Financial Transparency and Accountability GDP Gross Domestic Product GoE Government of Ethiopia GRM Grievance Redress Mechanisms GTP Growth and Transformation Plan GTP II Second Growth and Transformation Plan HEW Health Extension Workers ICR Implementation Completion Report IDA International Development Association IFR Interim Unaudited Financial Report IGFT Intergovernmental Fiscal Transfer IMF International Monetary Fund IP Implementation Progress IPF Investment project financing ISR Implementation Status & Results Report JRIS Joint Review and Implementation Support MDG Millennium Development Goals M&E Monitoring and Evaluation MIS Management Information System MOA Ministry of Agriculture MoEFCC Ministry of Environment, Forest and Climate Change MoF Ministry of Finance MOWE Ministry of Water and Energy OFAG Office of the Federal Auditor General ORAG Office of the Regional Auditor General PforR Program for Results PAD Project Appraisal Document PBS Promoting Basic Services Program (formerly, Protection of Basic Services) PDO Project Development Objectives PEFA Public Expenditure and Financial Accountability PFM Public Financial Management PPA Procurement and Property Administration Agency REF Results Enhancement Fund SAIP Social Accountability Implementing Partners SDR Special Drawing Rights SIL Specific Investment Loan TA Technical Assistance VfM Value for Money WASH Water, Sanitation and Hygiene WMS Welfare Monitoring Survey WoFED Woreda Office of Finance and Economic Development TABLE OF CONTENTS DATA SHEET ...........................................................................................................................1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 11 II. OUTCOME .................................................................................................................... 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 19 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 20 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 22 A. KEY FACTORS DURING PREPARATION ................................................................................... 22 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 23 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 24 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25 C. BANK PERFORMANCE ........................................................................................................... 26 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 49 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 51 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 53 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 62 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 66 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Ethiopia Promoting Basic Services Program Phase III P128891 Project Country Financing Instrument Ethiopia Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Related Projects Relationship Project Approval Product Line Supplement P129534-Ethiopia 06-Jun-2011 Recipient Executed Activities Protection of Basic Services Social Accountability Program Organizations Borrower Implementing Agency Federal Democratic Republic of Ethiopia Ministry of Finance Project Development Objective (PDO) Original PDO PBS 3 will contribute to the higher-level objective of expanding access and improving the quality of basic services by fundingblockgrants that ensure adequate staffing and operations, and by strengthening the capacity, transparency, accountability andfinancialmanagement of sub-national governments. The program defines basic services as education, health, agriculture, watersupply and sanitation and rural roads. Page 1 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing P128891 TF-99878 29,433,000 29,433,000 29,432,936 P128891 IDA-51620 600,000,000 600,000,000 593,374,798 P128891 TF-13952 615,743,000 453,640,000 453,456,021 P128891 TF-A5193 2,945,957 2,945,957 2,526,007 Total 1,248,121,957 1,086,018,957 1,078,789,762 Non-World Bank Financing 0 0 0 Borrower/Recipient 3,170,800,000 3,170,800,000 9,766,400,000 African Development Bank 250,800,000 250,800,000 253,800,000 UK: British Department for International Development 778,770,000 778,770,000 446,500,000 (DFID) EC: European Commission 66,350,000 66,350,000 37,400,000 ITALY: Dev. Coop. 12,270,000 12,270,000 9,100,000 Department (MOFA) AUSTRIA, Govt. of: Fed Chancellery - Dir Gen Dev 8,100,000 9,500,000 10,600,000 Coop. Total 4,287,090,000 4,288,490,000 10,523,800,000 Total Project Cost 5,535,211,957 5,374,508,957 11,602,589,762 KEY DATES Project Approval Effectiveness MTR Review Original Closing Actual Closing P128891 25-Sep-2012 27-Apr-2012 17-Nov-2014 07-Jan-2018 07-Jan-2019 Page 2 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 19-Jul-2013 289.57 Change in Loan Closing Date(s) Reallocation between Disbursement Categories 13-Dec-2013 292.64 Change in Results Framework Other Change(s) 12-Dec-2014 791.59 Change in Loan Closing Date(s) Reallocation between Disbursement Categories 23-Jul-2015 999.06 Reallocation between Disbursement Categories 23-Feb-2018 1060.16 Reallocation between Disbursement Categories 04-Oct-2018 1070.64 Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-May-2013 Satisfactory Satisfactory 184.34 08 05-Dec-2013 Satisfactory Satisfactory 292.64 09 03-Mar-2014 Satisfactory Satisfactory 489.14 10 01-Sep-2014 Satisfactory Satisfactory 696.10 11 03-Mar-2015 Satisfactory Satisfactory 881.65 12 09-Oct-2015 Satisfactory Satisfactory 1021.24 13 02-Jun-2016 Satisfactory Satisfactory 1034.62 14 07-Dec-2016 Satisfactory Satisfactory 1043.55 15 07-Apr-2017 Satisfactory Moderately Satisfactory 1045.22 16 31-Aug-2017 Satisfactory Moderately Satisfactory 1048.85 17 01-Mar-2018 Satisfactory Moderately Satisfactory 1060.16 Page 3 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) 18 24-Jul-2018 Satisfactory Moderately Satisfactory 1060.88 19 07-Jan-2019 Satisfactory Moderately Satisfactory 1079.06 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 21 Other Agriculture, Fishing and Forestry 21 Education 53 Primary Education 53 Health 15 Health 15 Transportation 5 Other Transportation 5 Water, Sanitation and Waste Management 6 Other Water Supply, Sanitation and Waste 6 Management Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 4 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Public Sector Management 40 Public Finance Management 10 Public Expenditure Management 10 Public Administration 30 Transparency, Accountability and Good 10 Governance Municipal Institution Building 20 Social Development and Protection 50 Social Inclusion 20 Participation and Civic Engagement 20 Social Protection 30 Social Safety Nets 30 Urban and Rural Development 10 Rural Development 10 Rural Infrastructure and service delivery 10 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Guangzhe Chen Carolyn Turk Senior Global Practice Director: Ritva S. Reinikka Michal J. Rutkowski Practice Manager: Lynne D. Sherburne-Benz Robert S. Chase Louise Victoria Monchuk, Task Team Leader(s): Qaiser M. Khan Yoseph Abdissa Deressa ICR Contributing Author: Julia Van Domelen Page 5 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Ethiopia, the second-most populous country in Africa, had created significant momentum in achieving development results in the five years prior to the third phase of the Promoting Basic Services (PBS) program.1 The country had moved faster toward reaching the Millennium Development Goals (MDGs) than all but two other developing countries.2 A strong focus on decentralized basic service delivery, efforts to maintain equity, and rapid economic growth had led to overall improvements in welfare. Economic growth averaged over 10 percent per annum, twice the average for Sub-Saharan Africa, resulting in a significant reduction in poverty. While 39 percent of Ethiopians lived in extreme poverty in 2004/05, five years later this was 30 percent, as measured by the national poverty line of less than US$0.6 per day. This was matched with impressive gains in human development outcomes, including a drop in child mortality from 123 per thousand in 2005 to 88 in 2010, increased vaccination coverage to 85 percent, and a doubling of contraceptive use. An increase in net primary school enrollment from 68 to 82 percent was supported by improvements in the gender parity index in primary school reaching 90 percent. Two-thirds of the rural population had access to potable water, and the average travel time taken to all-weather roads had progressively declined to 2.9 hours. These improvements were built on a strategy of decentralized service delivery based on Ethiopia’s Federal structure, a central element in its overall development trajectory. The 1995 Federal Constitution affirmed the roles and functions of federal vs. regional government. The federal government retained authority over a broad range of functions and responsibilities (e.g., fiscal and monetary policy, international trade, standards), and the regions and woredas (districts) were given responsibility for ensuring basic service delivery. In 2002, decentralization was extended to the woreda level, with more than 1000 woreda administrations expected to take on the bulk of service delivery responsibilities.3 Woredas receive block grants from their respective regional governments which determine distribution formulas using broadly similar methodologies to the federal grant. While Ethiopia's federal system is complex, its fiduciary and economic governance system is robust. Overall, the Public Expenditure and Financial Accountability ratings for Ethiopia placed it in the top tier of countries in Sub-Saharan Africa during the appraisal period. Despite the overall gains made during PBS 1 and 2, significant challenges remained in expanding access to and quality of local services at the time of appraisal of PBS 3, including: • Education: Expanded access, particularly in the lower primary levels, had been bolstered by previous PBS support, including the hiring of over 100,000 additional teachers. Nonetheless, upper primary net enrollment rates were still below 50 percent and not on track to reach the overall MDG of universal primary schooling. And, quality suffered from an inadequate number of qualified teachers and a lack of textbooks and supplies. 1The first two phases were termed Protecting Basic Services, with this third phase adopting the name Promoting Basic Services, focusing on improving the foundation upon which sectoral activities can better improve the quality of decentralized services. 2 Overseas Development Institute, Millennium Development Goals Report Card: Measuring Progress Countries, Sept. 2010. 3 The total number of woredas increased from 900 at appraisal to 1,023 at closing of PBS 3. Page 6 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) • Health: An impressive expansion in service facilities in the 5 years leading up to PBS 3, including a fivefold increase in health centers and threefold increase in health posts, contributed to significant improvements in health indicators. The PBS program had contributed significantly to these results through the funding of an expansion of health extension workers. Remaining challenges included reducing maternal mortality, sustaining gains in reducing child mortality, improved child nutrition and control of communicable diseases. • Agriculture: The Government’s strategy focused on improving agricultural productivity, especially of small- holders. The PBS program had supported the hiring of 45,000 Development Agents (DAs) to provide agricultural, livestock and natural resources management extension services. Key challenges remaining were identified as professional training and retention of staff, as well as improvements in agricultural data. • Water supply and sanitation: The share of the rural population with access to potable water increased from 46 to 66 percent from 2006-2010, reinforced by an increase in PBS-supported water supply, sanitation and hygiene (WASH) staff at the local level. Access to complimentary capital inputs, choice of appropriate technology, capacity to implement and sustain the systems’ functionality and lack of a well-functioning monitoring and evaluation system were identified as sector-wide issues at the time of appraisal of PBS3. • Rural roads: The expansion of rural roads led to a decrease in the average hours to an all-weather road for rural residents from 4.5 to 3.5 in the 5 years before appraisal. PBS provided funding for the establishment of over 500 woreda road desks to plan and supervise the rural road network expansion. To sustain the increased rural road network, additional local government staff and resources for road maintenance would be needed. Ethiopia’s progress was built on the priorities set in the Government’s Growth and Transformation Plan (GTP, 2009/10-2014/15), which served to guide PBS 3 preparation. The GTP aimed for Ethiopia to “extricate itself from poverty to reach the level of a middle-income economy as of 2020-23.” This was to be achieved through strong economic growth within a stable macroeconomic framework with agriculture and industry as the main drivers; provision of quality infrastructure and social services (particularly health and education); and good governance and democratization. The GTP had a strong focus on improving access to and quality of public services, with an emphasis on infrastructure and the social and human development sectors. The PBS program was considered an essential element of the GTP and contributed in two ways: (a) directly through ensuring funding to the local level for service delivery in the five core sectors; and (b) creating an enabling environment for development and good governance through transparency, fiduciary management and citizen engagement. The PBS program was central in the World Bank’s strategy for Ethiopia. PBS 3 was one of a number of World Bank operations to assist the Government in achieving the MDGs and reducing poverty. The Country Partnership Strategy (CPS) for Ethiopia (FY13-16), approved on August 29, 2012 was developed simultaneously with the preparation of PBS 3. Specifically, PBS would contribute to Pillar One: Fostering competitiveness and employment by supporting a stable macroeconomic environment; increasing competitiveness and productivity; increasing and improving delivery of infrastructure; and enhancing regional integration through its support to local infrastructure and agricultural extension services; to Pillar Two:. Enhancing resilience and reducing vulnerabilities by improving delivery of social services and developing a comprehensive approach to social protection and risk management by increasing access to quality health and education service; and the Foundation: Good governance and state building by enhancing space for citizen participation in the development process; and strengthening public financial management, procurement, transparency and accountability. Page 7 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Theory of Change (Results Chain) Results Chain for PBS 3 -% qualified primary school teachers PDO -%citizens who are informed about woreda budget -Student-teacher ratio (grades 5-8) Indicators -% citizens who report that woreda officials have actively -Population/health extension workers sought the views of citizens to improve basic services -# agricultural extension services beneficiaries -# of WoFEDs that have effectively rolled out IBEX -% non-functioning water supply schemes -% roads in fair and good condition Expanded access to and quality of basic Strengthened capacity, transparency, social PDO accountability and financial management of sub- services by funding block grants that Outcomes ensure adequate staffing and operations national governments to deliver and report on results -# woredas that publicly post budget, expenditure and procurement information -% of audit coverage at regional and federal levels and % woredas implementing recommendations Selected -# Health extension -# woredas using Social Accountability tools; % with action plans for service intermediate workers paid by block quality improvements grants -# regions with functioning Grievance Redress Mechanisms results -# of sectors with their indicators classified as official data (through EDQAF) -# of qualified primary indicators school teachers -# regions submitting quarterly Interim Financial Reports to COPCD without -# woreda road desks major deficiencies with appropriate staff -# woredas with improved MIS systems in education, health, agriculture, roads, water/sanitation -# woredas with improved environmental and social management capacity Improved Improved Improved availability Greater citizens' capacity at transparency Strengthened of qualified staff and engagement on local level to Intermediate and fiduciary M&E systems operational resources quality of basic manage results assurance in to measure for delivery of basic services environmental the use of results services delivery and social public funds risks -Training for woreda - Data quality - PFM training at citizens on assessments carried federal and regional -Environment Increased amount of budgeting, planning out in key sectors levels and Social Basic Service Block -Disclosure of local - Support for MIS - PFM manuals and Management Selected Grants transferred public budgets systems in training in woredas Operational to regions and - Social education, roads, Outputs - woreda Audit Manual woredas in a timely Accountability agriculture, health Committees disseminated fashion according to activities rolled out and water established to woredas transparent and in woredas - Publication of Established and staff equitable criteria - GRM capacity service facility maps -Procurement trained installed -National household Page 8 of 67 professionalization data collected The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Project Development Objectives (PDOs) “PBS 3 will contribute to the higher-level objective of expanding access and improving the quality of basic services by funding block grants that ensure adequate staffing and operations, and by strengthening the capacity, transparency, accountability and financial management of government at the regional and local authorities levels. The program defines basic services as education, health, agriculture, water supply and sanitation, and rural roads.” The PDO was developed as the third in the PBS series and reflects that evolution. The PDO statement evolved from PBS 2 “to contribute to expanding access and improving the quality of basic services in education, health, agriculture, water supply and sanitation, and rural roads delivered by sub-national governments, while continuing to deepen transparency and local accountability in service delivery”. While, the PBD 3 PDO is written using two by- statements, the entire PBS series has sought to achieve a dual objective: a) expanding access and improving the quality of basic services; and b) strengthening service delivery systems. General Bank guidance on PDOs evolved during implementation that called into question the use of by/for statements in PDOs. Taking into consideration initial intent and guidance at the time, this ICR chooses to assess the project against both of the objectives in the PDO even though by-statements were used. In addition, the PDO clarifies that the project contributes to higher level national objectives of access and quality of services as a way of emphasizing that many other Government initiatives (some supported by the WB and other partners) also contribute to improving access and quality in basic services. The set of PDO indicators chosen for the project identify changes in access and quality most closely associated with the provision of funding for adequate staffing and operations, where PBS 3 could be held most accountable. PDO indicators also identified outcome expectations for improvements to create transparent and accountable systems at the local level. In addition, PBS 3 also recognized the value in adopting the Government’s key goals and indicators from the GTP to ensure PBS 3 was aligned with the GTP and supported that overall program. Key Expected Outcomes and Outcome Indicators This ICR assesses the achievement of two main objective areas expressed in the PDO above, namely: Objective 1: Expanding access and improving the quality of basic services by funding block grants that ensure adequate staffing and operations. Key expected outcomes are expanded access to and quality of education, health, water and sanitation, roads and agricultural extension services provided at the woreda level. Corresponding project development outcome indicators in the five basic service sectors are: • Education: Student-teacher ratio (grades 5-8) and percent of qualified teachers • Health: Ratio of health extension workers to population • Agriculture: Number of agricultural services beneficiaries • Roads: Percentage of roads in fair and good condition • Water: Reduced non-functioning water supply schemes by region and urban/rural Objective 2: Strengthening the capacity, transparency, accountability and financial management of government at the regional and local authorities levels. Key expected outcomes include improved transparency and fiduciary assurance in the use of public funds, improved citizens' engagement in the quality of basic service delivery, and Page 9 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) strengthened M&E systems at central, federal, and sub-national levels. Corresponding project development outcome indicators are: • Transparency: Percentage of citizens who are informed about woreda budgets • Social accountability: Percentage of citizens who report that woreda officials have actively sought the views of people in their community on improving quality of basic services • Financial management: Percentage of woredas that have effectively rolled out the Integrated Budget Expenditure Management (IBEX) system. Components PBS 3 was appraised with the following components: A. Basic Service Block Grants (Sub-program A – Estimated component cost: US$ 6.2 billion, IDA US$0.555 billion; Actual cost: US$11.03 billion; IDA US$0.511 billion). Block grants are transferred through the decentralized fiscal system for recurrent expenditures (salaries, maintenance) in the five basic service sectors (education, health, agriculture, water and sanitation, and rural roads) based on Ethiopia’s Intergovernmental Fiscal Transfer (IGFT) formula. Regions and woredas determine the allocation between sectors based on local needs. IDA and development partners (DPs) funds are combined with the Government’s own resources which are then distributed to regional and woreda governments using the IGFT system (aka ‘the Block Grants’). This approach is consistent with the 2005 Paris Declaration on Aid Effectiveness which calls for donors to align with country strategies and systems, to coordinate their actions and harmonize procedures, and to focus on results. Donors’ support to these Block Grants are based on a set of core principles: Effectiveness based on adequate sectoral resource allocation, balanced intra-sectoral allocation, and results achieved; Sustainability to ensure that financing of basic services can be sustained over the long-term; Additionality to ensure the Government’s commitment to increase their contribution to block grant funding over the medium-term; Fairness reflecting that resource allocations are rules-based and transparent and that they are actually disbursed to regions and woredas; Equity to assess discrepancies in access to basic services among the different regions and woredas and between genders; Fiduciary probity based on robust fiduciary systems; and Predictability to confirm that DPs disbursements are aligned to commitments and accurately reflected in the Government’s budget. B. Strengthening Local Accountability and Transparency Systems (Sub-Program B – Estimated component cost: US$114.6 million, IDA US$40.0 million; Actual cost: US$117 million, IDA US$66.2 million) to foster an institutionalized system for transparency and accountability that is socially inclusive and responsive to local needs. This component seeks to improve the country’s decentralized, integrated system of fiduciary, administrative and information governance, focusing on the woreda level as the frontline administration with responsibility for basic service delivery. At appraisal the component encompassed three sub-components: 1. Citizen’s Engagement (Component B.1 – Estimated cost: US$35 million, IDA US$10.0 million; Actual cost: US$43.7 million, IDA US$10.5 million) to ensure Block Grant transfers are increasingly transparent, responsive to local needs and delivered in a socially accountable manner, focusing efforts on three areas: a. Financial Transparency and Accountability (FTA) Sub-component B.1.a: to raise awareness and understanding of public budget processes, and increase disclosure of information on budgets and service delivery to the public, including utilization of mass media, budget literacy training, and posting of resource utilization and service standards. Page 10 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) b. Social Accountability Program (ESAP) Sub-component B.1.b: supports the "demand side" of citizen's engagement, providing citizens, as public services users, with the means to assess, understand and monitor the planning, delivery and quality of basic services, and voice their needs and preferences. PBS social accountability activities are financed through a programmatic multi- donor trust fund administered by the World Bank though without IDA resources. The Ethiopia Social Accountability Program phase 1 (ESAP 1) began as a small pilot (2006–2009) during the first phase of the PBS Project. ESAP 2 followed as part of PBS 2 and PBS 3 (2010-2018). c. Grievance Redress Mechanisms (GRM) Sub-component B.1.c introduced in this phase of the PBS series, supports a formal system to provide due process and administrative justice for citizens who may have complaints regarding the implementation of projects and service delivery. GRM activities focus on strengthening the Ethiopian Institution of the Ombudsman (EIO) and its regional affiliates as well as local-level public complaints hearing structures and processes by developing a common standard of grievance redress procedures, capacity training for grievance officers and a public education program to inform citizens of grievance procedures. 2. Local Public Financial Management (PFM) and Procurement (Component B2 – Estimated cost: US$52.6 million, IDA US$20.0 million; Actual cost: US$50.3 million, IDA US$38 million). PBS implementation relies on a robust, decentralized financial management system that allows public resources to flow reliably from the Federal Treasury, through regional and zonal administrations to woreda-level and frontline service providers. Specific activities include developing standardized training materials and operational PFM manuals with capacity building delivered through regional management institutes, as well as the roll-out of automated IBEX financial management software to woredas. The audit function will be strengthened at all levels. Regarding procurement, the project aimed at establishing sound procurement system and organization at woreda level including staffing, preparing working documents and enhancing transparency on procurement decisions. At national level, the project support includes establishing procurement professionals training program. 3. Managing for Results (Component B3 – Estimated cost: US$27.0 million, IDA US$10.0 million; Actual cost: US$22.5 million, IDA US$17.2 million) to ensure that data, systems, and analytical capacity are strengthened for accurately and timely reporting on results. This included support for sectoral MIS systems, data quality assessments as well as collection of national service delivery data disaggregated by gender and selected surveys and studies. 4. Results Enhancement Fund (Sub Program C – Estimated cost: DFID-administered UK£50 million equivalent to US$78.2 million; Actual Cost: 0). The Results Enhancement Fund (REF), to be funded solely by DfID, would provide incentives for innovation and performance in a yearly prize to the three sub- national administrations that made the most progress in relaxing constraints to delivering decentralized basic service results. The detailed arrangements for this component were to be finalized during PBS 3 implementation. However, this component was not implemented. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION There were six restructurings during PBS 3: July 19, 2013: Change in Closing Date; Reallocation between Disbursement Categories December 13, 2013: Change in Results Framework December 12, 2014: Change in Closing Date; Reallocation between Disbursement Categories July 23, 2015: Reallocation between Disbursement Categories February 23, 2018: Reallocation between Disbursement Categories Page 11 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) October 4, 2018: Reallocation between Disbursement Categories The restructurings that represented significant changes are described below: Revised PDOs and Outcome Targets and Revised PDO Indicators Restructuring December 13, 2013 (Report No. RES13069): An initial restructuring revised the Results Framework to reflect the availability of new and better data, including changes to certain baselines, targets and indicators in the PBS 3 Results Matrix. These changes were also to align with the Government’s revised GTP targets. The principal changes affected higher level indicators for education, health and water. PDO and intermediate results indicators were not changed but data sources and definitions were better clarified. Restructuring July 20, 2015 (Report No. 98461) amended the Results Framework in light of the new resource allocations and new activities (see discussion on revised components below). Revised Components Restructuring July 20, 2015 noted above also revised two components: a) DfID withdrew from PBS 3 in 2015. As a result, it was agreed with the Government to cancel Sub-Component C: Results Enhancement Fund, which had been a fully-funded DfID initiative. b) A new component under Sub-Program B was added: B4 Risk and Safeguard Management Capacity to support capacity building efforts targeted to developing and implementing a nationwide Environment and Social Safeguards framework for woreda governments. This was part of agreed upon actions from the Management Response to an Inspection Panel review of PBS 3, a detailed discussion of which is presented in the section below on Environmental and Social Compliance. Other Changes Restructuring December 13, 2013 noted above allowed the Ethiopian Health and Nutrition Research Institute to implement the health census and facilities surveys under Managing for Results Component B3. Restructuring July 20, 2015 noted above (a) reallocated funds under Sub-Program A and Sub-Program B in light of the withdrawal of DfID funding; (b) changed implementation arrangements to enable Government fiduciary agencies to strengthen monitoring activities; and (c) eliminated the additionality testing under Sub-Program A. Rationale for Changes and Their Implication on the Original Theory of Change The theory of change was not significantly modified as a result of these changes. An additional intermediate result indicator was added corresponding to the new sub-component B.4. Page 12 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating PBS 3 was a core part of the Government’s overall development strategy as set forth in the GTP. PBS 3’s support for basic service delivery in the core sectors of education, health, water and sanitation, roads and agricultural extension supported the GTP’s main objectives of attaining the MDGs in the social sectors and sustaining and supporting equitable growth with a focus on agricultural development and infrastructure. In addition, the GTP called for improved governance, including greater transparency and public participation at all levels of government, and improved capacity of the Government financial management system. PBS 3 provided important system-wide capacity building at the local levels for these Government priorities. PBS 3 was closely aligned with the World Bank’s strategy for Ethiopia. The project was the main vehicle for supporting improved service delivery (through block grants to basic services) within the Bank’s CPS and contributed importantly to the other focus areas of economic growth (through its support to recurrent expenditures for local roads and agricultural extension services) and good governance (through its capacity building investments in fiduciary management, transparency and accountability). The design of PBS 3 used and supported existing Government systems as opposed to parallel project structures which was consistent with the overall CPS goals. PBS 3 remained relevant to the Bank and Government strategies throughout its implementation period. The Government’s Second Growth and Transformation Plan (GTP II) issued in May 2016 specifically highlighted improved service delivery results and the advancements in financial transparency and accountability and citizen’s engagement that PBS 3 supported. The Bank’s new Country Partnership Framework (CPF) approved May 22, 2017 towards the end of PBS 3 confirmed the continued relevance, noting the exceptional impact in improved delivery of social services, as well as important outcomes in enhanced space for citizen participation, as well as enhanced public financial management, procurement, transparency, and accountability. The CPF further confirmed the Bank’s commitment to continue to leverage Ethiopia’s federal system. Considering the above, this ICR rates the Relevance of PDOs as High. Page 13 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome According to the theory of change, the PDO was composed of two areas of outcomes: Outcome 1: Expanded access to and quality of services by financing block grants that ensure adequate staffing and operations. Block Grants financed local recurrent costs in the basic service sectors (education, health, water and sanitation, roads and agricultural extension) for a total of US$11.03 billion almost twice the initial estimate of US$6.2 billion. IDA and development partners contributed US$1.2 billion (80 percent of expected amount) while the Government contributed US$9.8 billion, over three times the initial estimate, demonstrating the Government’s strong financial commitment to basic services and effective fiscal decentralization. The structure of PBS 3 was a commitment of IDA funding for the block grant component through the first 3 years of implementation, with a five-year horizon for the capacity building activities in sub-Program B. This permitted a transition to a follow-on Program for Results- based instrument for IDA funding to Block Grants in the final years of PBS 3. Total Block Grant transfers by the Government and development partners during the PBS 3 period (including the follow-on IDA ESPES project) is presented in Figure 1. Moreover, as discussed in greater detail in Section II.E Other Impacts below, Block Grants preferentially benefitted less well-off regions and woredas. The benefits of PBS-financed spending at the woreda level on health, education, and agriculture accrued to all income levels as these were pubic services. But, there was a greater concentration in services used by the poor. While, other Government initiatives including in the health and education sectors, also contributed to expanding access to and quality of services of basic services in Ethiopia, PBS 3 contributed to these objectives through its block grant contributions. Attaining these objectives required sufficient funding for staffing and operations. Sub-Program A had 6 PDO-level results indicators to measure achievement of outcomes of expanded access and improved quality (Table 1): Table 1: PDO Level Results Indicators: Block Grants that ensure adequate staffing End of Project % Area of and operations Baseline Target result achieved impact 1:2,578/ Access/ Population per health extension workers *1:2,144 1:2,500 1:2,570 97% Quality Proportion of qualified primary school Quality teachers 53% 97% 83% 86% Page 14 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Student-teacher ratio (grades 5-8) 51.0 41.0 35.0 117% Quality Agricultural extension services beneficiaries 9,400,000 14,900,000 16,975,113 114% Access Percent non-functioning water supply Access schemes 26% 8% / *10% 9% 106% Percent roads in fair and good condition 82% 88% 93% 106% Quality Source: Sectoral MIS systems.* Revised during restructuring. Restructured indicators are used to calculate % of target achieved. Table 1 shows substantial achievement of the objectives of the Block grant transfers, with 5 of the 6 indicators largely met or surpassed. By sector: a) Following Ethiopia’s significant expansion in education, the focus has been on reducing class size and increasing teacher qualifications to improve quality. The student teach ratio for grades 5-8 was reduced from 51 to 35, significantly surpassing the goal of 41. PBS 3 supported an increase in qualified primary school teachers from 142,000 to 401,000, surpassing the intermediate results target of 300,000. The proportion of qualified primary school teachers increased from 53 percent to 83 percent during PBS 3. While this fell short of the Government’s ambitious national goal of 97 percent, this should not overshadow the tremendous improvement in qualified primary school teachers in relatively short time. b) Ethiopia’s health extension workers (HEWs) per population target was substantially achieved at 97 percent of goal. There were 36,660 HEWs deployed at the end of project (December 2018) up from 30,900 at project start, resulting in a ratio of 1:2,570 population versus the goal of 1:2,500. However, the year before, in December 2017, the Ministry of Health recorded 39,878 HEWs for coverage of 1:2,314 which would have surpassed the end of project target. According to the MOH, in 2018 HEWs were being trained for level IV (in service) and may have been missed out from consideration by regions and well as HEWs that were promoted to diploma level shifting to nurses and midwifery but remained in the health service. c) The share of roads in good or fair condition increased from 82 to 93 percent, surpassing the goal of 88 percent. Block grant funding supported road maintenance supplies and labor as well as woreda level staffing in roads. The number of woreda desks with appropriate staff increased from 155 to 842, far surpassing the intermediate result target of 650. d) Smallholder agriculture forms the backbone of the Ethiopian economy, supporting about 85 percent of the country’s population. PBS 3 helped fund a large expansion in local access to agricultural extension services, with 9.0 million farmers benefitting from these services at the start of PBS 3 increasing to 17.0 million at project closing, far surpassing the goal of 14.9 million. e) The quality of local infrastructure improved with targets of functioning water supply schemes surpassed. The target was to reduce by almost two-thirds the share of water systems that were built but not functioning because of under-funded repair and maintenance issues. The goal was achieved with 91 percent of systems now fully operational. While PBS funded recurrent costs of staffing and other inputs, these achievements required complimentary capital to build classrooms or water systems for example, or sectoral inputs like preventive health supplies and textbooks, largely funded outside of the Block grants. Complimentary data also confirm overall improvements in access to and quality of services. In the health sector, the contraceptive prevalence rate (CPR) for Ethiopian women age 15-49 rose from 19.6 to 25.3 percent from 2011 to 2016. Births attended by skilled personnel increased from 10 percent to 28 percent and women receiving antenatal care almost doubled from 34 percent to 62 percent. The percentage of children Page 15 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) vaccinated (Penta 3) increased from 85 to 96 percent. Infant mortality rates dropped from 59 to 48 per 1,000 live births. Child mortality rate decreased from 88 to 67 per 1,000 live births over the same period.4 Access to improved sources of drinking water increased, from 42 to 57 percent of rural households. In 2011, 91 percent of rural households used non-improved sanitation facilities versus 56 percent in 2016. Rural water access within 1.5km radius (new standard) increased to 81 percent in 2016 from 41 percent in 2011. Access and quality in the education sector also point to positive trends during PBS 3. The total number of students in Grades 5-8 increased from 5.5 million to over 6.4 million, and the average number of textbooks per primary student increased from 1 to 3.8.5 Net primary enrolment rate in grades 5-8 increased from 47 to 57 percent, showing progress towards the national goal of 80 percent. Gender parity index grades 5-8 NER has achieved 0.98 by end of 2018. Per hectare productivity of major food corps increased from 17 to 24 quintals. Average time to the nearest all-weather road decreased from 3.5 to 1.4 hours. Outcome 2: Strengthening the capacity, transparency, accountability and financial management of government at the regional and local authorities levels Sub-Program B built durable systems to strengthen capacity, transparency, accountability and financial management around local service delivery. A total of US$116.6 million was disbursed on these activities, including funding executed directly by the World Bank through Multi-Donor Trust Funds. This was 60 percent of the estimated amount at appraisal of PBS 3, with the difference due to the cancellation of a Results Enhancement Fund anticipated to be supported by DfID. Taken together, the Sub-Program B components all increased capacity and were mutually reinforcing on the themes of transparency, accountability and financial management. The three PDO indicators for Sub-Program B were met or surpassed (Table 2). Table 2: PDO Level Results Indicators: End of Strengthened capacity, transparency, accountability and financial project % management of sub-national governments Baseline Target result achieved Percent citizens who are informed about woreda budget 19% 25% 42% 168% Percent citizens who report that woreda officials have actively sought 36% 55% 55% 100% the views of people in their kebeles on improving basic services WoFEDs that have effectively rolled out IBEX 0 900 983 109% Sources: COPCD, FTA Impact Assessment These three PDOs do not fully capture the transformative nature of these systems changes and improvements in governance. Evaluations identified additional outcomes to further support the achievement of the PDO. Citizen dissatisfaction with services provided by the woredas decreased significantly over the period of PBS 3, decreasing from 67 percent in 2013 to 11 percent in 2017.6 Sixty-nine percent of citizens voiced that their understanding of the local budget process had grown versus 29 percent in 2013. Sixty-three percent of survey respondents took part in budget meetings in the previous 12 months, versus 30 percent in 2013. Ninety percent 4 Demographic and Health Surveys (DHS) 2011, 2016. Data for water and sanitation access is also drawn from DHS. 5 Ministry of Education Statistical Abstracts. 6“Impact Assessment on Financial Transparency and Accountability (FTA) Implementation”, BDS Center for Development Research 2009, 2013 and 2017. Page 16 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) of citizens agreed that there is a GRM in place for complaints and disagreements. Budgets, audit findings and procurement templates are now routinely posted in woredas, with the most recent verification finding 94, 86 and 92 percent of compliance respectively. These outcomes were supported by intermediary results in each of the components of Sub-Program B. A full listing of outputs and intermediate results is presented in Annex 1, with highlights by component presented below. Taken together, these efforts were ambitious and transformative, reinforcing decentralized service delivery. This was done in a challenging political context and in a diverse country with over a thousand local government districts. PBS was the main basis for citizen voice and engagement as well as government transparency and accountability at the woreda level during this time period. Sub-Program B.1: Citizen’s Engagement Citizen’s Engagement aimed to ensure that block grant transfers are increasingly transparent, responsive to local needs and delivered in a socially accountable manner, focusing efforts on three areas: financial transparency and accountability, social accountability, and grievance redress mechanisms: Financial Transparency and Accountability (FTA) – supported the "supply side" of citizen's engagement, with information and communication activities from government to citizens on service standards, public budgets and expenditures. Budget literacy training was provided to citizens, council members, and sector staff reaching 2.6 million woreda citizens, of which 1.1 million were women. FTA service facility and budget data were disseminated through public posting, mass media and culturally relevant information channels. At the beginning of PBS 3, 70 percent of woredas were confirmed as having posted the refined and simplified budget and expenditure template to the public, growing to 94 percent by the end of PBS 3. Basic service units that post the standardized service delivery template increased from 25 to 62 percent. And, woredas that publicly post all procurement award decisions increased from five to 66 percent. Social Accountability Program (ESAP 2) - Designed to support the ‘demand-side’ of service delivery, PBS 3 provided citizens with the means and training to voice their service delivery needs and preferences. Piloted in PBS 1, this second phase of ESAP was implemented through 139 Civil Society Organizations (CSOs), supported by a Management Agency recruited internationally. CSOs act as intermediaries to facilitate between service providers and users of basic services using social accountability tools. Under PBS 3, all 223 participating woredas developed joint action plans for service quality improvements based on interface meetings between service users and providers. Social Accountability Committees (SACs) were established in all program woredas consisting of citizens, representatives from vulnerable groups, women’s groups and service providers. There were delays in completing the design of ESAP 3 and in finalizing donor funding agreements. An ESAP 2 Bridging Phase was extended through 2018 to consolidate and deepen activities in the 223 ESAP 2 woredas until the successor program was in place. This delayed expanding to the original target of 400 woredas foreseen for ESAP 2. However, the deepening of activities during the Bridging Phase resulted in 369,155 citizens participating in social accountability activities, exceeding by more than four times the original target. Grievance Redress Mechanisms (GRM) - PBS 3 supported capacity building, training and standardized manuals and guidelines for Ethiopian Institution of Ombudsman (EIO) staff and stakeholders at federal and branch office levels and established three new EIO branch offices. At the beginning of PBS only two regions had functioning GRM units consistent with international best practice; this grew to nine by the end of PBS 3. Media campaigns informed the public of where to go and how to access GRM mechanisms. Of the more than 32,000 complaints registered, over half have been resolved. Of note, the GRM mechanism helped lower-income citizens who Page 17 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) would not have had redress through formal court systems or did not have the resources to travel far to lodge administrative complaints. Sub-Program B.2: Local Public Financial Management and Procurement This component aimed at ensuring that Block Grant transfers were well managed from a fiduciary perspective. One of the most important accomplishments of PBS 3 was the roll out of the IBEX system in all woredas, which provided the financial management software to automate the country’s woreda-level financial administration. IBEX incorporates modules on budgeting, accounting and reporting aspects of fiduciary transactions. Further improvements include the installation of woreda net, the government internet backbone, in all woredas to support even faster and more secure data transmission, where connectivity is currently available. This was reinforced with PFM training and manuals in all woredas. This automation resulted in timelier, more accurate and transparent processing and reporting of the financial transactions around service delivery expenditures. The project also supported the regional and federal levels with Public Financial Management Institutionalized Training (PFMIT). The Office of the Federal Auditor General (OFAG) now audits all Federal level public bodies, up from 30 percent before the program and the backlog of public audit completion which was four years at the outset of PBS 3 has been cleared. At the regional level, audits by ORAG now cover 63 percent of woredas compared to 30 percent at the start of PBS 3 and 42 percent of woredas implemented prior period audit recommendations versus five percent at project start. Federal and Regional procurement regulatory bodies enhanced their oversight capacity including increased procurement audit coverage. Only five woredas had a procurement specialist at the beginning of PBS 3; 500 woredas had such staff by project completion. Federal Procurement and Property Administration systems were strengthened, and a Procurement Professionalization and Accreditation Program established, the first of its kind in Ethiopia. Sub-Program B.3: Managing for Results Managing for Results activities strengthened data, systems, and analytical capacity for accurately and timely reporting on service delivery results. The scope was ambitious, encompassing activities across many agencies. PBS 3 supported development of sectoral management information systems in education, agriculture, water and sanitation and roads. This was largely achieved, with some delays in roll-out of the WASH system and delays with the Ethiopian Road Authority MIS that resulted in its postponement to the successor project. Data quality assessments by the Central Statistics Agency were carried out at least once in all basic service sectors. A curriculum was developed and implemented for the Ministry of Education on indicators definitions and collection methods, with training delivered to all woredas. The Ministry of Agriculture developed software on productivity and impact of development agents. Four sectors had their indicators classified as official data through implementation of EDQAF, surpassing the intermediate results target of 2. National level data on service delivery aspects were collected by the Central Statistics Agency, including: development of geo-referenced service facility maps for all regions, a mini-Demographic Household Survey (DHS), completion and publication of national household income and expenditure survey (with welfare monitoring component), and support to disaggregation of data by gender at decentralized level (in all 11 regions) and publication of country report on Gender Disaggregated Development Data, among others. Sub-Program B.4: Risks and Safeguard Management Capacity Added in response to the Management action plan following an Inspection Panel case, this component focused on boosting woreda level capacity to manage environmental and social risks. The Ethiopian government expressed its commitment to mainstreaming environmental and social issues into program/project design and implementation through various proclamations and guidelines, and by establishing environmental offices at Page 18 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) different levels. Environmental and Social Impact Assessments (ESIAs) were required through Government directives and guidelines. The PBS 3 project undertook an assessment of ESIA capacity in all regions looking at the institutional arrangements, existing regulations, and staff capacity to effectively carry out this charge. Working with the Ministry of Environment, Forest and Climate Change (MoEFCC) and regional Environmental Protection agencies, PBS 3 resulted in about 200 trainers prepared to train in environment and social management. An Environmental and Social Management Operational Manual was developed and customized at regional level in all 11 regions and is currently being rolled out at the woreda level, reaching 4,768 woreda staff, far exceeding initial goals. Justification of Overall Efficacy Rating 8 of the 9 PDO indicators were met and many substantially surpassed. Beyond this narrow set of metrics, the Government has shown a strong commitment to scaling up funding through the fiscal block grant system. Complimentary evidence of improved access to and quality of services at the national level is compelling. There were some minor shortcomings, including not fully reaching the health personnel target as well as less than full execution of activities planned for under Sub-Program B, mainly in the Managing for Results sub- component and delays in launching the third phase of ESAP. Some of these had to do with overly ambitious targets, which either reflected national goals or faced implementation challenges, like procurement delays. Given the significant achievements particularly of the PDOs, this ICR rates overall efficacy as Satisfactory. C. EFFICIENCY Assessment of Efficiency and Rating Efficiency of PBS 3 was assessed along four domains (see Annex 4): a) Assessment of the overall fiscal framework: Strong economic growth translated into a significant increase in government revenues and poverty-related expenditure during the PBS 3 period. At the outset, annual Block Grant transfers were about US$1.95 billion, more than doubling to US$4.4 billion by the end of PBS 3, reflecting Government prioritizing support for local service delivery. Overall, the donors’ share of the Block Grants fell from 23 percent at the beginning of PBS 3 to 7 percent at project closing, reducing reliance on external funding.7 The core principles of Block Grant transfers were consistently met during implementation, including effectiveness, sustainability, additionality, fairness, equity, fiduciary probity, and predictability. b) Efficiency of using Government systems: By making use of government systems, PBS provides efficiency gains in the delivery of basic services. While it is difficult to provide precise estimates of the impact of PBS 3’s capacity building activities, the direction of their effect is clear: spending efficiency is improved through better capacity, more transparency, and greater accountability to citizens. c) Cost effectiveness analysis: Cost effectiveness was confirmed through a Value for Money study that estimated a benefit cost ratio for PBS 3 of 1.43 and an internal rate of return of 23.1 percent.8 A Poverty and Social Impact Analysis (PSIA) found that, for education, an increase of US$1 per capita in spending by each woreda is associated 7 These overall figures on Federal fiscal transfers include funding outside of the PBS 3 project envelope, including the successor PforR IDA project which overlapped with the last two years of PBS 3 implementation. 8 DfID/ITAD. “Value for Money Assessment of the Promotion of Basic Services (PBS) Programme, February 2015. Page 19 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) with a 3.7 percent increase in the net primary enrollment rate within that woreda.9 Every additional US$1 of per capita spending by the woredas on health is associated with a 7.5 percent increase in the contraceptive prevalence rate and an 12.4 percent increase in deliveries by skilled birth attendants (two interventions that can reduce maternal mortality dramatically), as well as a 4 percent increase in antenatal care (which can reduce infant and child mortality significantly). And, for every additional US$1 per capita spent, the probability that a field in a given zone will benefit from extension services increases by about 0.2 percent. d) Social cost-benefit analysis – poverty orientation of PBS expenditures: The PSIA assessed equity on two levels and found that: (a) the relative distribution of spending on woreda-level services is pro-poor, and (b) the geographical distribution of resources by poverty level of the region/woreda favors poorer regions. e) Project-level efficiency: Because of the scale of PBS 3 and the multi-donor configuration, administrative costs were relatively low and shared among many agencies. The project closing date was extended when additional activities were added to the capacity building program. In addition, government contribution to the Block Grant program increased significantly yet unpredictably during the project. Several of the PDO indicators were surpassed but they were not doubled, nor were the indicator targets changed. From an efficiency point of view this requires an explanation. At appraisal, the project adopted the Government’s goals of access to and quality of basic services as set forth in the GTP. The funding needed to achieve each indicator was not based on a linear input –output model so it is not expected that achievement of specific PBS 3 PDO indicators would be directly correlated with levels of Block Grant transfers, for several reasons detailed in Annex 4. Given the aspects above, the Rating for Efficiency is Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING The overall outcome rating for PBS 3 is Satisfactory. The operation was highly relevant and efficient and achieved or surpassed its Project Development Objectives. Ninety-six percent of the project’s funding was in Sub-Program A, the basic service Block Grants, where results were highly satisfactory, significant improvements in access to and quality of basic services at a national scale and notable advancements in well-being of rural areas and poorer households. Sub-Program B implemented important capacity building and system development, with notable achievements in citizen engagement and PFM. Because PBS 3 worked through existing Government systems, system-wide and durable improvements in institutional capacity through PBS 3 sought to change how the country’s institutions operated in delivering services. There were a few activities that were either not completed or completed partially. While these did not materially affect the meeting of the project’s development objectives they were intended benefits that did not fully realize, and so classified as a minor shortcoming. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender PBS 3 had significant impacts on women in several areas: Expansion of basic services positively impacting women: Improvements in the gender parity index for grades 5-8 reflect expanded education achievement for girls. Significant increases in pregnant women receiving antenatal care 9 Khan, Q., Faguet, J-P., Gaukler, C., and Mekasha, W. “Improving Basic Services for the Bottom Forty Percent”, World Bank. 2014. Page 20 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) rose from 34 percent in 2011 to 62 percent in 2016 as measured in successive DHS surveys. However, female farmers are less likely to benefit from extension service where eighty-seven percent of Ethiopia’s farmers are male and access to extension services was found to favor men. Better gender-disaggregated data: PBS 3 developed better gender-disaggregated data, including expanding gender tracking on service delivery and project outputs, like training and meeting attendance. PBS 3 supported the publication ‘Gender-Disaggregated Development Data’ by the Central Statistical Agency in 2015. Provision of local platforms and training for women’s voice to influence budgets and service delivery: Citizens Engagement activities sought to bring women into the development process. Social Accountability Committees (SACs) specifically recruited members of both genders including representatives of local women’s groups. Women’s participation in social accountability meetings varied throughout the implementation, with a high of 47 percent and an end of project level of 34 percent. Women’s participation is higher among the citizen groups (35 percent) compared to service providers (26 percent) and councilors (28 percent). Surveys found that the proportion of female respondents who had information about budget, strategic plan, agenda of their council and decisions made is 56 percent, 43 percent, 39 percent and 41 percent respectively. Access to woreda budget information by the men and women is relatively equal, with data showing that 63 percent of the male respondents received information on the source of woreda budget versus 60 percent for women. Institutional Strengthening Because of the way PBS was designed to work through existing government systems and in all woredas of the country, the institutional impact of the project was extensive: Fiduciary management. The IBEX financial management system has substantially improved the capacity of the decentralized financial transfer system. Other systemic improvements include enhanced internal controls in the system. Though the rollout has not been finalized, simplified procurement directives are enhancing procurement efficient and compliance. Oher supports include the establishment of procurement professionalization training program, improved audit coverage and procedures, and PFM modules implemented in all woredas. Increased engagement of citizen oversight and transparency of public institutions. The general scope of PBS 3’s citizen engagement is to create permanent forms of productive interface between the public, woreda officials and service providers. ESAP’s Social Accountability Committees (SACs) have taken up a more permanent presence in the institutional architecture at the local level. FTA activities of dissemination of budget and service delivery information to the public have been supported through formal FTA directives at the federal, regional, and woreda levels. FTA directives have been implemented in 50 federal budgetary institutions and 7 regions. A module on FTA is now included in standard training of woreda accountants. A government directive has been issued to guide consultations on budget formulation at the woreda level. New grievance offices, procedures and mechanisms have strengthened Ethiopia’s GRM system. Beyond these specific system changes, PBS 3 supported development of a culture of continued joint review and dialogue that brings together basic service stakeholders from across levels and sectors of government. Improved capacity to report on results: MIS systems in ministries and standards issued for data quality, with monitoring to verify, have strengthened the ability to report on service delivery results. The development of an Ethiopian Data Quality Assessment Framework has created capacity to validate and improve data, reinforced by the establishment of a Directorate of Data Quality within the CSA. Page 21 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Environmental and social assessments at the local level: One of the positive results from the Inspection Panel process was the decision to more broadly use PBS 3’s local platform to mainstream simplified environmental and social screening expectations and tools applied to all woreda investments. Institutional development within the Ministry of Finance: During PBS 3 the evolution/upgrade of COPCU to COPCD becoming a full-fledged directorate from essentially a project unit is an example of institutionalizing coordination capacity at the national level. Poverty Reduction and Shared Prosperity PBS 3 was a core part of the Government’s poverty reduction strategy. Block Grants preferentially benefitted less well-off regions and woredas. In terms of expenditure per capita, the current system broadly favors Ethiopia’s historically disadvantaged regions. Block grant transfers also appear to target less well-off woredas within regions. Analysis shows that for a 1 percent increase in the poverty head count, the per capita allocation will increase by about ETB 0.17. A woreda with a poverty rate of one standard deviation above the mean (42 percent) is predicted to receive about ETB 2.2 10 The benefits of PBS-financed spending at the woreda level on health, education, and agriculture accrued to all income levels as these were pubic services. However, there appears to have been a greater concentration in services used by the poor. Woreda-level spending on health and education is particularly pro-poor: 58 percent goes to the two bottom wealth quintiles. Citizen engagement efforts explicitly sought to reduce the exclusion of particular groups, often lower in income, in engaging in social accountability and transparency efforts. Social Accountability Committees required representation from women and disabled citizens. The dissemination of public budget and service delivery information was both in written and oral forms (e.g. via radio) and in local languages affording better penetration of information to poorer households. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION The preceding two phases of PBS (PBS 1 May 2006-December 2009 and PBS 2 May 2009 -January 2013) provided a rich basis for lessons learned and to optimize project design. Experience of the previous six years had created a very close collaboration between the Government, the World Bank, and other DPs. A harmonized approach between Government, DPs, and other complementary programs is essential to strengthening decentralized service delivery. Preparation of PBS 3 was consistent with development effectiveness principles of Government ownership and donor alignment behind key objectives and a harmonized results framework. The PBS program continued with dedicated resources for a PBS Secretariat. During preparation it was recognized that the quality of decentralization and the related institutional arrangements would impact the effectiveness of the PBS program. The Government issued a Policy Letter on Decentralization and Sustainability, received during project appraisal, which further clarified the central role of decentralization as contributing to the basic service delivery objectives. PBS 3 was identified as providing a critical platform for systemic capacity building and improved economic governance at the woreda level. Building on PBS 2 experience, a Grievance 10 Khan, Q., Faguet, J-P., Gaukler, C., and Mekasha, W. “Improving Basic Services for the Bottom Forty Percent”, World Bank. 2014. Page 22 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Redress Mechanisms was included as an important complement to social accountability measures piloted previously. The choice of instrument was a relevant issue in project preparation. From the PBS program inception in 2006, the World Bank used investment lending instruments to support PBS. The choice of an IPF was critical to supporting capacity building at the sub-national government level, particularly on PFM and transparency/accountability aspects. It also allowed strengthening the M&E systems and the use of associated Bank fiduciary safeguards, which facilitated co-financing from donors. PBS 3 was ready for implementation upon approval. The funding gap was considered acceptable given the past history of Government and donor support to PBS, with several donors restricted from making longer-term commitments. Key risks identified were (i) stakeholder risk (high), (ii) capacity (moderate), (iii) governance (low), (iv) social and environment (low), program and donor (moderate), delivery, monitoring and sustainability (moderate) and overall implementation risk which was rated high. The higher risk ratings reflected that the project flowed through and support government systems on a national scale. Risk and mitigation measures were adequately identified based on the experience of the previous two phases of PBS. B. KEY FACTORS DURING IMPLEMENTATION PBS 3 was a complex project with a country-wide focus. Several factors influenced implementation: Key factors outside of the implementing agencies’ direct control: Ethiopia is a large, diverse and complex country with political dynamics resulting from its unique history. The general atmosphere of political unrest and ethnic tensions influenced PBS 3 implementation. Widespread protests in parts of the two most populous regional states of Amhara and Oromia with hundreds of reported deaths resulted in the Prime Minister declaring a State of Emergency in October 2016 authorizing the military to enforce security nationwide, which was lifted in August 2017 after the easing of protests. The Prime Minister resigned in February 2018, with a State of Emergency re-instated for a few months in 2018. Ethnic tensions have continued, leading to an estimated 1.4 million internally displaced people, exacerbated by an influx of refugees from neighboring countries. This situation created a challenging environment for basic service delivery and country-wide capacity building efforts to continue to operate unimpeded. Charges made against PBS 3 for its potential links with the Government’s Community Development Program and related charges of human rights violations had a critical influence on implementation. Safeguards issues are discussed in Section IV.B on Environment and Social Compliance below. The allegations, though not substantiated, brought the program under intense international scrutiny. Charges brought in an international tribunal, though ultimately dismissed, were an influencing factor in DfID exiting PBS 3. Other donors like the EU ultimately redirected aid to more ring-fenced sectoral operations. This shifting of donor engagement affected what had been a model of aid harmonization and effectiveness, calling for a reconfiguration of donor coordination. The inclusion of the Risk and Safeguard Management Capacity sub-component resulted in the institutionalization of simplified Environment and Social Safeguard Frameworks at the woreda level. In terms of factors under the implementing agencies’/Government’s more direct control: PBS 3 was characterized by strong Government commitment to improved service delivery. This positively affected implementation in several ways: Continued solid macroeconomic performance and Government commitment to poverty reduction created a stable policy environment. The Government scaled up Federal block grant transfers significantly. This better enabled woredas and regions to increase coverage and quality of services. Page 23 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) The Government fully supported transparency and civic engagement, noteworthy given the overall political environment. The commitment to good governance and pro-poor spending to improve local services was a countervailing force against international concern with open society issues in Ethiopia at the time. For example, given the political unrest, the Government had a fairly restrictive posture vis a vis CSOs, looking to limit their participation in human rights issues. PBS 3’s ESAP program created a safe space for CSOs to operate on civic engagement and brought together civil society, local government officials and service delivery staff. The Government was fully engaged in the donor coordination mechanisms of PBS 3, taking a leadership position in the common platform, organizing joint supervision missions, presenting studies and findings, and following up on recommendations. The Government invested in the PBS 3 capacity building platform with a commitment to help strengthen national systems and build capacity across key transversal systems which supported the full scope of Ethiopia’s decentralized governmental structure. Despite these strengths of Government commitment and involvement in PBS 3, there were some challenges during implementation. There were delays in contracting and procurement of several planned activities, including the ERA MIS, a cost of service delivery study, and the 2nd Independent Procurement Audit. Staff turnover was a systemic concern voiced at all levels of Government and in all sectors. Although the Government raised salaries during the implementation period, there continues to be an overall need for civil service reform and better compensation for local government and service providers. It was a challenge to engage the federal level sectoral ministries since PBS 3 was not a specific sectoral operation under their control. Moreover, given the federal structure of Ethiopia, sectoral ministries at the federal level have little direct responsibility for service delivery. Key factors under Bank control: PBS 3 implementation benefitted from effective mechanisms for coordination among a large number of development partners. Multi-donor support for the PBS Secretariat enhanced supervision and implementation capabilities. Twice annual JRIS missions provided an organized supervision structure, bringing consensus and mutual agreement on implementation issues and actions. The PBS Secretariat supported various working groups of government on specific topics (transparency and accountability, financing and sustainability, PFM, Managing for Results, etc.) and provided a platform for shared analytical work. The PBS transformed into the Service Delivery Secretariat in 2015, seeking to retain (and has since retained) the positive functions of aid harmonization and donor coordination even if not under a single project. The coordination arrangements also allowed for a unified and shared analytical agenda. For example, a 2016 PBS 3 Process Evaluation was a useful joint effort by the Bank, Secretariat and DPs to have shared understanding of PBS 3 performance. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design PBS 3 design had a very clearly identified results framework built on a theory of change. The results framework was well specified, focusing on specific service linkages to woreda level recurrent spending. PDO indicators blended service access and quality issues. The expected results were built on country systems of data reporting in line with the PBS program approach to strengthen existing mechanisms to ensure that national data is credible, timely and Page 24 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) available rather than setting up parallel structures. Beyond specific measurement of PBS 3 results, the project sought to build overall system capacity to monitor and evaluate service delivery results. The Managing for Results sub-component sought to improve sectoral MIS systems as well as nationwide service delivery data. M&E design was challenged in setting targets. The PDO was formulated to support Government’s higher-level objectives and included two ‘by-statements’. Following current Bank practices of PDO formulation, it had some weaknesses. Given the close linkages with Government policies, national goals were adopted that in several instances were overly ambitious. In certain cases, not meeting a specific numerical goal overshadowed the large gains made. In several of the areas of capacity building, for example citizen’s engagement, there was little experience in what were realistic targets for such innovative and difficult to measure outcomes and intermediate results. Moreover, there was an acknowledgement that sectoral data was of relatively low quality in Ethiopia hence the investments in improving results measurements under PBS 3. This includes imprecise population estimates, particularly given significant internal population displacements, which affects accuracy of several service access indicators and targets. M&E Implementation M&E data was systematically collected and analyzed through project supervision. In addition to the routine monitoring of service delivery outcomes through national systems, several assessments and evaluations were carried out, in particular on FTA, ESAP, PFM institutional training and other areas of the capacity building agenda of PBS 3. Results indicators were updated through formal project restructurings as data systems improved or indicators required improved specifications. The results capacity building agenda was ambitious, with significant impacts despite some limitations in the full list of activities accomplished. CSA provided a central role in helping to improve the quality of sectoral data. This has helped instill a practice of data system validation. This improved ability has helped build the conditions necessary to move to more results-based lending. M&E Utilization One hallmark of the PBS 3 project was its consistent use of M & E data on performance and results to inform decision-making about local service delivery and capacity building. The results framework was regularly reported on during JRIS missions. In addition, this information was used to assess national service delivery performance and country-wide capacity building efforts more generally. Service delivery results have pointed out specific regions or indicators that are lagging, providing orientation for more targeted efforts. Specific assessments were useful in gauging the effectiveness of transparency and accountability efforts, creating a justification for their further expansion and systematization in national codes and practices. Justification of Overall Rating of Quality of M&E This ICR rates the quality of M & E as substantial. The M&E system was sufficient to assess the achievement of the objectives and test the links in the results chain. It also made notable improvements in the national M & E capacity. Some activities were not achieved in the Managing for Results component. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social The initial safeguard assessment determined that World Bank safeguard policies were not triggered, with an Environmental category C applied. This was largely because PBS 3 concentrated financing on recurrent expenditures for basic services. Although there could be concurrent Government financing for capital development in the basic Page 25 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) services sectors, this was not necessary to achieve the objectives of PBS 3. In July 2013, the Board authorized an Inspection Panel investigation following claims made by Anuak community members that PBS 3 was linked to human rights abuses that allegedly occurred under the government’s Commune Development Program (CDP) of population resettlement in the Gambella Region. The Panel’s Investigation Report, dated November 21, 2014 concluded that the alleged harm which the Panel had investigated could not be attributed to PBS 3. However, considering the magnitude of the operation, the nature of block grant financing, and the overlapping implementation area between PBS III and CDP, the Panel found that Management did not carry out the required full risk analysis, nor were its mitigation measures adequate to manage the concurrent roll-out of the villagization program in four PBS III regions. The Panel found that there was an ‘operational interface’ between the PBS and CDP due to geographical and sectoral overlap and nature of block grant financing to local governments. In response, the Management Report found that staff had followed applicable safeguard and operational policies, with the findings accepted by the Bank’s Board of Directors. A Management Action Plan was approved on February 27, 2015 to address concerns with regard to environmental and social risk assessment and financial management, including: (a) a screening tool was to be developed for use by woreda staff when preparing proposals for Bank financing under programmatic operations in order to manage the risks from CDP; (b) PBS 3 would continue to build the capacity of woreda staff to implement the Bank’s safeguard policies, including through a woreda level safeguards manual that could be used across the country; and (c) recruitment of a specialist within the PBS Secretariat to help implement the safeguard policies and provide continuous support to woreda-level staff. All three actions were completed. An Operations Screening Tool and the Environmental and Social Management System Operational Manual were developed, and training provided to woreda staff. A Safeguard Technical Working Group was established, chaired by the Environment, Forestry and Climate Change Commission. In addition, the World Bank carried out an Enhanced Social Assessment and Consultation for PBS 3 in 2015 with results used to address the area of indigenous peoples and vulnerable groups. Fiduciary Compliance From a project financial management and procurement lens, fiduciary compliance was effectively monitored throughout implementation. Led by financial and procurement Bank staff in the Ethiopia Country Office, the Bank did comprehensive FM and procurement supervision, quarterly reviews of IFRs, procurement reviews, and annual audit reports as well as participation in the multi-donor semi-annual JRIS missions. Fiduciary aspects of the project were rated as satisfactory or moderately satisfactory throughout the implementation of PBS 3. Issues identified during implementation largely centered on strengthening needed in internal audit procedures and procurement delays. Of note, a second Independent Procurement Audit was not concluded prior to closing. PBS 3’s capacity building component included an ambitious agenda for PFM improvements across the local level of Government decentralized service delivery. The PBS Secretariat provided additional PFM supervision combined with technical assistance, for instance by working directly on capacity building with regional BoFEDs. Because PBS 3 was based on national systems as opposed to parallel project systems for PFM, some of the issues identified were systemic ones. For example, while there were significant improvements in audit coverage and timeliness, internal audit functions were weakly staffed at the woreda level. Overall governance of PFM issues was reviewed by the PFM Working Group, a donor-government dialogue body chaired by World Bank and State Minister, with leadership from the Ministry of Finance. A similar arrangement has been established and is functional for procurement (Procurement Working Group), with representatives from Government, Development Partners and the private sector, chaired by World Bank and Director General, PPA. This promoted cross-cutting issues being raised and addressed within the Government’s PFM and Procurement systems. C. BANK PERFORMANCE Page 26 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Quality at Entry The World Bank prioritized continuing its commitment to the PBS series of operations because of the centrality of local service delivery to reaching the MDGs and helping achieve the country’s development goals. The Bank had a leadership role in PBS 1 and 2 and continued to help coordinate overall engagement by development partners with the Government in the design of PBS 3. Lessons learned from PBS 2 were clearly identified and incorporated into PBS 3. The project retained the commitment to working through government systems as the best way to support decentralized service delivery and leave a lasting impact on national systems and institutions. With strong coordination systems and a harmonized approach between all parties, the Bank helped ensure that PBS 3 stood as a model for aid effectiveness principles that were internationally recognized. The safeguard risk assessment done at the time of appraisal raised no red flags and was consistent with Bank practice for operations of this type. It was difficult to foresee that the scope of PBS 3 support to local service delivery would make it indirectly linked to all programs implemented in woredas, including the CDP. The Inspection Panel concurred that there was no direct harm done by PBS 3, and Bank Management and Board found that all relevant safeguards and operational policies had been followed. Quality of Supervision The Bank supervision team was robust, represented technical skills and experience across the domains of PBS 3 activity, and focused on development impact throughout the implementation period. Supervision was carried out through semi-annual JRIS missions as well as through continuous PFM supervision, and ongoing engagement in implementation of ESAP, FTA and other capacity building activities. The Bank further supported analytical and evaluation work on PBS 3, including a PSIA and research papers on decentralization and service delivery aspects, as well as evaluation work on ESAP. It is worth highlighting that the TTL and other program staff were field based, which led to opportunities for continuous implementation support and dialogue with Government and DPs. The World Bank played a central role in managing the complex donor coordination arrangements during PBS 3. As the host of the PBS Secretariat and the administrator of the multi-donor trust funds supporting all aspects of PBS 3, the Bank provided a backbone function to implementation. All of the donors were interviewed as part of the ICR process cited good performance by the Bank. There were some differences of opinion on how narrowly the scope of social accountability efforts should be focused on service delivery, with some donors looking for broader engagement on open society and democratization/human rights issues. But, it was recognized that focus on service delivery was appropriate for the Bank and the Government and made ESAP more viable program. Performance ratings were adjusted through implementation. The team routinely updates outcome and intermediate results indictors to track progress toward goals. Towards the end of the project, the Managing of Results component was downgraded to moderately unsatisfactory in an effort to draw attention to the need to speed up the contracting and finalization of certain analytical or capacity building pieces. However, looking over the entire performance of the sub-component and what it achieved, the last ISR should have rated that Moderately Satisfactory given the relatively small weight of the outstanding items in the overall achievements. The Bank team effectively oversaw transition to the follow-on operation. Justification of Overall Rating of Bank Performance The overall rating for Bank Performance is Satisfactory. This review rates overall Bank performance as Satisfactory. The Board agreed with Management that World Bank Directives were followed in the Inspection Panel case. The attention to building strong relationships with the Page 27 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Government and DPs and providing leadership through its Ethiopia-based Task Team and the PBS Secretariat was widely appreciated by the Government and Development Partners. The Task Team consistently reported on project progress and outcomes. Moreover, the project had a wide number of initiatives and topical areas, from service delivery to citizen engagement, PFM aspects, capacity building for results and safeguards training. Bank supervision fielded a multidisciplinary team able to effectively review this broad scope. D. RISK TO DEVELOPMENT OUTCOME There are several strong indications of a longer-term, sustainable impact. Regardless of any specific donor-funded initiative, the Government’s own commitment to increased funding through the intergovernmental fiscal transfer system is ensuring that basic services continue to improve at the local level in Ethiopia. In addition, the approach of PBS 3 in using existing Government systems resulted in the institutionalization of many of the capacity building efforts, such as the IBEX public financial system, formal directives on PFM and environment and social safeguard screening, the expansion of formal grievance redress mechanisms, and the change in expectations that local governments and service providers provide financial and service delivery information to the public and that citizens can engage in discussions on quality of services, among others. A follow-on IDA operation Program-for-Results (PforR) Enhanced Shared Prosperity through Equitable Services (ESPES) for US$600 million was approved August 19, 2015. The PforR continued Bank support to Subprogram A of the PBS 3 to finance woreda block grants through the intergovernmental fiscal transfer system but using a results-based instrument, rather than additional financing through the IPF-format of the PBS 3 project. This reflects the system improvements achieved during PBS 3 which underpinned the ability to move to a PforR format. In addition, ESPES identifies outcomes in the weaker performing woredas as a key Disbursement Linked Indicator to promote more equitable outcomes and incentivize investment in basic services in the poorer regions. This is the natural progression towards results-based financing foreseen at the appraisal of PBS 3 and builds on and extends the focus on improved local service delivery through the basic service block grants. Additional Financing of US$700 million was approved on August 22, 2017. This continues the PforR block grant funding and includes an allocation of US$100 million as a grant for an investment component to provide technical assistance and capacity building for service delivery system performance, continuing the system strengthening of PBS 3. V. LESSONS AND RECOMMENDATIONS Under PBS 3 considerable results were achieved both in improvements in service delivery and capacity building within a decentralized system, with key lessons learned in several areas: Lessons for working in a decentralized setting: Devolution of power and responsibilities to regions and woredas must be matched with resources. PBS was carried out in a Federal system where the core basic services are essentially the responsibilities of the local levels of Government. This Ethiopian fiscal federalism works because of the Block Grant transfer system. Ethiopia’s rapid expansion in basic service results like access to health and education could not have been achieved without these fiscal transfers to local government. However, this creates greater complexity for service delivery project design and implementation. It is important to identify how to engage sectoral agencies within such a decentralized structure . PBS engaged mainly with Federal sector agencies to support their systems for generating reliable data. Local service providers were integrated mainly through local budgets for staffing and operations as well as citizen engagement processes. An added focus on sectoral service delivery planning at the regional and local levels should be considered to ensure coordination of all sectoral inputs and service standards. Page 28 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Over time a decentralized system will need more diversified instruments than a single base general Block Grant to address issues of equitable access and quality. Ethiopia has developed or is considering adding capital grants, special purpose block grants, equity block grants etc. as well as earmarked sectoral transfers for woredas lagging behind on results. This is especially relevant for some of Ethiopia’s lowland areas where the local capacity and structures for effectively managing basic service delivery through Block Grants is a challenge and basic service outcomes are lagging much behind other parts of the country. In the Ethiopian context, the regional level is crucial. Regional government must be involved in the design and implementation as they serve as a critical link in the system, often as a locus for capacity building systems as well as the interface with secondary level service facilities like hospitals, high schools and regional roads. Lessons learned on large-scale program working through local government: As the Bank moves to provide more funding to local service delivery, for example through PBS-type or PforR operations, task teams would benefit from clearer guidance for navigating the practical implementations of funding local government budgets. The experience of the Inspection Panel case had a significant if unforeseeable influence on PBS 3. PBS 3 was implicated largely because its use of government systems, including fiscal transfer mechanisms. Bank guidance on environmental and social assessment in investment operations had not really envisioned a case such as PBS 3, where due to these factors it was necessary to have an operational interface to screen certain local government operations. This has broader implications for other Bank operations outside of Ethiopia. Specific lessons learned in respect to the screening tool designed for Ethiopia include the requirement for the procedure to be simple and, embedded within existing Environmental and Social Management Frameworks already in use. It should enable the Bank to support local government programs whenever possible by (a) managing the operational interface, (b) being able to demonstrate that it has taken all reasonable steps to consider the implications of the interface, and (c) avoiding involvement in non-viable or seriously deficient programs. In addition, continued environmental and social system strengthening at local level will have a transformational impact in managing environmental and social risks at local level regardless of the type of project and/or funding source. Lessons for large-scale programs working with multiple donors: Such a large program presents both opportunities and challenges. The scale of PBS 3 was very large, both in the number of sectors and the geographic scope (all regions and woredas), as well as the amount of funding. The scope contributed to moving national-level indicators of service delivery. Economies of scale allowed for relatively large scale capacity building efforts and provided significant pooled funding for supervision activities. However, as identified in the PBS 3 process evaluation, there were risks of such a broad lens. The scope of institutional reforms under Sub-Program B embraced many challenging areas, taking on perhaps too many specific activities. PBS 3 is an outstanding representation of the aid effectiveness principles of the Paris Declaration. This required resources and buy-in. A permanent secretariat was the core feature, supported by JRIS missions twice per year, monthly development partner meetings, shared results frameworks and common analytical work agenda. Beyond just reducing the transaction costs of project management for partners and the Government, this format created a platform for coherent discussions on service delivery issues, particularly those that are transversal, affecting all sectors. This arrangement has held even as donors have diversified away from a single instrument to varied instruments to support decentralized service delivery. Lessons for improving the transparency and accountability of local service delivery: Improvements in basic services rely on important cross-cutting systems that may be overlooked in more narrow sectoral operations. This includes that funds flow efficiently and reliably to finance service delivery, that service delivery is built into local budgeting, that people for whom the service are intended have a voice in providing feedback on those services, and that information on budgets and service is made publicly available. All of these aspects translate into better Page 29 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) quality, more relevant and more efficient service delivery. The PBS series represents a highly intensive and innovative approach to building this supportive environment and systems. For example, the social accountability approach was mutually reinforcing from the supply side (FTA) and demand side (ESAP). This went beyond just posting information and getting citizens’ feedback on services, to bringing together woreda officials, service providers and citizens into a formal process of consultation and joint action plans to improve local services. Lessons learned in aligning with country systems and donor coordination: Despite the higher risks, there were significant advantages in PBS’s reliance on national systems. By working through Government fiduciary systems, PBS capacity building affected all government spending rather than focus on only project funding. Relatively simple investments, like IBEX to automate financial processing and reporting in over 900 woredas built a more reliable fiduciary base for more resources to flow to the local level. This approach also gave leverage to engage in discussions not just regarding donor funding but on the larger pool of national funds channeled through the block grant transfer system. And finally, all of the financial, safeguard, procurement, transparency and accountability measures achieved have a far greater likelihood of permanence because they are embedded into the country’s systems and directives. The lesson learned on a project like PBS is to be ambitious and build in capacity and system change across the entire local service delivery level, but through simplified tools and adequate training. The PBS series is an outstanding example of effective donor harmonization as well as Government/donor collaboration. Key elements included common results framework, shared supervision and analytical work, full integration with Government sectoral targets and fiscal decentralization systems, a multi-donor Trust fund to pool resources, and support for a PBS secretariat which provided a dual function of donor coordination which relieved . transaction costs on the government and technical assistance to Government implementing agencies. Page 30 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Expand access to/quality of basic services by funding block grants that ensure adequate staffing/ope Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Ratio of health extension Text 1:2144 1:2500 1:2570 workers to population 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Student Teacher Ratio (Grade Number 51.00 41.00 35.00 5-8) 30-Dec-2011 30-Nov-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 31 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Agricultural extension services Number 9044000.00 14900000.00 16975113.00 beneficiaries 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Roads in fair and good Percentage 82.00 88.00 93.00 condition 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Reduced non functioning water Percentage 20.00 8.00 9.40 supplies schemes (disaggregated by regions + 30-Dec-2011 30-Dec-2011 10-Dec-2018 Urban/Rural) Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of qualified primary Percentage 47.20 97.00 83.00 Page 32 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) school teachers (Disaggregated 30-Nov-2011 30-Nov-2011 10-Dec-2018 by gender + Regions) Comments (achievements against targets): Objective/Outcome: Strengthen capacity, transparency, accountability and FM of governments at regional and local levels Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Citizens who are informed Percentage 19.00 25.00 42.00 about Woreda budget 30-Dec-2011 30-Nov-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Citizens who report that Percentage 48.00 55.00 54.80 Woreda officials have actively sought the views of people in 30-Dec-2011 30-Nov-2011 10-Dec-2018 their Kebele on improving quality of basic services Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 33 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Number of Woredas producing Number 0.00 900.00 983.00 IBEX and IFMIS based financial management reports 30-Nov-2011 30-Nov-2011 10-Dec-2018 Comments (achievements against targets): A.2 Intermediate Results Indicators Component: Expand access to/quality of basic services by funding block grants that ensure adequate staffing/ope Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Health extension workers Number 38298.00 40000.00 36660.00 20-Jul-2012 20-Jul-2012 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Qualified primary school Number 141696.00 300000.00 401172.00 teachers 20-Jun-2012 20-Jun-2012 10-Dec-2018 Comments (achievements against targets): Page 34 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woreda road desks with Number 155.00 650.00 842.00 appropriate staff (Disaggregated by Regions) 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Development agents with Text 2102 n.a. n.a. This indicator is specialization on Agro- no longer tracked in ecological zones and high value the MoANR MIS crops system so no data are available. 30-Dec-2011 30-Dec-2011 14-May-2018 Comments (achievements against targets): Component: Strengthen capacity, transparency, accountability and FM of governments at regional and local levels Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woredas that post the refined Percentage 70.00 100.00 87.00 and simplified budget and expenditure template to the 30-Dec-2011 30-Dec-2011 10-Dec-2018 Page 35 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) public Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Basic service units that post Percentage 25.00 70.00 62.00 the standardized service delivery template 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Regions with functioning GRMs Number 2.00 11.00 9.00 consistent with best international practice 20-Jul-2012 20-Jul-2012 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of regions preparing Text 3 11 10 and adopting standardized manual and guidelines for 20-Jul-2012 20-Jul-2012 10-Dec-2018 GRMs Page 36 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Audit Coverage: Office of Percentage 30.00 60.00 69.00 Regional Auditor General 20-Jul-2012 20-Jul-2012 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Audit Coverage: Office of Percentage 90.00 100.00 100.00 Auditor General 20-Jul-2012 20-Jul-2012 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woredas that implemented Text 5 60 41.99 prior period audit recommendations 20-Jul-2012 20-Jul-2012 10-Dec-2018 Comments (achievements against targets): Page 37 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Regions submit the PBS Number 4.00 11.00 9.00 quarterly and Annual IFRs to COPCU without major 20-Jul-2012 20-Jul-2012 10-Dec-2018 inconsistencies Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woredas that post all Number 5.00 900.00 590.00 procurement award decisions 20-Jul-2012 20-Jul-2012 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of woredas staffed Number 5.00 500.00 500.00 with procurement specialists with regional procurement 20-Jul-2012 20-Jul-2012 10-Dec-2018 training and with clear career path Page 38 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woredas where PBS Social Number 50.00 400.00 223.00 Accountability Implementing Partners (SAIP) facilitate the 30-Dec-2011 30-Dec-2011 10-Dec-2018 use of Social Accountability tools Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Social Accountability Number 0.00 400.00 223.00 Implementing Partners woredas that have developed 30-Dec-2011 30-Dec-2011 10-Dec-2018 joint action plans for service quality improvement based on interface meetings between service users and providers Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 39 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Number of sectors with their Number 0.00 2.00 4.00 indicators classified as official data through implementation 30-Dec-2011 30-Dec-2011 10-Dec-2018 of Ethiopia Data Quality Assessment Framework Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woredas that rolled out the Number 0.00 900.00 1019.00 new Education Management Information System 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Improved community health Percentage 72.00 100.00 100.00 management information systems 30-Dec-2011 30-Dec-2011 30-Mar-2019 Proportion of health centres Percentage 72.00 100.00 96.00 implementing the new Health Management Information System Page 40 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Woredas that rolled out the Number 50.00 900.00 450.00 new WASH Managment Information System 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Strengthened Ministry of Text No Management Continued Achieved Agriculture Management Information System implementation of Information System new MIS 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Development of Ethiopia Road Text No web based Continued ERA has initiated roll Authority Management Management implementation of out of the MIS to all Information System Information system new MIS regions and provided training Page 41 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) 30-Dec-2011 30-Dec-2011 10-Dec-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Environmental and Social Yes/No N Y Y Management System (ESMS) Operational Manual developed 20-Jul-2012 20-Jul-2012 14-May-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of Woreda-based staff Number 0.00 100.00 4768.00 trained in environmental and social management based on 20-Jul-2012 20-Jul-2012 30-Mar-2019 ESMS manual Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of regions with Number 0.00 2.00 11.00 Page 42 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) customized Environmental and 20-Jul-2012 20-Jul-2012 10-Dec-2018 social Management System Operational Manual Comments (achievements against targets): Page 43 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Expanded access to and quality of basic services by funding block grants that ensure adequate staffing and operations 1. Proportion of qualified primary school teachers 2. Student-teacher ratio (grades 5-8) 3. Population per health extension workers Outcome Indicators 4. Number of agricultural extension services beneficiaries 5. Percent non-functioning water supply schemes 6. Percent roads in fair and good condition Intermediate Result: Improved availability of qualified staff and operational resources for delivery of basic services 1. Number of health extension workers deployed (end of project target of 36,660 was 92% achieved) 2. Number of qualified primary school teachers (end of project target of 300,000 was 134% Intermediate Results Indicators achieved) 3. Number of woreda road desks with appropriate staff (end of project target of 650 was 130% achieved) Key Outputs by Component 1. US$11.03 billion block grants for basic services transferred from Federal level to all regions (linked to the achievement of the and woredas during the PBS 3 implementation period. Objective/Outcome 1) Objective/Outcome 2: Strengthened capacity, transparency, accountability and financial management of sub-national governments to deliver and report on results 1. Percent citizens who are informed about woreda budget 2. Percent citizens who report that woreda officials have actively sought the views of people in Outcome Indicators their kebeles on improving basic services 3. Number of WoFEDs that have effectively rolled out IBEX Intermediate Result: Improved citizens' engagement on quality of basic services delivery Page 44 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) 1. Number of woredas where PBS Social Accountability Implementing Partners (SAIP) facilitate the use of Social Accountability tools (end of project target of 400 was 56% achieved) 2. SAIP woredas that have developed joint action plans for service quality improvement based on interface meetings between service users and providers (end of project target of 400 Intermediate Results Indicators was 56% achieved) 3. Regions with functioning GRMs consistent with best international practice (end of project target of 11 was 82% achieved) 4. Number of regions preparing and adopting standardized manual and guidelines GRMs (end of project target of 11 was 91% achieved) Financial Transparency and Accountability: 1. Roll-out and implementation of audit and procurement templates to disclose activities at woreda levels 2. 2.6 million woreda citizens trained to raise awareness and understanding of public budgeting and planning processes and participate in pre-budget planning 3. Disclosure of local public budgets through mass media and culturally relevant information channels 4. Printing and dissemination of FTA templates 5. Procurement of bill-boards and suggestions boxes for posting budget and service delivery information. Key Outputs by Component 6. Impact assessment survey on budget literacy of citizens (linked to the achievement of the Social Accountability: Objective/Outcome 2) 1. Social Accountability activities rolled out in 223 woredas implemented by 49 lead CSOs or Social Accountability Implementing Partners (SAIPs) 2. 200,000 citizens’ representatives directly supported to make own choices, have control over their own development and hold decision makers accountable for the services, far exceeding end of project target of 40,320. 3. 38,104 service providers and 18,621 local council representatives participated in social accountability sessions, also far exceeding targets. 4. In 63 percent of participating woredas local government resources were allocated to implement Joint Action Plans. Grievance Redress Mechanisms: Page 45 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) 1. Production of guidelines and manual for GRM at regional and Federal level through Ethiopian Institution of Ombudsman (EIO) 2. Capacity building training for EIO staff at Federal and branch office level 3. Establishment of 3 new branch offices of EIO : New branch EIO offices are established in regions of Gambella (in 2013), Afar (in 2017), and Benishangul Gumuz (in 2017). 4. Standard procedural manuals prepared in English and Amharic languages are disseminated in hard and soft copies to all regional EIO and GRM offices. Intermediate Result: Improved transparency and fiduciary assurance regarding the use of public funds 1. WoFEDs connected with woreda net (end of project target at 1,023 woredas was 100% achieved) 2. Woredas that post the refined and simplified budget and expenditure template to the public (end of project target at 100% of woredas was 94% achieved) 3. Basic service units that post the standardized service delivery template(end of project target of 70% was 89% achieved) 4. Audit coverage: ORAG (end of project target of 60% was 115% achieved) Intermediate Results Indicators 5. Audit coverage: OFAG (end of project target of 100% was 100% achieved) 6. Woredas that implemented prior period Audit recommendations (end of project target of 60% was 70% achieved) 7. Number of woredas staffed with procurement specialist with regional procurement training and with clear career path (end of project target of 500 was 100% achieved) 8. Woredas that publicly post all procurement award decisions (end of project target of 900 was 66% achieved) 1. Public Financial Management Institutionalized Training (PFMIT) implemented at federal level with 1,318 PFM professionals from federal budget bodies trained Key Outputs by Component 2. PFMIT training in seven regional states and Addis Ababa City Administration reached 6,949 (linked to the achievement of the regional staff. Objective/Outcome 2) 3. Audit Committees established and trained in 400 woredas 1. Workshops on PFM activities for woreda council and cabinet members, BoFED heads and deputy heads (2,161 woreda council members attending) Page 46 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) 2. Woreda-level manuals on PFM (cash management, performance audits, procurement standard bidding documents) developed and rolled out in 1,023 woredas 3. PFM reinforced at Federal and Regional levels through support to 549 contract employees Intermediate Result: Strengthened M&E systems at central, federal, and sub-national levels 1. Regions Submit the PBS the quarterly Interim Financial Reports to COPCU without major deficiencies (end of project target of 11 was 82% achieved) 2. Number of sectors with their indicators classified as official data through implementation of EDQAF(end of project target of 2 was 200% achieved) 3. Woredas that rolled out the new EMIS(end of project target of 900 was 113% achieved) Intermediate Results Indicators 4. Proportion of health centers implementing the new HMIS(end of project target of 100% was 96% achieved) 5. Woredas that rolled out the new WASH MIS(end of project target of 900 was 50% achieved) 6. Strengthened MoA's MIS(end of project target of an operational MIS was fully achieved) 7. Development of ERA's MIS(end of project target of an operational MIS was not achieved) Ministry of Education: 1. Data Quality Assessment carried out 2. Service Facility/Delivery Survey (SDI 2) carried out 3. Nationwide training for woreda staff on EMIS conducted. 4. IT support (server and network installation) at Ministry of Education completed Key Outputs by Component Ministry of Health: (linked to the achievement of the 1. Service Provision Assessment (SPA+) carried out. Objective/Outcome 2) 2. IT equipment procured Ministry of Agriculture: 1. Development Agent tracking survey conducted 2. Training for phase 2 regions on new MIS 3. Semi-annual reviews on agricultural service delivery with stakeholders 4. Recruitment of M & E and IT experts Page 47 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Ministry of Water, Irrigation and Energy 1. Service Delivery and Facility Survey conducted 2. Purchase of computers and equipment for roll out of woreda-level MIS in 390 woredas. Ethiopian Road Authority 1. Baseline data collection on road quality for all regions 2. Design of ERA MIS Central Statistics Agency 1. Implementation of data quality assessments by Central Statistics Agency in in each of the 5 basic service sectors (and twice in education and health sectors). 2. Development of georeferenced rural service facility data for all regions and publication of regional service facility maps 3. Mini-Demographic Household Survey (DHS) carried out 4. Completion and publication of national Household expenditure survey (with welfare monitoring component) 5. Support to disaggregation of data by gender at decentralized level (in all 11 regions) and publication of country report on Gender Disaggregated Development Data (2015). Intermediate Result: Improved capacity at local level to manage environmental and social risks (added during implementation) 1. Environmental and Social Management System (ESMS) Operational Manual developed (fully achieved) 2. Number of Woreda-based staff trained in environmental and social management based on Intermediate Results Indicators ESMS manual (end of project target of 100 was 4,768% achieved) 3. Number of regions with customized Environmental and Social Management System Operational Manual (end of project target of 2 was 550% achieved)( 1. Environment and Social Impact Assessment (ESIA) Capacity Review carried out and action plans developed for Emerging Regions Key Outputs by Component 2. Environmental and Social Management Operational Manual developed and customized at (linked to the achievement of the regional level in all the 11 regions. The dissemination at woreda level is being done through Objective/Outcome 2) training of staff. Training has been delivered in 872 woredas, reaching 4,768 woreda staff 3. 187 Regional staff trained in environment and social management (training of trainers’ model). Page 48 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Louise Victoria Monchuk, Yoseph Abdissa Deressa Task Team Leader(s) Demelash Demssie, Binyam Bedelu Mekbib Procurement Specialist(s) Abiy Demissie Belay Financial Management Specialist Qaiser M. Khan Team Member Qays Hamad Team Member Sofia Said Team Member Wendmsyamregne Mekasha Team Member Eric Brintet Team Member Laura Campbell Social Specialist Tamene Tiruneh Matebe Environmental Specialist Samuel Mulugeta Team Member Adiam Berhane Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 34.103 213,892.32 FY13 3.430 52,694.05 Total 37.53 266,586.37 Page 49 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Supervision/ICR FY13 233.899 806,673.30 FY14 170.127 1,031,055.87 FY15 181.826 1,122,014.54 FY16 125.461 707,355.51 FY17 63.457 348,111.96 FY18 50.118 248,471.33 FY19 40.650 331,442.41 Total 865.54 4,595,124.92 Page 50 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) ANNEX 3. PROJECT COST BY COMPONENT Amount at Percentage of Actual at Project Components Approval Approval Closing (US$M) (US$M) (US$M) (A) Basic Service Block Grants $6,193 $11,030 178% (B1) Citizen's Engagement (Strengthening Local $35 $43.7 125% Transparency and Accountability Systems) (B2) Local Public Financial Management and Procurement (Strengthening Local Transparency $50.3 96% $52.6 and Accountability Systems) (B3) Managing for Results (Strengthening Local $27 $22.5 83% Transparency and Accountability Systems) (C) Results Enhancement Fund $78.2 0 0% (B4) Risk and Safeguard Management Capacity 0 $0.5 n.a Total $6,390.8 11,147.0 174% By Funding Source: Component Total costs GoE IDA* DPs Appraisal Closing Appraisal Closing Appraisal Closing Appraisal Closing Sub-Program A: Basic Service Block Grants $6,193 $11,030.0 $3,171.0 $9,796.5 $555.0 $511.0 $984.2 $722.5 Sub-Program B: Strengthening Local Transparency and Accountability Systems Bl: Citizen's Engagement $35 $43.7 $10.0 $10.5 $8.6 $33.2 B2: Local PFM & Procurement $52.6 $50.3 $20.0 $38.0 $32.6 $12.3 B3: Managing for Results $27 $22.5 $10.0 $17.2 $12.0 $5.4 Sub-Program C: Results Enhancement Fund $78.2 $0.0 $0.0 $78.2 $0.0 B4: Risks and Safeguard Management Capacity (added at Mid-Term) n.a. $0.5 n.a. $0.5 Unallocated $5 0 $5.0 $0.0 Total (US$ Millions) $6,390.8 $11,147.0 $3,171.0 $9,796.5 $600.0 $577.2 $1,115.6 $773.3 *At time of drafting the ICR, funds remained in the special account to be reconciled against expenditures effectuated in the final IFR, with a lag time for reporting through the Government’s fiscal transfer system. ICR data as of May 23, 2019 Page 51 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Details on Financing by Development Partner DPs of which: Germany Component Dfid AfDB EU Italy Austria Ireland (KfW) Sweden Appraisal Closing Appraisal Closing Appraisal Closing Appraisal Closing Appraisal Closing Appraisal Closing Appraisal Closing Appraisal Closing Sub-Program A: Basic Service Block Grants $664.6 $417.4 $237.9 $253.8 $61.4 $33.0 $12.3 $9.1 $8.1 $9.2 Sub-Program B: Strengthening Local Transparency and Accountability Systems Bl: Citizen's Engagement $7.0 $11.5 $1.5 $4.6 $1.4 $1.4 $8.9 $6.2 $0.5 B2: Local PFM & Procurement $21.9 $12.3 $11.4 B3: Managing for Results $7.0 $5.4 $5.0 Sub-Program C: Results Enhancement Fund $78.2 $0.0 B4: Risks and Safeguard Management Capacity Total (US$ Millions) $778.8 $446.5 $250.8 $253.8 $66.4 $37.6 $12.3 $9.1 $9.5 $10.6 $0.0 $8.9 $0.0 $6.2 $0.0 $0.5 Page 52 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) ANNEX 4. EFFICIENCY ANALYSIS Economic and financial analysis to assess the efficiency of PBS 3 covers four aspects: A. Assessment of the overall fiscal framework in which PBS 3 was implemented B. Efficiency of using Government systems C. Cost effectiveness analysis D. Social cost-benefit analysis – poverty orientation of PBS expenditures A. Fiscal framework Economic context of PBS Ethiopia has maintained high and inclusive growth for more than a decade, achieving commendable progress in reducing poverty and improving living standards. Over the PBS 3 implementation period, annual GDP growth averaged 9.5 percent (Table 1 below). This high and sustained growth rate generated a significant increase in Government revenues, which averaged 14.1 percent of GDP annually over the PBS 3 period. Government’s commitment to poverty reducing expenditures remained steadfast, with poverty-oriented expenditures holding very consistent as a percentage of GDP over the PBS 3 period. As GDP rose substantially, this translated into a significant increase in revenues and poverty-related expenditure during the PBS period. As a result of this sustained economic growth and pro-poor spending, Ethiopia’s poverty rate fell from 36.8 at the beginning of PBS 3 to 23.5 in 2015/2016, the most recent year with a poverty head count, despite a significant drought in 2015. Moreover, these improvements were widespread as poverty decreased in all the regions, most notably in Somali, Dire-Dawa and Addis Ababa. Table 1: PBS 3 Implementation Period, Selected Economic Indicators PBS 3 2012/2013 2013/2014 2014/2015 2015/2016 2016/2017 2017/2018 average GDP growth rate (%) 9.8 10.3 10.4 8 10.7 7.7 9.5 Government revenue (% GDP) 14.3 14 14.4 15 14.3 12.8 14.1 Poverty-reducing expenditure (% of total expenditures) 12.5 12.1 11.4 12 11.3 10.7 11.7 Poverty head-count ratio (%) 36.8 23.5 Source: IMF data. Federal fiscal transfers have played an important role within this overall fiscal context. The PBS program was based on intergovernmental fiscal transfers using block grants for basic service delivery at the woreda level. At the outset of the PBS 3 period, total block grant transfers were about $US 1.95 billion, more than doubling to $US 4.4 billion by the end of PBS3. This far exceeded expectations at the beginning of PBS 3, which according to the Project Appraisal Document estimated Government contribution to block grants at ETB 129 billion when in actuality the Government contributed ETB 313 billion. The share of funding going to basic services held relatively steady with this increase in overall funding. Donor contributions to the block grants, estimated at $US 1.5 billion at appraisal, resulted in US$1.8 Page 53 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) billion over the PBS 3 period. Overall, donor share of the Federal block grants fell from 23 percent at the beginning of PBS 3 to 7 percent at project closing. Of note, these block grant transfers funded recurrent costs which were the focus of PBS3. At the same time, the Government and DPs funded capital transfers through the MDG/SDG fund transfer which helped capital investment in basic service sectors at woreda level. Table 2: Regional Block Grant Transfer from Federal Government (in million birr) Average annual 2012/13 2013/14 2014/15 2015/16 2016/17 2017/18 increase 2012/2013- 2017/2018 Tigray 2,474.4 3,036.1 4,583.3 5,203.7 7,170.0 6,929.8 36.0% Afar 1,131.2 1,339.9 1,864.3 2,272.1 3,196.0 3,466.6 41.3% Amhara 8,212.5 9,839.7 15,132.5 16,931.2 23,221.0 24,847.3 40.5% Oromiya 11,575.4 13,815.5 20,222.4 23,660.3 32,751.0 39,703.2 48.6% Somale 2,908.6 3,457.6 4,751.1 5,919.6 7,968.0 11,505.1 59.1% Benishangul 746.1 897.7 1,359.5 1,540.9 2,205.0 2,101.6 36.3% SNNP 7,187.9 8,521.7 12,109.7 14,656.3 19,719.0 23,164.5 44.5% Gambella 541.0 630.3 979.0 1,088.8 1,512.0 1,511.6 35.9% Harari 361.0 429.2 590.8 729.6 966.0 861.2 27.7% Dire Dawa 417.2 497.7 688.1 847.6 1,143.0 1,013.3 28.6% Total birr million 35,555.3 42,465.3 62,280.8 72,850.1 99,851.0 115,104.1 44.7% Total US$ million $1,953.9 $2,236.1 $3,099.2 $3,457.1 $4,448.2 $4,408.7 25.1% PBS DPs' Contribution to Block Grants (in million USD) 2012/13 2013/14 2014/15 2015/16 2016/17 2017/18 Total 444.6 450.6 369.5 284.5 310.8 320.0 DPs share of Block Grants 22.8% 20.2% 11.9% 8.2% 7.0% 7.3% Page 54 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) The overall efficiency and effectiveness of the program is tied to the performance of these transfers. PBS 3 adopted several core principles to ensure the efficiency and effectiveness of these transfers: Effectiveness – fiscal transfers translate into service delivery outcomes. Status: The bulk of PBS block grant component supported salaries of public service delivery workers in the five basic service sectors; health, education, water and sanitation, rural roads and agriculture. This spending accounted for about 64 percent of woreda spending, and resulted in a significant expansion of services at the local level, as tracked in the PDO indicators and other relevant data: • The number of pregnant women receiving pre-natal care increased from 2.4 million to 3.5 million; • The number of teachers Grades 5-8 increased from 144,979 to 184,804 during the PBS 3 period, resulting in a pupil/teacher ration declining from 51 students per teacher to 35, a significant quality improvement. • Agricultural extension service beneficiaries increased from 9.4 million to 17.0 million during PBS 3, and the productivity per hectare of major food crops increased by 40 percent; • The percentage of roads in fair or good condition increased from 82 to 93 percent; and • Non-functioning water systems decreased from 26 to 9 percent. Sustainability – seeking to ensure that financing of basic services can be sustained over the long-term even without the PBS Program. Status: During implementation, domestic resource mobilization was a frequent theme of discussions. A Domestic Resource Mobilization (DRM) study was carried out and a study on the cost of basic services commissioned to help inform the Sustainability Review. The study projected DRM potential in different scenarios and that to realize a positive outcome, it emphasized the need to accelerate structural transformation of the economy and to strengthen tax administration. During the PBS 3 period, domestic resource mobilization via taxes rose in real terms at an annual average of 11.5 percent. Additionality – ensuring increasing Government funding for the federal block grant and its predictable flow to subnational levels. The additionality test has two verification criteria: i) the proportion of federal block grant from federal government spending does not show a decline; and ii) FBG allocation to regions in real terms does not show a declining trend. Page 55 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Status: Federal block grant transfers have risen significantly and steadily through the PBS 3 period, averaging a 45 percent annual increase (in birr), built on increased national revenues and economic growth. The Government of Ethiopia’s contribution to the block grant transfers increased from 77.2% to 92.7% of transfers during the PBS 3 period, far surpassing initial expectations (Figure 1). In real terms, there has been a generally increasing trend in the value of Government financing of Federal block grants (Figure 2), with a real annual average of 26.4 percent over the 5 year implementation period despite years of relatively stronger inflation rates. Figure 1: Federal Block Grant Transfers by Source (in million current birr) 150,000.0 100,000.0 DPs 50,000.0 GoE 0.0 2012/13 2013/14 2014/15 2015/16 2016/17 2017/18 Figure 2: GoE Funding of Federal Block Grant Transfers in constant 2011/12 birr (millions) 70000 60000 50000 40000 30000 20000 10000 0 2012/13 2013/14 2014/15 2015/16 2016/17 2017/18 Fairness – resource allocations from the federal government to the regions and from regional governments to districts are rules-based, transparent, and delivered as budgeted. Status: The fairness test ensures resource transfers to regions and to woredas are according to budget formula and that the budgeted resources are actually transferred. The Fiscal Decentralization Strategy (2011-12) outlines the national approach to both federal-regional and regional-woreda grant systems. The Strategy lays out a number of criteria for the allocation formulae of region-woreda block transfers, including a focus on minimum standard of service and expenditure need analysis. Throughout PBS 3, the Government consistently and satisfactorily met the fairness test as verified during supervision. Block grant allocations from the Federal government to regional governments and then to the woredas tended to be largely fully disbursed through the PBS 3 period. Equity –in access to basic services among the different regions and districts and between genders. Page 56 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Status: Block grant transfers appear to target less well-off woredas within regions. Analysis shows that for a 1 percent increase in the poverty head count, the per capita allocation will increase by about ETB 0.17 (PSIA 2015). The PSIA also found that block grants preferentially benefit less well-off regions, supporting higher levels of expenditure per capita in regions that were historically disadvantaged. And, sectoral expenditure favors the bottom 20th percentile (in terms of poverty headcount) and those comprising the bottom 20th percentile in terms of outcomes, for both health and education. For further evidence of equity impacts see Section D Social Cost-Benefit Analysis below. Transparency – to provide stakeholders with more information about resource flows, standards and results. Status: Supported through Sub-Program B capacity building investments. Based on the activities and outputs under the Citizens’ Engagement sub-component, including social accountability, financial transparency and accountability and grievance redress mechanisms, key achievements include: a) 94 percent of woredas were found to have posted the budget and expenditure template to the public and 89 percent of service delivery units had posted service standards and service delivery data. b) Budget literacy training was provided to citizens, council members, and sector offices’ staff reaching 2.6 million woreda citizens, of which 1.1 million women. In citizens’ understanding of budget process, the majority, 69 percent, voiced that their understanding has grown versus 29 percent in 2013. c) 63 percent of survey respondents took part in budget meetings in the previous 12 months, showing an improvement compared to 30 percent in 2013. d) 90 percent of citizens agreed that there is a GRM in place for complaints and disagreements. Fiduciary Probity – using a robust fiduciary system reaching from the federal level through regions to local administrations, with numerous mechanisms to maintain the strength of that system. Status: Capacity building improved PFM systems across all woredas. Key achievements include: (a) the roll out of the IBEX system in all woredas, which provided the financial management software backbone to automate the country’s woreda-level financial administration; (b) manuals on PFM aspects (cash management, performance audits, and procurement standard bidding documents) were developed and implemented in all woredas with training; and (c) the audit function was strengthened resulting in significantly increased coverage and timeliness of public audits, reducing the average backlog of 4 years at the start of the project to no backlog at present, with now covering all Federal level public bodies, up from 30 percent before the program. Predictability – mutual accountability for predictable resource flows for basic service delivery results. Status: The Medium-Term Expenditure Framework lays out the overall fiscal framework for basic services. Donor commitments were fully disbursed, with the exception of DfID. B. Efficiency of using existing Government systems PBS channels resources through existing Government systems. By making use of government systems, in particular the FBG, the PBS provides efficiency gains in the delivery of basic services and provides incentives to improve public financial management. Although difficult to quantify, this heightens the efficiency of resource transfers in several ways: a) Reduced transaction costs of setting up parallel systems for donor financing b) Use of existing infrastructure to transfer and account for funds at the local level Page 57 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) c) Capacity building efforts of PBS 3 focused on strengthening budgeting and expenditure management and creating stronger arrangements for transparency and accountability. This should increase the efficiency of fiscal transfers. d) A relatively small investment in capacity building increased the efficiency of a large flow of funds. A total of $US116.6 million in capacity building activities had significant impacts on public fiduciary management, transparency, citizen engagement and results measurement of $11.03 billion in basic service block grants, or a 1 percent investment. e) Because the capacity improvements are systemic, efficiency gains would have spill-over effects onto all resources used at the woreda level, for example including locally-generated resources and not solely federal fiscal transfers. While it is difficult to provide precise estimates of the impact of the capacity building activities, the direction of their effect is clear: Spending efficiency is improved through better capacity, more transparency, and greater accountability to citizens. C. Cost-effectiveness analysis Cost effectiveness was assessed during the implementation of PBS both by DfID and by the World Bank. DfID updated a Value for Money Study in 2015 that was based on previous analysis DfID carried out in PBS2.11 As part of its business case for PBS 3, DFID performed a benefit cost analysis of the local service delivery component. The study examined the potential benefits in each of the five sectors of the project along different dimensions. For example, in the health sector, it extrapolated monetized benefits from additional productive years due to increased life expectancy and decreased amount of time spent ill to determine an overall benefit in the health sector. In education, grade completion effects on lifetime earnings were used to quantify benefits. The impact on household consumption of access to roads was used to dimension benefits of roads. And in water, time savings and increased income were measured. These estimated rates of return on investments in service delivery identified an average rate of return for each of the five sectoral investment areas and then calculated a composite internal economic rate of return by weighting that sector’s share in the overall PBS portfolio of woreda-level spending. DFID’s Business Case uses cost benefit framework to estimate a benefit cost ratio for PBS (excluding the results enhancement fund) of 1.43 and an internal rate of return of 23.1 percent. And second, a Poverty and Social Impact Analysis (PSIA) was carried by the World Bank in 2014 aimed to assess the specific impact/ cost effectiveness of PBS 3 by using econometric modelling to assess the association of increased woreda level recurrent spending with the expansion of basic services and outcomes. The Bank study focused on the three main sectors of PBS 3 spending. Education, health, and agriculture account for 97 percent of woreda spending, which in turn constitutes 97 percent of PBS-financed basic service block grants. The findings by sector are presented in the table below. Sector Findings 11DFID Ethiopia. “Value for Money Assessment of the Promotion of Basic Services (PBS) Programme, Final Report:. February 2015. Page 58 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Education Both net enrollment rate and pupil-teacher ratio show a strong and significant relationship with woreda-level per capita recurrent expenditure, after controlling for the effects of rural/urban percentage and ethnicity. For every additional Ethiopian birr (ETB) per capita of woreda education spending, net enrollment rate increases by 0.20 percent. These results are all significant at the 1 percent level and robust to changes in specification. The results for pupil-teacher ratio are similar. For education, an increase of US$1 per capita in spending by each woreda is associated with a 3.7 percent increase in the net primary enrollment rate within that woreda. Similar results are seen for the pupil-teacher ratio. Health Increased health expenditures by woredas appear to improve the rates of Penta 3 vaccination, antenatal care, contraceptive use, and deliveries by skilled attendants. All of these results are significant, and all are robust to changes in specification. Every additional US$1 of per capita spending by the woredas on health is associated with a 7.5 percent increase in the contraceptive prevalence rate and an 12.4 percent increase in deliveries by skilled birth attendants (two interventions that can reduce maternal mortality dramatically), as well as a 4 percent increase in antenatal care (which can reduce infant and child mortality significantly). Agriculture The PBS Program supports agriculture by financing recurrent costs at the woreda level, mostly consisting of the costs of employing agricultural extension workers. For every additional US$1 per capita spent, for example, the probability that a field in a given zone will benefit from extension services increases by about 0.2 percent. Coefficients for the use of fertilizer and improved seed are smaller, but these practices remain positively and significantly associated with spending. Of the eight categories of crops grown in Ethiopia, five—cereals, vegetables, enset, fruit, and coffee, which represent about 85 percent of national agricultural production—show a positive and significant relationship with agricultural spending. The effect of woreda-level spending on agricultural extension workers is associated with higher yields for major crops, including cereals, vegetables, enset, coffee, and fruit. Spending on agricultural extension workers increases the probability that farmers, regardless of the size of their plots, will use improved farming techniques. Other indications of cost effectiveness include: a) Existing evidence in health indicate that Ethiopia is one of the few countries in SSA that has achieved a significant reduction in under-five mortality with a limited increase in expenditure per capita. It is also one of the few countries in SSA that have life expectancy higher than predicted, while per capita expenditure is lower than predicted, indicating an overall high efficiency in the health sector b) A PBS-funded Education Service Delivery Indicator (SDI) study assessed a small but representative sample of 222 rural schools and 125 urban schools. Teacher absenteeism was found to below at 12 percent (10.1 percent rural; 14.1 percent urban). This compares favorably with documented absence rates in other SSA countries. D. Social cost-benefit analysis Social cost benefit analysis includes consideration for the distribution of benefits, costs, and net benefits across population groups. Benefits that accrue disproportionately to the poor increase the social rate of return. The PSIA assessed this equity aspect on two levels: (a) the relative distribution of spending as pro-poor at the local level by sector, (b) the geographical distribution of resources by poverty level of the region/woreda. Woreda-level spending by sector concluded that: Page 59 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) a) For education, woreda-level spending appears to be pro-poor. The bottom 40 percent benefited from 56 percent of that expenditure, and the top quintile benefited from 13 percent. The incidence of per capita woreda-level spending on education was more than 2.5 times higher for the bottom quintile than for the top quintile. b) 63 percent of the health expenditure at the woreda level accrued to the bottom 40 percent, whereas 10 percent accrued to the top quintile. For someone in the bottom quintile, the benefit incidence of per capita woreda-level health spending was more than three times higher than it was for someone in the top quintile. a) In agriculture, woreda-level spending (primarily for agricultural extension workers) drives increases in output and the adoption of new, improved methods across all asset quintiles, although the magnitude was smaller for the bottom quintile, perhaps because of a lack of financing to purchase productivity-enhancing inputs. In addition to its general tendency to be pro-poor, the PSIA also noted that spending on basic services appears to be reaching females, especially spending on health and education. In fact, much of health spending promotes women’s access to services that have a strong impact on reducing maternal mortality, including contraception, antenatal care, and assisted deliveries. In looking at regional distribution of resources, the PSIA found that: a) In terms of basic service expenditure per capita, the current system broadly favors Ethiopia’s historically disadvantaged regions compared to the historically dominant ones. More than 50 percent of the woredas in Gambella and 30 percent in Beneshangul-Gemuz, which are the two most disadvantaged regions of the country, spend more than 110 percent of the national average on the basic service sectors. b) Spending also appears to favor some historically disadvantaged ethnic groups, in accordance with constitutional mandates. Five majority-Anyiwak woredas are noteworthy for receiving the most public resources of all woredas in the nation. The exception occurs among the Somali groups. Federal transfers to Somali Region appear to be fine, but the region transfers a much smaller share to the woredas—49 percent, compared to the national average of 73 percent (excluding Addis Ababa) E. Project-Level Efficiency Per ICR guidance, other aspects of design and implementation also contribute to efficient or inefficient use of resources: Actual vs Estimated Costs: As presented in Annex 3: Project Costs by component the variance between appraised versus final costs varied widely between component: Sub-Program A: Overall final costs of this component at US$ 11.0 billion were 178 percent of the appraised amount. This almost doubling of the Government’s federal fiscal transfers for recurrent costs in the basic service sectors compared to the estimate at appraisal showed the Government’s commitment to funding decentralized service delivery in the country. While many of the PDO indicators were surpassed, they were not almost double so this requires some explanation from an efficiency viewpoint. At appraisal, the project adopted the Government’s goals of access to and quality of basic services as set forth in the GTP. The funding needed to achieve each indicator was not based on a linear input–output model so it is not expected that achievement of specific PBS 3 PDO indicators would be directly correlated with levels of Block Grant transfers, for several reasons, including: Page 60 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) • Unit costs of service delivery vary greatly between the 5 basic service sectors. The ultimate mix of sectoral basic service spending is determined at the local level annually by over 900 woredas. It would not be possible to predict expenditure distribution ex-ante. • Ethiopia experienced an average annual inflation rate of 8.5 percent over the project period. In addition, there were teacher salary increases, necessary given the relatively low wages in the sector were creating difficulties attracting and retaining teachers. Current costs would therefore rise without a corresponding increase in access or quality. • Block grant transfers are taken globally and therefore contain service delivery elements that are not represented in the project PDOs. For example, the Government contribution to block grants in basic service sectors include expenditures on primary education grades 1-4, which continue to expand with population and enrollment rate growth. • And finally, unit costs of service delivery vary significantly across regions and population densities in Ethiopia making linear extrapolations of spending and outputs difficult. Rather than direct correlation between money spent per unit of service delivery output, the guiding tool has been the medium-term expenditure framework in Ethiopia which helps ensure that key services are adequately funded. Budget preparation is guided by the Macro-Economic and Fiscal Framework (MEFF) prepared by MOFED. THE MEFF forecasts government revenues and expenditures, its distribution between federal and regional levels, and the split pf federal expenditures between recurrent and capital expenditures on a three-year rolling basis. This sets the framework for the block grant transfers to regions and woredas. Sub-Program B: Overall final costs of this component at US$ 197 million were 59 percent of appraised amount. The variance was largely due to the cancellation of DfID funding of $78.2 million for the proposed Results Enhancement Fund. Other variations were relatively small, with some additional spending on Citizen’s Engagement activities which were prioritized by donors and funded almost exclusively through a grant-funded multi-donor trust fund. Here the activities to engage citizens at the woreda level and provide training required more time than expected, with the decision to deepen within program woredas (to kebele levels) prior to expanding to more woredas, leading to delays in launching ESAP 3. Impacts were extensive and better assured with the more intensive approach. Actual administrative costs: Because of the scale of PBS 3 and the multi-donor configuration, administrative costs were relatively low and shared among many agencies. For example, semi-annual joint supervision missions (JRIS missions) reduced transaction costs for the Government and allowed for sharing of technical personnel across funding agencies. Joint analytical work was also an important process efficiency. Time overruns: time overruns (project extensions) may impact efficiency, depending on the underlying reasons. In the case of PBS 3, The expected implementation period at appraisal was 5 years and 3 months. The actual closing date was extended =at the end of 2014 for one year to fully execute the capacity building items in Sub-Program B which had been expanded as committed to under the Management Response on the Inspection Panel investigation. There were some delays in the procurement of MIS equipment for sectoral ministries, in particular the Ethiopia Roads Authority, which represents a minor inefficiency given the extremely small share of project costs taken up by this activity. Page 61 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS The Government submitted minor corrections on a draft of the ICR. “While the ICR notes that the ERA MIS was not achieved during PBS 3 due to delays in procurement of the required computer hardware and software, the Government commented that “base line road data base on web- based application system was established in all Regions and Woredas. The road network system is operational and all data from woredas are collected and captured in the system. Except hardware purchase, the MIS is established”. Synopsis of the Government’s ICR: The Government of Ethiopia submitted its own Implementation Completion Report on June 5, 2019. The following is a synopsis of the Government’s report: PBS3 origins PBS3 needs to be understood in terms of its evolution from earlier phases of the programme, not least because key design features continued through all three phases. Before closure PBS3 had already been partially succeeded by the Enhancing Shared Prosperity through Equitable Services (ESPES) project, and continuing components of PBS3 have been incorporated into the next phase of ESPES. PBS3 management PBS3 funding is provided to the Federal Ministry of Finance through a World Bank Multi-Donor Trust Fund (MDTF). PBS implementation is facilitated by a Secretariat, located in the World Bank but supported by participating donors. Within the Ministry of Finance, PBS has been managed by the Channel One Programmes Coordinating Directorate (COPCD). Channel One coordinators at regional level were introduced during the transition from PBS2 to PBS3. The social accountability element of PBS (ESAP) is financed through a separate WB recipient executed trust fund. It is implemented by a consortium of civil society organisations, headed by a management agent. DPs contributing to PBS3 at its outset included the World Bank, the African Development Bank (AfDB), the UK’s Department for International Development (DFID), the European Union (EU), Austria, and Italy. In addition, the social accountability component was being supported by DFID, KfW (Germany), Irish Aid, and the EU. Adjustments during implementation Interacting factors led to some substantial adjustments of PBS3 during implementation. Adjustments are reflected in various documents. Some changes were non-radical (e.g. to adjust the results framework in line with GTP II), but the major changes can be understood as responses to: • the findings and recommendations of a WB Inspection Panel; • the withdrawal of DFID and a changing configuration of DP support; • the migration of PBS3 components to ESPES. Project results The agreed results indicators for PBS3 are naturally at the centre of any assessment of project performance. Based on indicators' performance against targets, Sub-Program A achieves the highest available rating of "highly satisfactory" at both Project Development Objective (PDO) and intermediate results levels. Sub-Program B is Page 62 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) also rated highly satisfactory at the PDO level, but the rating at intermediate results level is only "moderately satisfactory". This results in an overall efficacy rating for PBS3 of "Satisfactory". However, the results indicators are not by themselves a complete theory of change for PBS3. A full assessment needs to consider the strength of evidence for the causal assumptions that underpin its logical framework and also take account of intended project benefits that are not well captured in the results framework: Citizens' engagement: GoE has demonstrated, and DPs have acknowledged, its commitment to the strengthening of transparency and accountability as a means of strengthening the relevance and effectiveness of basic services delivered at local level. PFM and procurement: This component has been a prime example of PBS' ability to support the strengthening of the fiduciary systems that ensure proper and effective use of the FBG from woreda level upwards. Managing for results: The M4R component had three main elements: reporting on PBS3 results, strengthening data systems in the basic services sectors, and supporting other analytical work. The results reporting process is considered to have worked well, with results regularly reported for each JRIS, and the results framework appropriately updated as new data or better indicators became available. The work on data quality is especially noteworthy. As a multi-sector programme PBS3 was well placed to work with the Central Statistical Authority (CSA) on systematic data quality analyses and improvement plans for each of the basic services sectors, strengthening the validity and coverage of available data. Additional observations on performance PBS3 was a complex programme, with the bulk of funding channeled through sub-program A, but with the complementary components under sub-program B requiring much more intensive management and supervision. It is unsurprising that some components encountered difficulties and delays but sub-program B nevertheless made substantial progress. Factors affecting performance and outcomes Global factors: PBS3 was designed and implemented during a challenging period for international aid. There were positive aspects, including the transition from the MDGs to the broader and more ambitious SDGs. But many DPs were experiencing increased pressure to justify aid, both politically and in terms of its effectiveness. Focusing on results has been part of the aid effectiveness agenda since the Paris Declaration, and is a worthy principle. In practice, however, concern to demonstrate the results of aid has often led DPs to retreat from joint programmes where each partner shares in overall results, towards more fragmented approaches that make it easier to claim specific outcomes as a direct result of the DP's own inputs. Ethiopia factors: Strong economic growth throughout the period was a strongly positive factor which facilitated rapid growth in GoE funding of basic services. At the same time the Government had to cope with major national and regional droughts from 2016. Service delivery was also affected by the unrest associated with two states of emergency during the PBS3 period. Weaknesses in local capacity have consistently been a challenge to the implementation of PBS3, but they are also part of its raison d'être. As noted earlier, PBS3 has had significant success addressing weaknesses in institutional capacity. In this regard, it is relevant that support to basic services via the FBG has been shown to reduce disparities between more and less advanced regions and woredas. Borrower's performance Through its GTPs and sector strategies, GoE has provided a strong and appropriate policy framework for poverty reduction and service delivery, made more practical by exceptionally strong macroeconomic performance (which inter alia enabled GoE to complement PBS3 with large-scale capital finance though its MDG fund). GoE Page 63 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) fully supported the complete PBS3 strategy, including the commitment to transparency and citizens' engagement. It led and fully engaged with the coordination structures surrounding PBS3 and its successor projects. Assessment of WB and DP performance GoE acknowledges the WB's pivotal role in supporting PBS3. Longstanding features of the design, including the role of SDS and the fact that TTLs have been resident in Ethiopia are very helpful. Overall, PBS3 has been very successful, and all partners deserve credit for their share in its success. GoE recognises that DPs have had to cope with the external pressures already noted. Nevertheless, the adaptations PBS3 has had to make represent an overall decline in aid quality, and for that reason GoE would assess their performance as only satisfactory. Lessons learned Hindsight on risk assessments: PBS was originally designed to work as much as possible through Government systems. Even though this was correct and in line with the WB’s policies and guidance, in retrospect it was unfortunate. It would be better for programmes where reputational risks are high to err on the side of more reviews and safeguards than fewer. Appropriateness of the PBS approach: The PBS experience (since 2006) has demonstrated that, where a government is committed to poverty reduction and the expansion of effective basic services, providing additional financial support and working through government systems can be a highly effective and efficient way of delivering aid. Linking the FBG support to a supporting programme that addresses PFM, accountability and institutional capacity can be highly effective. Challenges of cross-sector working: GoE agrees with the WB ICR that working across sectors in a federal system can be difficult. A programme like PBS has major external benefits in strengthening system on which all the basic services sectors depend. But it can be difficult to gain full engagement from sector ministries (and, indeed, sector specialists within DP organisations) because the financial benefits of the FBG do not flow directly to the sector ministries concerned. Future operation of the programme Although technically PBS3 has closed, ESPES-AF has been reformulated as its continuation. ESPES had, in effect, taken over financial aid component of PBS from 2015. Under ESPES-AF, the supporting components of PBS3 will continue under the ESPES umbrella. Similarly, the supervision arrangements that developed under PBS are set to continue, including the roles for SDS and BSDG. The same applies to GoE management arrangements, including the role of COPCD. Page 64 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Page 65 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) ANNEX 6. SUPPORTING DOCUMENTS Project documents: BDS Center for Development Research. “Impact Assessment on Financial Transparency and Accountability (FTA) Implementation”, August 2017. DfID/ITAD. “Value for Money Assessment of the Promotion of Basic Services (PBS) Programme, February 2015. Edjeta, Buli. “Quick Assessment of the Effectiveness of the Grievance Redress Mechanism in Ethiopia”, November 2018 EnDev. Environmental and Social Impact Assessment (ESIA) Capacity Building for Local Governments in Ethiopia – Final Report. December 2014. Government of the Federal Democratic Republic of Ethiopia. Promoting Basic Services Program – Phase III. Program Operational Manual (POM). Revised March 2014. Inspection Panel. Ethiopia: Promoting Basic Services Phase III Project (P128891), Investigation Report Nov 21, 2014, Report No. 91854-ET IPE Global. “Promoting Basic Services Phase III(PBS3) Project in Ethiopia - Process Evaluation”, March 2016. Khan, Q., Faguet, J-P., and Ambel, A. “Blending Top-Down Federalism with Bottom-Up Engagement to Reduce Inequality in Ethiopia” World Development Vol. 96, pp. 326–342, 2017. Khan, Q., Faguet, J-P., Gaukler, C., and Mekasha, W. “Improving Basic Services for the Bottom Forty Percent”, World Bank. 2014. Mesfin Temesgen. Public Financial Management - Institutionalized Training (PFMIT) Assessment, May 2018 Ministry of Finance and Economic Development (ED) Promoting Basic Services Program Phase 3, Enhanced Social Assessment and Consultation – Final Report”, December 2014. Progress Report No.1 on the Implementation of Management’s Action Plan in Response to the Inspection Panel Investigation Report (report No. 91854-ET) on the Ethiopia Promoting Basic Services Phase III Project (Credit No 51620), February 5, 2016. Progress Report No. 2 on the Implementation of Management’s Action Plan in Response to the Inspection Panel Investigation Report (report No. 91854-ET) on the Ethiopia Promoting Basic Services Phase III Project (Credit No 51620), January 17, 2017. Progress Report No. 3 on the Implementation of Management’s Action Plan in Response to the Inspection Panel Investigation Report (report No. 91854-ET) on the Ethiopia Promoting Basic Services Phase III Project (Credit No 51620), January 26, 2018. Project Appraisal Document, Ethiopia: Promoting Basic Services Phase III Project (PBS 3), August 29, 2012, Africa Region, Report No: 69689-ET Project Appraisal Document, Ethiopia: Protection of Basic Service s Project Phase II (PBS 2), April 22, 2009, Africa Region, Report No: 45186-ET Project Appraisal Document, Ethiopia: Program-for-Results Enhancing Shared Prosperity through Equitable Services (ESPES), August 19, 2015, Africa Region, Report No. 98687-ET Page 66 of 67 The World Bank Ethiopia Promoting Basic Services Program Phase III Project (P128891) Project Appraisal Document, Ethiopia: Additional Financing for Program-for-Results Enhancing Shared Prosperity through Equitable Services (ESPES-AF), August 22, 2017, Africa Region, Report No: 115228-ET Restructuring Paper, Ethiopia: Promoting Basic Services Phase III Project (PBS 3), December 13, 2013, Africa Region, Report No: RES13069 Restructuring Paper, Ethiopia: Promoting Basic Services Phase III Project (PBS 3), July 16, 2013, Africa Region, Report No: RES11190 Restructuring Paper, Ethiopia: Promoting Basic Services Phase III Project (PBS 3), July 20, 2015 Africa Region Report No: 98461 Restructuring Paper, Ethiopia: Promoting Basic Services Phase III Project (PBS 3), October 2018, Africa Region REPORT NO.: RES34259 SPA INFOSUV East Africa Ltd. Independent Procurement Audit, Consolidated Final Report, June 30, 2016 World Bank. Ethiopia Social Accountability Program Phase 2 (ESAP2) Impact Evaluation Report, October 2018 Asmelash Haile Tsegay (DECSU) with inputs from Fitsum Zewdu Mulugeta (GED01) and Laura Campbell (GSP01). Other reports and relevant documents: Central Statistical Agency, Demographic and Health Survey, 2011, published March 2012. Central Statistical Agency, Demographic and Health Survey, 2016, published July 2017. Government of the Federal Democratic Republic of Ethiopia. Ministry of Finance and Economic Development. “ Growth and Transformation Plan I (2010/11-2014/15)”. November 2010. Government of the Federal Democratic Republic of Ethiopia. Ministry of Finance and Economic Development. “ Growth and Transformation Plan II (2015/16-2019/20)”. May 2016. IDA, IFC, MIGA Country Partnership Strategy for the Federal Democratic Republic of Ethiopia, August 29, 2012 Ethiopia Country Management Unit Africa Region Report No. 71884-ET IDA, IFC, MIGA Country Partnership Framework for the Federal Democratic Republic of Ethiopia for the period FY18-22, May 22, 2017, Ethiopia Country Management Unit Africa Report No. 115135-ET Implementation Completion and Results Report. Ethiopia: Protection of Basic Services (PBS) Phase II Project, December 20, 2013, Africa Region. Report No: ICR2574 International Monetary Fund. IMF Country Report No. 18/354 the Federal Democratic Republic of Ethiopia, 2018 Article IV Consultation, December 2018. Ministry of Education, Educational Statistics Annual Abstract, 2008 E.C. (2015/16) World Bank. Ethiopia: Public Expenditure Review, Report No: ACS14541. April 2016. Also consulted in the preparation of this ICR were all aid-memoires of JRIS Missions, Implementation Status Reports, financial and procurement reviews, Mid Term Review, and other internal reports. Page 67 of 67