Document of The World Bank Report No: ICR2897 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-75620) ON A LOAN IN THE AMOUNT OF US4.0 MILLION TO THE HASHEMITE KINGDOM OF JORDAN FOR A SOCIAL PROTECTION ENHANCEMENT PROJECT February 28, 2014 Human Development Sector Department Middle East and North Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective 11/30/2013) Currency Unit = Jordanian Dinar JOD 1.00 = US$1.41 US$1.00 = JOD 0.708 FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy CCSS Coordination Commission of Social Solidarity CPS Country Partnership Strategy DOS Department of Statistics EPP Enhanced Productivity Program FDI Foreign Direct Investment FM Financial Management HIES Household Income and Expenditure Survey ICT Information and Communication Technology ISSC Integrated Social Service Center ISTD Income and Sales Tax Department LAN Local Area Network (refers to institutional internet/ intranet connectivity) MIS Management Information System MOPIC Ministry of Planning and International Cooperation MoSD Ministry of Social Development MTR Mid-term Review NAF National Aid Fund NUR National Unified Registry and Outreach Program for Targeting Social Assistance Project (2013) PCU Project Coordination Unit PDA Portable Digital Assistant (Software) PISU Project Implementation and Support Unit RIC Research Information Center SIL Specific Investment Loan SPEP Social Protection Enhancement Project WAN Wide-Area Network (refers to a system for internet/intranet connection) Vice President Inger Andersen Country Director Ferid Belhaj Sector Manager Yasser El-Gamamal Project Team Leader Setareh Razmara ICR Team Leader Setareh Razmara ICR Authors Amr S. Moubarak / Suzana N. de Campos Abbott ii Hashemite Kingdom of Jordan Social Protection Enhancement Project CONTENTS DATA SHEET.............................................................................................................................................. IV 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN ............................................................... 1 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES .............................................................. 6 3. ASSESSMENT OF OUTCOMES ................................................................................................................. 16 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME ............................................................................ 23 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE...................................................................... 23 6. LESSONS LEARNED ............................................................................................................................... 26 7. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS ........................ 28 ANNEX 1. PROJECT COSTS AND FINANCING ............................................................................................. 29 ANNEX 2. FULL DESCRIPTION OF PROJECT’S ORIGINAL COMPONENTS .................................................... 30 ANNEX 3. COMPARISON TABLE OF ORIGINAL AND RESTRUCTURED PROJECT COMPONENTS, FINANCING, AND OUTPUT ............................................................................................................................................ 35 ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES ........................... 38 ANNEX 5. BENEFICIARY SURVEY RESULTS .............................................................................................. 40 ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS .................................................................. 41 ANNEX 7. SUMMARY OF BORROWER'S ICR .............................................................................................. 42 ANNEX 8. COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS.................................. 52 ANNEX 9. LIST OF SUPPORTING DOCUMENTS ........................................................................................... 53 MAP ........................................................................................................................................................ 54 iii Data Sheet A. Basic Information Jordan Social Country: Jordan Project Name: Protection Enhancement Project Project ID: P100546 L/C/TF Number(s): IBRD-75620 ICR Date: 02/28/2014 ICR Type: Core ICR HASHEMITE Lending Instrument: SIL Borrower: KINGDOM OF JORDAN Original Total USD 4.00M Disbursed Amount: USD 0.968M Commitment: Revised Amount after USD 3.18M restructuring: Environmental Category: C Implementing Agencies: Ministry of Social Development Co-financiers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 11/22/2006 Effectiveness: 12/01/2008 12/01/2008 Appraisal: 03/12/2008 Restructuring(s): 02/27/2013 Approval: 06/03/2008 Mid-term Review: 12/05/2011 02/06/2012 Closing: 08/31/2013 08/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Highly Unsatisfactory Risk to Development Outcome: High Bank Performance: Unsatisfactory Borrower Performance: Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Implementing Quality of Supervision: Highly Unsatisfactory Unsatisfactory Agency/Agencies: Overall Bank Unsatisfactory Overall Borrower Unsatisfactory iv Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 75 75 Other social services 25 25 Theme Code (as % of total Bank financing) Administrative and civil service reform 33 33 Social safety nets 67 67 E. Bank Staff Positions At ICR At Approval Vice President: Inger Andersen Daniela Gressani Country Director: Ferid Belhaj Hedi Larbi Enis Baris (acting SD)/Yasser Michal J. Rutkowski/ Roberta Sector Director/ Manager Aabdel-Aleem Awny El-Gammal Gatti Project Team Leader: Lire Ersado Aleksandra Posarac ICR Team Leader: Setareh Razmara Amr S. Moubarak / Suzana Nagele ICR Primary Author: de Campos Abbott F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) Project development objective is to improve the management and operations of the cash social assistance programs and to improve the access to and quality of social care services. v Revised Project Development Objectives (as approved by original approving authority) N/A (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Coverage (C) of the target poor population by NAF programs reaches at least 70 Indicator 1 : percent; error of inclusion (EI) of the non-poor does not exceed 30 percent At Year 5: Value 70% (C) / 30% 25% (C) / quantitative or 19% (C) /80% (EI) 60%(C)/30%( (EI) Unidentified (EI) Qualitative) EI) Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 There is inconsistency between the PDO indicator and the Intermediate Indicator # 7 (see below). The objective was to ensure that 70% of NAF beneficiaries are poor compared to 19% at the baseline. The measure was reported and communicated at Supervision and Restructuring Comments in very unclear terms. In 2012, Bank staff estimates showed that two-thirds of (incl. % benefits went to the bottom two deciles of the population, making NAF program achievement) one of the best targeted social assistance programs in the region. However, at restructuring the reported baseline was 25% (C), and 75% (EI) and was not updated based on the latest analytical work. Further, the formally revised target value does not add up to 100% -- 60% (C), and 30% (EI), and at closing the original coverage rate of 25% was reported. Fifty percent increase in the number of people benefitting from the MoSD social Indicator 2 : work and care services in localities served by pilot centers 100% of Pilot Value TBD (to be added at pilot At Year 5: Centers are quantitative or None preparation) 50% increase fully Qualitative) functional Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments Since baseline was not identified at pilot preparation, it is unclear how the (incl. % increase was measured and based on which baseline value. No value reported at achievement) closing. Indicator 3 : Improved user satisfaction with the NAF and MoSD services Baseline survey to be Value conducted during the first quantitative or Satisfactory (Not revised) None year of the loan Qualitative) effectiveness Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments No baseline survey was conducted. No definition was provided for “satisfactory” (incl. % rating. No value reported at closing. achievement) vi (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years The MoSD adopts a Strategic Development Plan and a Human Resources Indicator 1 : Development Strategy At Year 2: Both Value documents (quantitative None Dropped N/A approved by or Qualitative) MoSD Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments Consultant reports were submitted to MoSD from contracted consultants. Project (incl. % did not adopt a Strategic Development Plan nor a Human Resources Strategy achievement) Indicator 2 : The M&E Unit of the MoSD issues its annual report Value At Year 5: (quantitative None Second M&E Dropped N/A or Qualitative) report issued Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments Several delays in hiring an M&E Specialist in the PISU. M&E Unit at the MoSD (incl. % was not established. achievement) At least two-thirds of the MoSD staff completes training envisaged in the Human Indicator 3 : Resources Development Strategy Value At Year 5: Training sporadic and (quantitative 66% of staff Dropped N/A not planned or Qualitative) receives training Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments No Human Resources Development Strategy adopted. No trainings were held on (incl. % HR Strategy. achievement) Standard packages for statistical data entry, cleaning, imputation, management Indicator 4 : and analysis introduced and staff trained in using them At Year 5: Value “Staff Routinely (quantitative Relatively Weak None uses provided or Qualitative) packages” Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments (incl. % achievement) NAF adopts an Operations Manual governing its business processes and Indicator 5 : performance Value At Year 1: (quantitative None None “completed” or Qualitative) vii Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments (incl. % No record of an operational manual at supervision. achievement) NAF HQ and MoSD/NAF local branches adequately staffed and staff skills Indicator 6 : improved Value At Year 5: (quantitative None 100% “adequately Dropped N/A or Qualitative) staffed” Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments (incl. % No baseline collected. No trainings held. achievement) The number of households included in the NAF data base reaches at least 80 Indicator 7 : percent of poor households Value At Year 5: (quantitative None At Year 5: None 80% or Qualitative) 30% Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments (incl. % Original project target is inconsistent with PDO indictor 1. See above. achievement) Indicator 8 : At least 50 percent of NAF beneficiaries use smart cards to collect assistance Value At Year 5: (quantitative None Dropped N/A 50% or Qualitative) Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments No record on development or use of smart cards for NAF payments at (incl. % effectiveness or in ISRs achievement) At least 60 percent of key standards of care adopted and staff trained to Indicator 9 : implement them Value At Year 5: At Year 4: (quantitative None None 60% 60% or Qualitative) Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments (incl. % Standards of care developed but not adopted. No implementation on record. achievement) Indicator 10 : Three residential institutions transformed to resemble a family like setting Value At Year 5: (quantitative None None 3 or Qualitative) Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments (incl. % All three centers received some funding for rehabilitation and furniture. achievement) Indicator 11 : Three pilot integrated social work and care centers fully functional and viii adequately funded by the Government Value At Year 5: (quantitative None 1 None 3 or Qualitative) Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments None of the integrated social work and care centers were fully functional. (incl. % Standards of care developed but were not adopted for implementation. achievement) Three pilot resources and information centers for persons with disability fully Indicator 12 : operational and adequately funded by the Government. Value At Year 5: (quantitative None 1 None 3 or Qualitative) Date achieved 06/03/2008 06/04/2008 02/27/2013 08/31/2013 Comments During site-visit (September 2013), one center was rehabilitated center for (incl. % persons with disability. It was financed by the project, and was deemed out of achievement) compliance with building code standards and subsequently abandoned in 2013. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 11/05/2008 Moderately Satisfactory Moderately Satisfactory 0.00 2 06/18/2009 Moderately Satisfactory Moderately Satisfactory 0.00 3 12/29/2009 Moderately Satisfactory Moderately Satisfactory 0.00 4 05/26/2010 Satisfactory Satisfactory 0.30 Moderately 5 12/26/2010 Moderately Satisfactory 0.50 Unsatisfactory Moderately Moderately 6 10/26/2011 0.61 Unsatisfactory Unsatisfactory Moderately Moderately 7 07/11/2012 0.85 Unsatisfactory Unsatisfactory Moderately Moderately 8 07/21/2013 0.99 Unsatisfactory Unsatisfactory ix H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions See attached Annex 3 for table 02/27/2013 N MU MU 0.85 summary of key changes made I. Disbursement Profile x 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Background. At the time of Appraisal of the Social Protection Enhancement Project (SPEP, the Project) in 2007, the Hashemite Kingdom of Jordan had been achieving above-average development outcomes in terms of economic growth and human development, compared to other lower middle-income countries. Despite a poor resources base (with limited agricultural land, no oil resources, and scare water resources), regional uncertainties and shocks stemming from a steep increase in oil prices, Jordan had achieved a remarkable decline in poverty, and good economic performance. Between 1997 and 2002-2003, poverty declined from 21.3 to 14.2 percent and the annual GDP growth rate reached 5-7 percent. The broad-based economic growth was supported by comprehensive structural reforms, increased foreign direct investments from oil-rich Gulf countries and an expansion of manufacturing and exports. Consistent levels of spending on, and a continued commitment to, human development, in particular education and health, placed Jordan’s human development outcomes above those of other lower middle-income countries globally. With a young, urbanized population of 5.7 million inhabitants, and a declining but still high population growth rate of 2.6 percent per year, the Government of Jordan (GOJ) was facing increasing demands for further expansion of social services. 2. Jordan continued to face challenges, however, including a tight fiscal situation (requiring fiscal discipline). Rising oil prices were putting pressure on inflation and the state budget given its subsidies of fuel products. In the labor market, unemployment was hovering around 13-15 percent, particularly among the youth. The country continued to experience geographically uneven economic development, and a large influx of refugees from neighboring countries, especially from Iraq increased demand on infrastructure and housing. The majority of jobs created through Foreign Direct Investment (FDI) demanded low skills, and were being filled by the huge influx of immigrant workers. Jordan had yet to follow a growth path that would create higher productivity jobs. Moreover, there was a concern about maintaining previous economic gains and progress in poverty reduction. Particularly, there was a need to reduce disparities between the poor and non-poor in education and health and to support and protect the poor through affordable, sustainable, efficient and effective social safety net interventions—a need that had become all the more urgent with increases in oil prices and the GOJ’s gradual removal of the fuel subsidy. 3. Sector Background. Broadly matching the country’s poverty and vulnerability profile, Jordan’s social safety net system was comprised of most of the programs found in relatively well-developed and diversified systems. The system included: (i) income support to poor and vulnerable families, implemented by two key institutions, the National Aid Fund (NAF) and the Zakat Fund; (ii) social care services to vulnerable groups such as people with disabilities, children, youth, families and women in distress, and others; and (iii) economic empowerment interventions through skills and asset development, the most important of which was the Enhanced Productivity Program (EPP) 1 hosted by the Ministry of Planning and International Cooperation (MOPIC). A multitude of public and private providers were involved in safety net program delivery. Total public spending on safety nets was estimated at more than one percent of GDP (excluding subsidies), with about one half spent through the NAF, and more than one quarter on the EPP. The total number of beneficiaries was tentatively estimated at about 8-10 percent of the population and the coverage of the poor (at baseline) at project approval was 19 percent. 4. Several issues had emerged with respect to the performance of the GOJ’s safety net programs. First, the poverty impact of cash transfers had been found empirically to be modest; a major segment of the poor were not reached and the leakage of resources to the non-poor was substantial. Second, the demand for social care services remained prevalent but largely unmet. Third, anecdotal evidence suggested that programs aimed at creating and enhancing economic opportunities for the poor tended to be captured by the non-poor. Analytical support provided by the World Bank had pointed to several factors that impacted the GOJ’s safety net in terms of its efficiency and effectiveness, including: (i) an incomplete and somewhat incoherent policy and institutional framework; (ii) overlapping institutional mandates; (iii) fragmentation and duplication of programs; (iv) inefficient and ineffective targeting methods (some programs, such as community social work and care services, were largely missing); (v) discretionary decision making and weak technical, institutional and human resources capacity resulting in inefficient and ineffective implementation; (vi) weak monitoring and evaluation mechanisms; and (vii) modest use of a management information system (MIS) in managing programs. 5. Government Strategy. The GOJ had adopted a National Poverty Agenda (2006), an overarching poverty reduction strategy aimed at increasing the welfare of Jordanians, reducing poverty and creating jobs through sustained growth while achieving fiscal sustainability. The reform of the social safety net system was one of the National Agenda’s main priorities, articulated among its three pillars: (i) provision of case assistance to increase and smooth consumption of the poor; (ii) the establishment of the Coordination Commission for Social Solidarity (CCSS) to ensure coordination among various stakeholders involved in poverty reduction efforts; and (iii) the development of community-based social care services and economic empowerment programs. 6. The renewal of the NAF was at the heart of the National Agenda. The NAF was established in 1986 to provide cash social assistance to the poor. Its restructuring had taken on added urgency in 2005-2006 when Jordan lost its oil grants and had to raise fuel prices significantly. At the time of appraisal, the GOJ had, in response, eliminated the oil subsidy, a decision that was expected to contribute to increased inflation, thus affecting the welfare of many, especially those below the poverty line. 7. Through broad participatory processes, the GOJ had also adopted several strategies targeting specific vulnerable groups. These included: (i) a National Plan of Action for Children 2004-2015, approved in 2004; (ii) a National Strategy for Persons with Disabilities, approved in 2007; (iii) a National Strategy for Youth, approved in 2 2003; (iv) a National Strategy for Women, approved in 2004; and (v) a National Strategy for Family Protection, approved in 2005. While addressing specific groups, these strategies had several common elements, especially in that they all called for the development of community-based family focused social work, social care and rehabilitation services, tailored to the specific needs of different vulnerable groups. 8. Rationale for Bank Involvement. The rationale for Bank involvement was strong. As background for the GOJ’s National Agenda, the Bank had prepared several studies, including a Poverty Assessment (2004) and “Jordan: A Note on Strategy for Modernization of the Social Safety Nets” (2007). The Bank brought knowledge and experience in the development and reform of safety nets from countries across the world. It was playing a catalyst role in building consensus and support for safety net reform by providing cutting-edge technical advice on policy design. Overall, the Social Protection Enhancement Project (SPEP) was a clear response to a GOJ priority and the project’s design (including components) mirrored the diagnosis of issues and the GOJ’s preferred policy directions. In addition, it was fully consistent with the Bank’s 2006 Country Assistance Strategy (CAS). Responding to the GOJ’s National Agenda, the CAS included reforming social protection and expanding inclusion as one of its four pillars. Specifically, the CAS proposed analytical and financial assistance in support of: (i) a comprehensive social protection and safety net strategy; and (ii) the implementing safety net reform and developing robust targeting and monitoring and evaluation systems. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 9. The Project’s Development Objective (PDO) was to improve the management and operations of the cash social assistance programs and to improve access to and quality of social care services. The Project’s progress towards accomplishment of its PDO was to be measured by progress towards the following Key Indicators:1  Coverage of the target poor population by NAF programs reaches at least 70 percent; error of inclusion does not exceed 30 percent;  50 percent increase in the number of people benefiting from social work, and care services in pilot districts; and  Improved user satisfaction with the NAF and Ministry of Social Development (MOSD) services. 10. In addition, the Project’s Results Framework included 12 Intermediate Outcome Indicators, by component (Annex 3). 1 The wording of the Outcome Indicators differs slightly between that listed on page 4 of the PAD and the Results Framework on pages 25-26 of the PAD. The Indicators provided below are from the Project’s Results Framework. 3 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 11. A Level 2 Project Restructuring was approved on February 27, 2013. The restructuring did not modify the PDO, but did revise the Project’s Key Outcome and Intermediate Outcome Indicators (Annex 3). The Loan’s August 31, 2013 Closing Date was not extended as part of the restructuring. Overall, the restructuring was not intended to be substantial given the approaching Closing Date and the weak implementation capacity. 12. The Project had been facing several bottlenecks that were limiting its progress and resulting in Moderately Unsatisfactory ratings for both Implementation Progress (since December 2010) and the Achievement of Development Objectives (since October 2011). The factors that led to this less than satisfactory performance are described in detail in Section 2. In order to address the difficulties the Project was facing, the GOJ and the World Bank agreed to restructure the Project to focus on activities that were progressing well and/or those with strong ownership and high probability of successful implementation. In addition, the share of activities financed by the World Bank loan increased to 100 percent in order to avoid delays in procurement, payment and transaction processing, and availability of counterpart funding. In the context of the restructuring, the GOJ agreed to put in place several enhancements to its financial management (FM) and procurement functions intended to expedite implementation. 1.4 Main Beneficiaries 13. As identified in the Project Appraisal Document (PAD), target groups were very poor households and vulnerable groups of the population, including persons with disabilities, ‘dysfunctional’ families, youth and women at risk and others who count on provided assistance.. In addition, through its institutional strengthening objective, the Project’s beneficiaries also included the MOSD, the NAF, the CCSS, and residential care institutions, the integrated social service centers, and resource and information centers that would improve the effectiveness and efficiency of social service delivery, thereby multiplying the number of ultimate beneficiaries benefiting from the Project. 1.5 Original Components (as approved) 14. The US$10.9 million Social Protection Enhancement Project (SPEP) was financed by a US$4.0 million Specific Investment Loan (SIL) approved by the World Bank’s Board on June 3, 2008. The Government provided counterpart funding for the remaining amount. The Project comprised four components. Annex 2 provides a comprehensive description of all four original components of the project as well as relevant sub-components and envisioned activities. 15. Component A: Institutional Development and Capacity Building for Social Safety Net Policies Formulation and Implementation (estimated Total Cost US$3.3 million, World Bank financing US$1.2 million). This component aimed at 4 strengthening and developing institutions involved in analysis, design, coordination, monitoring and evaluation and implementation of social safety net policies and programs, so that they perform their function and deliver services efficiently and effectively. More specifically, this component included three sub-components: (i) capacity building of the CCSS; (ii) strengthening institutional and human resource capacity of the MOSD; and (iii) strengthening the capacity of the Department of Statistics (DOS). 16. Component B: Renewal of the National Aid Fund (NAF) (estimated Total Cost US$2.9 million, World Bank financing US$1.1 million). This component aimed to improve the operations and management, coverage of the targeted poor population and targeting efficiency and effectiveness of the cash assistance through the NAF. The objectives were to establish an automated, and up-to-date database on poor and vulnerable population in Jordan; to improve skills of NAF staff to perform their tasks; and strengthen the capacity of NAF to monitor delivery of assistance and carry out knowledge-based adjustments in both targeting and implementation arrangements. More specifically this component included the following three sub-components: establishment of an MIS for a database on poor and vulnerable in NAF; strengthening NAF technical and administrative capacities for benefit delivery; and improving the targeting mechanisms of NAF. 17. Component C: Improving Access to and Quality of Social Work and Care Services (estimated Total Cost US$3.8 million, World Bank financing US$1.4 million). This component aimed to improve access to and quality of services to vulnerable groups such as persons with disabilities, women and children in difficult circumstances, dysfunctional families, youth with behavioral problems, youth in conflict with law, etc. It would support the MOSD in implementing a number of actions envisaged in the GOJ’s strategies targeting vulnerable groups (Section 1.1). Under this component, MOSD would develop core standards of social care, adopt an action plan for their implementation, and launch its implementation. The capacity of service providers to provide better care and rehabilitation would be strengthened, family centered, community based care and rehabilitation model would be piloted, service options available to vulnerable populations in six localities would be increased to include family counseling and a referral system, and monitoring and evaluation would be introduced. More specifically, this component included the following sub-components: (a) developing standards of service and monitoring mechanism; (b) upgrading the model of care in three residential care institutions; (c) piloting integrated social service centers; (d) piloting resource and information for persons with disabilities; and (e) piloting for MOSD social innovations programs. 18. Component D: Project Management (estimated Total Cost US$0.8 million, World Bank financing US$0.3 million). This component aimed to strengthen the capacity of the MOSD and other institutions involved in project implementation (NAF, CCSS and DOS) to efficiently and effectively implement the Project, while adhering to the Bank’s fiduciary requirements, particularly with respect to procurement and financial management. It was anticipated that technical assistance would be delivered in close 5 collaboration with relevant departments/individuals, so that the transfer of knowledge and hands-on training could take place immediately. 1.6 Revised Components 19. Through the Project Restructuring, approved on February 27, 2013, the four components were not revised. However, activities under each of the components that were stalled or not progressing were eliminated from their respective components. The following activities were cancelled in the restructuring (categorized by component): Component A: Institutional Development and Capacity Building  MOSD activities, including formulation of a strategic development plan, human resource development, monitoring and evaluation, and establishment of an MIS, and  Activities not yet carried out that were to be implemented by the DOS pertaining to advanced poverty analysis. Component B: National Aid Fund Renewal  Activities not yet carried out (planning the mechanism, collecting household information, carrying out the NAF survey, etc.) pertaining to implementing a more efficient system of targeting the poor. Component C: Improving Access and Quality of Social Welfare and Care Services  Establishment of two (out of an original three) Integrated Social Service Centers (ISSC)  Establishment of two (out of an original three) MOSD Resource Information Centers (RIC)  Piloting of the Social Innovation Fund Component D: Project Management  Financing for a Financial Officer, Project Assistant and Secretary in the PISU 1.7 Other significant changes 20. The Project Restructuring (Section 1.3) made the following additional changes: (i) revised the project activities and the relevant targets; (ii) revised the financing arrangements to include the reallocation of loan proceeds and increasing the Bank’s disbursement to 100 percent; (iii) cancelled US$831,797 from the total World Bank loan of US$4.0 million, bringing the total loan amount to US$3.18 million; and (iv) modified the Project’s Results Framework. The changes introduced in the Project Restructuring are presented in Annex 3. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 21. Background Analysis. Project design and preparation were based on previous extensive analytical work (Section 1.1) and were responsive to the Government’s policy priorities, as articulated in its National Agenda (2006), and as part of a joint strategy 6 developed by the MoSD and the World Bank for the enhancement of the social protection sector in Jordan. The Project’s design was clear and relevant. It took into consideration lessons of earlier operations, in general, and of countries that had successfully implemented cash social assistance programs and integrated social care service centers and resource and information centers for persons with disabilities, in particular. Bringing knowledge and experience in supporting social safety net reform from around the world, the World Bank had been supporting Jordan in building consensus by providing technical advice on policy design, targeting methods, monitoring and evaluation systems and implementation. Project activities such as building the MoSD’s monitoring and evaluation systems, the creation of a dynamic Management Information Systems (MIS), and the adoption of better targeting approaches to cash transfers by the NAF were based on the recommendations of the modernization strategy. 22. Government Commitment. The Government’s commitment at entry was clear. The MOSD worked closely with the World Bank on a number of trust funded technical assistance and sector work activities (including a Policy and Human Resource Development Grant funded by Japan), and in designing a project which responded to its policy priorities. World Bank engagement during preparation had resulted in a strong dialogue through the provision of technical assistance to the NAF. However, the first Implementation Status and Results (ISR) Report highlighted that “political support for and commitment to the NAF reform is yet to be clearly formulated and expressed [by MOSD].” 2 23. Project Design. The Project’s design was ambitious, in terms of the number of components and activities, all of which were well detailed and described in the PAD. Moreover, the Project’s design did not sufficiently take into account the feasibility of the implementation of a set of complex activities. The design was over-optimistic given the multiple objectives and actors, weak technical capacity of the implementation agency, and fragmented level of coordination among different levels of Government. Moreover, given the evident difficulties to coordinate institutional arrangements among agency prior to implementation, the PDO and intermediate indicators were overly optimistic. 24. The Project encompassed a mix of activities aimed at enhancing the processes, governance, and institutional capacity of the MOSD, the NAF, CCSS and DOS, and of increasing the efficiency of their operations. These interventions would have a medium- to long-term impact on Jordan’s social protection sector, while at the same time incorporating activities that would directly have an impact on end-user services (ISSCs, RICs and Social Innovation Fund). 25. The design was complex in terms of required policy decisions for the Project to achieve successfully its expected outcomes. Particularly, the Project over-estimated the required, sustained Government commitment and the need for policy decisions and high- level political intervention for reforms to move forward across all three components. Although structured as an investment operation, achievement of the Project’s expected outcomes, in reality, required major policy decisions with potentially difficult 2 See ISR dated November 5, 2008, Section 1: “Note for Management Attention”. 7 implications for Jordanians and which the GOJ found difficult to implement. Policy decisions faced challenges both in terms of the potential implications as well as in terms of institutional capacity constraints of MOSD, and NAF to implement it (e.g., improved targeting of the system of cash social assistance).3 26. The Project’s institutional arrangements were optimistic, given capacity constraints that the implementing agencies faced and the fact that oversight of implementation was entrusted to a Project Implementation Support Unit (PISU) headed by the MOSD Secretary General. The PISU included (i) a project director (external consultant) to manage the day-to-day project activities in coordination with the relevant program departments; (ii) a financial management specialist; and (iii) a procurement specialist. Fiduciary consultants were expected to handle the Project’s fiduciary management functions in coordination with relevant MOSD implementing departments for the first year, after which, in time, these functions would be absorbed by MOSD staff. In addition, the PISU was expected to hire term consultants with various technical capacities, as needed. The PISU was, inter alia, responsible for planning project implementation schedules, reviewing and issuing reports, and supporting implementation in accordance with annual plans. The Project aimed to strengthen institutional capacity of implementing agencies, while at the same time, these agencies would be carrying out other activities under the Project (especially improvements to NAF targeting under Component B and investments under Component C). 27. Finally, there was a clear disconnect in the implementation approach of the design. The design empowered the PISU to undertake steps in both designing reforms that dealt with policy issues while at the same time supporting the government in their implementation. This posed a client-clientele dilemma. On the one hand, PISU was supposed to work under the direction of the agencies while on the other hand, it was expected to direct a series of policy reforms within the MOSD. This included human resource development strategy, standard of care for residential facilities, targeting methodology for NAF, among others. To that end, the design underestimated the frictions that these institutional arrangements could create where the PISU played both a directive and support role to different departments in the ministry. Furthermore, these arrangements did not help the Project’s institutional objective of capacity building since the project management was entirely outsourced to a temporary PISU. 28. The Project’s results framework, especially the expected project outcomes, was overly optimistic (see above) given the level of reforms required. In addition, although analytical work was well developed prior to the project preparation, there was no linkage to the baseline data. For component C, the performance of NAF was to rely on the DOS Household Income and Expenditure Survey (HIES) for coverage (through benefit incidence analysis), targeting efficiency and poverty impact, as well as a special survey 3 In effect, the ICR for the Recovery under Global Uncertainty Development Policy Loan, Report No. 1986 dated June 26, 2012 reports that the GOJ did not succeed in increasing the efficiency and enhancing the poverty impact of Jordan’s social safety net by implementing the NAF Renewal Plan. The expected outcome for this policy measure under that operation was to increase the coverage of the targeted poor population by NAF assistance to 40 percent of those under the NAF threshold. This was well below the target that had been established under the Project, financed by an investment loan. 8 of NAF beneficiaries (that was never carried out). The assumptions of improvement in coverage over time were also overly optimistic. The results framework assumed that: i) NAF coverage would increase in a short period of time from 19 to 70 percent of the poor without putting in place measurable intermediate indicators that deal with outreach and enrollment for the poor who are not covered, and ii) NAF leakage would decrease without any graduation schemes or services for beneficiaries who do not qualify for benefits. As for the remaining components, measurement relied mostly on outsourced monitoring and evaluation by a third party and assumed that baseline information would be collected when the loan became effective. With these evaluation arrangements, neither the Ministry of Social Development (MoSD) nor NAF were expected to develop monitoring and evaluation capacity even though the project aimed at strengthening the government’s capacity in such areas. 29. Assessment of Risk. As highlighted in the PAD, the Project was not without risks—in fact, its Overall Risk Rating was considered Substantial. All of the factors that impacted implementation were correctly identified as risk factors, although, in retrospect, the risks of “weak project implementation capacity” and “increased regional political instability that might shift Government focus and slow down reform implementation,” both rated Moderate, were underrated during preparation. This is especially true in the case of implementation capacity since this was the first externally financed project of this scale to be implemented by the MOSD, and although the Project’s design recognized this and included activities to address this challenge, the measures were not sufficient. It might have been preferable to limit further the Project’s objectives, components and activities to a scale more in line with implementation capacity of the implementing agencies. Regional political instability undoubtedly impacted project progress as well, as described below. 2.2 Implementation 30. The US$4.0 million loan for the Project was approved by the World Bank’s Board on June 3, 2008, signed on July 8, 2008, and became effective on December 1, 2008. Implementation had a slow start as the Project Implementation Plan was only agreed to in March 2009. The Project made very little progress towards the accomplishment of its Project Development Objectives (Section 3.2), as it suffered difficulties throughout the implementation period. During the Mid-Term Review, which took place in February 2012, the scope of the Project restructuring, as well as next steps, were discussed although the restructuring was only finalized in March 2013. Nevertheless, the Project’s Development Objective and Implementation Progress ratings, which had already been downgraded to Moderately Satisfactory even before effectiveness, were temporarily upgraded to Satisfactory in 2010, particularly reflecting the startup of the disbursement. The rating for Implementation Progress again deteriorated and remained at Moderately Unsatisfactory throughout most of the Project’s implementation (even after restructuring). Most of the project ratings (financial management, project management, procurement, and monitoring and evaluation) and Component ratings were likewise judged to be Moderately Unsatisfactory throughout implementation. The main factors that affected implementation are described below. 9 31. Institutional Arrangements. Throughout implementation, the PISU faced increasing challenges to coordinate the implementation of various project components implemented by different agencies. The Project’s institutional arrangements did not emphasize the role of a higher level steering committee to provide overall policy direction, and direct the work of the PISU in terms of coordinating, as opposed to implementing the Project – a function beyond the mandate of a unit established with mostly external consultants, with little guidance and/or support from policymakers in government. Increasingly, the PISU was distanced from the staff of the Project’s implementing agencies and confronted with increasing resistance to what MoSD perceived as an externally funded PISU entity attempting to direct its work. The fact that the NAF operated as a semi-autonomous agency from the General Secretariat of the MOSD, with its operations functioning independently at the regional level, only exacerbated the PISU’s difficulties in coordinating and guiding the Project’s implementation. The PISU’s mandate could not have required it to direct the central and regional management of different agencies. When asked if a steering committee to guide the Project’s implementation had been established, in an interview conducted during the ICR preparation, two stakeholders made reference to a loosely formed group that had met only at the beginning of implementation. 32. During implementation, the GOJ took the decision to abolish the Coordination Commission for Social Solidarity, or CCSS. CCSS was responsible for implementation of Component A .1that was to provide support to assist it in fulfilling one of its three core functions: to research social issues of relevance for social policy development and where information gaps were identified. The Jordanian cabinet of ministers decided through Decision Number 2307 dated September 13, 2010 (reaffirmed on May 10, 2011) to abolish the CCSS and transfer its mandate fully to the MOSD. 33. Government Ownership. At the preparation stage, MoSD and the Council of Ministers supported the planned reforms. However, at later stages of the project, this support was less apparent. During later stages of the project, the Government showed passive commitment to, and ownership of, the renewal of NAF, which required a policy mandate from the highest levels of government. An idea that at one point – during project preparation – had gained approval from the Jordanian Cabinet seemed to waiver and slowly become a primary obstacle for the implementation of the Project’s second component, Component B, Renewal of the NAF. Similarly, interviewed stakeholders pointed to World Bank implementation support as increasingly focused on the implementation of this component, almost to the detriment of the other components, reiterating almost exclusively the adoption of proxy means testing (PMT) as the only method of targeting assistance provided through the NAF. To a large extent, this information is corroborated with several ISR notations submitted internally at the World Bank.4 Lack of government ownership, or more specifically, disconnect between different levels of Government, for a new targeting methodology is signaled as a hindrance to overall project progress. The PAD made no reference to the adoption of PMT as the targeting mechanism required to achieve the expected outcomes. Despite the 4 See World Bank ISR “No.2” to ISR “No.5” (2009- 2011) 10 successful piloting of about 6,500 households screened using the PMT formula to ascertain their eligibility, there was a lack of understanding, and therefore commitment, at all levels of the GOJ on the costs and benefits of adopting PMT at the national level. With no clear decision by the GOJ to adopt PMT, the issue appeared to have become a sticking point between the GOJ and the World Bank’s team. 34. Staffing Changes. The difficulties in implementing the Project were exacerbated by frequent changes in the leadership of both the MoSD (five ministerial changes from 2007 until closing in 2013), as well as changes in the Director General of NAF that resulted in lack of clear ownership, as well as frequent turnovers in staffing. These changes almost inevitably had the effect of diminishing ownership and commitment. Similarly, the PISU, already by design, without much power to put implementation on sound footing, remained without a director for intermittent periods during the Project. Moreover, the Project was not assigned a director for the first nine months of implementation mainly because of disagreement between the World Bank and MoSD officials on the selection process.5 35. The staffing changes in the GOJ, including the Project’s implementing agencies, were exacerbated by the frequent changes in the World Bank’s task manager for the Project. Throughout the Project’s five-year implementation period, task management for implementation support changed a total of four times, with task managers of different specializations and capabilities assuming responsibility. It is important to point out, however, that the World Bank brought in additional specialized staff as implementation faltered. 36. Lack of Internal Communication. Given that the Project did not have an integrated steering committee, communication also became a significant problem at implementation. Specifically, the Project lacked sufficient communication within agencies at different levels (e.g., implementing arms of MoSD, the PISU, and policy makers). In an interview with NAF at the conclusion of the Project, the Fund’s leadership team stated that it was not fully briefed of the type of activities that were agreed upon within its component at appraisal. In another field visit, two of the ISSC managers were unable to identify any of the objectives of the Project or the type of activities, services and reforms that they were to be implemented at their project site. A series of site-visits and interviews with management and staff of centers revealed that they were not well informed about the Project’s objectives and activities. 37. Insufficient Project and fiduciary management capacity. The Project was implemented by the MoSD, and supported by the PISU that was to provide general oversight of project implementation, including fiduciary support. The long delay in hiring the Project Director for the PISU affected early implementation, but even with the Director in place, almost throughout, the PISU lacked capacity to carry out this function, with key weaknesses especially in fiduciary management. Since the MoSD had no previous experience with Bank-financed projects, the PAD had assessed procurement and 5 It is important to note that the ICR team was unable to interview several of the Project Directors after repeated requests for meetings. Only one former Project Director was interviewed. 11 financial management risks as “moderate”, and identified several mitigating measures, including the contracting of external specialists and training of MoSD staff, to ensure that fiduciary arrangements worked smoothly. Nevertheless, implementation was constrained by lack of capacity in overall project management and frequent changes of FM and procurement consultants who were hired to mitigate fiduciary risks. 38. Administratively complex financing arrangements. The Project was to be financed by the US$4.0 million Bank loan, and GOJ counterpart funding of approximately US$6.8 million equivalent. Disbursements from the Bank loan were, therefore, set to represent 40 percent of all expenditures under the Project, and, as a result, the Bank’s Loan Agreement stipulated that the Bank’s Procurement Guidelines would apply to all expenditures financed under the Project. The PISU found implementation of this arrangement cumbersome, and coupled with weak capacity, procurement processing suffered extensive delays as a result. This was later rectified in the Project restructuring which set Bank disbursements at 100 percent of eligible expenditures (Section 1.7). 39. Misalignment of financing to Project activities. Although financing arrangements in terms of expenditure categories (e.g. procurement, consulting, non- consulting, other services) were aligned to the actual expenditures projected, the Project financing was misaligned to planned project activities. A number of activities did not receive allocations of funding and were expected to be implemented by the Government fully while other activities were earmarked a significant portion of the funds. For example, no Bank funding was allocated to the set-up of the M&E unit beyond the terms of reference written for assessment. Further, little support was dedicated to the set-up of a “one-stop shop” pilot in a regional MoSD office. 40. Counterpart funding requirements. The Project’s counterpart funding requirements were high (see above). Jordan’s previous strong macroeconomic performance—growth had averaged 6.5 percent in 2000-2009—was severely affected by the global economic crisis in 2009 that led to a sharp decline in growth in 2010. The downturn significantly affected public finances. In 2011 , higher world fuel and food prices, combined with growing political uncertainties in the region, and frequent disruptions of gas supply from Egypt led to a larger import bill, a decline in foreign direct investment, tourist revenues and foreign currency reserves. 41. Overall Deterioration of Bank portfolio performance in Jordan. There was noticeable decline in the overall performance of projects in the MNA region between 2008 and 2013. For Jordan, the percentage of unsatisfactory projects increased from 14%, in 2008, to 43% in 2013 as compared to 8% and 29% for the whole region in the same time period.6 This can be linked, to Jordan-specific challenges in policy reform and frequent ministerial changes and, to some extent, to the political economy of the region as a whole. 6 Calculation by the authors. Portfolio data includes IBRD/IDA projects only. 12 42. Arab Spring. Events in other countries in the region since February 2011 had an effect on the GOJ and prevented progress on some reforms that may have had the potential of causing social unrest (e.g., refining the targeting of NAF or Human Resource reform of MOSD). Moreover, since 2010, political frustrations ran high as there were frequent changes in the cabinet. Although the social challenges faced during this period make SPEP’s objectives more relevant as a response mechanism to popular social concerns, the country’s political economy made it difficult to implement components of the project which required policy reform. 43. Mid-Term Review. The Project’s Mid-Term Review took place from February 6-9, 2012. As part of preparation for the Mid-Term Review, the World Bank commissioned an independent, external evaluation of the Project’s progress and the issues it faced.7 The evaluation highlighted that the PDO was still achievable without justifying this assertion, especially given the short time remaining to the closure of the Project, and suggested that the PDO indicators should be revisited based on its findings. The external consultants stressed that the Project’s weak progress and performance was the “result of several factors including but not limited to, the weak performance of the PISU management team, the unsatisfactory level of support by MoSD and the Ministry of Planning and International Cooperation to PISU activities, the resistance to change that exists in certain entities such as resistance to NAF new targeting mechanism, hesitance to make some high level decisions related to achieving the Project’s objectives by cabinet and MoSD, in addition to frequent change of ministers and officials at key institutions working with the Project.” The report highlighted that there was a misunderstanding of the Project and its activities by key stakeholders, especially those that were its beneficiaries and implementers. This was to a large extent the result of the non-existence of a communication plan that should have been put in place by the PISU. The external evaluation concluded “the level of cooperation and coordination among the stakeholders of the SPEP is minimal. Instead, a circle of blame among stakeholders exists regarding who is responsible for the weak performance of the Project.” The evaluation also provided a detailed analysis of the status of implementation of each of the Project’s components, issues regarding their progress, and recommendations for improvement. 44. Discussions surrounding the Mid-Term Review dragged on (an Aide-Memoire of the Mid-Term Review was never finalized), and eventually led to the Project restructuring approved on February 26, 2013 (Sections 1.3, 1.6 and 1.7). But, the restructuring was “too little, too late”. The restructuring paper did not address the disconnect between different levels of Government, or propose new institutional arrangements. For the World Bank, the Project seemed to represent an important vehicle for sector policy dialogue on an issue that continued to be very central to its assistance strategy (Section 3.1). This presented the dilemma between closing a failing project or remaining engaged in a priority sector. It seems that cancellation was not an option, as neither the GOJ, nor the World Bank wanted to cancel a social protection operation. At the same time, neither party contemplated a closing date extension. The Project was left on the books to close as originally scheduled, with little, if any, results to show upon completion (Section 3.2). 7 Mid-Term Review of Social Protection Enhancement Project, Hashemite Kingdom of Jordan, March 25, 2012. 13 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 45. Design. Each of the Project’s components was to have built-in monitoring and evaluation activities. The MoSD, under the direction of the PISU Project Director, would prepare and make available to the public an Annual Project Implementation Report consolidating progress in implementation from MOSD, NAF, DOS and CCSS to allow for monitoring of the Project. One of the key sources of information to monitor project outcome indicators was to be a nationally representative Household Income and Expenditure Survey (HIES) that was used for analysis of the living standards of the Jordanian population, including poverty measurement, poverty profile, and NAF benefit incidence analysis. In addition, a beneficiary assessment survey was to be carried out during implementation to assess MoSD and NAF operations, as well as to obtain information on satisfaction of NAF and MoSD beneficiaries. The Project also contemplated third-party evaluation studies to assess some of its expected outcomes. The second source of information was to be administrative data that would be generated by the upgraded/new MIS in MoSD and NAF (to be financed by the Project), and integrated by the CCSS. Both sources of information were expected to be complementary and ensure adequate monitoring and evaluation of the Project’s outcomes. 46. Apart from the fact that baseline data for outcome indicators was not available when implementation began, the Project’s results framework presented several issues (Section 2.1). Further, a key baseline and data sources for monitoring were not in place at the time implementation began, and would be financed under the Project. 47. Implementation and Utilization. The PISU compiled information on the progress of each component through utilizing information submitted in the context of disbursement processing and of site-visits to monitor intermediate outcome indicators. The DOS routinely carried out HIES, and its capacity for household data collection and poverty measurement and monitoring was strengthened under the Project (Section 3.2). 2.4 Safeguard and Fiduciary Compliance 48. During project preparation, weak project implementation, including fiduciary capacity, was identified as a potential risk and several mitigating measures were identified as needed (and actions were taken) before effectiveness (Section 2.2). Nevertheless, weak fiduciary management capacity, both for financial management and procurement, led to delays in implementation almost throughout. The PISU opted to provide fiduciary support, through progressively and internally appointed positions. This was seen as a commitment to the sustainability and institutionalization of the Project, and its results. Yet, frequent staff turnovers in the PISU, especially during in 2010 through mid-2011, left a notable vacuum in knowledge of World Bank FM and procurement requirements and procedures that affected the PISU’s ability to comply with fiduciary requirements in a timely manner, and resulted in a lack of coordination and alignment of the FM and procurement functions. As a result, the World Bank, during the project restructuring supervision mission, agreed upon a set of actions that needed to be 14 addressed for the restructuring to move forward, including submission of delayed reports, staffing, etc. 49. The above-mentioned capacity constraints led to frequent unsatisfactory Financial Management (FM) ratings in the Project’s ISRs. The issues encountered included delays in issuance of loan withdrawal applications (on one occasion a nine-month delay—two years after implementation began only one had been submitted), submission of quarterly interim unaudited financial reports, and delivery of audited financial statements. In general, however the PISU maintained an acceptable automated accounting system that was used to capture the Project’s financial transactions and generate the quarterly interim unaudited financial reports.8 It also maintained adequate internal controls (detailed in an acceptable FM and disbursement Manual that was part of the Project Implementation Manual), including regular reconciliation of bank accounts and adequate segregation of duties, and performed monthly reconciliation of disbursement summaries with project accounting records. 50. All of the Project’s audit reports until the end of 2012, were unqualified audit opinions. Many, however, were consistently submitted with delays. The most recent audit report (for year ending December 31, 2012) received a qualified auditor’s opinion. A “Disclaimer of Audit Opinion” was issued on the statement of fixed assets. The Disclaimer stated that the Project Implementation Support Unit “PISU” did not prepare the statement of fixed assets. Despite several requests from the Bank, this statement was not prepared. At project’s closing, a management letter reported ineligible expenditures of US $21,371 representing taxes financed from the World Bank loan proceeds.9 This amount is considered ineligible owning to the fact all related taxes should be financed by the MoSD’s own budget as per the signed Loan Agreement. The MoSD is in the process of providing justification for eligible expenses and refunding the World Bank for ineligible balance. The last audit report will cover the period from January 1, 2013 to Closing Date and given the grace for period ending December 31, 2013, it is due for submission by February 28, 2014. 51. Staffing constraints, especially a lack of continuity of qualified staff, similarly affected the Project’s procurement function, almost throughout implementation. Issues included a lengthy procurement process (on occasion, up to nine months to contract a consultant), delays in approving awards due to internal procedures (at one point, each contract award required the approval of the Prime Minister’s office), and non-conformity to acceptable procurement practices (such as evaluating bids on an item-by-item basis, leading to slicing a single bid into numerous awards). The PISU utilized the procurement plans more for reporting purposes than as a planning tool to package efficiently the activities and items being procured for purposes of observing economies of scale for project procurement purchases. Finally, on occasion, activities to be procured were 8 The accounting software was developed by a local firm and was utilized by other World Bank-financed projects in Jordan, yet the software was modified to keep accounts of the Project. The software combines procurement, accounting and budgeting modules to account for and report on Project activities. 9 World Bank letter to Ministry of Social Development concerning ineligible expenditures on the Social Protection Enhancement Project (December 24, 2013) 15 processed through contract award, only to be rebid (on occasion without the Bank’s no- objection) if the Minister did not approve the award. 52. The Project did not trigger any of the Bank’s safeguard policies. 2.5 Post-completion Operation/Next Phase 53. The Project’s PDO to improve the management and operations of the social assistance programs remain relevant in Jordan (Section 3.1). In December 2012, the GOJ announced a broadly targeted cash transfer compensation scheme to accompany the fuel subsidy removal administered by the Income and Sales Tax Department (ISTD). All households with an aggregate annual income below JD 10,000 (approximately US$14,100 equivalent) excluding foreigners and Jordanians living abroad are eligible for the program, which covered up to 70 percent of the population in 2013. To assist GOJ in its continuing efforts to compensate lower and middle-income groups following reduction of fuel subsidies, a US$9.5 million grant “Support to Implementation of a National Unified Registry and Outreach Program” (the NUR Project), financed by the Deauville Partnership Transition trust fund, was approved in January 2013. At the demand of the Government, the NUR Project is expected to improve the accuracy of targeting, to provide options to assess more accurately the welfare of households, and to rank all applicants for the compensation program according to a new welfare criterion for distribution of social assistance. In addition, NUR Project is expected to introduce a national unified registry that will help reduce fragmentation of databases across various agencies and help increase the quality, reliability and accuracy of the existing national database. Although the NUR Project is not the continuation of SPEP and its target population is broader than the target population of NAF, the development of additional targeting mechanism through the NUR Project could help in the future to improve the accuracy of the existing categorical targeting used by NAF. Moreover, the development of the National Unified Registry is critical for improving targeting of the compensation scheme for fuel subsidies reform, as well as other safety net programs in the future. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: High 54. Relevance of Objectives. Despite the Project’s implementation challenges, the SPEP’s objectives remain valid today and consistent with the GOJ’s priorities as well as with the World Bank’s assistance priorities. The relevance of its objectives is high. Jordan’s economy was affected by both the global economic crisis and the political upheaval that swept the Arab region. Sharp increases in oil and food prices, the removal of fuel subsidies, and the global economic downturn have put pressure on the living standards of many Jordanians, particularly those at the bottom of the income distribution. As a result, Jordan’s weakening economy is undermined by social challenges, including unemployment and poverty. Yet, the adverse changes in the world economy have led to an increase in Jordan’s fiscal deficit that has restricted fiscal space for implementing 16 employment generation and social protection programs. So, at this time of growing social unrest, the GOJ is challenged to adopt a mix of fiscal consolidation measures which include subsidy reform as an integral part (to save money for the Government), and to reinstate social protection policies to protect the poor and vulnerable. The GOJ’s commitment to this objective is reflected in its recently launched National Poverty Reduction Strategy (PRS) with an overall goal of “containing and reducing poverty, vulnerability and inequality in the current socio-economic environment of Jordan, between 2013 and 2107, through the adoption of holistic and results-oriented approach, which targets poor and below middle class individuals.” The PRS includes among its nine objectives the following that are consistent with the Project’s PDO: (a) to better harmonize all public, private and civil society poverty reduction programming, and (b) to deliver expanded, increasingly aligned and better-targeted social protection measures to members of poor and vulnerable households. 55. The Project’s objective remains relevant and is consistent with the World Bank’s FY12-15 Country Partnership Strategy (CPS) for Jordan.10 Specifically, it is consistent with the first and third pillars of the CPS, which aims to: Strengthen fiscal management and increase accountability and Enhance inclusion through social protection and local development. The CPS mentions that the World Bank group will support efforts to better target social assistance to the neediest while simultaneously improving access to, and quality of, social care services and improving health insurance coverage. 56. Relevance of Design. The relevance of design is rated low. Although the Project’s objective continues to be relevant and reflects an important development priority for Jordan, its design was not. There was demand from the GOJ for support in improving the delivery of cash assistance programs at the time of preparation, but an eventual disconnect surfaced in government ownership, resistance or understanding of what reform would entail. The GOJ was not prepared for the type of reform the Project entailed, and the Project’s complex design was complicated with both activities to improve the capacity for delivering social services and to address issues with the cash social assistance program, at the same time. 57. The GOJ’s strategy for addressing its objective of protecting the poor and vulnerable has not changed, but the mechanism for doing so has. A new targeting system is expected to address the fragmentation of databases across various agencies that exist at present, including the NAF, and establish the NUR where all applicants will be ranked according to a new welfare criterion for distribution of social assistance (Section 2.5). 58. Relevance of Implementation. The World Bank’s implementation assistance was not supportive in addressing the GoJ’s changing needs, institutional changes (e.g.., decommissioning of CCSS), and competing priorities. As a result, it is rated low. Implementation assistance did not address in a timely manner the bottle-necks faced by the Project. The Mid-Term Review was conducted too late in the Project’s 10 World Bank, Country Partnership Strategy for the Hashemite Kingdom of Jordan for the Period FY12-FY15, Report No. 58114 17 implementation to address the chronic issues identified in ISRs almost from the onset of implementation. Although in practice, mid-term reviews can be used as tools to assess project challenges and identify potential avenues to address them, the mid-term review process was delayed in the case of this Project. Further, little, if any, proactive action was taken to restructure the Project early---the project restructuring was approved during its last year of implementation. 3.2 Achievement of Project Development Objectives Rating: Highly Unsatisfactory 59. The achievement of Project Development Objectives is rated Highly Unsatisfactory. There is no evidence to suggest that the Project succeeded in improving the management and operations of the cash social assistance programs or improving access to and quality of social care services as intended in the Project Development Objectives. Quite apart from the inconsistency of targets established for its Key Development Indicators, both in the PAD and later in the restructuring (Sections 2.1 and 2.3), the Project did not achieve its PDO, even in small measure. Overall, the Project financed only a dispersed set of activities, some of which were initiated but not completed, others of which were completed for institutions that were abolished, and yet others that financed mere inputs for social centers that were not transformed in a way that reflects the intended type of transformation specified in the PAD and intermediate indicators. Furthermore, the lack of measurable baseline, and lack of any relevant information with which to monitor the Project’s impact (PDO indicators 2 and 3, see Section 2.0), further complicates the ability to measure any progress on several indicators. Due to the lack of observed outcomes and poor measurement, this section merely describes the inputs and services that were financed, by project component and intermediate outcome. Component A: Institutional Development and Capacity Building Intermediate Outcome: The MoSD capacity to formulate and implement safety net policies enhanced; DOS routinely uses standard software and statistical packages. Rating: Highly Unsatisfactory 60. The Project contemplated activities to strengthen the capacity of CCSS, MoSD and DOS. CCSS completed the activities that were envisaged, i.e., procuring five packages of software (Statistical Package for the Social Sciences, SPSS) and providing training to its staff on its usage, and delivering training on data collection and research design, evaluation indicators, monitoring and Strength, Weakness, Opportunity, Threat analysis of CCSS, strategic planning and policy analysis and how to formulate poverty policies and strategies. Further, indexes related to institutions serving the social care centers were developed but not applied. However, CCSS was abolished in 2010 as part of Jordan’s public expenditures consolidation. 61. For DOS, the Project financed SPSS and personal data assistants, and training of staff in their use. The Project also equipped a training center for DOS staff. There is no 18 information on whether new hand-held enumeration devices envisioned in the project were utilized for surveys by DOS. 62. A second sub-component was dedicated for the adoption of a Strategic Development Plan and a Human Resources Development Strategy. Based on interviews with stakeholders, no reference was made to the adoption of a Strategic Development plan at project closure. For the Human Resources Strategy, implementation was hindered at the onset of the project when MoSD did not endorse the Strategy. Alternatively, the PISU in collaboration with the MoSD leadership drafted an assessment of needs and the establishment of standard position descriptions for MoSD that was discussed at Social Care Centers but not implemented at regional offices of MoSD. The assessment was expected to include training and capacity building requirements and standards based on defining staff responsibilities and competencies. During the ICR mission, there was no evidence of changes in Strategic Planning or HR policies at MoSD. Further, no trainings were undertaken to the Human Resources Development Strategy (sub-component 4). 63. Finally, a third sub-component was dedicated to the set-up of an M&E Unit to coordinate monitoring and evaluation across programs and issue annual reports for the ministry. This sub-component was also hindered at initial phase of the project where MoSD did not approve the recruitment of an M&E consultant. No payments were made for this sub-component. The ICR team was unable to identify reasons for why the ministry did not move forward with establishing framework or parameters of the unit even though delays were apparent at the recruitment stage. 64. In sum, Component A did not achieve the results for the PDO indicator which was to measure progress. Further, it did not deliver on any of the intermediate outcome indicators that were prescribed to it at both the pre- and post- restructuring stage. Component B: National Aid Fund Renewal Intermediate Outcome: Coverage of the target poor population by NAF assistance increases; errors of exclusion declines; NAF performs its functions efficiently and in a transparent manner. Rating: Highly Unsatisfactory 65. During preparation, a pilot involving about 6,500 households, an updating of the PMT targeting formula utilizing the 2008 HIES, approval for hiring of 50 new social workers, revisions of the PMT and NAF questionnaires and preparation of technical requirements for information technology hardware and software needs had been completed. Still, this component faltered upon effectiveness as the World Bank insisted in adoption of PMT methodology at the national level as the only targeting mechanism for reducing errors of exclusion and improving coverage, and the GOJ was unable to reach a decision to adopt the new methodology. Some activities related to installation of MIS-related software and hardware, and voice and data communications in the central and local NAF offices, as well as local area network (LAN) and wide area network (WAN) connections were financed, and MoSD headquarters are connected to the national 19 Secure Government Network. Beyond these institutional changes, there is no evidence of the Project’s impact on increased coverage of the poor or reduction in errors of inclusion. Component C: Improving Access and Quality of Social Welfare and Care Services Intermediate Outcome: More people have access to good quality social work and care services. Rating: Unsatisfactory 66. Integrated Social Services Centers: The Project financed consultants to select the locations of pilot ISSCs, and to pilot three ISSCs in Amman, Russayfeh and Irbid. Consultants identified priority areas and services needed in each center, determined the main target groups and designed the centers accordingly. A master plan was designed and submitted to MoSD by the hired consultant. The Project trained staff working with people with disabilities at three RICs to provide services based on individual care and rehabilitation plans, based on a case management approach. Three centers financed under the Project are operational: Amman, Aqaba and Irbid. A consulting firm to assist in rolling out the new care model was selected, but was not contracted. Finally, an operational manual for the MoSD Social Innovations Program was developed. 67. Residential Care Facilities: All residential institutions run by the ministry were transformed to resemble family-like settings. This was particularly encouraged by the decision of MoSD ex-minister to move forward in making most centers follow new residential models. The Bank’s financing (below $50,000 per center) played a role in purchase of furniture, computers and electronics, and some physical rehabilitation for Ministry owned properties. However, there is not clear indication that Bank project played a central role in the transformation. 68. Disability Center: One pilot center which was financed by the World Bank pre- and post- restructuring was abandoned (Umm Nuwarah Center) and no longer in use for any purpose. This was largely due to an error during the rehabilitation process were it was not done according to International or Local Disability Building Code Standards. Component D: Project Management11 Rating: Highly Unsatisfactory 69. The Project financed staffing of the PISU (including project director, project procurement officer, financial officer, auditor, assistants, administrative assistants, and the staffing for a NAF technical support unit including two technical experts and IT consultants). The majority of expenses went for supporting PISU staffing and no expenses were accrued beyond PISU activities for this component. The NAF technical assistance unit worked under the direction of the project director in practice. Of the successful activities undertaken for this Component, the PISU established separate books for procurement and financial management, and upgraded financial management software. However, the capacity developed is not expected to be maintained as the PISU was dissolved upon completion of the Project. 11 No intermediate outcome was defined for this Component. 20 3.3 Efficiency Rating: Unsatisfactory 70. Economic and Financial Analysis. The Project’s ex-ante economic and financial analysis presented in the PAD justified the Project’s economic impact mostly on the basis of an expected increase in efficiency in the use of public resources allocated to NAF. The direct economic benefits from better access and improved quality of social care services to vulnerable populations were expected to materialize in the medium- to long- term, and were considered more difficult to quantify. Project activities aimed at capacity building of MoSD, CCSS, DOS and NAF were expected to result in overall efficiency gains in service delivery and a reduction in administrative costs for the Government. The Project’s impact on activities that were expected to increase efficiency was negligible. Therefore, an economic analysis of efficiency gains (cost-benefit, cost-effectiveness) for the MoSD and GoJ as a result of the Project is not feasible. To that end, this section looks at the overall breakdown of costs, and disbursement. 71. The Project’s overall implementation was inefficient from a cost-benefit perspective. Annex 3, shows a financial breakdown of the Project’s costs by component. During the Project’s five years of implementation, only one quarter of the loan was disbursed (US$967,905.51 out of US$4.0 million). The 25 percent disbursement does not correlate to the Project’s limited achievements. For the first PDO indicator, lack of improvement in the coverage of NAF beneficiaries (coverage of poor beneficiaries 25% compared to 70% target) was attributed to both differences in measurement of the baseline and lack of substantive activities. For the second PDO indicator (increase in number of beneficiaries receiving social care services), no baseline was reported and the Bank’s limited disbursement for the project (see section 3.2, above) is unlikely to have generated any increase in enrollment capacity (there is no data by which to measure this, however). Finally, no baseline was collected for beneficiary satisfaction (the third PDO indicator); therefore, measuring overall satisfaction is not feasible. 72. Loan disbursements were correlated to component inputs and financing arrangements. When comparing the World Bank’s portion of expenditures to those of the MoSD, the ratio between Bank and MoSD expenditure reflects the financing agreement. The World Bank financed 32.4 percent of total project costs; the GOJ financed 67.7 percent. 73. The Loan’s low disbursement is the result of the Project’s limited implementation, and the financing of only a few, disparate activities. This was especially the case for components where implementation challenges at the policy or inter-ministerial level affected implementation. Only 7.3 percent of the appraised amount was actually disbursed for Component B (Renewal of National Aid Fund). Conversely, for areas where a number of activities were implemented, expenditure levels were higher (i.e., for Component A 33.0 percent of the appraised allocation was disbursed). As expected, the expenditures for Project Management represented the highest share of project expenditures (55 percent). 21 3.4 Justification of Overall Outcome Rating Rating: Highly Unsatisfactory 74. The Project’s Overall Outcome Rating is considered Highly Unsatisfactory. Although its PDO continues to be relevant, challenges in the Project’s design and implementation contributed to the unsatisfactory outcome. The Project did not succeed in achieving its PDO, as measured by key outcome indicators. At completion, the Project had financed only some of the inputs that would be needed to achieve the intermediate outcomes. The project was constrained by the Government’s and implementing agencies’ limited commitment to making critical decisions and adopting new approaches, based on the inputs financed. In addition, the World Bank’s implementation support was not effective in working with the Government to restructure the Project in a way that its design and implementation would support the achievement of the PDO. Consequently, the project’s implementation was not satisfactory (given the lack of achievement of any development objectives) and did not result in the efficiency gains that would have made it an economically viable investment. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 75. The Project was to target poor and vulnerable Jordanians. Nevertheless, as it produced only minimal, if any, development outcomes. Its only impacts on poverty reduction are remote and limited to possible future marginal impacts that may result from assisting the DOS in improving its poverty measurement and monitoring approaches. (b) Institutional Change/Strengthening 76. The Project as designed had a very strong institutional development focus, as evident in the design of Components A and B. Nevertheless, it had very little impact, if any, on building institutional capacity within MoSD or any of the other implementing agencies. The capacity building activities for CCSS were implemented, but the institution was abolished and its responsibilities transferred to MoSD; it is not clear whether the staff trained were absorbed within MoSD. Support to DOS under Component A (3) through the provision of training in the use of advanced statistical tools will improve its capacity in the area of HIES. Many of the enhancement, capacity building, training and other activities which involved establishing standards of care for social care centers, were all uncompleted or never moved beyond the conceptualization stage. In field visits conducted during preparation of this ICR, general managers of residential units and social care services expressed that the Project’s activities in no way increased their capacity to deliver more effective services to their beneficiaries. Finally, the renewal of the NAF was never implemented, following the pilot carried out during preparation. (c) Other Unintended Outcomes and Impacts (positive or negative) 22 N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A 4. Assessment of Risk to Development Outcome Rating: High 77. The Risk to Development Outcome is High. The Project had few, if any, results at completion. Hence, development outcomes have not and are not expected to materialize from the Project. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 78. Bank Performance in Ensuring Quality at Entry is rated Moderately Unsatisfactory. The Project was responsive to the GOJ’s development assistance priorities, and to the Bank’s CPS. Working with the Government, the World Bank had carried out extensive analytical work, and the findings and recommendations of that work were incorporated in the Project’s design. The Project’s design incorporated lessons of experience, especially those of other countries that had tackled the same difficult issues. However, World Bank Performance in Ensuring Quality at Entry was ineffective in several areas. These included the definition of the Project’s institutional arrangements, the ambitiousness of targets for key indicators in the Project’s Results Framework, the lack of baseline data, the definition of the Project’s financial arrangements (with World Bank disbursements at 40 percent of all expenditures), and an overestimation of Government commitment to politically difficult decisions (including implementation of new HR strategy, adoption of a new targeting formula, potential exclusion of NAF beneficiaries, etc). Nevertheless, these were issues in the Project’s design that could have been easily addressed (and some of which were, albeit belatedly) to allow the Project to proceed towards the accomplishment of its PDO. (b) Quality of Supervision Rating: Highly Unsatisfactory 79. Quality of Supervision is rated Highly Unsatisfactory. The Project faced problems at entry, but not necessarily more than many projects face. These challenges could have been addressed through intensified implementation support and early proactive action. Instead, task management for implementation support of what had been rated as a problem project was assigned to a rotating number of task managers (a total of 4 task managers during implementation period of 5 years), each of which faced the challenges of becoming familiar with the Project, the issued it faced and its 23 counterparts, and that increasingly focused on the absence of a governmental decision to modify the NAF’s targeting formula as the only issue affecting implementation. Supervision missions failed to internalize that there was no consensus within the GOJ to adopt PMT (and the Project did not specify PMT as an agreed methodology), especially since there were apparently important concerns regarding the population that would lose benefits, an issue that must have become all the more important in light of the events sweeping through the region, and the need to change legislation that would have been politically difficult. 80. Differing views by changing task managers resulted in occasional optimism about the Project’s prospects, but in reality, the Project was in problem status for over 4 years, with little significant proactive action (restructuring) taking place only six months before closing. Project reporting was extremely deficient: the Aide-Memoire of the Mid-term Review (MTR) mission was never finalized, ISR reports, and were few and focusing mostly on Component B (National Aid Fund Renewal) and the final ISR incorporates outdated data to report on results. The main, significant document identifying progress, issues, challenges and recommendations was the evaluation prepared by external consultants in preparation for the MTR. Several task managers reported that the World Bank was not willing to take stronger action to close the loan as the Project represented an important vehicle for sectoral policy dialogue in the area of social protection. But in retrospect, keeping the Project on the books did little even in furthering the dialogue within the context of implementation support—the policy dialogue proceeded in parallel to the Project. (c) Justification of Rating for Overall Bank Performance Rating: Unsatisfactory 81. Overall Bank Performance is rated Unsatisfactory. Performance at Ensuring Quality at Entry was Moderately Unsatisfactory, but the issues presented could have been remedied if they had been addressed earlier, either through a major restructuring, or through scaling down the Project, changing institutional arrangements, or reorienting activities to only those that responded to the GOJ’s needs. The World Bank’s implementation support somehow failed to internalize the institutional and other realities the Project faced, was not able to document them in a comprehensive manner for the whole of the Project, and was not receptive and willing to restructure the Project in a proactive and timely manner. As a result, implementation support was not conducive to supporting the adjustments that could have helped produce a satisfactory outcome. 5.2 Borrower Performance (a) Government Performance Rating: Unsatisfactory 82. The Government’s Performance is rated Unsatisfactory. During preparation, the GOJ worked closely with the World Bank through analytical work, and the design of the project mirrored diagnosis of issues as well as the Government’s policy directions. 24 However, the Project was implemented in a broader environment involving several external factors (the global economic crisis, fuel prices, Arab Spring) that affected not only the country as a whole, but the Project. These factors, on the one hand, made the Project’s PDO more relevant (i.e., through its expected impact on poverty), while at the same time, they have made the Government’s ability to adopt decisions and focus on project activities more difficult. This was in part the result in extensive staffing changes at all levels of government and the challenges in adopting decisions that would impact a share of the population. Thus, these factors resulted in the GoJ not being able to clearly focus on the objectives of the project, even though its objective remained a central component of its development priorities. The Project suffered from lack of support by MoSD, resistance to change, and hesitancy to take high-level political decisions relating to the Project’s objectives, and lack of communication with key stakeholders. (b) Implementing Agency or Agencies Performance Rating: Unsatisfactory 83. The implementing agencies’ performance is rated Unsatisfactory. The Project had five main implementing agencies for purposes of this rating: MOSD, NAF, DOS, CCSS, and the PISU (within MoSD). Each of these had varying amounts of activities and involvement in the Project, and consequently in the eventual achievement (or not) of its PDO. DOS and CCSS had relatively small, contained activities, and performed these efficiently and well. While important, the influence of these activities on the achievement of the PDO was to be relatively limited. MoSD’s performance in the implementation of its own capacity building activities under Component A(2) and works and capacity building under Component C fell short of expectations, mostly since MoSD staff lacked knowledge about the Project’s objectives and components, a reflection of the weak project communication plan. NAF management expressed resistance to adopting a new targeting mechanism, in part because the components and objectives were not clear to NAF staff, and in part because staff felt that a new, standard targeting methodology would affect the level of authority of some NAF staff, particularly field managers for deciding on who would benefit from NAF assistance. This was compounded by management changes in NAF that resulted in the fund being without a director for almost one year. 84. The PISU’s performance (and its actual role in the Project) was not conducive to good implementation. Quite apart from staffing issues, the relations between the PISU team and MoSD management and staff were, as reported in the external evaluation “interrupted and suffer from lack of proper interaction, cooperation and communication tools.” The PISU did not perform well in planning project activities with responsibilities and deliverables by component, monitoring and evaluating the Project, communicating with key stakeholders or providing a solid management structure for the Project. The lack of communications negatively impacted stakeholder buy-in to the Project and its objectives. Although the PISU counted on qualified technical staff, they were not empowered, and, were apparently perceived as highly remunerated in comparison to MoSD staff, further isolating the PISU as a separate project unit within the Ministry. Although the PISU itself appears to have been committed to the Project and its 25 objectives, its structure, governance arrangements, the shortcomings described above and lack of familiarity with issues that became central to the Project (targeting), precluded it (and the Project) from accomplishing its objectives. (c) Justification of Rating for Overall Borrower Performance Rating: Unsatisfactory 85. Overall Borrower Performance is rated Unsatisfactory. Despite the Government and implementing agencies’ close cooperation during preparation, the issues that surfaced during implementation, both on the side of the Government and of the implementing agencies, and the lack of action to address them justify this rating. 6. Lessons Learned 86. With few results, the Project offers few lessons learned on the implementation of a social protection operation. It does, however, offer important lessons for the World Bank in terms of how to provide implementation support for a project that is performing well below expectations. The lessons are: 87. Problem projects can become successful, but they require intensive implementation support and a thorough reflection as to the very issues that affect implementation. Several lessons are relevant. First, frequent changes in task manager lead to discontinuity in implementation support. Since each new task manager is likely unfamiliar with the implementation issues, the specific country, sectoral and project context. When change in task manager is inevitable, there is need for a systematic and smooth hand-over, Second, a problem project should not remain on the books for over two years (at the most) without some sort of proactive action (i.e., improving, restructuring, closing, etc.). Third, a Mid-term Review (MTR) for a project experiencing implementation delays and issues should not be delayed because implementation is not sufficiently advanced to review progress. To the contrary, the MTR for a problem project should be advanced in order to address issues that are impeding implementation and agree upon actions needed to improve it. Then, if those actions are adopted and successful, a follow-up review, a second MTR (for lack of a better name) can be conducted to review progress. Fourth, supervision reporting for a project facing implementation issues should be frequent and comprehensive with respect to the entirety of a project. Finally, restructuring should not be done for the simple sake of restructuring a project. In the case of the Project, the restructuring was done six months before closing, when neither the Government nor the World Bank intended to extend the Closing Date. It was clear that, in retrospect, even with the restructuring approved in February 2013, the Project would not be able to meet its PDO or any of its revised indicators. Restructuring can be an important tool to help reconfigure a problem project but it needs to be timely (after an early MTR) and realistic. Alternatively, a wise decision would be to recommend the cancellation of the project if restructuring is not feasible. 88. Implementation Support for investment lending can be used effectively to reinforce sectoral policy dialogue, but, unless there is clear commitment by the 26 Government to sectoral policy reform, it should not be the driving force behind that dialogue. In the context of ICR preparation, several Task Managers expressed that despite issues in implementation, the World Bank chose to maintain a poorly performing project on the books allegedly because it was an important vehicle in its social protection policy dialogue with the GOJ. In reality, keeping the loan on its books did little to advance the World Bank’s policy dialogue as the implementation support team was apparently fixed on the sectoral issues that were to be supported by the Project, losing sight of the bigger picture of issues and proposals the GOJ was considering. There is a need to ensure consistency in policy dialogue across assistance instruments, and a very clear division of roles in the policy dialogue between Task Team Leaders of ongoing projects and those under preparation, and other World Bank staff involved in defining and conducting policy dialogue through other vehicles, and defined work arrangements to ensure the consistency among all involved in delivering sectoral assistance. In the case of the Project, perhaps, if implementation support had benefited on the inputs of a broader social protection policy dialogue, at the highest levels, it could have internalized the constraints, issues, priorities and realities the GOJ faced, as well its commitment or not to sectoral policies, and in response taken appropriate proactive action to ensure a satisfactory outcome for the Project. Conversely, if the sectoral policy environment is not appropriate, it does not make sense to keep a project open for “policy dialogue” purposes; non-lending technical assistance would probably represent the assistance vehicle of choice. 89. One size does not fit all with respect to targeting of social protection assistance. PMT may be considered the best standard in targeting methodology, specifically for its ability to predict poverty status and ranking all beneficiaries in a given registry. However, PMT is not the only acceptable and not always the appropriate method of targeting. In many countries a combination of various targeting mechanisms are used (e.g. geographical, categorical, community, PMT). The World Bank’s implementation support focused almost exclusively on adoption of PMT, at the expense of addressing implementation of other components and the issues the Project was facing, without realizing both that the Government had serious issues with the methodology. The Project under implementation lost an opportunity to engage with the Government in the broadest sense and to help in consensus building, in the context of implementation support for the Project, on discussion of alternative methodologies, demonstrating flexibility in terms of different means of achieving very similar objectives, i.e., improved targeting efficiency. 90. Combining what is in effect policy conditionality in an investment loan/credit/grant should only be done in the context of a clear commitment by the Government. Not only should there be upfront agreement on any policy conditionality (i.e., improve targeting, etc.), but also there should be upfront agreement on process and methodology for how the reform will take place (i.e., PMT targeting mechanisms adopted), accounting for institutional buy-in and capacity to implement at all levels. 91. Poor institutional arrangements can have an adverse impact on the prospects for success in project implementation. To expect a project implementation unit, staffed by mostly outside consultants who both play a role of coordinating as well as directing 27 the project implementation, was unrealistic. The PISU was not expected to succeed in engaging several agencies in actually implementing a project while there was no support at the higher level or commitment across relevant agencies for realizing reform during the timespan of the project. Even with limited institutional capacity, it is generally best to have project implementation entrusted to those agencies that are actually responsible for the activities being carried out, with support provided directly to them by outside consultants if needed, and with a coordination unit to facilitate handling of administrative requirements only. 92. Social protection projects, especially those that aim at increasing the efficiency of existing systems, in particular cash transfer programs, face large challenges when exogenous factors impact the livelihoods of the covered population. These exogenous factors may increase the relevance of a government’s social protection policies and programs, by increasing the number of people who may expect and require government assistance. Yet, it is this very increased relevance that makes any effort to promote greater targeting efficiency more needed (in terms of fiscal sustainability) but more difficult to implement (in terms of popular reaction). Especially in an environment of increasing protests, it is difficult to expect that any government would have the political wherewithal to exclude some beneficiaries from existing programs under these circumstances. 93. Building Institutional capacity in implementing agencies presents challenges between utilizing government civil servants or relying on external consultants. The Bank’s fiduciary policies require compliance with, preparation and submission of, documentation for which government civil servants often do not have the capacity to fulfill. However, exclusive reliance on external consultants to fulfill these fiduciary requirements, results in little capacity building for civil servants. This presents a dilemma for Bank implementation of projects with Ministries that have limited fiduciary capacity. On the one hand, fiduciary compliance is needed, but on the other hand, delegation of tasks to ministries can build capacity. Therefore, careful attention should be given to designing of contracting arrangements for external consultants; the design both ensure fiduciary compliance as well as provide a capacity building opportunity for involved implementation agencies. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies No Comments or reaction from the Government was received. (b) Cofinanciers Not Applicable. (c) Other partners and stakeholders Not Applicable 28 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Component Appraisal Actual/ Latest Percentage of Estimate12 Estimate Appraisal (USD (USD millions) millions) A. MoSD, CCSS and 1.2 0.396 33.0% DoS Capacity Building B. National Aid Fund 1.1 0.080 7.3% Renewal C. Social Care 1.4 0.325 23.2% Services Upgrading D. Project 0.3 0.165 55.0% Management 4.0 0.967 24.1% Total Baseline Cost Physical Contingencies Not specified Not specified - Price Contingencies Not specified Not specified - Front-end Fee 0.01 0.01 10.0% Front-end Fee IBRD NA NA - Total Financing Required 4.0 - - (b) Financing Appraisal Actual/Late Type of Co- Estimate st Estimate Percentage of Source of Funds financing (USD (USD Appraisal millions) millions) Borrower 6.9 2.02 29.3% International Bank for 4.00 0.967 24.1% Reconstruction and Development 12 Table (a) reflects the appraised and actual expenditure amounts of the Bank’s contribution only. Physical and Price Contingencies were for the overall project cost, and therefore not listed in the table. 29 Annex 2. Full Description of Project’s Original Components Component A: Institutional Development and Capacity Building for Social Safety Net Policies Formulation and Implementation (Estimated Total Cost US$3.3 million, Bank financing US$1.2 million). This component aimed at strengthening and developing institutions involved in analysis, design, coordination, monitoring and evaluation and implementation of social safety net policies and programs, so that they perform their function and deliver services efficiently and effectively. This component included three sub-components: Sub-component A (1): Capacity Building of the CCSS. The Project would assist in building the CCSS capacity to apply qualitative and quantitative methods in researching social issues of relevance for social policy development and where information gaps were identifies. Standard statistical packages such as STATA and SPSS would be purchased and relevant staff trained in their use. Sub-component A (2): Strengthening Institutional and Human Resource Capacity of the MoSD. This sub-component was to assist MoSD in strengthening and further developing its capacity to formulate, regulate and implement social development and safety net policies and programs through: (i) formulation of the MoSD strategic development plan; (ii) transformation of the MoSD local offices into a one-stop shop for implementation of an integrated approach to service provision with focus on inclusion and activation of the poor and vulnerable population; (iii) development and implementation of monitoring and evaluation function; (iv) strengthening of the policy analysis and development unit of the MoSD; and (v) upgrading and further development of the existing MoSD management information system (MIS) into a real time WAN and a LAN, including a comprehensive database on vulnerable populations and beneficiaries of the MoSD programs. The database was to be linked in real time to other major databases relevant for the MoSD work and was to feed into the integrated social data system that would be managed and maintained by the CCSS. This sub-component would also support building of the MoSD human resource competencies, including: (i) a formulation of a strategy for the MoSD human resource development with detailed implementation plan, and (ii) a number of training activities to implement the plan, including to strengthen and develop relevant staff capacities to assess and analyze social problems; formulate social development and safety net policies; develop legal framework; plan, coordinate, manage and monitor MoSD activities and programs; use MIS system; perform day-to-day tasks better, and others. Sub-component A (3): Strengthening the Capacity of the Department of Statistics (DOS). This sub-component was to support strengthening of the household data collection and capacity building for poverty measurement and monitoring. Activities would include introduction into the DOS operations of (i) a standard software platform for statistical data entry and data management (e.g., BLASE, SPSS DATA ENTRY, etc.), and (ii) a standard software for data cleaning and data imputations. Standard software packages for statistical data analysis (e.g., STATA, SPSS, etc.) would be provided and staff trained in using them routinely as a tool for data processing, reporting and analysis. 30 Personal Digital Assistant (PDA), software for portable devices during enumeration phase of the household budget survey would be introduced. Relevant DOS staff would be trained in construction of consumption aggregate, poverty lines estimates and simulations, poverty and inequality measurement techniques, methodology for estimating value of home produced goods, and price collection methodology and calculation of price indices. Component B: Renewal of the National Aid Fund (Estimated Total Cost US$2.9 million, Bank financing US$1.1 million). This component aimed to improve the operations and management, coverage of the targeted poor population and targeting efficiency and effectiveness of the cash assistance extended to the population through the NAF. An automated, live, up-to-date database on poor and vulnerable population in Jordan would be established and functioning; the NAF staff would have improved skills to perform their tasks, and NAF would have the capacity to monitor delivery of assistance and carry out knowledge-based adjustments in both targeting and implementation arrangements. This component included the following activities: a) Implementation of an MIS to Support Establishment of a Database on Poor and Vulnerable Population and to Improve Targeting. NAF MIS was to use information and communication technology (ICT) as a means of achieving transparent, efficient and effective and client oriented service delivery. The activities were to include: (i) Installation of MIS-related hardware and voice and data communications at the central and in local NAF offices; (ii) MIS-related software development and implementation at the central and in the local offices; (iii) capacity development for hardware maintenance and software support and development; (iv) linking NAF with other data bases in the country for cross checks and verification of information on households; (v) linking NAF with institutions handling the payment of assistance to beneficiaries (post offices, banks, etc.) and (vi) training of the staff to use the MIS. Using WAN and LAN connection, local NAF offices, NAF headquarters and all NAF departments would be linked into an integrated MIS. The feasibility of the use of smart card technology would be examined and, if appropriate, would be introduced for delivery of assistance in those parts of Jordan where public access to financial institutions and their Automated Teller Machines was possible. The NAF data base and the MIS was to feed into/be part of the national integrated social data system to be managed by the CCSS. b) Strengthening NAF Technical, Administrative and Benefit Delivery Capacity. This activity was to include the establishment and capacity building of a technical and analytical unit, training of manager, training of the local offices staff, training of accountants and financial officers and improvements in financial management system, diversification of modes of delivering assistance to beneficiaries (smart cards, bank accounts, etc.). The staff in local MOSD offices not involved in performing NAF benefits related tasks would also be included in training activities relevant for their jobs (accounting, financial management, office management, etc.). This would also include support for collection of information from households for the database on poor and vulnerable population. 31 c) Improving Targeting through, but not limited to: (i) improved implementation and administrative arrangements; (ii) streamlined business processes; (iii) strengthened monitoring of staff performance; (iv) development of an operations manual to guide staff performance and timely and orderly execution of business processes; (v) strengthened collection of information on applicant households; (vi) strengthened eligibility testing procedures, including cross-checks and verification of information; (vii) streamlined complaints and applications renewal procedures; (viii) staff training; and (ix) planning and implementation of public information programs related to the NAF programs. Component C: Improving Access to and Quality of Social Work and Care Services (Estimated Total Cost US$3.8 million, Bank financing US$1.4 million). This component aimed to improve access to and quality of services to vulnerable groups such as persons with disabilities, women and children in difficult circumstances, dysfunctional families, youth with behavioral problems, youth in conflict with law, etc. It would support the MoSD in implementing a number of actions envisaged in the GOJ’s strategies targeting vulnerable groups (Section 1.1). Under this component, MoSD would develop core standards of social care, adopt an action plan for their implementation, and launch its implementation. The capacity of service providers to provide better care and rehabilitation would be strengthened, family centered, community based care and rehabilitation model would be piloted, service options available to vulnerable populations in six localities would be increased to include family counseling and a referral system, and monitoring and evaluation would be introduced. This component included the following activities: a) Developing Standards of Service and Monitoring Mechanism for Compliance with Standards. The MoSD would be assisted in developing core standards of services for a number of vulnerable groups and variety of institutions. The standards were to be adhered to by all providers, public and private, and were to serve as a basis for licensing service providers. The Project was to support: (i) an assessment and update of the existing standards; (ii) the development of the key missing standards, including those regarding physical facilities, equipment, staff qualifications and technical content of services provided by different institutions; (iii) training of the relevant MoSD staff in standards development and compliance; (iv) dissemination, including printing materials, organizing workshops, etc., and (v) setting up a mechanism for monitoring the compliance with standards. b) Upgrading the Model of Care in Three Residential Care Institutions. For residential care, the Project was to support transformation to a more individualized and family-like setting model. The new care model would emphasize outreach and family involvement in the provision of services; the services would be provided based on individual care and rehabilitation plans and case management approach in their implementation. In addition to upgrading the programs, the Project would support improvements in the centers’ governance and management organization and practice. The centers’ management reform was expected to be an integral part of upgrading the model of care. 32 c) Piloting of the ISSC. The Project would support piloting of three ISSCs. These centers were expected to fill in the gap in social work, care and rehabilitation services in Jordan that pertained to the scarcity of professional counseling for vulnerable individuals and families and lacked an integrated approach to the provision of services through an organized, well-functioning referral system. The ISSCs would be piloted in three localities, one in the North of Jordan, the second in the South, and the third in Amman. The Project was to finance setting up of the centers, including: refurbishment of facilities; preparation of an operational manual that would guide operations and provision of services and would include terms of reference for staff and methodological guidelines and instructions; training of staff; equipment; vehicles; technical assistance and operating costs for two years of operations. In the final year of project implementation, the financing of ISSCs would be assumed by the MoSD. The establishment of the ISSCs was to be documented and their functioning and performance evaluated by a third party, as well as by the MoSD. The lessons learned would be taken into account for improvements in their functioning, as well as for their national roll out. d) Piloting of the RIC for Persons with Disabilities. The Project was to establish three RICs that would be used as focal points to develop and strengthen services for persons with disabilities through a community-based rehabilitation approach. These focal points would serve as a “one-stop-shop” for persons or organization of persons with disabilities. In cooperation with community-based and disabled persons’ organizations, community workers were to be identifies to be part of a network of trained individuals able to support persons with disability, their families and caretakers, and to find appropriate solutions to removal of barriers to independence and inclusion of persons with disabilities. The Project was to finance the setting up of the center, including: refurbishment of facilities, preparation of an operational manual that would guide functioning of the centers and provision of services, training of the centers’ staff, master trainers and community rehabilitation workers; equipment, vehicles, technical assistance and operating costs for two years of operations. In the final year of project implementation, the financing of RICs would be assumed by the MoSD. The process of establishment of the centers would be monitored and documented and their functioning and performance assessed by a third party, as well as by the MOSD. The lessons learned would be taken into account for improvements in their functioning, as well as for their national roll out. The location of the centers was to be decided by the MOSD, based on the then recently completed census of persons with disability in areas with the highest concentration of persons with disability. e) Piloting MoSD Social Innovations Program. The Project was to assist the MoSD in designing a competitive program that would provide seed money for implementation of innovative projects in social work, care and rehabilitation. The fund was to be open to all public and private service providers and would encourage public-private partnership. The Project would support technical assistance for the program design and would finance the innovations grants for the first year of the program’s implementation. The MoSD was planning to attract other donors’ contributions to the program. 33 Component D: Project Management (Estimated Total Cost US$0.8 million, Bank financing US$0.3 million). This component aimed to strengthen the capacity of the MOSD and other institutions involved in project implementation (NAF, CCSS and DOS) to efficiently and effectively implement the Project, while adhering to the Bank’s fiduciary requirements, particularly with respect to procurement and financial management. The Project contemplated that when technical assistance was utilized, it was to be delivered in close collaboration with relevant departments/individuals, so that the transfer of knowledge and hands-on training materialized immediately. 34 Annex 3. Comparison Table of Original and Restructured Project Components, Financing, and Output ITEM ORIGINAL (PAD) Revised (RP) ACTUAL AT CLOSING Loan amount (in millions of 4.0 3.18 .968 US dollars) % IBRD Financing of Project 36.6% 100% N/A Project Development “to improve the management and Unchanged N/A Objectives operations of the cash social assistance programs and to improve the access to and quality of social care services.” Project Component: A RESTRUCTURED (Institutional Development and Capacity Building for (1) Capacity Building of the (1) Subcomponent completed (1) N/A. CCSS committee was SSN Policies Formulation and Coordination Commission for dissolved in 2010. (No Implementation) Social Solidarity information is available on (2) Strengthening Institutional and (2) Cancelled level of support). Human resource Capacity of the (2) (i) Strategic Development plan MoSD developed by consultant. (3) Strengthening the capacity of (3) Some activity occurred, remaining However, ICR visit shows that DoS “advanced poverty analysis work” the plan was not adopted by the cancelled Ministry. Human Resource Strategy was not adopted. (ii) No evidence of changes in business operations at local offices following ICR field visit (iii) M&E Unit was not established. (iv) No trainings were carried out for new HR strategy (v) Some IT and software up-grading purchases were made. (3) STATA and SPSS software licenses have been purchased and staff trained. PDAs were purchased and will be used for data entry. Project Component: B (1) Implementation of a MIS to RESTRUCTURED (Renewal of the National Aid support database on poor and (a) (i) completion of a pilot involving 35 Fund) vulnerable population and to (a) designing/implementing a more efficient about 7500 households; (ii) improve targeting system of targeting the poor (e.g., PMT), and updating of the PMT targeting (2) Strengthening NAF Technical developing an MIS system formula utilizing the 2008 HIES; Administrative and Benefit (b) developing efficient systems (financial and (iii) revisions of the PMT and Delivery Capacity management/business processes, etc.) to NAF questionnaires. (3) Improving targeting administer the NAF program, (b) No information (c) building human resource/institutional (c) No information capacity All the remaining activities under (b), (c) and (d) related to improving the current targeting system and related business processes will continue, while activities under (a) pertaining to implementation of the new PMT targeting will be cancelled as this targeting mechanism was not approved by the Government. Project Component C RESTRUCTURED (1) Sub-component completed (Improving Access to and prior to restructuring. No Quality of Social Work and indication of adoption of Care Services) (1) Developing standards of services (1) Subcomponent completed standards during field visit and and monitoring mechanism for interviews at ICR compliance with standards (2) 3 centers received funding. (2) Upgrading the model of care in (2) Ongoing (at time of restructuring) Amount of financing was three residential care institutions limited to transform the three centers. All centers adopted (3) Piloting of the integrated Social (3) Reduced to 1 pilot center residential care models through Service Centers (ISSC) MoSD funding. (4) Piloting of the Resource and (4) Reduced to 1 pilot center (3) No ISSC functional. However, Information Centers (RIC) for a consulting firm was hired Persons with Disabilities during the project to work with MOSD on the ISSCs. The firm (5) Piloting MoSD Social Innovation (5) “all activities pertaining to Social finalized the locations of each Program Innovation Fund were cancelled” center, and designed the centers accordingly. (4) One firm was selected for 36 rehabilitation and the awarded contract. Work was completed. Building abandoned as the rehabilitated space was below local building code standards for persons with disability. (5) N/A Project Component D (Project Strengthen capacity of the MoSD and The project will support the hiring of a At closing, the Project did not have a Project Management) other institutions involved in project project director and procurement office, and Director. High Turn-over was cited during implementation to efficiently and will continue with the M&E activities. ICR interviews as a hurdle to effectively implement the proposed project implementation. MoSD employed civil servants for procurement and FM functions. Financing Arrangements  Specific Investment Loan (SIL) is  Fully finance the project N/A the instrument to support the  Cancel US$831,797 from the loan modernization program. amount – revised amount $3.18M  IBRD financed US$4.0 including retroactive financing of US$400,000. Total estimated project cost for SPEP is US$10.9M Institutional and  MoSD responsible for the overall  Additional fiduciary management  At closing the FM and Procurement Implementation project implementation. PISU to by FM and procurement specialist specialists were civil servants from Arrangements, FM, and lead the work headed by (consultants). MoSD procurement Secretary General. PISU is  New FM Consultant hired  FM and Procurement were rated responsible for FM, procurement,  Revision of procurement practices moderately unsatisfactory TORs, announcing bids, and bidding standards. organizing evaluation, and project planning.  Hiring includes: Project Director, Project Assistant, FM, Procurement Specialist. Closing date August 31, 2013 Unchanged Closed on August 31, 2013 37 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Program Afifa Alia Achsien Senior Program Assistant MNSHD assistant Ghassan N. Alkhoja Senior Operations Officer MNSSP Operations Financial Diana C. El Masri Consultant MNAFM Management Lina Fares Senior Procurement Specialist MNAPC Procurement Iqbal Kaur Sr Social Protection Specialist MNSSP Co-TTL Michael M. Lokshin Manager DECRG Technical Aleksandra Posarac Country Sector Coordinator EASHS TTL Haneen Ismail Sayed Lead Operations Officer MNSSP HD Coordinator Hjalte S. A. Sederlof Consultant IEGPS Technical Eileen Brainne Sullivan Health Specialist EASHH Operations Supervision/ICR Name Title Unit Responsibility Aleksandra Posarac Country Sector Coordinator EASHS TTL Haneen Ismail Sayed Lead Operations Office MNSSP HD Coordinator Eileen Murray Country Manager MNC01 TTL Lire Ersado Sr. Economist MNSSP TTL Afifa Alia Achsien Senior Program Assistant MNSHD Operations Ghassan N. Alkhoja Senior Operations Officer MNSSP Operations 38 Robert Bou Jaoude Program Coordinator SACPK Operations Amy Champion Operations Analyst MNSHH Operations Lina Fares Senior Procurement Specialist MNAPC Operations Iqbal Kaur Sr Social Protection Specialist MNSSP TTL Renata Lukasiewicz Program Assistant CPFCI Operations Sr Financial Management Jad Raji Mazahreh MNAFM Operations Specialist Eileen Brainne Sullivan Health Specialist EASHH Technical Setareh Razmara Lead Social Protection Specialist MNSSP ICR TTL/ Lead ICR Co- Leader Amr Moubarak Social Protection Economist MNSSP / Co-author Suzana Abbott Consultant MNSSP ICR co-author (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) Number of staff weeks USD Thousands (including travel and consultant costs) Lending FY07 45.0 230,419.06 FY08 36.0 224,470.56 FY09 13.8 62,510.95 FY10 12.6 58,116.47 FY12 3.7 15,933.63 Total 111.2 591,450.67 Supervision/ICR FY09 3.8 30,866.64 FY10 11.6 78,033.41 FY11 15.7 90,645.38 FY12 13.3 116,973.20 FY13 19.6 96,065.90 FY14 10.7 66,736.39 Total 74.7 527,686.90 Lending and Supervision 1,119,137.57 Total 39 Annex 5. Beneficiary Survey Results Not Applicable 40 Annex 6. Stakeholder Workshop Report and Results None 41 Annex 7. Summary of Borrower's ICR “An Evaluation of the Social Protection Enhancement Project from the view of the Government of Jordan”13 Prepared December 2013 Ministry of Social Development Introduction: 1. The Government of Jordan agreed to a project for the total amount of $ 10.880 million US which was to be financed by the World Bank in amount of US $4.0 million totaling around 40%. The remaining amount was financed by MoSD. The agreement was signed on 8 July 2008. 2. The project has three major goals: i) enhancement of the capacity of the ministry, ii) improvement of targeting mechanisms for the poor, iii) improvement of social care centers. 3. To that end the project had 4 components: A) Enhancement of the institutional capacity of the ministry for social projection, B) Renewal and modernization of the National Aid Fund, C) Enhancement of the capacities for social care for women and children, D) Project management of the Social Protection Enhancement Project. Overall accomplishments of the project from effectiveness until restructuring: Component 1: Update the National Assistance Fund A. Enhancement of the Capacity of Ministry of Social Development 4. Social Solidarity Coordination Commission (SSCC): The commission component was responsible for purchase of SPSS for DOS. The commission also appointed an expert on poverty to enhance the capacity of the coordination commission. The commission also received trainings to increase its overall capacity. However the commission was disbanded and it became a part of the ministry’s overall structure. 5. Department of Statistics: This component also move forward with purchase of statistical packages. The Project also financed the purchase of 75 PDA for enumeration in a mobile way on the ground. Some of the DoS employees also received training from the 13 Editor’s note: This is an English summary of the original report which was prepared by Ministry of Social Development (MoSD) in Arabic and was received by the World Bank on 2/24/2014. The original Arabic report is 32 pages in length and was filed on WB Documentation Database (WBDocs). The summary includes some editorial revisions. 42 project. Some of the project participants were also given the opportunity to attend trainings in Egypt. 6. The project also spent on recruitment of a statistics expert and a procurement consultant in the purchase of some of the goods for the project (including the furnishing of a computer lab in DoS). 7. Building the Capacity of the Ministry in Human Resources and Monitoring and Evaluation : The project attempted to recruit an expert in human resources since 2010, however an expert was only hired in March 2011. The Project Implementation Unit requested that the Bank raises the budget for this component (and payment of expert from $50,000 to 100,000 US). This came after the Minister approved this. This was not approved by the World Bank. 8. Although the World Bank did not approve the increase the allocation of this component in the project it did contribute in this component in the following ways: i) A ministerial strategy was developed for servicing cases of delinquent and special needs kids ii) A national strategy was developed for servicing cases of orphan children iii) Recruitment of a Monitoring and Evaluation expert (which was paid for by the ministry). 9. Building the Capacity of the Ministry in MIS: The project moved forward with the appointment of an MIS consultant to work on increasing the capacity of the technology and information department of the ministry. The main duties of the consultant were: i) connect MoSD and NAF with a global server network managed by the Ministry of Telecommunication ii) Provide a study of the needs to modernize the ministry’s information systems, iii) Provide a study and recommendations on the needs of the provincial offices of the ministry, iv) provide a study of the needs of the headquarters offices of the ministry. 10. A second consultant was also hired to manage the overall operations of the implementation. However this work was hindered since it did not have an allocation in the budget. Alternatively, some of the assistance from this component went to provide similar studies to NAF. There, NAF leadership appointed a consultant. The consultant reported on the preliminary steps needed to provide services and register cases via MIS. 11. 60 ministerial processes were also expected to be modernized over the course of the project. This modernization process involved the purchase of a number of computer programs at regional offices, as well as the purchase of servers and there was allocation for funding for training to increase the capacity. However, these processes were not modernized at the time of the writing of the report. Component 2: Renewal the National Assistance Fund 43 B. Modernization of Targeting Mechanism for the National Aid Fund 12. Phase 1 – Pilot phase: During the early phases of the project, NAF brought in a local expert to develop a PMT formula based on the 2006 Household Income and Expenditure Survey. The expert specified 75 variables in determining the poverty level of households using the formula. Accordingly, the ministry hired a private software firm to put the PMT formula into the NAF database. 13. In 2009, NAF moved forward with developing a survey questionnaire for 100,000 families and the appointment of 50 social outreach workers to collect and input the information in the NAF database for purposes of targeting. Between 2009- 2010, Social Workers brought 7,500 household questionnaire. This pilot was discussed with the World Bank. Main outcomes of pilot phase were: i) discussion of graduation and case handling mechanism for those excluded ii) Upgrading of formula used to determine eligibility of beneficiaries starting in 2008, iii) World Bank insisted on using a shorter survey of no more than 67 variables and that field visits are a part of evaluation, iv) World Bank team agreed to evaluate the 2008 formula for use in NAF targeting. 14. Phase 2 - Using 2008 Data for Targeting: NAF went forward with using the HEIS data of 2008 as a way to improve its overall targeting. In 2010, a local expert presented a new PMT formula using 75 variables to identify and target poor households. The expert also presented overall estimates of the costs for administration of programs to reduce poverty in Jordan. This was presented and discussed with the Council of Ministers after consultation with the World Bank. 15. Main outcomes of Phase 2: i) a new questionnaire was developed, which was divided into two questionnaires for both house visit and at center applications, ii) A draft operational manual for training social workers on the new questionnaire was developed, iii) The project implementation unit requested of the Council of Ministers to appoint Outreach Workers to move forward with house visits (the council later rejected the proposal), iv) Council of Ministers approval was attained, however, for a communication campaign so that the public understands the new formula developed. 16. Phase 3- Scenario Planning Phase: During this phase, the ministry was aware that in order to close the poverty gap JD 105.5 million per year is needed based on the 2008 data. This is with the assumption that the medium expenditure of NAF beneficiary is JD 23 per month. This required an increase of JD 16 million in the budget. Further, recommendations were cited that NAF can be used as a utility for cash compensation of 44 subsidies. In fact, utilization of NAF would provide JD 200 million in savings. This was in agreement with the World Bank expert. 17. These recommendations were discussed at length with the Secretary General of MoF. To that end, MoF and NAF agree on the following next steps: i) roll-out of a public awareness and communication campaign, ii) education of the NAF employees on their potential change in responsibilities, iii)beginning of a pilot communication campaign in 3 areas to explain the new PMT formula. The gradual approach for introduction of a new targeting formula was accepted and consultations with the parliament and media were en route. The work was stopped by the Minister of Social Development at a later stage. 18. Concurrent activity – enhancement of MIS for NAF: The project went forward with enhancement of the overall MIS of NAF. This was done after a rapid needs assessment in 2010. Some purchases of hardware and software were secured. Component 3: Improve the Quality and Methods of Obtaining Services and Social Welfare D. Modernization of Social Care Services to residential fashion 19. There centers received some assistance in furnishing and reconstruction. They were: i) Madaba center for incidents (rented property minimum assistance received given that it is not owned), ii)Irbid Center work stopped due to security incidents in center, however, some renovation was done, iii) Girls Center in Resayfeh: this contract was not completely carried out and was therefore partially procured by the contracted company, iv) trainings for employees in the three centers on principals of residential social care was envisioned, however, the contract did not move forward. F. Establishment of Centers for Integrated Social Service Delivery (3 Centers) 20. The ministry identified 3 centers to be renovated into integrated social service delivery centers. They were Umm Nuwarah in Amman, Rehsyifeh Center in Amman, and Karameh Center in Aqaba. 21. A company (Maharah Consulting) was hired to develop the standards of operations and training manuals for the employees of the newly envisioned integrated service centers. However, the contract was partially suspended in 2012 after standards of operations were submitted and some training material. This was due to the inability of the ministry to provide the needed human resources and capacity to receive training from the hired firm and to restructure these 3 centers. The contract was also partially suspended the project restructuring effort which was proposed by the Bank. This included the reduction of number of centers (from 3 to 1). The contract was finally closed. 45 G. Establishment of Information Centers for the Disabled (3 Centers) 22. There were three centers under consideration for this component: i) Manar Center in Rehsyifeh (only renovation to center, no furnishings purchased), ii) Manar Center in Irbid (contract stopped), iii) Disabled Persons Center in Aqaba (new center built and it needed little renovation assistance). H. Social Innovation Fund 23. A special committee was created to follow-up on issues related to standards for service of private charities and organizations that apply to the innovation fund. The social innovation fund was later consolidated to become part of the working of the Charities Fund to streamline implementation. 46 Annex 7 - Table 1: Social Protection Enhancement Project – activities under original and restructured project14 Original Project Restructured Project Explanation Component 1: Institution development and capacity building 1.1 Coordinating body for 1.1 Coordinating body for No change, activities were social solidarity (CCCSS) social solidarity completed and the coordinating body for social solidarity no longer exists 1.2 Ministry of Social Cancelled For more than two years, Development: Human the ministry of Social resource development plan; Development did not establishment of a approve the monitoring and evaluation subcomponents’ contracts unit, and management and action plans information system (MIS) There is no clear evidence that there is ownership and that the Ministry of Social Development will continue circulating the results of these activities in the same manner. 1.3 Department of Statistics Cancelled The government does not plan to utilize the funds for the remaining activities under the sub-components of the department of statistics Component 2: Renewal the National Assistance Fund 2.1 Database and Targeting Database and Targeting No change Planning Mechanism Cancelled This activity is related to the adoption of the targeting test based in family expenditure, therefore there is no need for the adoption of the above-mentioned previous test. Planning instructions, Cancel: printed models Only some outstanding 14 The next two tables of this summary are translated fully as presented by the Government. 47 printed models, media activities to continue to campaigns, and availability Keep: Awareness provide strong ownership of training campaigns and and ability to unite the managers/supervisors development instructions National Assistance Fund Add: Stationary Collecting household Cancelled This activity is related to information the adoption of the targeting test based in family expenditure, therefore there is no need for the adoption of the above-mentioned previous test. Monitoring and Evaluation Monitoring and Evaluation No change, unit unit Only some outstanding activities to continue to provide strong ownership and ability to unite the National Assistance Fund National Assistance Fund Cancelled This activity is related to survey measures the adoption of the targeting test based in family expenditure, therefore there is no need for the adoption of the above-mentioned previous test. 2.2 Management Cancel: Independent outstanding activities to Information Systems evaluation of third party continue to provide strong ownership and ability to Keep: all remaining unite the National Assistance Fund Add: Conservation (Archiving) 2.3 National Assistance National Assistance Fund No change, Fund capacity building capacity building Only some outstanding activities to continue to provide strong ownership and ability to unite the National Assistance Fund 48 Component 3: Improve the quality and methods of obtaining services and social welfare 3.1 Development of basic Completed standards of services 3.2 Update forms of care in Keep three welfare centers No change, three welfare centers Continue and resume related activities 3.3 Establish three integrated Keep one center and cancel One center in Amman, social service centers two which will continue to benefit from renewing the work of the remaining activities under the project. Cancel the project support of the other two centers as there has been no decision on employment in the long- term 3.4 Establish three Keep one information Continue renewing and information centers for the center for the disabled, and hiring at the Rasefia disabled as part of the cancel the other two (Almanar) center and ministry of Social benefit from the revenues Development of the project according to the plan. Cancel project support for the other two centers (they have not been identified), and decisions were not made with regards to long- term employment 3.5 Social Innovation Fund Cancelled Lack of ownership and improvement Component 4: Project Management 4.1 ministry of Social Keep: Project manager and Continue hiring for the Development administration Procurement Officer remaining period of the project Project manager, Monitoring Cancel: Financial officer, , and Evaluation officer, Project management 49 Procurement Officer, Project assistant, and secretary management assistant, and secretary 4.2 Technical unit in the No change: continue hiring National Assistance Fund experts 4.3 Monitoring and Monitoring and Evaluation No change: activities under Evaluation unit unit the individual and budget component were combined. The initial plan was to evaluate each component separately, but this is no longer the case, evaluation of activities will be done as one evaluation study for all project components. 4.4 Periodic costs and Periodic costs and No change incidental expenses incidental expenses 50 Annex 7 - Table 2: Reallocation of loan revenues (since the restructuring date):15 Expenditure Category Allocations Funding Percentage (%) Current Revised Current Revised Goods and Labor, advisory 3,727,500 2,895,703 40% 100% services and operating costs Loan management fees 10,000 10,000 Not specified 262,500 262,500 Sub-total amount 4,000,000 3,168,203, Amount cancelled after 831,797 restructuring Original total amount 4,000,000 4,000,000 15 Editor’s note: Information presented in this table does not correspond to fiduciary reports submitted to the Bank during ICR. 51 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not Applicable 52 Annex 9. List of Supporting Documents  World Bank Country Assistance Strategy (2006)  World Bank Country Partnership Strategy (2011)  Project Appraisal Document for Social Protection Enhancement Project (SPEP) - P100546 (2007)  World Bank Mission Aide-Memoires, Social Protection Enhancement Project – P100546 (2007-2013)  World Bank ISR, Social Protection Enhancement Project – P100546 (2007-2013)  Jordan National Poverty Agenda (2006)  National Plan of Action for Children 2004-2015 (2004)  National Strategy for Persons with Disabilities (2007)  Jordan National Strategy for Youth (2003)  Jordan National Strategy for Women (2004)  National Strategy for Family Protection (2005)  World Bank Poverty Assessment for Jordan (2004)  “Jordan: A Note on Strategy for Modernization of Social Safety Nets”, World Bank Group (2007)  Restructuring Paper, Social Protection Enhancement Project (SPEP) (2013)  Mid-Term Evaluation Report, Social Protection Enhancement Project, CONSULT-US MENA Group (2011)  Jordan’s National Aid Fund: Categorical vs. Poverty-based Targeting, Ersado, Lire, Victoria Levin and Haneen Sayed(2012) 53 IBRD 33424 35°E 36°E 37°E 38°E 39°E 34°N 34°N n Se anea r a er dit JORDAN Me 33°N 33°N To To Zefat Damascus Lake To Tiberias Baghdad Um Qais Irbid IRBID IRBI D Ar Ruwayshid AJLUN Ajlun Al Mafrak River JARASH Jarash MAFRAK n Arda Mahattat al Halif Jorda BALQA Az 32°N As Salt 32°N Zarka AMMAN ZARKA Azraq ash - - Shishan Madaba To Jerusalem MADABA AMMAN Dead Sea To Al Jawf Al Mazra’ah - Al Qatranah To Al Karak Beersheba KARAK -- As Safi 31°N 31°N Ard TAFILAH as At Tafilah Sa w - Al Rashadiyah - Ba’ir aa n Ash Shawbak MA'AN Petra Al Jafr 0 0 50 Kilometers Ma'an 0 25 50 Miles 30°N Ra’s an Naqb 30°N 38°E 39°E AQABA Ad Disi To Nuweiba JO R D A N Jabal Ram Aqaba (1,734 m) SELECTED CITIES AND TOWNS aba GOVERNORATE CAPITALS Al Mudawwarah of Aq This map was produced by NATIONAL CAPITAL To the Map Design Unit of The World Bank. The boundaries, Al B'ir RIVERS Gulf colors, denominations and any other information shown 29°N on this map do not imply, on MAIN ROADS the part of The World Bank To Group, any judgment on the RAILROADS Al B'ir legal status of any territory, or any endorsement or acceptance of such GOVERNORATE BOUNDARIES boundaries. INTERNATIONAL BOUNDARIES 35°E 36°E 37°E JANUARY 2005