Document of The World Bank Report No: ICR00002859 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H1620 IDA-H5920 TF-92544) ON A GRANT IN THE AMOUNT OF SDR 26.2 MILLION (US$40 MILLION EQUIVALENT) AND ADDITIONAL FINANCING GRANT IN THE AMOUNT OF SDR 13.1 MILLION (US$20 MILLION EQUIVALENT) AND AFGHANISTAN RECONSTRUCTION TRUST FUND GRANT IN THE AMOUNT OF SDR 3.1 MILLION (US$5.0 MILLION EQUIVALENT) TO THE ISLAMIC REPUBLIC OF AFGHANISTAN FOR THE STRENGTHENING HIGHER EDUCATION PROJECT DECEMBER 26, 2013 Human Development Sector South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective December 26, 2013) Currency Unit = Afghani (AFN) AFN 1.00 = US$ 0.0176 US$ 1.00 = AFN 56.799 FISCAL YEAR 21 December- 20 December ABBREVIATIONS AND ACRONYMS AFN Afghani ANDS Afghanistan National Development Strategy ARTF Afghanistan Reconstruction Trust Fund DGA Development Grant Agreement EC Executive Committee EERDP Emergency Education Rehabilitation and Development Program HEMIS Higher Education Management Information System ICR Implementation Completion Report IDA International Development Association KPI Key Performance Indicator M&E Monitoring and Evaluation MoHE Ministry of Higher Education MoF Ministry of Finance MTR Mid-Term Review NATO North Atlantic Treaty Organization NCDC National Curriculum Development Committee NHESP National Higher Education Strategic Plan PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective QA Quality Assurance RF Results Framework SDR Standard SHEP Strengthening Higher Education Program TSS Transitional Support Strategy UNESCO United Nations Education, Scientific, and Cultural Organization USAID United States Agency for International Development Vice President: Philippe H. Le Houerou Country Director: Robert J. Saum Sector Manager: Amit Dar Project Team Leader: Harsha D. Aturupane ICR Team Leader: Kimie Tanabe ICR Authors: Roshini Ebenezer/Kimie Tanabe AFGHANISTAN Strengthening Higher Education Project (SHEP) CONTENTS DATA SHEET A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design............................................................ 1 2. Key Factors Affecting Implementation and Outcomes ........................................................... 4 3. Assessment of Outcomes ....................................................................................................... 11 4. Assessment of Risk to Development Outcome ..................................................................... 19 5. Assessment of Bank and Borrower Performance .................................................................. 19 6. Lessons Learned .................................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ....................... 23 ANNEXES Annex 1. Project Costs and Financing....................................................................................... 24 Annex 2. Outputs by Component .............................................................................................. 25 Annex 3. Economic and Financial Analysis .............................................................................. 28 Annex 4. Bank Lending and Implementation Support/Supervision Processes ......................... 34 Annex 5. Beneficiary Survey Results ........................................................................................ 36 Annex 6. Stakeholder Workshop Report and Results ............................................................... 36 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR .................................. 39 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders .................................... 44 Annex 9. List of Supporting Documents ................................................................................... 45 MAP A. Basic Information Afghanistan: Strengthening Higher Country: Afghanistan Project Name: Education Project Project ID: P089040 L/C/TF Number(s): IDA-H1620,IDA-H5920,TF-92544 ICR Date: 12/15/2013 ICR Type: Core ICR ISLAMIC REPUBLIC OF Lending Instrument: ERL Borrower: AFGHANISTAN Original Total Commitment: USD 40.00M Disbursed Amount: USD 55.7M Revised Amount: USD 65.00M Environmental Category: C Implementing Agencies: Ministry of Higher Education Co-financiers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 04/19/2004 Effectiveness: 07/14/2005 07/14/2005 Appraisal: 02/21/2005 Restructuring(s): 05/07/2010 Approval: 05/19/2005 Mid-term Review: 04/08/2008 04/24/2008 Closing: 06/30/2010 06/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation Indicators QAG Assessments (if any) Rating Performance Potential Problem Project at Yes Quality at Entry (QEA): Moderately Satisfactory any time (Yes/No): Problem Project at any time No Quality of Supervision (QSA): None (Yes/No): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 4 10 Tertiary education 96 90 Theme Code (as % of total Bank financing) Administrative and civil service reform 33 33 Education for the knowledge economy 67 67 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Praful C. Patel Country Director: Robert J. Saum Alastair J. McKechnie Sector Manager: Amit Dar Julian F. Schweitzer Project Team Leader: Harsha D. Aturupane Keiko Miwa ICR Team Leader: Kimie Tanabe ICR Primary Author: Roshini Ebenezer F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective of the project is to progressively restore basic operational performance at a group of core universities in Afghanistan. Revised Project Development Objectives (as approved by original approving authority) N/A (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value* Revised approval Completion or Target Values documents) Target Years Indicator 1 : Percent of project faculties using revised and updated curriculum Universities with100% 65% of the selected 100% of the selected revision of Value 0% (Curriculum is dated by faculties in 6 faculties have revised curriculum quantitative or at least 30 years in most universities with their curricula and completed and Qualitative) faculties.) revised curriculum these are currently being used in in use. being implemented. selected faculties. Date achieved 04/25/2005 07/14/2005 06/27/2010 06/30/2013 Comments ACHIEVED (100%). Moreover, the entire public higher curriculum (for all faculties) (incl. % has now been revised and this will be used by all public universities in the new achievement) academic year. Indicator 2 : Increased percentage of faculty members with at least a Master's degree 38% of the faculty at the 6 50% in Balkh, 38% project universities has at in Herat, 61% in 50% of faculty 49% of faculty at the Value least a Master's degree Kabul, 95% in at the 12 project twelve project quantitative or (40% in Balkh, 18% in Kabul Polytech, universities universities have at Qualitative) Herat, 41% in Kabul, 90% 51% in Nangarhar have at least a least a Master's in Kabul Polytech, 41% in and 40% in Master's degree degree Nangarhar, 32% Kandahar). Kandahar . Date achieved 04/25/2005 06/01/2005 04/15/2010 06/30/2013 ACHIEVED (99%). The target for increasing the percentage of faculty with a Master's Comments degree was almost fully achieved with the SHEP support for graduate (Master's level) (incl. % education for 345 faculty members at the 12 universities. The unit of measurement for achievement) the target had to be revised at the ICR stage (it was listed as an absolute value in the AF document) to keep consistency with the unit of measurement for the indicator itself. Percent of administrators at the university and the MoHE with management Indicator 3 : training 40% of 20% administrative administrative 50% of staff from 6 staff from 12 administrative staff at Value There was no management universities and universities and MoHE and the quantitative or training provided. MoHE to receive MoHE to project universities Qualitative) management receive have received training. management management training. training. Date achieved 04/25/2005 06/01/2005 04/15/2010 06/30/2013 THIS TARGET WAS SURPASSED BY 10%. Under SHEP, 1,166 staff at the MoHE and the project universities received management training through training workshops Comments and seminars; increasing the overall percentage of those with management training to (incl. % 50%. The targets were revised slightly at the ICR stage to ensure consistency in the unit achievement) of measurement between the indicator and the target (as these were also listed as absolute numbers in the original documents). Increase in the number of operational labs, libraries and other facilities in selected Indicator 4 : project faculties Computer, science Labs and labs and libraries Limited use of lab facilities teaching and have become (There were 25 libraries, 32 learning operational in science labs, and 3 70% increase in the facilities in selected faculties of Value computer labs across the number of selected all 12 universities (12 quantitative or twelve project universities operational lab faculties project universities Qualitative) at baseline, however it is facilities. become have a total of 84 unclear whether these were operational in science labs, 50 fully operational. the 12 computer labs, and universities 37 libraries which are fully operational) Date achieved 04/25/2005 06/01/2005 04/15/2010 06/30/2013 Comments ACHIEVED. The original indicator identified measurement in terms of a percentage (incl. % rather than an absolute number but the targets were in absolute values, so the unit of achievement) measurement for the indicator was changed from percent to number at the ICR stage Number of project universities that have at least one more functional academic Indicator 5 : building All twelve project All twelve project universities universities have Functional physical Value construct or constructed or infrastructure at the project quantitative or renovate at least renovated at least one universities was very Qualitative) one new new building for limited. building for teaching and teaching and learning. learning. Date achieved 04/25/2005 06/30/2013 06/30/2013 Comments ACHIEVED. This indicator was introduced at the ICR stage in order to more (incl. % accurately assess the achievement of the PDO. achievement) Indicator 6 : The development of a national higher education sector plan A five-year National Value A higher No higher education Higher Education quantitative or education sector strategic sector plan exists. Strategic Plan has Qualitative) plan developed. been developed. Date achieved 04/25/2005 06/30/2013 06/30/2013 Comments ACHIEVED. This indicator was introduced at the ICR stage in order to more (incl. % accurately assess the achievement of the PDO. The development of the National Higher achievement) Education Strategic Plan (2010-2014) was a seminal accomplishment. Indicator 7 : Number of project universities with a functional student service center A functional All 12 project Value No student service centers student service universities have a quantitative or at any of the project center at all 12 functional student Qualitative) universities. project service center. universities. Date achieved 04/25/2005 06/30/2013 12/02/2013 Comments ACHIEVED. This indicator was introduced at the ICR stage in order to more (incl. % accurately assess the achievement of the PDO. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Number of project universities with strategic plans developed and made Indicator 1 : operational Value Only Kabul University had All 6 universities All 12 All 12 universities (quantitative a strategic plan in place. implementing universities have developed and or Qualitative) strategic plans. implementing implemented strategic plans strategic plans. Date achieved 04/25/2005 06/01/2005 06/27/2010 06/30/2013 Comments (incl. % ACHIEVED (100%) achievement) Higher Education Management Information System (HEMIS) developed and Indicator 2 : introduced A Higher Education Information Value HEMIS Management System HEMIS developed (quantitative No HEMIS in existence. developed and (HEMIS) has been and in place. or Qualitative) introduced. developed and introduced to 9 project universities. Date achieved 04/25/2005 06/01/2005 04/15/2010 06/30/2013 Comments PARTIALLY ACHIEVED (75%). This is a work in progress, and as such it (incl. % determined to be partially achieved. achievement) Indicator 3 : Number of project universities that have carried out an internal self-evaluation All 12 All 12 universities Value None of the project 6 universities carry universities have carried out an (quantitative universities carried out an out an internal self- carry out an internal self- or Qualitative) internal self-evaluation. evaluation. internal self- evaluation. evaluation. Date achieved 04/25/2005 06/01/2005 06/30/2013 06/30/2013 ACHIEVED (100%). This indicator was part of the original Results Monitoring Comments Framework approved in 2005. However, it was not monitored during the project. The (incl. % ICR re-introduced this indicator as it is useful in measuring the achievement of the achievement) PDO. Establishment of a national Quality Assurance (QA) body with national standards Indicator 4 : in place for QA A National QA Commission has been Established established and Value No national QA body exists National body national standards for (quantitative and there are no National for QA with QA are established or Qualitative) QA standards in place. QA standards in and have been place. introduced to all 12 project universities. Date achieved 04/25/2005 06/30/2013 06/30/2013 Comments ACHIEVED. This indicator was introduced at the ICR stage in order to more (incl. % accurately assess the achievement of the PDO. achievement) Indicator 5 : Establishment of a National Curriculum Development Committee. Established A National Value National Curriculum (quantitative No such committee exists. Curriculum Development or Qualitative) Development Committee has been Committee established and is functioning. Date achieved 04/25/2005 06/30/2013 06/30/2013 Comments ACHIEVED. This indicator was introduced at the ICR stage in order to more (incl. % accurately assess the achievement of the PDO. achievement) Indicator 6 : Number of project universities offering a new student orientation All project All twelve project Value None of the project universities universities have now (quantitative universities offer a new offer a new started offering a new or Qualitative) student orientation student student orientation orientation Date achieved 04/25/2005 06/30/2013 06/30/2013 Comments ACHIEVED (100%). This indicator was introduced at the ICR stage in order to more (incl. % accurately assess the achievement of the PDO. achievement) The baseline indictors, where presented, were taken from the first ISR as the original * Bank Results Framework (RF) template did not provide for baselines and as such they were not included in the RF original project document. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 11/02/2005 Satisfactory Satisfactory 0.03 2 04/28/2006 Moderately Satisfactory Moderately Satisfactory 1.53 3 10/17/2006 Moderately Satisfactory Moderately Satisfactory 1.65 4 04/18/2007 Moderately Satisfactory Moderately Satisfactory 2.82 5 10/10/2007 Moderately Satisfactory Moderately Satisfactory 3.91 6 04/16/2008 Moderately Satisfactory Moderately Satisfactory 9.83 7 10/14/2008 Moderately Satisfactory Moderately Satisfactory 13.23 8 04/16/2009 Moderately Satisfactory Moderately Satisfactory 20.41 9 10/15/2009 Moderately Satisfactory Moderately Satisfactory 25.15 10 04/29/2010 Satisfactory Satisfactory 32.22 11 11/22/2010 Moderately Satisfactory Moderately Satisfactory 35.28 12 06/25/2011 Moderately Satisfactory Moderately Satisfactory 38.47 13 02/01/2012 Moderately Satisfactory Moderately Satisfactory 40.63 14 10/25/2012 Moderately Satisfactory Moderately Satisfactory 44.42 15 06/26/2013 Moderately Satisfactory Moderately Satisfactory 49.45 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions 05/07/2010 N S S 32.22 The government requested additional US$ 20.00 million to scale up the SHEP project. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. In December of 2001, a newly installed Afghanistan Interim Administration (AIA) was confronted with rebuilding a country reeling from protracted conflict and drought, an increasingly unstable economy, and a state that had become virtually non-functional in terms of policy making and service delivery. In addition, over five million people who had been displaced as refugees in neighboring countries were beginning to return in large numbers, adding to the general instability in the country. Despite these enormous challenges, the years that followed were marked by a series of significant political and social achievements including the successful establishment of a constitutional Loya Jirga and the peaceful completion of the country’s first ever presidential election, which culminated with the inauguration of the new Cabinet in December of 2004. 2. At the start of 2005, significant challenges remained; among them the restoration of countrywide security, rehabilitation of key economic and social infrastructure, and the establishment of functional state institutions across the country. Another serious challenge to the pace and scope of reconstruction was the massive skill-deficit that cut across all institutions in Afghanistan, from public administrators to teachers, engineers, technicians, administrators, accountants, agriculturists, and business leaders. To address this need, the Government had made notable efforts to revive the higher education sector in parallel with its efforts to rebuild primary and secondary education. As a result, eighteen higher education institutions reopened their doors and enrollment jumped from 4,000 students in 2001 to 37,000 in the fall of 2004. The rapid increase in enrollment provided clear evidence of the demand for higher education. With students returning from Pakistan and other countries, combined with students graduating from local high schools, demand for higher education continued to rise. In addition to the increased demand for access, there was a growing demand for quality and relevance in the higher education system. Most, if not all, institutions in the sector had no quality or quantity inputs since the beginning of country’s conflict in the 1980’s. As such, even as institutions opened, they were lacking in most measures of basic operational performance. Some of the key issues noted at the time included: a) The low quality of the higher education sector as a result of: i) a lack of qualified academic staff. More than half (57%) of the professors in the 18 tertiary education institutions in Afghanistan had only Bachelor’s degrees, 38% had a Master’s, and only 5% had Ph.Ds. and; ii) Curricula, teaching materials and pedagogical methods were grossly outdated. b) Fragmented and inefficient higher education institutions. Given the size of the country and its economy, there were far too many higher education institutions, and many of them would not qualify as tertiary level institutions by established international norms and standards. Enrollment in these institutions ranged from 50 students to 1,983 students. c) A basic lack of appropriate physical facilities: In all cases, physical infrastructure was dramatically short of what was required for a university. Electricity and water supply were insufficient and/or not provided. Classroom, laboratory and library facilities were very rudimentary and most of them were unable to operate. Basic computer facilities are lacking for both instruction and administration. The majority of public universities did not equip female bathrooms and dormitories. d) Outdated governance and management. The governance and management structure and norms were outdated. The notion of strategic planning, management was both foreign and weak in the higher education system and in each institution. Furthermore, there were no quality assurance mechanisms to ensure a basic level of quality in the sector. 3. To address these issues, the Bank worked with the Government to develop the Strengthening Higher Education Project (SHEP). Following on the heels of the Emergency Education Rehabilitation and Development Project (EERDP), which included a higher education component, SHEP was the Bank’s first project completely focused on the higher education sector in Afghanistan. SHEP’s objective was to restore basic operational performance to a core group of universities through support for strategic planning and management, improvement 1 in the quality of academic staff and curricula, and improvements in physical infrastructure, laboratories, libraries and other facilities. 4. The project was approved in June 2005 with a grant of US$40 million to support higher education system development and program quality improvement in six universities. The Afghanistan Reconstruction Trust Fund (ARTF) provided additional resources for the project with a grant of US$5 million in 2008. The Bank extended its support for SHEP in 2010, with an Additional Financing Grant of US$20 million. Rationale for the Bank's Involvement: 5. While other development partners were in agreement on the importance a functional higher education sector, few were actually involved in this area. The Bank was uniquely positioned to the task of supporting this sector because of its: a) technical expertise in the higher education sector globally, which enabled it to introduce a comparative global perspective on tertiary education reform and development; b) extensive practical experience with tertiary education projects, which provided relevant and concrete implementation lessons, vital to addressing the higher education sector issues in Afghanistan and; c) concrete experience in the higher education sector in Afghanistan through its previous support for the EERDP. Moreover, the proposed project was clearly aligned with the Bank’s overall interim Transitional Support Strategy (TSS-2003) for Afghanistan. The TSS, developed within the framework of the Government’s own reconstruction strategy, was also the beginning of the Bank’s transition from a short-term emergency orientation to a long-run development orientation in Afghanistan; and SHEP was well-aligned with this framework. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 6. The PDO, as stated in the Technical Annex for SHEP, was “to progressively restore basic operational performance at a core group of universities in Afghanistan.” The Key Performance Indicators (KPI) associated with this PDO at approval (impact and output indicators) were: 1. Improved employment rates among future graduates of participating universities/faculties (impact indicator) 2. Revised and updated curriculum in use; 3. Increased number of faculty members with a higher degree and relevant training; 4. Increased number of well-used labs, libraries and other facilities in six universities; 5. Simple strategic plan prepared and implemented in six universities; 6. Management information system developed and used; 7. The number of universities that carried out the self-evaluation and; 8. The efficient and accountable use of block grants by six universities. 7. The objective was worded slightly differently in the Development Grant Agreement (DGA) for SHEP. The DGA stated that “the objective of the project is to assist the Recipient in restoring the basic operational performance of Higher Education institutions so as to enhance the quality of tertiary education”. This wording essentially clarified the intended project outcomes of restoring and enhancing the quality of tertiary education in Afghanistan as project indicators clearly imply. The DGA PDO is broader version of the Technical Annex PDO and while it clarifies the intended project outcomes, the ICR has chosen to use the PDO in the PAD Technical Annex because it is more operational in its construct and as such, lends itself better to assessing the project’s outcomes. Moreover, it is also the PDO used in all other project monitoring documents (this includes the Supplemental Letter to the GA, the ISRs etc.) As such, any reference to the PDO in this document hereafter is a reference to the PDO as stated in the PAD Technical Annex, 2005. 1.3 Revised PDO and Key Indicators, and reasons/justifications (as approved) 8. The PDO remained the same under the 2010 Additional Financing. However, indicators approved under the Additional Financing were somewhat changed from the list in the original PAD Technical Annex for SHEP. This is because three of the eight KPIs were discontinued in the first ISR and the remaining five were used to monitor 2 the progress of the project from the first ISR onwards. In addition to these five, one more indicator was introduced in the first ISR and maintained throughout the project. Therefore, the following six KPIs were approved under the 2010 Additional Financing. 1. Percent of project faculties using revised and updated curriculum; 2. Increased percentage of faculty members with at least a Master’s degree; 3. Percent of administrators at the university and MOHE with management training; 4. Increase in the number of operational labs, libraries and other facilities in selected project faculties; 5. Number of project universities with strategic plans developed and made operational; and 6. Management Information System (HEMIS) developed and introduced. 9. At the ICR stage, the ICR team added seven supplemental indicators (three PDO indicators and four intermediate outcome indicators) in order to measure the project’s achievements more fully and accurately (the rationale for selection of these indicators is discussed more fully in Section 2.3). These actually include one intermediate outcome indicator “the number of universities that carried out an internal self -evaluation”, which was part of the original set of KPIs but which was officially discontinued in 2010. Below is the list of the performance indicators (both PDO and Intermediate Outcome Indicators) included at the ICR stage: this makes a total of 13 project indicators that are used to measure the project outcomes, as listed in the ICR Datasheet: Supplementary PDO Indicators1:  The number of project universities that have at least one more functional academic building.  The development of a national higher education sector plan.  The number of project universities with a functional student service center. 1.4 The Main Beneficiaries of this project were:  12 universities (there were initially 6 universities at the appraisal stage, and another 6 were added under Additional Financing);  University faculty and administrative staff from 12 universities and staff at MoHE who received management/ academic training and/or completed or are in the process of completing a Master Degree, facilitated by SHEP;  Administrators and Higher Education officials benefitting from the use of the HEMIS; and;  Students at the 12 project universities (through efficient use of labs, libraries and other university facilities). 1.5 Original Components at Appraisal Stage 10. Component 1: Program Development and Quality Improvement in Key Higher Education institutions (US$38.5 million) a) University Partnership Program (US$7.5million): This sub-component was designed to facilitate and finance partnership programs between six Afghan universities (two in Kabul and the four regional project universities) and established foreign universities (especially those within the South Asia region). Financial and technical contributions from various foundations were actively sought in the process. The partnership program was primarily designed to provide support for: (i) curricula revision and preparing 1 Supplementary Intermediate Outcome Indicators added at the ICR stage: (a) Number of project universities that have carried out an internal self-evaluation; (b) Establishment of a national Quality Assurance (QA) body with national standards in place for QA; (c) Establishment of a National Curriculum Development Committee; and (d) Number of project universities offering a new student orientation; 3 facility plans; (ii) training programs for Afghan faculty at partner institutions; and (iii) visiting professorships in Afghanistan for partner institutions. b) Block Grants (US$31 million): The primary purpose of the block grants was to enable universities to implement those activities prioritized in their strategic plans in order to restore the basic operational performance of universities. Block grants were expected to support the installation of basic administrative systems and procedures and to improve academic programs in order to enhance the performance of universities, especially in selected faculties. They could also be utilized for the establishment of new programs or rehabilitation of physical facilities, in line with the strategic needs of the university. 11. Component 2: Higher Education System Development (US$1.5 million) a) Support for a new governance system (US$0.75 million). This sub-component was designed to provide technical assistance for strengthening governance in all higher education institutions through: (i) defining the relationship between the Ministry and higher education institutions in order to facilitate/promote the autonomy and accountability of the institutions, (ii) identifying the information flow between universities and the MOHE, and (iii) reviewing the financing strategies for higher education, including developing funding formulae and mobilizing external resources. b) Support for a quality assurance system (US$0.75 million) This sub-component was designed to support the development of an autonomous accreditation and quality assurance system for both public and private universities. 1.6 Revised Components 12. There was no change in the components except for the scaling up of activities under the Additional Financing in 2010, as discussed in the next section. 1.7 Other significant changes 13. In June 2008, SHEP received an Additional Grant of US$5 million from the Afghanistan Reconstruction Trust Fund (ARTF). This grant was to provide increased financial support for four (Nangarhar, Balkh, Kandahar, and Kabul Polytechnic) of the six project universities to enable them to further improve their physical teaching/learning facilities. 14. In June 2010, the Board approved Additional Financing (from the IDA Crisis Response Window) of US$20 million, and extended the Closing Date of the project from June 30, 2010 to June 30, 2013. Under the Additional Financing, the scope of the project expanded to cover six additional universities. The activities of both project components were scaled up to accommodate these additional universities but the design of the components did not change. The expectation was that the Additional Financing would also provide a catalytic platform for the effective use of domestic and external sources for further development of the higher education sector. 2. Key Factors Affecting Implementation and Outcomes 2.1. Project Preparation, Design and Quality at Entry 15. The Government had made the development of the higher education sector a clear priority; reopening 18 public higher education institutions in the earliest years of reconstruction and drafting of new higher education law even prior to SHEP’s preparation. The draft law stipulated the responsibilities of the state and both public and private education providers, with an emphasis on increased institutional autonomy and accountability of higher education institutions. This commitment from the Government was especially important in light of the challenging implementing environment of SHEP at the time. The preparation, design and quality at entry of SHEP reflected multiple strengths as well as a few limitations. It’s strengths include: 4  A strong evidenced-based approach: The Bank team made ample use of the available resources to ensure that the project design was informed by a solid understanding of the country context and the state of the higher education sector; as well as lessons from the Bank’s experience in higher education both globally and in Afghanistan. The project was informed by: a) The World Bank’s 2005 Higher Education Sector Assessment2; b) Global lessons learned and; c) lessons from the IDA-funded Emergency Education Rehabilitation and Development Project (EERDP), just prior to SHEP. The following lessons in particular were incorporated into the project design: a) Block grants (used in the EERDP) were useful measures in providing universities with the flexible resources needed early in the rebuilding process; b) Reform efforts in higher education require comprehensive intervention strategies and sustained long-term engagements. c) Political consensus is critical to the higher education reform agenda. d) Providing incentives can stimulate change in higher education.  A strong participatory approach to planning: The project also benefitted from strong engagement with all stakeholders involved in higher education in the country. Multiple consultations were held during the course of the project preparation and after the project became effective in June, 2005. This included consultations with development partners such as USAID and UNESCO as well as the Ministry of Higher Education (including the leadership and key directorates), beneficiary universities and the Ministry of Finance.  A strong capacity building approach: Effective capacity building was not only a strength of the project but critically important in the country context. The project had strong capacity building measures built into the design through activities designed to increase the expertise and organizational capacity of the Ministry and the universities. These activities mainly consisted of training workshops/seminars aimed at training staff in procurement/financial management and general tertiary education management. These workshops and seminars were supplemented by one-on-one mentoring provided by specialists (i.e. international procurement specialists/higher education specialists) in these areas who worked with the relevant departments on full time basis to increase capacity in these areas.  Balanced targeting criteria: The project’s targeting effectively balanced equity and efficiency criteria. Equity was a key consideration in choosing the project universities. The first six universities were chosen so as to provide an equitable focus (regionally and hence ethnically) and ensure that the project supported the different regions of the country3. To maximize efficiency, the target universities chosen were all located in the capital cities.  A highly appropriate PDO, which is clearly reflected in the components and sub-components. The post-conflict reconstruction context in which SHEP was prepared created a complex set of challenges for the design of the project. There was an obvious need to restore basic functionality and core operations to the higher education sector to enable the sector to provide quality education. The two components also clearly operationalized the project objective because the process of restoring operational performance required inputs at two levels. Appropriately, the first component of the project was given more weight with the majority of financing as this component supported the acute needs of the sector at the time. The second component intended to develop the underlying management system by improving governance and quality assurance processes. 2 “Investing in Afghanistan’s Future: A Strategy Note on the Education System in Afghanistan” was published in 2005 and contrib uted to the highly targeted design of SHEP. 3 Although the main ethnic groups are fairly integrated country wide, there are ethnic concentrations in certain areas of the country- for example Balk is predominantly Uzbek and Tajik while Kandahar is primarily made up of Pashtuns. These six universities were comprised of Herat in the West, Balkh in the North, Nangarhar in the South East, Kandahar in the South and Kabul and Kabul Polytechnic in the capital. 5 16. While the design of the project was very appropriate, particularly for the country context, it could have benefitted from:  Explicit inclusion of the development of a higher education strategic plan under Component Two. The absence of a comprehensive sector strategy in the early stages of implementation was later cited as a bottleneck for project planning in terms of prioritization and specific targeting of activities. Therefore, at a later stage in the implementation, the project included direct support for the development of the National Higher Education Strategic Plan (2010-2014).  A stronger Results Monitoring Framework (RMF): While the components and corresponding inputs clearly reflected the PDO, the Results Monitoring Framework for the project could have been more effectively designed in terms of capturing more fully the elements of the PDO. This issue is discussed in greater detail in the Monitoring and Evaluation Section (2.3). In addition, the design of project implementation arrangements would have benefitted from more explicit attention to Monitoring and Evaluation mechanisms, both for the project and the sector as whole.  More flexible implementation arrangements. The implementation arrangements of the project stipulated that the first tranche of the block grants were to be released upon fulfillment of a specific set of activities. These activities included the development of the university strategic plans with technical support from the University Partnerships Program. Regrettably, the establishment of University Partnerships took approximately two years which delayed the implementation of the block grants activities. 2.2. Implementation 17. Institutional arrangements for implementation: The responsibilities for management and implementation of SHEP were divided between the Ministry of Higher Education (MoHE) and the project universities. A Steering Committee (SC) at the MoHE, comprised of the Minister, Deputy Ministers, key directors and PCU staff were responsible for oversight of the project. A Project Coordination Unit (PCU) at the MoHE was responsible for overall implementation of the project under the guidance of the SC. The PCU was headed by a Project Coordinator and included procurement, finance, administrative, audit, human resource and academic affairs staff, as well as other support staff. The PCU reported to the SC on project implementation and provided technical support to the universities. Based on procedures outlined in the Project Implementation Manual, each project university established an Executive Committee (EC) to oversee implementation of the project at the university- level. The university EC was led by the Chancellor and included senior administrative and academic staff at the university. ECs worked closely with the MoHE PCU on project planning and day-to-day implementation. Each university EC was supported by a small university-level PCU. Initially, the main function of the university-level PCUs was to assist in procurement and financial management for SHEP and as such, these units mainly consisted of a procurement and finance officer. As the capacity for planning and management increased at the university level, the mandate of the EC’s expanded to include support for programme needs and student services at the University. 18. Factors outside the control of the Implementing Agency: As with the project design, the post-conflict reconstruction context presented a particular set of challenges for implementation. Afghanistan was going through a transition with several reforms in administration and governance. At the same time, there were deeply entrenched bureaucratic structures in place and higher education was highly centralized like many other sectors in the country. This context created a set of challenges that were beyond the control of the project. They included:  Constant reforms in the administrative apparatus of the country: The policies and procedures guiding the project were a combination of those directed by the donor and the Afghan government. The Government itself was in a period of transformation and new administrative and financial policies were being developed throughout the course of the project. For the implementing agency, the (MoHE PCU and 6 the University SCs and PCUs), this added another layer of complexity to the already challenging task of adapting to the Bank’s fiduciary procedures.  Similar to other public sectors, bureaucracy in the higher education sector slowed implementation: Public universities lacked fiscal and operational autonomy and nearly all administrative and academic matters were channelled to the Ministry for endorsement and ultimate approval. Academic partnerships with overseas universities, for example, required a level of independent decision making by their counter- part Afghan universities in meeting certain deliverables, including hiring of additional technical and administrative staff. The lack of fiscal autonomy also inhibited universities from undertaking research, linking up with the industry, and generating income. To counter this, universities often directly approached the PCU requesting financing for some of their basic activities/needs that would otherwise take a disproportionate amount of time to implement (if processed through the Ministry’s ordinary budget).  Changes in university leadership: The lack of stability of leadership at the universities was often a challenge. Similar to the context in the Ministry, the role and involvement of strong leadership and management had a direct impact on the project implementation. Any change to specific university leadership/management often entailed considerable time and effort on the part of the project teams to ensure that the new leadership supported the commitments made under the project or by their predecessors. 19. Early Implementation Challenges: Moreover, the difficulty of implementing a donor-funded development project in a challenging environment was evident, particularly in the early stages of the project. SHEP was the first major development project in higher education in the country. Naturally, this affected the pace of implementation as the project had no precedent for addressing implementation challenges4. Early challenges included:  Delays in establishing university partnerships. The university partnerships took well over a year to establish, in part because the deteriorating security situation made it difficult to find foreign universities willing to partner with their Afghan counterparts. Because they were a pre-requisite for most of the block grant disbursements at the university-level, they in turn, delayed the implementation of the block grants.  Lack of capacity at the Ministry and universities to plan, manage and implement development projects. This capacity was particularly lacking in the areas of financial management and procurement. There were also delays in setting up the university PCUs at the beginning of the project because it was a challenge to find qualified staff.  Lack of familiarity with the project policies and modus operandi among project staff at the universities. This being the first interaction with the Bank for many of the project staff both at the Universities and the MoHE, project management and implementation staff across were unfamiliar with project policies and the Bank’s modus operandi and this created a challenge for implementation early on.  Lack of strategic guidance for identifying and prioritizing needs. Strategic planning at the university- level was built into the design of the project. However, technical assistance for developing university- level strategic plans was to be channeled through the University Partnerships Program. Because these university partnerships took a significant amount time to establish, strategic planning was an early challenge. Another bottleneck early on was the lack of strategic guidance at the national level. The Bank team recognized this need and facilitated the planning process for the National Higher Education Strategic Plan (NHESP) later in the project. 20. Implementation progress: Despite these early challenges, SHEP made remarkable progress. In the first few years of the project, the Bank team supported efforts to build capacity at the Ministry and University-level. To contend with the delays in establishing the university partnerships, an international higher education specialist 4 While the EERDP had included interventions in the higher education sector, these were emergency-focused and such the implementation arrangements were fairly different 7 was hired to work with the universities to develop their strategic plans. By 2007, all universities had developed strategic plans, which provided much-needed guidance for the implementation of block grant activities. Implementation capacity also improved at the local level with the establishment of PCUs at each university in the first two years of the project. Several rounds of training sessions and workshops as well as on-the- job mentoring helped to educate the staff at the university PCUs on the Bank’s guidelines and modus operandi, as well as procurement and financial management issues. By 2008, most of the staff at the PCUs had attained an appropriate level of familiarity with SHEPs objectives and implementation procedures. Procurement capacity also improved at the central PCU with the recruitment of an international procurement specialist, which in turn improved the pace of implementation. 21. In 2008, a Mid-Term Review (MTR) was conducted, which highlighted many of the above-mentioned challenges and made recommendations to address them. These recommendations included capacity building for better monitoring of the civil works under the project. To oversee civil works, a team of engineers was hired and housed in the Planning & Policy Development Directorate. The team was initially led by an international specialist who helped to build capacity for monitoring civil works. As a result of the capacity building, in 2010, the responsibility for oversight of civil works was eventually handed over to local engineers. 22. Implementation following the Additional Financing - 2010-2013: The emphasis during this phase shifted slightly more to higher education system development, which had been given less attention previously. The Higher Education Strategic Plan (2010-2014) was developed and launched, and was instrumental in providing strategic guidance for this process. Activities such as the development of Quality Assurance System and the Higher Education Information Management System began to take shape in this phase, while the block grants continued to support “hardware” development at the universities. 23. In 2010, there was a significant leadership vacuum at the MoHE. With the departure of key project staff and a transitional period in MoHE leadership, project implementation slowed down again. However, in 2012, a new Minister of Higher Education was appointed who was very committed to developing the sector. Under his leadership, new staff was appointed at the PCU (including a new Project Coordinator). With this renewed commitment, the pace of project implementation and disbursements picked up significantly, and despite time- constraints, the Ministry and the universities were able to utilize a considerable amount of the project’s resources to complete activities planned under both components of the project. 24. Disbursement: Overall, approximately US$39.4 million of the Original Financing for SHEP was disbursed and US$746,446 was cancelled. Of the Additional Financing grant, approximately US$12.2 million was disbursed and US$3.9 million was cancelled. From the ARTF Grant, approximately US$4.1 million was disbursed and US$877,443 was cancelled. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 25. Given the state of the higher education sector in Afghanistan, it was not surprising that M&E was a challenge because the higher education sector, like much of the country lacked the basic infrastructure for data collection and information management. This meant that SHEP had the responsibility of developing basic capacity in this area as well utilizing M&E processes to monitor the progress of its development objectives. Therefore, M&E design, implementation and utilization is best discussed on two levels: M&E for SHEP and Information Management for the Higher Education Sector as a whole. 26. M&E Design, Implementation and Utilization for SHEP: The original Results Framework (RF) identified five outputs for the project: 1) Improved quality of education in the project universities; 2) Improved physical/pedagogic/academic facilities based on strategic planning at the project universities; 3) Increased institutional autonomy and accountability; 4) Improved efficiency in the management of the higher education sector (data-based decision making) and; 5) Quality assurance system developed and implemented. Each of these outputs reflects a basic element of the PDO and together they provide a clear picture of what constitutes basic 8 operational performance in the higher education sector. As discussed earlier, the original RF identified eight indicators to monitor progress towards the PDO: but only five of these were used to monitor the project. The project documents during implementation did not explain why the other three indicators were dropped. However, it is pertinent to note that at the time SHEP was prepared the RF was in itself an evolving concept: the original Bank project document template did not provide for baselines and targets in the RF, only a listing of indicators. The RF template in the ISR did, however, make a provision for baselines and targets and this may have contributed to the challenge of transferring indicators from the original RF in the Technical Annex to the RF template in the ISR, as not all of the original indicators were easy to measure. As such, in the first ISR, these indicators were reassessed, and five were chosen, presumably because they were amenable to measurement. In addition, another indicator was introduced in the first ISR “Percent of administrators at the university level and MoHE staff with management training”. As a result, six indicators with specific targets were identified in the first ISR and monitored throughout the project. The RF could have been developed further still to fully reflect the expected outcomes of the project, particularly in the Additional Financing stage. The RF would also have benefitted from more consistency between the unit of measurement for the indicators, the baseline values and the target values (e.g. some indicators were identified in percentages while the targets in absolute numbers). As such, at the ICR stage, this RF was supplemented with 3 supplemental PDO indicators5 and 4 supplemental IO indicators6 in order to fully assess the project achievements. The selected indicators were derived on the basis of expected achievements under the originally described components and better reflect what the project meant and did accomplish. 27. Implementing M&E was a challenge in the beginning but improved significantly over the course of the project. In the early stages of the project, there were no systematic mechanisms for collecting data within the sector; the project was constrained both by a scarcity of data and constraints on data collection. As such, in the beginning, data for monitoring the project outcomes was a challenge. However, over time, the Bank team and government counterparts made arrangements for more systematic and reliable information management; they improved the capacity for record keeping and data collection both at the University as well as the MoHE (through the development of the HEMIS as well as technical support from the Bank-side to improve these processes for monitoring project outcomes). In a fragile context such as Afghanistan, it is not surprising that the ability to verify data is a challenge as security concerns severely inhibit otherwise standard means of verification, such as site visits, random third party monitoring etc. The Bank team was appropriately cautious about these security constraints and made efforts to ensure the reliability of the information management process; the inclusion of a Video Conferencing system (to directly connect the MoHE with project universities and particular the more remote ones) under HEMIS was part of the effort to ensure a means of connecting with project universities, monitoring activities, and improving the information management process. As the project progressed and HEMIS was developed, the universities and the MoHE were able to provide more systematic data for monitoring the progress of the project. 28. Information Management for the Higher Education Sector: The information management processes in the higher education sector were virtually non-existent at the beginning of SHEP. Data collection was ad-hoc and 5 3 supplemental PDO indicators were added at the ICR stage for the following reasons: The restoration of basic capacity was a core aspect of the PDO (physical/ academic/ and pedagogic). A such, the ICR team felt it was important that the PDO indicators sufficiently capture each of these aspects (i.e. the restoration of both physical and academic facilities in addition to pedagogic facilities) and so added two PDO indicators (PDO indicator 5 and 7). The significance of the HE sector plan in restoring basic functionality to the HE system has been repeatedly noted, and so it was obvious to the team that that there should have been an indicator included at the AF stage, to ensure comprehensive assessment of the PDO. 6 4 supplemental IO indicators were added at the ICR stage for the following reasons: 1) There were only two IO indicators in the original RF, given that “progressive” restoration was the objective it was clear that more was need to assess the process of restoring capacity particularly in the areas of QA (which was a basic dimension of the PDO but there were no indicators to measure progress on this and so IO #3 and 4 were included) and; 2)The process indicators were also important in verifying that the restoration process was not a result of external inputs but sustainable systemic development as such IO indicator 5 was included and finally, as the ICR included a PDO indicator to assess the restoration of academic facilities, the team felt it was important to include a process indicator to assess the functionality of this input. 9 there was no central database for collecting, collating and evaluating higher education data for efficient management and planning in the sector. Therefore, improving the information management system was a priority for the project; and under SHEP, a basic Higher Education Information Management System (HEMIS) was developed. It has a central database housed at the MoHE and end users at each of the project universities (these will later be extended to non-project universities). The system collects basic information on the higher education system such as enrollment data. It is currently only implemented in 9 of the 12 universities, and in the process of being extended to the rest of the project universities (i.e. they have functioning system for data collection which is linked to the central database at the MoHE). For further discussion of the HEMIS see Section 3.2. 2.4 Safeguard and Fiduciary Compliance 29. The environmental safeguards rating for the project was Category C because only minor civil works were planned to be carried out under the block grants sub-component; as such, no negative environmental impacts were expected during project preparation. Indeed, in 2005, when the project was prepared, Category C was typically assigned for projects with such minor rehabilitation works, and particularly in cases of projects that had demand- driven components, such as block grants. This is because the inherent flexible design of block grants makes it impossible to know in advance what types of activities would be eventually financed. This was especially the case in this project because in addition to the inherent uncertainty of the block grants sub-component, it was prepared in a highly volatile environment against a background of major security concerns, which made it more difficult for the Bank team to gauge precisely what universities could potentially request under block grants. Therefore, the Bank team allowed for a very broad definition of block grants in the Grant Agreement which included financing for a variety of different types of expenditures, including civil works. During project implementation, however, block grants did end up financing rehabilitation and renovation works to bring university buildings into a functional condition in line with the PDO. These works were handled in compliance with the existing Government’s Environmental Management Framework. There was no land acquisition as all buildings were on government land and there was no resettlement or relocation undertaken. Further, no complaints were received during implementation. However, it must be noted that given the more extensive nature of the works carried out, the Bank team should have changed the rating to Category B through project restructuring. In particular, this could have been done in 2010, when the AF was prepared to scale up project interventions in additional 6 universities. According to Government counterparts, no adverse impacts were noted and all university buildings are operating in compliance with the existing country environmental standards. However, the Category C rating was maintained at that time and was reflected in the updated Integrated Safeguards Data Sheet (ISDS). 2.5 Financial Management 30. SHEP financing was implemented through the government’s central treasury, involving the Ministry of Finance, Mostofiats (i.e., MoF provincial directorates) and Da Afghanistan Bank (i.e., Central Bank). Despite several reforms in this process, financial transactions still needed to follow a system that required considerable paperwork and adherence to cumbersome procedures, resulting in slow disbursement, particularly at the provincial level. This included several additional layers of bureaucracy, including funds transfers through the Mostofiats and various procurement approvals. These circuitous processes created several challenges for funds flow and financial management of SHEP. 31. SHEP was also challenged by a lack of financial management capacity at all levels of project management; particularly in the early stages of project implementation. Taking into account the challenging implementing environment, the project supervision team assessed the financial management of the project as Satisfactory in the first few years, with the hope that capacity would increase over time. However, in 2009, financial management had to be downgraded from Moderately Satisfactory to Moderately Unsatisfactory because the internal audits of SHEP by the MoHE revealed unsatisfactory performance. The annual audits were not submitted on time and there were irregularities and delays in the bids for the selection of service providers. The internal controls over expenditures were also noted as weak, in particular petty cash expenditures. 10 32. Following a Financial Management Review in 2010, a new financial management specialist was appointed and a fiduciary action plan was developed. The Bank team made a concerted effort to engage and support the MoHE in finding practical solutions to address the concerns raised by the internal audits. The project also worked closely with the Deputy Minister for Admin and Finance as well as the Office of the Director for Admin and Finance. During the last few years of the project, as a result of enhanced capacity, more familiarity with Bank procedures, and sustained implementation support, these issues were addressed satisfactorily and the project ’s FM performance was upgraded to Moderately Satisfactory. 2.6 Procurement 33. The PCU at the MoHE had the overall technical responsibility for program implementation. The PCU-MoHE unit handling procurement consisted of an Internal Procurement Specialist, a Procurement Manager and a Procurement Officer. The unit provided technical support on all procurement issues to the MoHE and universities under the program. A Procurement Officer was also placed within each university PCU. Procurement and general administrative capacity at the PCU was weak from the beginning. Because of extremely weak procurement capacity at the University-PCUs, universities were given limited procurement values and procurement was to be carried out primarily through the MoHE PCU. However, procurement capacity at the MoHE was not much stronger. In 2007, an international procurement specialist was hired to improve capacity and support large scale procurements for the project. This was useful in increasing capacity and clearing some of the procurement-related bottlenecks. Training courses were also held to develop procurement capacity at all levels. However, procurement continued to be a concern over the course of the project due to staff turnover, a lack of well-qualified procurement staff at the PCUs, political instability and the lack of security. These procurement challenges caused several delays in implementation over the course of the project. 34. In 2009, procurement was downgraded from Moderately Satisfactory to Moderately Unsatisfactory during the supervision process, and then to Unsatisfactory in 2011 as a result of: a) unsatisfactory results from the internal audits of SHEP by the MOHE; b) delayed submission of annual audits; and c) delays in the submission of evaluation of bids for selection of service providers. Problems in procurement were also exacerbated with the departure of key procurement staff; both the international procurement specialist and the chief National Procurement officer departed in 2010. Over the course of 2011 and 2012, measures were taken to address the issues disclosed by the audits, and the Ministry and the PCU were supported in developing procurement capacity, which improved the overall procurement process. Several of these procurement issues were also resolved when the ARDS of the Ministry of Finance took control of major procurement in 2011. In 2011, the MoHE also formed a joint committee called the National Procurement Board, which consisted of 4 directors, 2 deputy ministers and the SHEP PCU to oversee the Procurement process. In 2012, the rating moved up from Unsatisfactory to Moderately Unsatisfactory. While challenges existed, procurement continued to show steady improvement through the end of the project, but the Bank team maintained the rating in view of the overall performance of the procurement; and to caution against complacence in this area, which would require continued support in the context of the implementing environment. 2.7 Post-completion Operation/Next Phase Development of a New Higher Education Program 35. The Government of Afghanistan (GoA) has a national development strategy in which higher education is a key policy priority. SHEP supported the MoHE in developing the National Higher Education Development Strategic Plan (2010-2014). This is now being updated for the period 2015-2019. The MoHE has a team of consultants working to design a new higher education program for the implementation of this strategy and this should be completed by 2014. The MoHE is also seeking technical and financial assistance from the Bank in developing and implementing this new program. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance: High 11 36. Objectives: The project development objective of SHEP remains highly relevant to the country’s development objectives, as outlined in the Afghanistan National Development Strategy (ANDS 2008-2013), the World Bank’s Interim Strategy Note for Afghanistan (2012-2014) and the National Higher Education Strategic Plan (2010- 2014). SHEP is also strongly aligned with Afghanistan’s National Development Strategy (2008-2015) (the ANDS), which identifies increase in the quality and independence of the Higher Education system as an education sector policy priority. SHEP is not only aligned with the higher education sector priorities in the (NHESP 2010- 2014) but was instrumental in facilitating the strategic planning process that led to NHESP’s development. Although this activity was not explicit in the original design, it became a cornerstone of the higher education system in Afghanistan. 37. The World Bank’s current Interim Strategy Note for Afghanistan (2012-2014) identifies the following three pillars as foundational to the Bank’s strategy for supporting Afghanistan: 1) Building the legitimacy and capacity of institutions; 2) Equitable service delivery; and 3) Inclusive growth and jobs. SHEP is clearly aligned with this strategy because a strong tertiary education sector is an essential prerequisite for the success of this strategy. 38. Design and Implementation: The design of SHEP was and remains highly relevant to the challenges facing the higher education sector and to the priorities outlined in Bank’s Interim Strategy Note (2012 -2014). As previously discussed, the project was informed by a 2005 Education Sector Review of Afghanistan, which provided a situation analysis of the key issues facing the higher education sector. These findings were incorporated into the design of SHEP and the result was a project design which was highly responsive to the needs of the sector and by extension, to the needs of the country as a whole. SHEP also took into account the lessons learned from the EERDP and its own global experience in tertiary education reform and development, which also informed the design of the project. 39. In light of the extremely low quality and operational capacity in the sector in 2005, the project’s development objective, “to progressively restore basic operational performance at a core group of higher education institutions” was highly appropriate. 3.2 Achievement of Project Development Objectives Rating: Satisfactory 40. It is important to underscore the achievements of this project with a reflection on the context in which the project was implemented. As previously noted, the project was launched as an emergency operation in a fragile, post-conflict, Afghanistan. This by itself enormously challenged the stability of the implementing environment. There were also several rounds of changes in the project management team both from the Bank side as well as the Government. SHEP’s achievements in this context are quite remarkable and despite the challenges of the operating environment, SHEP achieved its objective by substantially improving: the capacity of academic staff; the quality and relevance of the higher education curricula; key physical infrastructure; learning facilities such as science labs, computer labs and libraries; and by making significant contributions to the reform and development of the country’s higher education system through support for greater university autonomy, quality assurance, and strategic planning. 41. The PDO was “to progressively restore basic operational performance at a core group of universities in Afghanistan”. Quality improvement of the sector was an underlying principle in the restoration process. This is alluded to in the Technical Annex (2005), the PDO as expressed in the Grant Agreement, 2005 and other key project documents. As such, in order for the sector to be restored to operational performance implied that basic dimensions of quality in the higher education sector had to be addressed in the restoration process. Thus, restoring operational performance meant: 1) Improving the quality of education by improving the quality of academic staff and curricula; 2) Restoring physical/academic/pedagogic facilities on campus; 3) Increasing institutional autonomy and accountability for planning and management; 4) Improving efficiency in the management of the higher education sector; and 5) Increasing Quality Assurance (QA) and Enhancement Processes in the sector. 12 Below, Table 1 provides a summary of the achievements against the 13 project indicators identified to monitor the achievement of the PDO; this is followed by a discussion of the specific achievements in each dimension of the PDO. Table 1: Summary of Achievements against Project Indicators Over-achieved Fully Achieved Partially Not achieved (Surpassed Target) achieved PDO Indicators-7 1 6 Intermediate Outcome 5 1 Indicators - 6 Sub-objective 1: Improved quality of education in project universities- Achieved Indicators (PDO and Intermediate Outcome)  Percent of project faculties using revised and updated curriculum –Achieved.  Increased percentage of faculty members with at least a Master’s degree- Achieved (fell short by only 1%).  Establishment of a National Curriculum Development Committee- Achieved. 42. Two basic factors that affect the quality of higher education are the quality of academic staff and the quality of the higher education curriculum. SHEP significantly improved both of these. Baseline data for the project indicates that only 38 percent of faculty in the twelve project universities had a Master’s degree or above. To restore basic operational performance at the project universities meant increasing the proportion of faculty with at least a Master’s degree in those universities. The project aimed at increasing the proportion of faculty with Masters Degrees to 50 percent. Through the University Partnership Programs, SHEP sent a total of 345 faculty members from the twelve universities to study for their Masters. Of these, 295 completed their education and had returned to teach by the time SHEP closed. Another 50 are expected to return to their teaching positions upon completion of their Masters programs next year. Under Additional Financing, SHEP was supported faculty to purse their studies through “in-house” Masters Programs at Kabul University, Kabul Polytechnic and Kabul Education University. This was another indication of the both increased capacity within the sector to improve quality and as such, proved the sustainability of SHEP’s activities. Over the course of SHEP, the proportion of faculty members with at least a Master’s degree increased to 49 percent across the twelve project universities(out of 2518 total faculty at the 12 Universities, 1242 have a Masters or above), slightly under the target of 50 percent. 43. The relevance of higher education curricula is another important indication of its quality. In 2005, the Higher Education curriculum was dated by at least 30 years in most higher education departments across the country. A first step in restoring operational performance was to ensure that the public higher education curriculum was revised. SHEP targeted select faculties (based on strategic planning) at all 12 universities for curriculum revision. These faculties had been chosen because they were identified as a core part of the higher education sector and were particularly relevant to the needs of the country at the time (they included Engineering, Agriculture, and English Departments). By the end of the project, all targeted faculties had updated their curricula it is in the process of being implemented across all the project universities. Moreover, SHEP supported the establishment of a National Framework for Curriculum Reform and Development. With the establishment of the National Curriculum Development Committee (NCDC) as the body overseeing this process, the higher education sector made significant headway in revising the whole public higher education curricula in the country; in a more remarkable achievement the entire public higher education curricula has now been revised and will be used by all public universities by the beginning of new academic year in Afghanistan. 44. It is worth noting that in both these achievements, SHEP was not only able to address the acute needs of the sector but also laid the foundation for sustainable rehabilitation of the sector as a whole by: (i) establishing the 13 NCDC and implementing a national framework for curriculum reform and development, as well as; (ii) increasing the institutional capacity of the sector to improve the quality of its own academic staff. Sub-objective 2: Improved physical/pedagogic/academic facilities at project universities – Achieved Indicators (PDO and Intermediate Outcome)  Number of project universities that have at least one more functional academic building- Achieved.  Increase in the number of operational labs, libraries and other facilities in selected project faculties- Achieved.  Number of project universities with a functional student service center- Achieved.  Number of project universities offering a new student orientation- Achieved. 45. In order to function, any higher education institution requires basic physical infrastructure such as classrooms, bathrooms, etc. In addition, teaching equipment such as labs, library and IT facilities are essential to the operation of the modern university. SHEP’s inputs were targeted at restoring these basic facilities based on the needs of the universities as expressed in their strategic plans. SHEP undertook large projects to rehabilitate and expand the basic physical infrastructure at the 12 universities. These included construction and renovation of buildings for various academic departments as well as libraries. 46. In the depleted post- war environment of Afghanistan, the project universities lacked even the most basic infrastructure such as bathrooms, dormitories (particularly for female students), boundary walls and water supply systems. In addition to the construction of buildings for various academic departments, five of the universities also constructed bathroom facilities (two sets of female bathroom facilities and three sets of male bathroom facilities). Dormitories were also renovated at four of the project universities. Each of the project universities was improved through the rehabilitation and construction of key physical infrastructure. 47. At the start of the project, there were very limited science labs, ICT facilities, and libraries in the twelve project universities. Baseline data identifies 25 libraries, 32 science labs, and 3 computer labs across the twelve project universities. It is, however, unclear whether any of these were truly operational. Through the block grants, SHEP was able to significantly improve these facilities in all twelve universities. As a result, the 12 project universities now have a total of 84 science labs, 50 computer labs, and 37 libraries which are fully operational. The goal of SHEP was not simply to deliver these facilities but to make them operational and to this end, the project provided training sessions to university staff in order to build capacity for use of this modern equipment. During the ICR mission, discussions were held with the universities to gain a better understanding of the functionality of these facilities. While the training sessions were useful and helped to ensure the functionality of these facilities, the challenge lies in maintaining this lab equipment and ICT facilities, which require both knowledge staff and significant resources to ensure that they remain functional. 48. Another indication of a functional university is the availability of student service facilities. Student service facilities such as a student services center, which supports career planning and guidance post-graduation is a basic component of the modern university system. These service facilities are particularly important for the establishing linkages with the job market and facilitating the employability of students. SHEP supported the development of a student services department at each of the twelve project universities. An indication of the functionality of these centers is that by the time SHEP closed, all of them had conducted new-student orientation programs. The percentage of new students attending these orientations sessions has been remarkable for a first effort: approximately 30 to 64 percent of all incoming students at the twelve project universities took part in these sessions in the 2013 academic year. 49. Facilitating research is another function of the modern higher university and SHEP made headway in establishing the research bodies to encourage research at the project universities. This included supporting the establishment of a National Research Commission as well as Research Committees at all twelve universities. As a 14 result, so far, over 60 research proposals have been reviewed, 9 of which are being undertaken as research projects. Sub-objective 3: Increased institutional autonomy and accountability-Achieved Intermediate Outcome Indicator  Number of project universities with strategic plans developed and made operational- Achieved. 50. Institutional autonomy has been classified as a basic component of educational quality in the modern higher education system. In the highly centralized higher education context, it was challenge to integrate inputs that would increase the institutional autonomy and accountability as there were several factors involved in this process, many of which were outside the control of the project. Nevertheless, the project endeavored to increase the capacity of universities for management and planning, and thereby, to move them towards increased autonomy and accountability. To this end, SHEP supported all twelve universities in developing strategic plans, which enabled them to identify and prioritize their needs. All project universities now operate according to strategic plans; the activities included under the block grant component were also based on these strategic plans. 51. The Government took an important step in support of this increased institutional autonomy when it passed a by-law on university autonomy towards the tail end of the project. The by-law, outlined in two chapters and 16 articles, essentially allows universities to exercise fiscal autonomy for expansion and development purposes. For instance, the law allows universities to open bank accounts, secure and generate resources through grants, provision of services and other means. The by-law will initially be operationalized and implemented at the four Kabul-based public universities. Implementation will gradually be expanded to other public universities. Sub-objective 4: Improved efficiency in the management of the higher education sector-Achieved Indicators (PDO and Intermediate Outcome)  Percent of administrators at the university and MOHE with management training- The target was surpassed by 10%.  The development of a national higher education sector plan- Achieved.  Higher Education Management Information System (HEMIS) developed and introduced- Partially achieved. 52. Governance and management of the higher education sector was very weak at the start of the project. The structures in place were outdated or non-existent. SHEP aimed to improve various dimensions of higher education management, including strategic planning at the national level, management capacity in the sector, and higher education information management. Human resource capacity for administration both at the national level and the university level is, obviously, very important to ensure operational performance in the higher education sector. According to baseline data, prior to the SHEP, none of the administrative staff either at either the universities or at the MoHE had received management training. SHEP’s goal was to ensure that at least 40 percent of staff at both levels had received management training. SHEP facilitated management training for 1166 staff (approximately 50 percent of the total administrative staff at both the project universities and the MoHE (PCU). Building such capacity in an environment that was severely lacking was an important achievement for the project and improvements in the pace of implementation over the course of the project were indicative of this. 53. The development of a national higher education sector plan was not part of the original project plan. However, with the implementation of the project, the Bank team and the Government soon recognized the need for it and SHEP was instrumental in the preparation of the National Higher Education Strategic Plan (NHESP 2009-2014). This was a seminal accomplishment: it was the first Higher Education sector plan for Afghanistan and became the cornerstone of the higher education sector. The plan provided much-needed strategic vision and direction for the development of the sector as a whole. 15 54. The development of the HEMIS under SHEP was another key indicator of systemic development and an increased capacity for management in the sector. As a result of SHEP’s efforts, a web-based information management system for higher education was designed, with a central database at the MoHE and connecting databases at each of the universities. The development of HEMIS currently enables the project universities to collect data on key higher education indicators and feed it back to the MoHE for analysis and ultimately for planning. One of the key constraints of the system has been a lack of IT infrastructure, which interrupts the smooth functioning of the system. This constraint, in addition to the low technical capacity of the end-users has affected the utility of the system has affected the functionality of the system. The MoHE is currently building capacity for the end users through training workshops. 9 out of the 12 project universities have fully implemented this HEMIS and it is now being expanded to the remaining universities. The HEMIS also includes a platform for connecting the universities to each other and to the Ministry via a video-conferencing system set up under SHEP. This also supports more efficient management in the sector by enabling and promoting communication between universities and the MoHE. In a country with such weak physical infrastructure as Afghanistan, it is not surprising that it may be a while before the HEMIS functions smoothly. Even in the more highly developed countries such as system would take several years to implement and it is a creditable achievement that it is beginning to take shape in the higher education sector in Afghanistan now. Sub-objective 5: Improved Quality Assurance (QA) and Accreditation mechanisms in the higher education sector- Achieved. Indicators (Intermediate Outcome)  Established National Quality Assurance (QA) body with national standards in place for QA –Achieved.  Number of project universities that have carried out an internal self-evaluation- Achieved. 55. Quality Assurance (QA), which includes standards for accreditation of universities, is an important component in the operational performance of the higher education sector as it puts in place basic standards for quality across the sector. It is important to note here that in the severely depleted higher education context of Afghanistan in 2005, this may be viewed as a “higher order” dimension of basi c functionality, meaning that in order to begin this process, institutions must have the basic components of a higher education system in place: i.e. the factors discussed above including qualified teachers, a relevant curriculum, basic physical infrastructure, teaching and learning facilities. Nevertheless, it is an important component of a functional higher education sector and as such, it was important for SHEP to take initiatives to establish processes and mechanisms for QA. Given the “higher order nature” of this dimension, it is not surprising that although building QA processes was part of the original program design; its development really took shape in the second half of the project. In the first phase, SHEP did focus efforts on the Kankor (the national university entrance exam) both in implementing it and processing the results. By the end of the project, a QA Commission was established at the national level as a body that oversees QA processes; and a National Framework for Quality Assurance and Accreditation was developed with established National standards for quality assurance in the higher education sector. Indicative of the implementation of this framework and regulatory mechanisms, all twelve universities completed the self- assessment (evaluation) process. This evaluation was reviewed, both peer-reviewed and reviewed by the QA Commission and as a result, the accreditation process is beginning to take shape. Also, SHEP has been instrumental in supporting efforts to develop a credit system to facilitate the transfer and accumulation of credits between public universities; right now Kabul and Kabul Education University have begun the process of developing this system in some faculties. 56. The achievements of SHEP provide clear evidence the project’s substantial achievement of its objectives and as such the ICR rates the achievement of the PDO as Satisfactory. At first glance, there may appear to be a discrepancy between the ICR and ISR ratings in this regard since the ISR assessed the progress toward achievement of the PDO as Moderately Satisfactory for much of the project. It is important to note here, that the ICR added supplemental indicators to fully assess the project’s outcome, and all these taken together provide c lear evidence of the substantial achievement of the PDO. Indeed, even if the outcome were to be assessed solely on the basis of the original six indicators, there would be no real discrepancy in the actual assessment of outcomes. 16 Rather, what accounts for this difference is that the Bank’s Supervision team, particularly towards the end, took a cautious approach to the ratings, which we argue, may have served to improve the project’s performance over time, but may have been overly cautionary at times. This is particularly evident in the last ISR, which noted that the project had made substantial progress (and in fact achieved the targets set for each of the six indicators, with the exception of the HEMIS) but cites delays in the procurement of lab equipment as the main reason for maintaining a Moderately Satisfactory rating. With this in mind, the ISR could have been more generous in the final assessment. However, this missed opportunity should not obscure the project’s substantial achievements, and as such, the ICR, which determined its rating exclusively on the basis of the Project’s performance on the PDO/IO indicators, assesses the project’s performance in achieving its objectives as Satisfactory. 3.3 Efficiency Rating: High 57. This rating is endorsed by the following factors: i) internal efficiency improved as the total university enrollment at project universities increased from 12,325 to 28,315, and number of graduates from 1,564 to 11,256 between 2005 to 2013; however, although female enrolment is steadily increasing, it still lags significantly behind male enrollment: while male enrolment climbed up from 10,114 in 2005 to 23,090 in 2013 and the number of female students also increased from 2,211 in 2005 to 5,225 in 2013, the share of female students as per the total enrolment is still only 18.5% in 2013. ii) external efficiency also improved due to an increasing employment rate among youth and an increase in the high skilled labor force in the country, and by a decreasing inactivity rate among youth, especially young females (88.2% in 2005 and 86.9%in 2010); iii) the result of a cost-benefit analysis of the project (see Annex 3) shows a significantly positive internal rate of return to the project (23 percent), and a net present value of US$242 million. These results compare favorably with the analysis conducted at the additional financing stage in 2010. 58. While GDP in Afghanistan has been increasing dramatically since 2005, total government expenditure also has been growing between 2005 and 2013. The project cost for SHEP, which was less than 1% of the total government expenditure can gradually absorbed into the national budget. Moreover, some universities have secured research and service contracts with government organizations and private firms as a result of new equipment financed under SHEP, which has provided them with recurrent additional revenue. With this in mind, it is anticipated that fiscal sustainability will most likely be maintained. Figure 3-4: Youth and Employment Status in Afghanistan 2005-2010 Source: Using ILO Key Indicators of Labor Market. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 59. As the previous discussion indicates, the project design and objectives remain highly relevant to the development challenges faced by the higher education sector in Afghanistan today. The project was also highly 17 efficient. In the implementing context, its efficacy was remarkable: the project achieved its targets on 11 of the 13 project indicators. It surpassed its targets on one indicator (management training of administrative staff at the university and the MoHE). Finally, only one indicator was really partially achieved; the development of the HEMIS. Given the High ratings for relevance and efficiency and Satisfactory rating for efficacy, the overall project outcome rating should be Highly Satisfactory. However, since three of the project universities are still to be connected and the implementation of the HEMIS remains a challenge because of the IT infrastructure etc.; as well as the previously noted fiduciary/procurement issues that plagued project implementation, the ICR team assesses the overall outcome of the project as Satisfactory. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 60. Most of the project universities are located in provinces where the unemployment rate is higher than the national average. The inclusion of these universities in the program was very important not only from an equity perspective but also in terms of the impact on the local economy (by increasing local consumption and employment opportunities). Universities have increased the number of faculty members, administrative and support staff over the course of the project; which would certainly have had a positive impact on employment rates in the provinces where target universities are located. 61. SHEP was also attentive to the gender dimension of higher education in Afghanistan, where the participation of women was low both in terms of students and faculty. The project made a concerted effort to improve this through interventions to: a) increase the availability facilities such as bathrooms and dormitories for women (many of which were either non-existent or not usable prior to the project) and; b) promote “in-country” Masters programs (discussed in Section 3.2), which enabled female faculty members (who would not have been able to leave their families to study abroad) to pursue their Masters-level education. These two activities are important factors in encouraging female participation because both issues have been cited as obstacles to the participation of female students and female faculty at universities. The impact of these interventions to address the gender disparities in higher education will become clearer in the coming years as these interventions take effect. 62. Furthermore, SHEP has made very important contributions in the area of social and community development in Afghanistan. Among them: 1) Some of the project universities have established learning programs and training courses for people in the communities, using resources provided under SHEP, such as modern equipment, lab facilities and an updated curriculum. For example, the Engineering Faculty at Kabul University established NGO in 2009, which has been offering engineering consulting services, construction quality management certificate courses as well as construction safety trainings. Kabul Education University has developed a Service Learning Program that currently sends students to the communities so that students can offer classes to the people in the communities with the purpose of developing language and other skills of members of the community. A Service Learning Program offers opportunities to students and people in the communities in which they can work together in designing and implementing the activities. Medical faculties at some of the project universities offer primary health care services to underprivileged people in the communities. (b) Institutional Changes/Strengthening 63. Institutional strengthening was an integral aspect of SHEP’s strategy. As such, SHEP’s achievements include significant institutional development; enabling the country to build a foundation for long term development of the higher education sector in Afghanistan. These include the development of the sector plan, NHESP (2010-2014), the HEMIS, the various bodies established to oversee Quality Assurance, Curriculum Reform and Development, and Research. These have been discussed in greater detail in previous sections of this ICR. (c) Other Unintended Outcomes and Impacts 64. NA 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 18 65. A stakeholder and beneficiary workshop was conducted to assess the implementation experience of the SHEP program from November 30-December 1, 2013 in Kabul. There were a total of 80 participants, including representatives from the World Bank, MoHE as well as university staff, faculty and students from the 12 project universities attended the workshop. The universities had an opportunity to present their achievements under the project, discuss challenges and suggest ways in which future support could be strengthened. The overall outcome of the workshop was very positive: universities felt that they had gained a lot from the project and while they pointed out challenges, they were clear that the program had a strong and positive impact on their development. Students and faculty also had the opportunity to reflect on the impact of the project and these were also highly positive; students noted the new services provided by the student service centers as particularly beneficial. The main conclusions are summarized in Annex 6. 4. Assessment of Risk to Development Outcome Rating: Substantial 66. The sector risks to the development outcome are moderate because: (a) The Government of Afghanistan continues to show a very high level of commitment to developing the higher education sector, as evidenced by the various reforms in governance that have taken place over the course of SHEP (including the draft of new higher education law and the recent passage of by-law which extends greater autonomy to higher education institutions); (b) Higher education has been identified as an education sector priority in the Government’s national development strategy; (c) SHEP has significantly strengthened the institutional capacity of the sector but while the project increased overall capacity for financial management and procurement, there is still room for improvement in this area and the implementing environment does not guarantee the stability of these resources and; (d) the fiscal position of the sector, like most others in the transitional environment of the country, is not entirely secure. The country level risks are high in view of: (a) 2014 planned withdrawal of NATO forces and the transfer or security to the Afghan security forces; and, (b) anticipated political changes stemming from the 2014 Presidential election. In light of these events, which are likely to have a much larger impact than sector-specific issues, and may have a destabilizing effect on the country with negative impacts on security and social stability, the overall risk rating has been elevated to Substantial. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 67. Despite the post-conflict scenario of political instability and security constraints, and as such, a challenging environment for project preparation, the Bank team prepared a highly relevant, evidence based project, reflected in good quality project documents that outlined the challenges facing the higher education system in the country. There was a clearly articulated rationale for Bank investment in higher education given the country context and its critical needs. The project was clearly aligned with the four objectives of the World Bank’s Transitional Support Strategy (TSS) as well as other strategic documents discussed in earlier sections. 68. The Bank team was also deeply engaged with the Government counterparts during the design phase of the project. As noted in the Technical Annex, the project took into account lessons learned (from both the Bank’s experience in higher education as well as the preceding Emergency Education Rehabilitation and Development Project) in order to meet the needs of the country at that time. 69. However, there were some limitations, which could have been better addressed. As noted above, there were some inconsistencies in the PDO statements in the key project documents (discussed above) and the Results Framework could also have been more fully developed to reflect all aspects of the PDO. The project’s implementation arrangements could also have been more flexible, for example, taking into account the timing needed for the development of academic partnerships, upon which the disbursement of block grants were dependent. Lastly, compliance with safeguards as previously discussed, could have been addressed more 19 effectively during the project preparation, particularly at the Additional Financing Stage. For these reasons the Bank’s Quality at Entry is rated as Moderately Satisfactory. (b) Quality of Supervision Rating: Moderately Satisfactory 70. Despite several changes in the composition of the Bank team, and the general difficulty of finding staff to work in Afghanistan, the project maintained a good balance of specialists in education, higher education, economic and financial management, complemented by international and national consultants. The Mid-Term Review in 2008 was thorough and comprehensive, providing a fair analysis of project implementation up to that point and a set of valuable recommendations, which were later acted upon and improved implementation performance. 71. The Bank team also showed significant innovation and responsiveness to the needs of the sector as and when these needs became apparent, i.e. the team recognized the need for a sector strategy and responded with critical support for the development of the National Higher Education Strategic Plan 2010-2014(NHESP). Furthermore, where implementation arrangements were weak and hampering the pace of implementation, the Bank team was quick to bring in technical support to remedy the situation; e.g., the team recruited an international higher education specialist to support the development of the strategic plans at the university level. In hindsight, it is clear that this kind of engagement and responsiveness ultimately alleviated potential obstacles to achieving the project’s overall objective; and ultimately contributed to the remarkably successful outcome. Moreover, there was steady improvement in the quality of Bank supervision over the course of the project; as the project implementation progressed, the Bank team made concerted efforts to ensure more systematic data collection processes to monitor the progress of the project and to develop better information for the system as a whole. In another sign of improved quality in supervision; they exercised caution in the ratings process (i.e. even as procurement improved significantly, they upgraded the ratings judiciously (not too steeply) with a view to maintaining due diligence in this area. In view of these strong aspects of Bank performance, supervision would have been rated Satisfactory; but there were less satisfactory aspects of project supervision, which have been noted previously. 72. There are mainly two aspects that could have been better addressed in the supervision process. These are: a) attentiveness to M&E at early stages of the project; and b) appropriate compliance with Bank environmental safeguard policies. The turbulent environment in which the project was implemented and an accordingly, “fire- fighting” modus operandi, which is often the case in such implementing environments, may have contributed to these oversights. Ultimately, these limitations did not affect the outcome of the project per se. However, taking all of this into account, the otherwise Satisfactory Bank Supervision has been rated as Moderately Satisfactory. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 73. The Bank’s quality at entry is rated as Moderately Satisfactory and the quality of supervision is rated as Moderately Satisfactory. Therefore, the overall Bank performance is rated as Moderately Satisfactory. 5.2. Borrower Performance (a) Government Performance Rating: Satisfactory 74. The Government of Afghanistan made higher education a clear priority. Despite challenges to reform and to develop a higher education system from decades of conflict and a near-complete loss of intellectual and physical assets in the sector, the Government made notable efforts to develop this sector. The Government identified higher education as a key priority in the Afghanistan National Development Strategy (2008-2013) and permitted the establishment of the private higher education institutes for the first time in the country’s history and supported QA processes to ensure quality in the sector. Also, the Government continues to support reform (ongoing) of the 20 Kankor system (national university entrance examination) and the development of a Higher Education Management Information System (HEMIS). Most notably, a by-law on university autonomy was recently endorsed by the Cabinet. This is a significant step towards increasing the institutional autonomy in the sector. 75. Government commitment and buy-in for SHEP was consistent through much of the project with the exception of a two year period (2010-2012) when there was a vacuum in leadership at the MoHE which affected the pace of implementation. The performance of the Government was also adversely affected by several factors such as the administrative apparatus at the Ministry of Finance, which created several bottlenecks for the disbursement of funds and delayed project implementation at various times. However, these obstacles were relatively minor in comparison to the commitment and overall sound performance of the Government during most of the project. As such, the Government performance has been rated as Satisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 76. Although SHEP encouraged the MoHE and the target universities to take ownership of the project, this was challenging particularly at the early stages of the project. In part, the universities had a difficult time taking ownership because of the lack of autonomy they had within the sector. Over the course of the project this was addressed by involving the universities more directly in planning and implementation (through the University ECs). 77. High turnover of staff at the main PCU and changing leadership at the Ministry (and sometimes the universities) impacted the pace of project implementation where continuity and political support were important factors7. The capacity for overall project management, procurement and financial management fluctuated during the course of the project; with significant and steady improvement in the later stages. The current leadership at the MoHE and the PCU has been particularly engaged and very pro-active in implementing the project, which made a significant contribution to its successful outcome. In view of this, the performance of the implementing agency has been rated as Moderately Satisfactory. (c) Justification of Overall Borrower Performance Rating: Satisfactory 78. The overall Borrower Performance has been rated Satisfactory in view of the overall outcome of the project. The Government showed strong commitment to the project and made a concerted effort to see the project through. The bottlenecks in the government arose mainly from administrative procedures that were in place rather than an unwillingness to support the project. The weakness in procurement and financial management capacity was a reflection of broader systemic weaknesses that were inevitable in the country context. 6. Lessons Learned 79. As in other sectors, the implementation of a donor-funded development project in the Higher Education sector in Afghanistan was a complex task, given the very low baseline operational capacity of the sector. As such, SHEP faced several challenges but also had notable success. Both the challenges and successes provide useful lessons for both the Bank and the Borrower and these lessons may prove useful to other projects in Afghanistan as well as in other similarly challenging implementing contexts. These lessons include:  Strong-evidenced based approaches to project preparation and learning from preceding operations are crucial to effective project design. The highly targeted and relevant design of SHEP may be attributable to the strong evidenced-based approach to the preparation of the project. As previously noted, SHEP was 7 There were 3 Project Coordinators at different intervals during the 8 years of the project life. This is still a relatively moderate turn-over when compared with the other Bank funded projects. The Ministry’s leadership changed 4 times and there were several leadership changes at the project universities between 2005 and 2013. 21 informed by a 2005 education sector assessment, which provided valuable information on the state of the sector and potential approaches to the project design and targeting of project activities and inputs. SHEP was also well-informed by the Bank’s previous EERDP and as such, the block grants and technical assistance, instrumental to the success of the project, were core aspects of the design of the project.  Developing participatory approaches create a platform for reform. The participatory approach facilitated by PCU at MoHE and Steering Committee did have a significant impact on learning, dialogue and consensus building within the sector and supported the reform and development platform for the project; ultimately building a sense of “project ownership” which played a key role in the success and quality project implementation at all levels. From the beginning, the project made efforts to involve the Ministry and the universities in the decision making, planning and implementation phases. Through frequent dialogue, workshops, consultative mechanisms and extensive training activities, the project also made a concerted effort to broaden the participation of university stakeholders.  Government ownership and strengthening systems is likely to lead to sustainable outcomes. Project achievement can be attributed to the strong government ownership of the project because SHEP addressed the priorities of the government, was well-integrated into the Government structure, and was not seen as an external entity. Ultimately, this integration also proved to be important in building capacity in the sector as a whole, because many of the staff at the MoHE PCU and the Executive Committee at the Universities were well integrated into the management structure of the MoHE and the Universities. They did not work exclusively for the PCU but also acted as advisors to the MoHE and the University management. Subsequently, increasing capacity in the PCU and the ECs improved the implementation capacity of SHEP and ultimately contributed to the strong platform for the achievement of the project’s objectives, as well as laying the foundations for continuing reforms and systemic improvements.  Strengthening of governance arrangements has potential for broader impact. This ICR discusses numerous governance issues in the context of a challenging environment and how these issues were addressed. Among them is the strengthening of fiduciary capacity, and strengthening M&E, which improved transparency and accountability.  Flexibility is needed when implementing projects in a fragile environment. The fragile context posed unexpected challenges, including delays in the development of university partnerships (in some cases the security situation served created an obstacle to partnership formation with foreign universities), delays in creating University PCUs because of low staff capacity, and ultimately delays in implementing block grants, because of the delays in establishing university partnerships. Ensuring that the implementing arrangements are flexible enough to account for potential hurdles in project implementation is important in any context but even more so in fragile context such as Afghanistan. The early delays in project implementation may have been avoided if there had been more flexibility in the timing, roll out, and activities which the block grants were contingent on – this lesson was learned and incorported over time. It is useful to ensure implementing arrangements are kept flexible when designing projects in a fragile context.  The role of M&E in keeping the project on track. M&E is critical for tracking project progress, especially in a fragile context, where systemic mechanisms for M&E maybe weak. At early stages of the project, M&E mechanisms were not in place, either at the Ministry or at the project universities. Monitoring was mostly component/activity based and undertaken on an ad-hoc basis. Early planning for such mechanisms is critical for evidenced- based planning, policy formulation, service delivery and monitoring of project outcomes. 22  The importance of ensuring compliance with key Bank policies in an emergency context. Keeping track of compliance with key Bank policies is particularly important in an emergency context where compliance with such policies is often deferred to a later stage. It is also critical that teams pay attention to the project documentation and mission findings during transitions in the leadership of a project. Finally, safeguards specialists need to be involved in all phases of project preparation and supervision to ensure appropriate compliance with Bank policies. 7. Comments on Issues Raised by Borrower/Implementation Agencies/Partners 80. Overall the comments on the ICR were positive and Ministry of Higher Education (MoHE) appreciated the contribution made by the project in improving higher education in Afghanistan. MoHE felt that the ICR accurately and realistically reflected the achievements and challenges of the project. Finally, MoHE looks forward to working closely with the Bank to further strengthen the higher education sector in Afghanistan. 23 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD million equivalent) Components Appraisal Appraisal Appraisal Total Actual Estimate for Estimates for Estimates (USD /Latest Original IDA IDA for ARTF millions) Estimate Percentage Financing Additional Financing (USD of (USD millions) Financing (USD millions) Appraisal (USD millions) millions) 1. Program Development and Quality Improvement 38.5 13.9 5.0 57.4 49.6 86% in key Higher Education Institutions 2. Higher Education 1.5 6.1 7.6 6.1 System 80% Development Total Baseline 40.0 20.0 5.0 65.0 55.7 86 % Cost Total Project 40.0 20.0 5.0 65.0 55.7 86 % Costs Total Financing 40.0 20.0 5.0 65.0 55.7 86 % Required (b) Financing Source of Funds Appraisal Actual/Latest Percentage of Appraisal Estimate Estimate (USD (USD millions) millions) IDA Crisis Response Window (CRW) 20.0 12.2 61% Afghanistan Reconstruction Trust Fund 5.0 4.1 82% IDA GRANT FOR POST-CONFLICT 40.0 39.4 98.5 % 24 Annex 2. Outputs by Component Component 1: Program Development and Quality Improvement in Key Higher Education Institutions (US$ 47.6 million) Achievement: The core of the SHEP design focused on inputs designed to achieve the overall objective of restoring basic operational performance in higher education in Afghanistan and the majority of funding under SHEP was allocated to component one (nearly 84% of the total funding). Despite the constraints of the procurement capacity of the implementing agents, the target outputs linked to each indicator in this component were largely achieved and in some cases, exceeded. Each of the outputs identified in each sub- component can be directly attributed to investment under SHEP. Subcomponent 1.1: University Partnership Program- This component was designed to facilitate and finance partnership programs between key universities in Afghanistan and established foreign universities with the purpose of facilitating curricula revision and development, faculty and staff development and research and publication in higher education in Afghanistan. Key Outputs: Faculty Staff Development  345 faculty members participated in the Master Level degree programs (295 out of the 345 faculty have completed their MA/MSc. programs and returned to their posts as full time Faculty Members in Afghanistan; the other 50 are in the process of completing them).  134 of these were these Masters programs were completed in-country under the Additional Financing phase.  In-country teaching upgrading has taken place through the offering of short courses and exchange visits by Afghan and foreign professors.  674 staff received short term training. Curriculum and Materials  The curriculum was revised and updated in the targeted faculties of all 12 universities supported by SHEP.  The implementation of the new curriculum and study program for supported Bachelor degrees has begun. Capacity Building for Research  Research Committees have been established in the 12 partner universities.  Over all 60 proposals have been developed by the research committees of the 12 universities.  9 Proposals from KU, BU, KEU, HU have been selected for implementation, and contracts have been signed between MoHE and researchers. Subcomponent 1.2: University Block Grants. Under this subcomponent, the project universities were assigned a block grant to support complementary investments to the selected faculties benefiting from the Partnership technical assistance. It was designed to provide additional resources for university projects to improve laboratory and teaching tools, curriculum and research activities. Infrastructure, Teaching and Learning Facilities  Overall, there were 21 construction projects and 17 renovations projects under SHEP. These included: 25 -A new building was constructed for the English Department at Kabul University. -Two buildings (for a library and the engineering faculty) were constructed at Nangarhar faculty. -Two buildings (Economics and Literature) faculty building were constructed at Kandahar University (are near completion).  In addition to this, laboratories were renovated at 18 faculties in all 12 project universities and new equipment was provided.  Computer laboratories were refurbished and new computers purchased at the 12 universities.  Other basic infrastructure such as boundary walls and water supply systems were also rebuilt at several of the universities.  A student service department was established at each of the 12 universities for the first time in the history of tertiary education in Afghanistan, and they are actively supporting students.  Construction of bathrooms facilities at five universities (2 female and 3 male).  Dormitories were renovated at Kandahar, Nangrahar, and Bamyan Universities. Component 2: Higher Education System Development (US$ 5.1 million) Achievement: Under SHEP, there were hallmark achievements in the development of the higher education system as a whole such as the development of five- year National Higher Education Strategic Plan (2010-2014). In addition, strategic plans were developed for each of the twelve partner universities. These outputs contributed toward restoring the policy, planning and programming structures for the Ministry but also for individual universities. The establishment of a Quality Assurance Department, a Quality Assurance Commission, a Curriculum Development Committee, and a National Research Commission serve as markers of the rebuilding of basic institutional infrastructure necessary to higher education system development. More concrete evidence of system development are the outputs of the Quality Assurance System outlined below, including the conduct of self-assessments by universities and the progress under SHEP towards a functional accreditation system for a higher education in the country. The development of the HEMIS under SHEP was to another key indicator of systemic development and a key to ensuring high quality monitoring and evaluation in the country. As indicated below 6 out of the 12 project universities have fully implemented this HEMIS. The development of the HEMIS has enabled the 6 project universities and MoHE to conduct weekly video conferences; collect data on key higher education indicators; and feed it back to the MoHE for analysis and planning. The project is contingent on internet access provided by NATO, which has affected the roll out of the program. The system is web- based and internet access is only intermittently available in most of the country. This constraint, in addition to the low technical capacity of the end-users has affected the utility of the system. These issues are being addressed by the MoHE in order to make the HEMIS a fully functional information management system for the country. Subcomponent 2.1: University Governance and System Development The activities to support system development were aimed at: 1) strengthening key policy, planning and programming structures within the MoHE; 2) Improving University leadership, management and coordination capacity; and 3) setting up a national framework for curricula reform and development. The main outputs achieved here include:  A National Higher Education Strategic Plan (2010-2014) was developed and launched with SHEP support.  Under SHEP, 25 technical specialists were recruited to improve capacity at the MoHE. Among them: a Strategic Planning Manager, a Statistical Officer, Coordination Officer, a Program Officer, a Finance Development Officer, Engineers, a Change Management 26 Advisor to the Minister, a Medical Advisor, technical officers to Deputy Ministers, and assistants to IT Department at the MoHE.  A National Research Commission was established and the Commission has begun the process of developing research standards including; a checklist for the evaluation of research proposals and has preparation of guidelines for writing research proposals.  Regular communication between the MoHE and senior leadership at universities was instituted through weekly meetings, complemented by seminars and workshops (primarily through VC facilities provided under SHEP).  A Curriculum Development Committee (CDC) was established and has been meeting regularly to revise and update the curricula. They have now revised and updated the curricula at all public universities (in addition to the project universities).  Strategic plans were formulated in all twelve universities.  A basic Higher Education Information Management System (HEMIS) has been developed for collecting basic higher education data. The central database is at the MoHE and designed to be linked with databases at each of the public universities. Currently, 9 of the twelve universities under SHEP have set up the HEMIS and the other 3 SHEP partner universities are in the process of setting it up. In addition the HEMIS, is being rolled out in 10 other public universities.  Leadership and management training has taken place for a total of 20 high level administrators from the 6 universities targeted.  1166 administrators at university and the MoHE have received management training. Subcomponent 2.2: Developing Quality Assurance System  Design of Quality Assurance (QA) Framework and System.  Financing of improvements to Management Information System (MIS) for processing entrance examination.  A QA Department has been established.  Directors and two support employees have been recruited for the QA Department.  QA Training for Fifteen QA committee members was conducted in May-June 2013.  A QA National Commission has been established. The commission developed rules and regulations which were introduced to the universities through a series of formal gathering and workshops.  As part of the QA system, a self-evaluation process has been completed in 13 (including the 12 project universities) universities, while pilot self-assessment phase/process is on-going in 18 private higher institutes.  The QA Commission developed a check list based on which on the self-evaluation reports were evaluated.  Based on the results of the assessment, a 2 day-workshop was conducted, focused on training peer- reviewers and providing feedback to the members of the committees.  A peer- review process has been conducted in six universities (Nangarhar, Alberoni, Kabul, Kabul Education, Kabul Poly Technique, and Kabul Medical Universities). 27 Annex 3. Economic and Financial Analysis I. Introduction Afghanistan has survived more than two decades of conflict, recurring drought, and other natural disasters. Afghans are resourceful, resilient, creative, opportunity-seeking and entrepreneurial people as witnessed by the high incidence of labor migration, entrepreneurial activity wherever they are located, trading networks and remittances. The country’s achievements in the face of adversity are remarkable. Given this spirit of survival and independence, significant opportunities lie ahead as political stability returns, the economy grows, and state institutions begin to pick up the reins. Afghanistan’s economy has grown rapidly since 2001: real GDP has increased at 9 percent a year with fluctuations due to the volatility of agriculture production and relatively low inflation.8 What is now necessary is an improvement in the quality and quantity of skilled labor for sustainable economic growth. It was important for Afghanistan to invest more in the higher education sector while continuing to improve primary and secondary education system in the country. The Government’s development strategy is embodied in the twelve national development programs of the National Development Framework which is organized around three “pillars” of development: (i) human and social capital, (ii) physical infrastructure and natural resources, and (iii) private sector development, including security and rule of law and public administration reform and economic management. While all of these are important foundations for poverty reduction, education and training is most important for long term economic growth. In line with the Government’s Development Strategy and National Higher Education Strategy, SHEP was implemented in 2005. SHEP was the first major higher education investment aimed at enhancing the quality and management of higher education in Afghanistan. It provided financing to contract foreign universities to provide technical assistance for curriculum development, pedagogical and management advice and training of Afghan university staff. The project financed the construction of university facilities including libraries and classrooms, dormitories, female and male bathrooms, and computer and science labs. In addition, 12 universities were provided block grants to further develop their facilities based on key strategic development plans. SHEP made a major contribution by building a solid foundation for the reconstruction of Afghanistan’s higher education sector. A detailed economic and financial assessment of the SHEP project was undertaken at the completion of the project. The result of the economic analysis clearly shows that SHEP was an efficient project and a worthy investment. From a financial point of view, the result indicates that the project is financially viable and sustainable in the country’s economy. The purpose of this section is to present summary results of the economic and financial analysis SHEP in Afghanistan. II. Methodology Applying the Bank’s standard approach to economic evaluation, cost benefit methodology was used to estimate Net Present Value (NPV) of the project, and the Internal Rate of Return (IRR). The cost benefit analysis calculates the net benefits generated by each activity component on an incremental basis within the project period. The benefits of the project are equal to the difference between the incremental benefits and the incremental costs of three scenarios: “with” and “without” interventions. The “without” scenario considers how the quality of higher education service delivery would have continued unchanged ; i.e. without provision of new equipment, libraries and science and computer labs, and trainings for university staff and faculty members including MA or PhD programs. The “with SHEP” scenario is adjusted by direct investment and incremental annual costs, and average salary increase among graduates from target universities due to increased productivity. The social 8 Afghanistan in Transition, the World Bank 2014; and WDI Central database. 28 discount rate of 10 %, and employment rate of 80% among target university graduates was employed in the analysis. In the “with” scenario, the assumption of a 5% wage increase among target graduates was applied. The n, sensitivity analysis was conducted to measure the impact on the project’s efficiency assuming that the employment rate (80%) changes while all others remain constant (or a wage increase to be 3% or 7%). Costs The costs include the direct investment and incremental annual costs, and indirect costs such as forgone earnings. A student’s choice of studying at the university is associated with his/her opportunity costs of attending the school. Each student studying at the university forgoes the earnings of a person with high school education that is USD 2,8459 The total opportunity costs incurred by the activities is equal to the number of students enrolled multiplied by their individual opportunity costs. We are assuming that students did not have a paid job while they were at school. The total budget for all activities covered by SHEP was $ 52.7 million for an eight-year period of implementation beginning in 2005. The project cost is presented in the table below which covers both component of SHEP: (i) program development and quality improvement; and (ii) higher education system development. Expenditure for the activities of each component is broken down by the year of expenditure. Table 1: SHEP Project Costs 2005-2013 SHEP Project Costs 2005-2013 for IDA H1620 ( Original Financing ) (in USD million) 2005 2006 2007 2008 2009 2010 2011 2012 2013 Component I: Improvement of Quality in Key Higher Education Institutions University Partnership Programs 0.200 4.381 5.132 5.587 0.885 1.811 0.043 Block Grants support to universities 0.180 1.155 3.744 4.252 6.024 2.400 1.970 0.423 0.007 Total I 0.180 1.355 8.125 9.384 11.611 3.285 3.781 0.423 0.050 Component II:Development Of Higher Education System Support for a New Governance System 0.300 0.215 0.200 0.262 Support for Quality Assurance and Accreditation System 0.150 0.031 0.080 Total II 0.300 0.365 0.231 0.342 Total =Total I + Total II 0.180 1.355 8.125 9.684 11.975 3.516 4.123 0.423 0.050 SHEP Project Costs TF 92544 Component I: Improvement of Quality in Key Higher Education Institutions Support to University Partnership Programs 0.166 0.069 0.049 0.024 Additional Block Grant support to universities 0.744 0.896 1.400 0.709 0.093 Total 0.909 0.965 1.449 0.709 0.117 SHEPAdditional Financing IDA H592 Component I: Improvement of Quality in Key Higher Education Institutions Staff development (academic staff receiving MA and trainings) 0.206 1.806 0.742 Research (establishing research commttees; preparing research guidelines, etc.) 0.119 0.250 0.144 Infrastructure, teaching and learning facilities 0.058 0.465 Project Operation Expenses 1.330 0.336 Total I 0.383 3.851 1.222 Component II: Development Of Higher Education System Governance 0.059 1.257 0.723 Curriculam Revision and Material Development 0.120 0.419 Access Expansion and Higher Education Structure 0.110 0.257 0.132 Accreditation and Quality Assurances, Student Services 0.184 0.345 0.074 Total II 0.473 2.277 0.929 Total= Total I + Total II 0.856 6.128 2.151 Grand total= IDA H1620 + IDA TF92544 + IDA H592 0.180 1.355 8.125 9.684 12.885 4.481 6.428 7.260 2.318 Source: Data updated by PCU at MoHE in August 2013 9 The average annual salary for those with high school education was taken from the NRVA 2007/2008, adjusted by inflation in 2013. 29 Benefits The activities under SHEP were expected to yield benefits by increasing the number of graduates who are more productive in the labor market. The increased productivity of these graduates is captured by the compensation after graduation relative to what they could have earned without the additional education. We assume that the increased productivity remains over their entire life time. Without tracer surveys that could have provided us with the labor status of the graduates in Afghanistan, incremental benefits of the program were estimated by using the National Risk and Vulnerability Assessment (NRVA) Survey conducted in 2007/8 as a secondary source. The Afghan labor market has the typical characteristics of underdeveloped country: relatively high labor force participation rate and employment rate, and low unemployment rate which is compensated by underemployment, indicating that a majority of individuals can’t afford to be unemployed. As the figure 3 shows, the unemployment rate is as low as 7% at the national level; however, unemployment rates in most of the provinces where target universities are located are higher than the national average. SHEP intended to offer a positive externality in the labor market and economy in these provinces by reducing local unemployment and underemployment, and by increasing provincial public spending that ultimately contributes to poverty alleviation in these provinces. Most of the target universities are located in the provinces where the unemployment rate is higher than the national average. The inclusion of these universities in the program was very important not only from an equity perspective but also in terms of the impact on the local economy and the community. Universities have been increasing the number of faculty members and university staff during the project and that has certainly made a positive contribution to local employment opportunities in the provinces where target universities are located. Figure 1: Incidence of underemployment by selected provinces Unemployment Rate in Afghanistan 2008 18.0 16.0 14.0 12.0 % 10.0 8.0 6.0 4.0 2.0 0.0 Nangarhar Kandahar Bamyan Balkh Khost Herat Kabul Kapisa Takhar Jawzian Source: Using NRVA 2007/2008 Beneficiaries of SHEP were students, university graduates, faculty members and staff at 12 target public universities located in Kabul, Nangarhar, Kandahar, Bamyan, Balkh, Khost, Herat, Kapisa, Takhar and Jawzian province. Component I of SHEP financed the provision of new equipment, building of science and computer laboratories, new classrooms, and training programs for faculty members and university staff at the target public universities. Figure 4 shows that the number of students and faculty members who benefitted from the program quickly increased as the new equipment were installed and initial round of the training program took place. After Additional Financing, the number of beneficiaries rapidly increased between 2011 and 2013. 11,256 students and 30 2,708 faculty and university staff directly benefitted from the project, and 345 faculty members and university staff received training or graduate degrees in Afgahanistan or other countries according to MoHE. Figure 2: Number of beneficiaries of SHEP Beneficiaries from the program 2005-2013 16000 14000 Number of faculty members 12000 benefitted by the 10000 program 8000 Number of students benefited 6000 by the program 4000 2000 0 2005 2006 2007 2008 2009 2010 2011 2012 2013 Source: Data from PCU at MoHE The benefits come from increased number of graduates who became more productive in the labor market. Benefit streams used in this analysis were (a) increased earnings resulting from larger number of students completed university education; and (b) enhanced labor skills from the higher quality of university education. The activities in the component II are expected to increase graduates’ productivity over their life time. The increased productivity of graduates is captured by their compensation after graduation relative to what they would have earned without additional education. Total incremental earnings in (a) were adjusted by 80 percent of employment rate among university graduates for the first year of graduation; however, from the second year onwards, we are assuming full employment with all of them. It is assumed that graduates’ upgraded skills during the remainder of their entire working years, and on average, a graduate remains in the labor force for 40 years. The quality premium in benefit (b) applied to all students who enrolled during the project period. An additional 10 % of the average earning of high school graduates times the number of students enrolled in each year was used as an indirect measure of the quality premium. Table 2 : Descriptions of Key Parameters Value used Plausible for Descriptions range computation Notes Active working life time 40 years Employment rate among target university graduates 70%-90% 80% Data from MoHE Out of pocket university US$80- education (per student of 100 US$90 Data from MoHE 31 public university) Average monthly salary among high school graduates US$108.3 NRVA 2007/8 Average monthly salary among university graduates .. US$182.9 NRVA 2007/8 GDP deflator IMF World Economic Outlook 2013 Discount rate 10%-15% 10% Real effective exchange rate IMF Afghanistan Country Report III. Results of the Economic Analysis Table 3 below shows the results of the economic analysis. Base case scenario of which 80 percent of employment among target university graduates yielded an internal rate of return (IRR) of 24 percent and a net present value (NPV) of US$ 257 million, the results clearly indicate that the SHEP project was worth investing. The results of sensitivity analysis show that the project is more efficient with the assumption of a higher employment rate of 90 percent, and also shows slightly less favorable results with lower employment rate of 70 percent. Table 3: Summary results of the economic analysis 1. Earning 2. Earning 3. Earning increased by 3% increased by 5% increased by 7% due to quality and due to quality and due to quality and relevance relevance relevance improvement improvement improvement Scenarios NPV IRR NPV IRR NPV IRR 90% of employment rate 257 24 266 24 275 25 80% employment rate (base case) 248 23 257 24 266 24 70% employment rate 239 23 247 23 256 24 Note: NPV in millions US$. IV. Fiscal Sustainability Fiscal sustainability of the project depends largely on the country’s economic performance and the government’s willingness to continue supporting higher education institutions financed by SHEP. Economic performance since the fall of the Taliban regime has been strong, and macroeconomic stability has been maintained. During 2002/03–2006/07, real GDP growth averaged 15 percent per year, reflecting a recovery in agriculture, donor- funded postwar reconstruction, and initial yet promising growth of a range of private sector activities. Between 32 2008-and 2011, Afghanistan’s economy slowed down; however, it grew significantly in 2012, increasing from 7.3 percent in 2011 to 11 percent in 2012 mostly due to a good harvest. While economic growth in South Asia weakened considerably in 2012 to an estimated 5.4 percent, the economic performance of Afghanistan is remarkable, and recognized as fastest –growing economy in all of South Asia. Prudent monetary and fiscal policies facilitated a decline in inflation to single digits in 2006/07, but inflation has risen recently owing mainly to global commodity price increases. Revenue collection increased as a share of GDP from 4.7 percent in 2003/04 to 7.5 percent in 2006/07, before declining to 7.0 percent in 2007/08. Finally, it has been observed that universities are developing their ability to finance the operating expenditures of SHEP investments beyond the project’s life. For instance, in several cases, some unive rsities in Kabul province, have secured service contracts with government organizations or/and private firms, and have established partnerships with other universities in the US, Europe and Japan. This is a result of new equipment financed under SHEP, which could provide them with recurrent additional revenues. 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes Name Title Responsibility Specialization Keiko Miwa Country Manager Task Team Leader Education Scherezad Joya Monami Sr. Education Specialist Task Team Leader Education Latif Avil van Adams Sr. Advisor Lending Education Manoj Agrawal Financial Management Lending Financial Management Specialist William Experton Sector Leader Lending Education Quamrul Hasan Sr. Procurement Lending Procurement Specialist Rajendra Dhoj Joshi Sr. Education Specialist Lending Education Scherezad J.M.Latif Education Specialist Lending Education Hena Mukherjee Lead Education Lending Education Specialist Vikram Raghavan Lead Counsel Lending Legal William Saint Lead Education Lending Education Specialist Jamil Salmi Sector Manager Lending Education Rosita Maria Van Meel Sr. Education Specialist Lending Education Habibulla Wajdi Education Specialist Task Team Leader Education Amit Dar Sector Manager Supervision Education Deepal Fernando Sr. Procurement Supervision Procurement Specialist Venkatesh Sundararaman Sr. Economist Supervision Economics Kenneth O. Okpara Sr. Financial Supervision Financial Management Management Specialist Hasib Karimzada Program Assistant Supervision Program Assistant Rahimullah Wardak Procurement Specialist Supervision Procurement Xiaonan Cao Sr. Education Specialist Supervision Education Manoj Agrawal Consultant Lending Consultant Julie-Anne Graitge Program Assistant Lending/Supervision Program Assistant Md. Mokhlesur Rahman Sr. Operations Officer Supervision Operations 34 Benoit Millot Consultant Supervision Consultant Nalin Jena Sr. Education Specialist Supervision Education Asha Narayan Sr. Financial Supervision Financial Management Management Specialist Joel Reyes Sr. Institutional Task Team Leader Institutional Development Development Specialist Moasteen Jouya Education Specialist Supervision Education Veronica Milagro Minaya Consultant Supervision Consultant Lazarte Samantha De Silva Sr. Social Protection Supervision Social Protection/ Sector Specialist Coordinator Asif Quirishi ET Temporary Supervision Admin/Operations Harsha Aturupane Lead Education Task Team Leader Economics/Education Specialist Mariam Haidary Program Assistant Supervision Program Assistant Roshini Ebenezer Research Analyst ICR Team/ Author Research Surendra Agarwal Consultant Supervision/ Consultant Abdul Hai Sofizada Sr. Education Specialist Supervision/ Education ICR Team Zohra Farook Financial Management Supervision Financial Management Specialist Mohamed Abbas Procurement Analyst Supervision Procurement Kimie Tanabe Economist ICR Team Leader Economics /Author Maria E. Gracheva Sr. Operations Officer ICR Team Operations 35 Annex 5 & 6. Beneficiary and Stakeholder Workshop Report and Results A SHEP Beneficiary and Stakeholder Workshop was conducted at the MoHE in Kabul on November 30- December 1, 2013. The purpose of the workshop was to assess the state of project completion, and identify key lessons learned during the course of project implementation. The Workshop was focused on two critical outcomes of the project: (i) Quality improvement in key higher education institutions, and (ii) Higher education system development. The Workshop was generously supported by his Excellency, the Minister of Higher Education, the Project Coordinator of SHEP PCU and his team as well as IT staff and other staff at MoHE. The Workshop also included 80 participants representing the Ministry of Higher Education and those project universities (faculties, male and female students, and university staff) under SHEP and granted all participants an opportunity to: (a) present assessment of the project; (b) identify challenges which had emerged and measures taken to address these issues and; (c) participate in a fruitful discussion for further improvement of Higher Education in Afghanistan. The following is a summary of the main outcomes: Development of University Facilities, Equipment, and Student Service Centers:  During the three decades of war and conflicts, university buildings were destroyed, and universities badly suffered the loss of facilities including: classrooms, libraries, laboratories and other buildings. All representatives of universities appreciated the support of SHEP in financing the reconstruction and rehabilitation of university buildings, and the provision of equipment.  Lab and teaching equipment was provided for selected faculties at the project universities. Project universities were able to establish computer labs, and most of the faculties were provided with computers. Classrooms were equipped with basic hardware including chairs, projectors, sound systems etc.; which improved the overall classroom environment and contributed to an improvement in the quality of the learning experience. Other university facilities such as academic and administrative buildings, female bathrooms, dormitories, cafeteria, gymnasiums, auditoriums and sports grounds were also constructed, rehabilitated or renovated.  For the first time in history, project universities in the country established a Student Service Center in order to improve quality of education. Although some universities are much more active than the rest, each university has a Student Service Officer, and several committees were established to support students in various aspects of student activities and services including: i) academic counselling; ii) mental health counselling; iii) sports and cultural events; iv) conflict management; v) career services; vi) dormitory services; and vii) new student orientation. At some universities like Kandahar and Herat, students are very actively engaged in activities to support other students. Kandahar University has been developing a Student Service Center with a career development program, which is highly popular and appreciated by their students.  A National Curriculum Development Committee was established and number of workshops and trainings were conducted during the course of the project to revise the curriculum; and all 12 universities have revised and updated the curriculum. University Partnership Program, Fellowships and Trainings:  Many management/administration/ and faculty development workshops and training sessions were conducted for faculty and administrative staff of the universities and the MoHE. This has improved the governance; as well as the quality of education at these institutions. The administrative training have also 36 enabled better management and a better understanding of governance and university management procedures among the administrative staff at the universities.  A research committee was established at the universities. Many faculty members participated in short- term (academic workshop and seminars) and long-term (Masters and PhD programs) training and workshops conducted both in and out of Afghanistan. Professors who participated in such training sessions were able to learn new research methodology; and were able to draft research proposals and apply new teaching methodologies learned at the workshops. The number of professors with Masters and PhD also increased (SHEP supported over 300 staff in their further studies and those who completed their advanced studies are now back at their respective universities). Challenges:  The universities had to go through dual procurement process of Government and World Bank procedures, which was very complicated for university and MoHE procurement specialists. Very early training/capacity building efforts were misdirected at higher levels of university management and not at the staff actually dealing with procurement and financial management on a daily basis (mainly because of the cultural factors, such as a respect for hierarchy etc. This was rectified later as training was redirected at the appropriate level of implementation management. Early training programs could also have been supplemented with an introduction to the project and World Bank procedures, which would have provided a clear context for the trainings. As the project progressed, these issues were also addressed as training was redirected.  SHEP provided universities with new and updated lab equipment, but the universities might have benefitted from more “hands- on training” for the use of this new equipment. Although Student Service Centers (SSC) were established and actively function at all of the project universities, the Student Service Department at most of the universities do not have enough of a budget to fully cover activities including; career-oriented programs (internships; career fairs, workshop and seminars); career guidance and training; skills- development courses for students to attain the skills needed to meet the needs of the labor market. Most of universities also need their own physical space for the SSC. Promoting University Autonomy:  University autonomy has been promoted and increased over the course of SHEP; and it is an on-going process. Under the new by-law on university autonomy, universities will soon be allowed to have a bank account and will be able to have access to and manage their own budgets. Development & Challenges of HEMIS:  The development of HEMIS has been taken place in 2008 with funding of the World Bank SHEP project. The central system of HEMIS is located at the IT Department of the MoHE functioning for technical trouble shooting, development of IT, and managing information and data. Out of 12 project universities, 9 universities have been provided with the hardware equipment, and they are all able to connect with MoHE through a Video Conference (VC) system. The MoHE plans to include the other 3 project universities in the system shortly. HEMIS greatly promoted the information flow between universities and MoHE. The system has improved the accuracy and efficiency of data management; including student records, management indicators etc. Quality Assurance & Accreditation system:  The Quality Assurance Department was established at the universities for the first time in order to monitor and assess the overall quality of the teaching and learning process and other related issues. The committee was established and a head of the committee was appointed at each university. Universities had quality assurance workshops, and valuable information was exchanged among the senior staff of the MoHE and the universities. Accreditation system was introduced at the most of the universities. 37 University self-assessment system is in place in all 12 universities. All project universities have conducted two self-assessments in the last 5 years, and these reports were submitted to the MoHE. Challenges:  The support provided to research and quality assurance committees ended being less efficient due to the delay of budget transfer. Universities provided the research and academic activity plan based on the SHEP proposals; however, there was at times a lack of clarity on the schedule for implementation.  Development of HEMIS was a great achievement; however, some universities have had difficulties in collecting and analysing the data in the absence of a M&E specialist or a skilled IT specialist. 38 Annex 7. Summary of Borrower’s ICR The Development of SHEP (2005-2013) Component One: Program Development and Quality Improvement in key Higher Education Institutions The first phase of SHEP support was targeted at six universities. The six beneficiary universities were: Balkh University; Herat University; Kabul University; Kabul Polytechnic; Nangarhar University, and Kandahar University. There were several achievements under this component, with the first phase of project financing. These include: partnerships with overseas universities for academic support, block grants for university development, the preparation of strategic plans, modernization of curricula in key faculties, human resource development of faculty (Masters and PhD programs in foreign universities), training in administration, improved equipment for libraries, laboratories, and teaching-learning activities, and upgraded physical facilities. All universities benefited from the support of SHEP. Component Two: Higher Education System Development The Higher Education System Development component of the project assisted the development of a new governance system and a quality assurance and accreditation system. Followings are the major achievements under this component: (a) key specialist staff and consultants for the policy and planning directorate of the Ministry of Higher Education (MHE) have been appointed, the capacity of MHE to facilitate faculty and staff development has been strengthened, and regular interactions between the MHE, University Chancellors and other senior administrators have been improved; (b) the leadership, management and coordination capacity of the MHE and universities have been strengthened, (c) a national framework for curriculum reform and development has been set up; (d) self-assessment and structures for accreditation and quality assurance have been initiated; and (e) gender equality elements for higher education in Afghanistan’s present context have been promoted. Evolution of SHEP under ARTF and Additional Financing In 2008, ARTF provided a grant for US$5 million to upgrade physical infrastructure rehabilitation for four (Kabuk Polytechnic, Balkh, Nanagarhar and Kandahar) of the six universities supported by SHEP in 2005. The Additional IDA Financing in 2010 expanded the activities of the SHEP project to cover six more universities. The six universities added to SHEP under the Additional IDA Financing in 2010 were: Al-Beroni University; Bamyan University; Jawzjan University; Kabul Education University; Khost University; and Takhar University. Overall Progress of SHEP (2005-2013) under the Two IDA Grants and ARTF Component 1: Program Development and Quality Improvement 1. Staff Development  345 academic staff had received their Masters degrees since the start of the project  674 academic staff completed short term training courses 2. Curriculum Development and Implementation  A national Curriculum Development Commission (CDC) has been formed with representatives of faculty from around the country.  The CDC has designed and improved curriculum for publication 39  The curriculum has been reviewed and revised in 95 faculties, including in all twelve of 12 target universities and implemented in 9 so far. 3. Research  Research Committees have been established in all 12 SHEP universities  The National Research Commission was established and has prepared guidelines for writing and evaluating research proposals. 4. Infrastructure, Teaching and Learning Facilities  Construction, renovation and rehabilitation of academic buildings, libraries, dormitories, bathrooms etc.  Procurement of lab equipment, books, teaching materials, furniture (including classroom chairs and tables) and computer/IT facilities for various faculties in several universities  Preparation of the designs of several new faculty buildings. Because of the approaching closing date of the project and considering the long construction periods, it was decided not to seek additional ARTF financing for infrastructure improvements. Such construction could be undertaken in the future.  Preparation of bidding documents for the laboratory equipment for participating universities. The ICB process for most packages however was not successful, and contracts could be awarded only for a few packages and lots within those packages. Because of the upcoming closing date, it was decided that the laboratory equipment would be supported in the future. 5. Establishment of Student Services (Advising, Counseling, Mental Health, Orientation for New Students, and Placement  Student Services framework and policies established in Afghan universities (addressing mental health, gender, sports, residential and new student orientation issues) A Student Services framework and 2 different level committees were established – Central (5 members) and University Levels. Component 2: Higher Education System Development 1. Governance and Quality Assurance  Strategic plans have been formulated for all twelve universities  A Quality Assurance (QA) Department was established  The QA National Commission has also been established and has developed rules and regulations that were introduced to the universities through  seminars and workshops 2. Improvement of Higher Education Leadership, Management and Coordination Capacity  1166 university and MoHE staff/administrators received training in university management, governance and leadership skills;  60 non-academic staff received training in financial management and procurement issues; and  Higher Education Information Management System (HEMIS) was introduced in 9 universities. 3. Development of a Credit System to Facilitate Transfer and Accumulation of Credits between Universities  2 workshops were held on development of a credit system  Kabul and Kabul Education University have started a credit system in some faculties 40 Result of the Borrower’s Assessment of SHEP The World Bank contribution to higher education in Afghanistan is highly appreciated. A large number of academic and faculty staff have received their master degrees, which has had a very great impact on the quality of teaching and faculty development. For instance, the Vice Chancellor of Kandahar University along with a group of the lecturers from this university received their Master’s degree from AIT Thailand (under the SHEP University Partnership program). They have gone on to play a great role in the development of the above- mentioned university, which is located in one of the most unsafe but critical cities in Afghanistan. In addition, SHEP’s provision of short-term training for academic staff has improved the quality of services and has had a great impact on faculty development. Also, revision of the curriculum for all the project universities as well as the whole public higher education sector under the Curriculum Development Commission (which was established and supported by SHEP) was a great accomplishment. The revised curricula are not only used by the 12 universities under SHEP but also for the whole universities in the country. It is the first time in the history of the Higher Education sector in the country that the entire curriculum has been revised. The Higher Education system in Afghanistan was always criticized by the students and other academic staff for not being updated. The universities had been using highly outdated curricula (over 30 years old). For the first time the research committees have been established and supported under SHEP. The members of the National Research Committee were given national and international training in research methodology under SHEP. With this knowledge, they are now training universities research committees. This is a great achievement. Constructions of the new teaching facilities, libraries, auditoriums, press buildings, sports grounds and female toilets have been very effective in the development of higher education. Establishing the student service section at the universities has been provided an opportunity to support the students through advising, counseling, mental health and orientation of new students. This is a totally new development at the universities. The education sector in Afghanistan has been cruelly affected due to decades of war and civil conflicts in this country. Afghanistan had lost almost all of skilled manpower and institutional foundations. In the post-Taliban years, we have put the EDUCATION sector as one of our top priorities alongside security; to educate our people and build up higher education institutions for research and development in order to make bring Afghanistan up to international standards and more competitive with other countries of the world in the fields of science and technology. The Ministry of Higher Education is mandated with producing quality human resources and is committed to the timely implementation of its projects. The Ministry gives high value and consideration to the World Bank funded projects. We appreciate the World Bank’s efforts through the SHEP Project for the successful implementation of all the activities, which have been supported by World Bank during the last 8 years. World Bank support and activities aimed at improving the Higher Education system are quite effective for the people of Afghanistan. The activities towards good governance, quality assurance and accreditation sub-components have also been quite effective. The quality assurance and accreditation departments did not exist prior to SHEP. This department was established under SHEP. There were several changes in the PCU coordination and staff, which slowed down the pace of implementation at various points. However, the latest team had been so actively engaged in SHEP activities resulting in an accelerated pace in the later stages of project implementation. The resent team has completed most of the incomplete projects during the last 12 months. They equipped the 12 universities with around 35 labs. Many infrastructure projects, research, QA trainings, management and leadership trainings have also happened during the last 12 months. 41 42 43 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not Applicable. 44 Annex 9. List of Supporting Documents 1. World Bank (2005-2013), Documents in Project’s Electronic File, including Project Appraisal Documents, Grant Agreement, Aide Memoirs, Back-to-Office Reports, Project Status Reports, Financial Assessments, and Project Procurement Assessments; Washington DC. 2. World Bank (2003). Afghanistan: Transitional Support Strategy. The World Bank, Washington DC. 3. Keiko Miwa (2005). Investing in Afghanistan's future: a strategy note on the Education System in Afghanistan, The World Bank,Washington DC. 4. Government of Afghanistan, Ministry of Higher Education (2008). Mid-Term Review for the Strengthening Higher Education Project. Ministry of Higher Education, Kabul, Afghanistan. 5. International Development Association and International Finance Cooperation (2009), Interim Strategy Note for the Islamic Republic of Afghanistan (2009-2011). Report No: 47939-AF. 6. Government of Afghanistan, Ministry of Higher Education (2009). National Higher Education-Strategic Plan (2010-2014). 7. IMF (2010). Afghanistan Country Report 2010. No. 10/22. 8. International Development Association and International Finance Cooperation (2012). Interim Strategy Note for Islamic Republic of Afghanistan (2012-2014). Report No: 66862-AF. 9. World Bank (2010). Poverty Status in Afghanistan: A Profile Based on National Risk and Vulnerability Assessment (NRVA) 2007/08. The World Bank, Washington DC. 10. World Bank (2013). Afghanistan in Transition: Looking Beyond 2014. The World Bank, Washington DC. 45 IBRD 34014 62 ° 64 ° 66 ° 68 ° 70 ° 72 ° 74 ° 76 ° U Z B E K I S TA N TA J I K I S TA N CHINA 38 ° 38 ° T U R K M E N I S TA N Am uD ar y a AFGHANISTAN BADAKHSHAN Ko Faizabad kc rr idor JAWZJAN ha ir a n Co akh KUNDUZ m Pa W Taloqan Sheberghan Kunduz Mazar-i Sharif TAKHAR 74 ° BALKH Saripul Aybak Pol-e Khomri (Samangan) 36 ° Maimana 36 ° SAMANGAN FARYAB BAGHLAN SARIPUL Sh ek ar i ir j sh an PARWANP NURISTAN BADGHIS r Mahmud-e Raqi Nurestan na Charikar KAP AFGHANISTAN Ku ISA ar Qal’eh-ye Now Asadabad LAGH- ngh Bamyan KABUL MAN KUNAR STRENGTHENING Ari BAMYAN Kabul Mehtar Lam Chaghcharan d Kabul Herat H a ri R u Maidan Shahr WARDAK NANGARHAR Jalalabad HIGHER EDUCATION HERAT Pol-e 34 ° GHOR LOGAR Alam 34 ° PROGRAM u d PAKTIKA ra hR Ghazni Fa URUZGAN Gardiz PARTICIPATING UNIVERSITIES GHAZNI KHOST d an Khost lm He Sharan RIVERS LAKES Tarin SALINE LAKES PAKTIKA FARAH Far ah Rud Kowt ISLAMIC REPUBLIC OF IRAN Farah MAIN ROADS ZABUL Qalat PROVINCE CAPITALS a b k 32 ° 32 ° a nd rna NATIONAL CAPITAL gh Ta ra Ar Lu Rud PROVINCE BOUNDARIES Kandahar s h K ha Lashkar INTERNATIONAL BOUNDARIES Gah KANDAHAR HILMAND Zaranj PA K I S TA N NIMROZ 0 50 100 150 200 250 300 Hilm and KILOMETERS 30 ° 30 ° This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. 62 ° 64 ° 66 ° 68 ° 70 ° 72 ° MAY 2005