Document of The World Bank Report No: ICR00003859 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-10112; TF- 15895; and TF- 16992) ON A GRANT IN THE AMOUNT OF US$8.0 MILLION AND TWO ADDITIONAL FINANCING GRANTS OF US$9.1 MILLION AND US$9.0 MILLION TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A KHYBER-PAKHTUNKHWA EMERGENCY ROADS RECOVERY PROJECT (KP-ERRP) August 11, 2016 Transport and ICT Global Practice Sustainable Development Unit South Asia Region CURRENCY EQUIVALENTS Exchange Rate Effective as of January 29, 2016 Currency Unit = Pakistan Rupees (Rs.) Rs. 1 = US$ 0.0095 US$ 1.00 = Rs. 105. FISCAL YEAR July 01 – June 30 ABBREVIATIONS AND ACRONYMS AADT Annual Average Daily Traffic ADB Asian Development Bank AF Additional Financing CSR Consolidated Schedule of Rates CSS Customer Satisfaction Survey CSSR Customer Satisfaction Survey Report DD (E&R) Deputy Director (Environmental & Resettlement) DNA Damage Needs Assessment DO Development Objective EIA Environmental Impact Assessment EIRR Economic Internal Rate of Return EMP Environmental Management Plan EPP Emergency Project Paper ERRP Emergency Roads Recovery Project ESSAF Environmental and Social Screening and Assessment Framework EU European Union FATA Federally Administered Tribal Areas FHA Frontier Highways Authority FM Financial Management GoKP Government of Khyber-Pakhtunkhwa GOP Government of Pakistan ICR Implementation Completion Report ICT Information & Communication Technology IP Implementation Performance ISR Implementation Status & Results km Kilometer KP Khyber Pakhtunkhwa KP-ERRP Khyber Pakhtunkhwa – Emergency Road Recovery Project M&E Monitoring and Evaluation MDTF Multi Donor Trust Fund MTR Mid-term Review NCB National Competitive Bidding OP Operational Policy PCNA Post Conflict Needs Assessment PDO Project Development Objective PkHA Pakhtunkhwa Highway Authority PMU Project Management Unit TTL Task Team Leader UN United Nations USAID US Agency for International Development VOC Vehicle Operating Cost WB World Bank Vice President: Annette Dixon Country Director: Patchamuthu Illangovan Sector Manager: Karla Gonzalez Carvajal Project Team Leader: Zafar Iqbal Raja ICR Team Leader: A.K. Farhad Ahmed PAKISTAN KHYBER-PAKHTUNKHWA EMERGENCY ROADS RECOVERY PROJECT (KP-ERRP) CONTENTS DATA SHEET ..................................................................................................................... i A. Basic Information ............................................................................................................ i B. Key Dates ........................................................................................................................ i C. Ratings Summary ........................................................................................................... ii D. Sector and Theme Codes................................................................................................ ii E. Bank Staff ....................................................................................................................... ii F. Results Framework Analysis ......................................................................................... iii G. Ratings of Project Performance in ISRs ........................................................................ v H. Restructuring (if any) ..................................................................................................... v I. Disbursement Profile ...................................................................................................... vi 1. Project Context, Development Objectives and Design ................................................... 1 3. Assessment of Outcomes .............................................................................................. 12 4. Assessment of Risk to Development Outcome ............................................................. 18 5. Assessment of Bank and Borrower Performance ......................................................... 19 6. Lessons Learned............................................................................................................ 20 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 21 Annex 1. Project Costs and Financing ................................................................................ 1 Annex 2. Outputs by Component........................................................................................ 3 Annex 3. Economic and Financial Analysis ....................................................................... 4 Annex 4. Bank Lending and Implementation Support/Supervision Processes................... 8 Annex 5. Fact Finding and Consultation Workshops Report and Results ........................ 10 Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 13 Annex 7. List of Supporting Documents .......................................................................... 16 MAP – KP-ERRP ............................................................................................................. 18 DATA SHEET A. Basic Information Khyber-Pakhtunkhwa Country: Pakistan Project Name: Emergency Roads Recovery Project TF-10112; TF- 15895; Project ID: P125584 L/C/TF Number(s): and TF- 16992 ICR Date: 08/11/2016 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: MDTF Borrower: PAKISTAN/MOF Original Total US$8.0 million Disbursed Amount: US$25.62 million Commitment: Revised Amount: US$ 26.10 million Environmental Category: B Implementing Agencies: Pakhtunkhwa Highway Authority (PkHA) (formerly Frontier Highways Authority (FHA)) Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Phase I (Original project, TF- 10122) Concept Review: 06/01/2011 Effectiveness: 10/17/2011 10/17/2011 Appraisal: 07/21/11 Restructuring(s): Approval: 08/23/2011 Mid-term Review: 12/31/2013 None Closing: 06/30/2014 06/30/2014 Phase II (AF-1, TF-15895) Concept Review: Effectiveness: 01/15/2014 01/15/2014 Appraisal: 12/4/2013 Restructuring(s): Approval: 01/15/2014 Mid-term Review: Closing: 06/30/2015 06/30/2015 Phase III (AF-2, TF-16992) Concept Review: Effectiveness: 05/30/2014 05/30/2014 Appraisal: 04/14/2014 Restructuring(s): Approval: 05/30/2014 Mid-term Review: Closing: 06/30/2015 10/31/2015 i C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately satisfactory Risk to Development Outcome: Significant Bank Performance: Moderately satisfactory Borrower Performance: Moderately satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Moderately satisfactory Implementing Quality of Supervision: Moderately satisfactory Moderately satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately satisfactory Moderately satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project at Quality at Entry No any time (Yes/No): (QEA): Not Applicable Problem Project at any time Quality of Supervision No Not Applicable (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Rural and Inter-Urban Roads and Highways 73 100 Other social services 18 Public administration- Transportation 9 Theme Code (as % of total Bank financing) Conflict prevention and post-conflict reconstruction 100 E. Bank Staff Positions At ICR At Approval Vice President: Annette Dixon Isabel M. Guerrero Country Director: Patchamuthu Illangovan Rachid Benmessaoud Sector Manager: Karla Gonzalez Carvajal Michel Audigé ii Project Team Leader: Zafar Iqbal Raja Zafar Iqbal Raja ICR Team Leader: A K Farhad Ahmed ICR Primary Author: Kirit G Vaidya A K Farhad Ahmed F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The PDO is to enable the population along the Project corridor to benefit from year round improved access and mobility through reconstruction of priority damaged roads and bridges in the conflict hit areas. Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised. (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years 34%, 26% and 5% decrease in vehicle operating costs (VOCs) of 2-axle trucks, 3- Indicator 1: axle trucks and passenger buses Rs 23.9 (2-axle) Rs 15.8 (2-axle) Actual value data not Value Rs 24.76 (3-axle) Rs 18.3 (3-axle) available. See (quantitative) Rs 12.09 (passenger Rs 11.5 (passenger comments below and (Original grant, bus) bus) Section 1.3, 10.5km sub-section) paragraph 7. 07/21/2011 06/30/2014 Value Rs 30.9 (2-axle) Rs 20.4 (2-axle) Rs 20.5 (2-axle) (quantitative) Rs 31.8 (3-axle) Rs 23.5 (3-axle) (34% reduction) (Original grant, AF- Rs 37.2 (passenger bus) Rs 35.5 (passenger Rs 21.1 (3-axle) 1 & 2, total 40.9km) bus) (34% reduction) Rs 28.8 (passenger bus) (23% reduction) Date achieved 12/04/2013 06/30/2015 03/26/2016 Comments The results represent progress on the whole 40.9 km section of S3-B under the (incl. % original grant and AF-1 & AF-2. Achievements are 100% (2-axle trucks), 130% achievements) (3-axle trucks) and 460% (passenger buses). The achievements calculated are percentages of VOC savings (e.g., for 3-axle trucks the 34% reduction for trucks is 130% achievement of the target of a 26% decrease in VOC). For the original 10.5km road sub-section, information on values achieved for this indicator (and indicator 2 that follows below) is not available. However, this 10.5 km sub-section is part of the whole 40.9km section of the provincial highway S3-B reconstructed under the project. Please see Section 1.3, paragraph 7, and Section 1.7 for more information on the road sections completed under the original project and the additional financings and Section 3 (Assessment of Outcomes), and Annex 3 for more details. iii Indicator 2: 50% increase in operating speeds of commercial traffic. Value Actual value data not (quantitative) available. See 20km/hour 30km/hour -- (Original grant, Section 1.3, 10.5km sub-section) paragraph 7. 07/21/2011 06/30/2014 Value 38km/hour 57km/hour 63km/hour (quantitative) (Original grant, AF- 1 & 2, total 40.9km) Date achieved 12/04/2013 06/30/2015 03/26/2016 Comments The results represent progress on the whole 40.9 km section of S3-B under the (incl. % original grant and AF-1 & AF-2. 116% achievement of the target of 50% achievement) increase in average operating speed for commercial vehicles for the whole 40.9km section of S3-B reconstructed (please see comments box for Indicator 1, Section 3 (Assessment of Outcomes), and Annex 3 for more details). Indicator 3 : Level of customer satisfaction (High = 4.0, Low = 1.0) Value Information not Not available 3.0 -- (qualitative) available. Date achieved 07/21/2011 06/30/2015 10/31/2015 The customer satisfaction survey, effectively the beneficiary survey, conducted during September-October 2015 did not collect information that specifically Comments addressed customer satisfaction on the PDO as it was defined, i.e. “to enable the (incl. % population along the Project corridor to benefit from year round improved access achievement) and mobility through reconstruction of priority damaged roads and bridges in the conflict hit areas” (see paragraphs 51 and 60 to 62 for further details). (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years About 10.5km of the provincial highway S-3B from Sharif Abad (km 30+424) to Kanju Indicator 1 : (km 40+924) constructed and operational. Value 0.0 10.5km 40.9km 50.3km (qualitative) Date achieved 07/21/2011 06/30/2014 06/30/2015 10/31/2015 123% achievement of revised target value of 40.9km (50.3km constructed). The Comments project was extended by additional financings to reconstruct the whole Kanju to (incl. % Chakdara section of provincial highway S3-B (40.9km) and additional link roads achievement) (9.4km) were constructed from project cost savings (see “1.7 Other significant changes” for more information). Indicator 2 : Project Management Value Satisfactory Moderately Project start-up (quantitative performance of satisfactory iv or qualitative) construction supervision and contract administration activities and environmental and social safeguards activities. Date achieved 07/31/2011 During project At ICR appraisal While Project Management is not technically an indicator, given the emergency nature Comments of the project, the project paper included a management indicator for performance (incl. % monitoring. The “moderately satisfactory” rating partially reflects weaknesses in M&E achievement) as well as other issues detailed in the ICR. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (US$ millions) 1 12/10/2011 Satisfactory Satisfactory 2.57 2 06/15/2012 Satisfactory Highly satisfactory 3.99 3 11/12/2012 Satisfactory Highly satisfactory 6.49 4 05/16/2013 Satisfactory Highly satisfactory 8.00 5 12/02/2013 Satisfactory Highly satisfactory 8.00 6 05/26/2014 Satisfactory Highly satisfactory 15.54 7 01/05/2015 Satisfactory Highly satisfactory 22.46 8 07/20/2015 Satisfactory Highly satisfactory 26.10 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in US$ millions If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Against Formally Revised PDO/Targets Overall (weighted) rating v I. Disbursement Profile vi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The Khyber-Pakhtunkhwa Emergency Road Recovery Project (KP-ERRP) was conceived in the aftermath of military operations in large parts of Khyber-Pakhtunkhwa (KP) Province and Federally Administered Tribal Areas (FATA). In early 2009, the Government of Pakistan (GOP) launched military operations against pockets of militants in KP and FATA. Following these operations, large parts of FATA and KP were in serious need of restoration of economic activity and physical and social infrastructure. 2. The federal and provincial government response was to embark on rapid post-conflict recovery and reconstruction efforts with support from international and national agencies. GOP conducted assessments to determine the medium and long term needs as a part of these efforts. A Damage and Needs Assessment (DNA) was completed in 2009 with the Asian Development Bank (ADB) and World Bank (WB) provided support. A subsequent Post Crisis Needs Assessment (PCNA) was completed in October 2010 with the assistance of ADB, European Union (EU), United Nations (UN) and WB. 3. The Bank established a Multi-donor Trust Fund (MDTF) with development partners to support the implementation of a program for reconstruction and development, following the assessments. The Government of Khyber Pakhtunkhwa (GoKP) and Pakhtunkhwa Highway Authority (PkHA), an autonomous road organisation of the GoKP, had identified rehabilitation of the severely damaged single lane provincial highway S-3B as a high priority project which serves the local population of some 300,000 people on the right bank of Swat river and is of strategic importance for law enforcement agencies for access to some districts as an alternative to National Highway N-45. MDTF agreed to support reconstruction of this road under Pillar I of its financing strategy (Restoration of Damaged Infrastructure and Disrupted Services1). 1.2 Original Project Development Objective (PDO) and Key Indicators (as approved) 4. The project development objective (PDO) was: to enable the population along the Project corridor to benefit from year-round improved access and mobility through reconstruction of priority damaged roads and bridges in the conflict hit areas. 5. The PDO level results indicators were: a) 34%, 26% and 5% decrease in vehicle operating costs (VOCs) of 2-axle trucks, 3-axle trucks and passenger buses, respectively. b) 50% increase in operating speeds of commercial traffic. c) Favorable response by trade: customer satisfaction level = 3 (scale High = 4, Low = 1). 1.3 PDOs and PDO Level Results Indicators for the two Additional Financings 6. The project was extended through additional financings from the original section of 10.5km of provincial highway S3-B to a total length of 40.9km of the same road. In addition, two link roads were included adding 9.4km, bringing the total road length constructed under the project to 50.3 km (see 1.7 for details). The PDO and results indicators were not changed for the additional road lengths 1 The MDTF strategy has four pillars. The other three are: Pillar II Improving Governance and Service Delivery; Pillar III Supporting Livelihood and Creating Employment Opportunities, and Pillar IV Building Capacity and Institutional Strengthening. but the PDO was extended to more beneficiaries and the beneficiaries of the original project derived more benefits from the improvement of the additional road sections. 7. The design preparation for the additional road sections under AF-1 and AF-2 occurred during the completion phase of the original 10.5km section. Design surveys to establish baselines and targets for the new sections were carried out in 20132 under a single consultancy which measured and recorded data for the entire 40.9 km road. No distinctions were made between the individual sections. Therefore, no final actual values for PDO indicators 1 to 3 (VOCs, operating speeds and customer satisfaction) were ever established for the original 10.5 km section. The baselines and targets for the three combined sections became the baselines and targets for the project. This may have come about as a result of the Bank team not being able to visit the project site, and erratic communication with the Highway Authority because of the security situation (paragraph 16), or a misunderstanding of monitoring methodology and procedure. In any case, the ICR assessment was not able to identify any data relating specifically to the original 10.5km section other than the original baseline taken at appraisal. 1.4 Main Beneficiaries 8. The primary target groups are the road users and residents in the area served by the road, benefiting from reduced VOCs and travel time. Additional beneficiaries include goods and service providers and commercial transporters. 1.5 Original Components (as approved) 9. The original project was a 10.5km section of provincial highway S3-B. The original components included: Component 1 (US$ 6.417 million plus US$ 0.97 million (physical and price contingencies) = US$ 7.387 million): Infrastructure Rebuilding – civil works along the provincial highway S-3B (Chakdara - Madyan on the Right Bank of River Swat) comprising: a) Reconstruction and widening of about 10.5km of highway (7.3 meter wide asphalt concrete 2-lane single carriageway with 1.0 metre wide shoulders) and related structures (including reconstruction of a bridge) from Sharif Abad (km 30+424) to Kanju (km 40+924), and b) Associated relocation of utilities, land acquisition and resettlement. Component 2 (US$0.61 million): Project management including support for: a) Contract administration and construction supervision consultant services; b) Environmental and social safeguards consultant services, and c) Other Project management activities including procurement, financial management, audits, and other administrative tasks. 1.6 Any Components Revisions 10. The components and the technical specifications of the road were not revised but the project was extended with two additional financings to increase the length of road reconstructed (see Sections 1.3 and 1.7). Two bridges were constructed on the 40.9km of S3-B constructed. 2 Precise dates of study not available to the ICR team. 2 1.7 Other significant changes 11. The project ended up reconstructing almost five times the road length originally envisaged in the project proposal. Table 1 and the inset map provide details. The original project was to reconstruct a 10.5km section (Kanju to Sharif Abad) of provincial highway S3-B (Madyan to Chakdara). The project ultimately reconstructed 40.9km of the highway (Kanju to Chakdara) and a further 9.4km of link roads. The first 10.5km section and the extended length of 5km (Sharif Abad to Dadahara) funded from exchange rate depreciation and other savings) were completed on schedule. Subsequently, the project received additional financings 1 and 2 (AF-1 and AF-2). These additional financings enabled completion of the Kanju to Chakdara section of highway S3-B. In addition, link roads 1 and 2 were financed from AF-1 and AF-2 savings. The link roads extended the improvement of access and mobility to more people and improved access from road S3-B to Shamozai Bridge over Swat River. For the link roads, the specification of a 7.3 metre two lane carriageway was retained as far as possible but adjusted where necessary to eliminate the need to acquire assets and remove structures. The project closing date was extended from 30th June 2015 to 31st October 2015 to facilitate the completion of the link roads. Along with reconstruction of the Madyan to Kanju section of S3-B with USAID assistance, the project completed almost the whole Madyan to Chakdara road (about 88km). Table 1: Project Road Reconstruction Length and Funding Details Phases (1) Road sections Length Grant Road Project Completion (km) (US$ m) rebuilding management date (US$ m) (2) (US$ m) Phase I Kanju – Sharif Abad 10.5 8.0 7.4 0.6 Oct-13 Phase Ia (3) Sharif Abad – 5.0 Dadahara Phase II (AF-1) Dadahara – Shamozai 14.7 9.1 8.3 0.8 Jun-15 Phase III (AF-2) Shamozai – Chakdara 10.7 9.0 8.7 0.3 Jun-15 Link road - 1 Shamozai – Zarkhela 8.0 Oct-15 Link road - 2 Shamozai – Shamozai 1.4 Bridge Total 50.3 26.1 24.4 1.7 Of which S3-B 40.9 Notes: (1) Phases I, II and III are Packages I, II and III in the inset map. The link roads at Shamozai and are not shown on the inset map but are shown on the project map at the end of the document. (2) Includes physical and price contingencies of US$ 0.97m (rounded) for Phase 1. (3) The additional 5.0 km of road S3-B constructed during Phase I from savings (see paragraph 13). 3 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 12. The project made a rapid start. The Government of Khyber-Pakhtunkhwa (GoKP) and Pakhtunkhwa Highway Authority (PkHA) had placed high priority on the project. They had used their own resources to commission surveys and prepare designs for the rehabilitation the whole 88km of the Madyan to Chakdara road. Assistance was secured from USAID and the Government of Pakistan for the reconstruction of approximately 43km of S-3B from Madyan to Kanju. GoKP and PkHA sought assistance from the MDTF for the remaining sections. Pre-improvement road condition 13. The Bank and MDTF were swift in responding to requests for financing and the Emergency Project Paper (EPP) was comprehensive in including aspects to be addressed in a post-conflict context. The EPP was prepared and approved by the Bank in July 2011. The Project and Grant Agreement documents were signed on 23rd August 2011 and the project became effective on 17th October 2011. Project implementation formally commenced on 1st November, 2011. The exceptionally early mobilization and implementation of the works was made possible as the selected consultants and the contractor were working on a road project (USAID supported Madyan to Kanju section) in the vicinity of the project site. They were therefore familiar with the local conditions including the security situation and had plant and equipment nearby. The following table summarises the key points. Overall Key Points ☼- Information - Accomplishments X - Concerns Rating Satisfactory ☼ There was an urgent need for rapid response reconstruction. ☼ The Bank Task Team could not visit the project site and at times could not visit the city of Peshawar because of the security situation. ☼ PkHA was unfamiliar with Bank’s M&E, fiduciary and safeguards requirements.  The client had identified S3-B road as a high priority project.  PkHA had commissioned project survey and design using its own resources.  Bank and MDTF responded rapidly in preparing and approving Phase I and extending to Phases II and III.  USAID funded the remaining section of S3-B (Madyan to Kanju).  PkHA’s rapid response, with task team support, in setting up the PMU as specified, and the PMU team’s rapid learning about Bank fiduciary and safeguards requirements compensated for their initial lack of familiarity.  The bid winning contractor was able to mobilize quickly as equipment and plant were available from a nearby site. x The Bank’s systems to compensate for the obstacles to supervision were limited. 4 2.2 Implementation 14. The project’s physical works were implemented successfully in spite of potential security risks associated with a post-conflict situation. KP-ERRP successfully navigated a range of difficult and adverse issues during implementation. The serious security situation during much of the implementation period was a challenge for the contractor, the supervision consultant and the PMU staff. Security staff were required at the contractor’s and consultant’s facilities, labor A section of the completed road camps and locations of works. Maintaining contact with the police and the army was essential. The security situation affected the Bank Task Team’s (hereinafter referred to as the Task Team) ability to support and advise the Pakhtunkhwa Highway Authority on the supervision of works, compliance with safeguards, and monitoring and evaluation. The lack of familiarity of the Government of Khyber-Pakhtunkhwa and Pakhtunkhwa Highway Authority with Bank procedures and compliance requirements was also a concern. 15. In spite of the circumstances outlined above, each phase of the project was completed relatively quickly and within budget enabling reconstruction of additional road lengths. Factors contributing to the quick completion of the physical works within budget were: (a) the preparedness of the client; (b) efficient establishment of the Project Management Unit with PkHA staff; (c) highly committed and able leadership of the Project Management Unit (PMU); (d) familiarity of the contractor and the supervision consultant with the local conditions and availability of the contractor’s plant in the locality as noted in 2.1, and (e) quick decisions taken by the Task Team. The following table summarises key points on implementation. Overall Key Points ☼- Information - Accomplishments X - Concerns Rating Satisfactory ☼ Project implemented under potentially risky and challenging conditions associated with a post-conflict situation.  The initial readiness of PkHA with project design and setting up a PMU.  Clear and relatively simple project design with which the contractor and supervision consultant had experience.  Speedy preparation and approval of Phases II and III proposals  The “remote” and yet still responsive support and monitoring by the Bank Task Team. x “Remote” support for implementation (detailed in next section) should have been complemented by: o more formal reviews and record keeping by the Bank, and o more frequent use of independent technical inspections. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 16. An M&E plan and implementation arrangements were set up for: (a) the progress and quality of physical works; (b) compliance with environmental and social safeguards, and (c) fiduciary compliance. In the field, (a) and (b) were the responsibility of the supervision consultant under the supervision of the PMU. The PMU was responsible for the project’s fiduciary compliance with Bank and GoKP requirements. M&E of (a) is considered here with M&E of (b) and (c) addressed in Section 2.4. A distinction is made between supervision and monitoring of works undertaken by the supervision consultant and the PMU, and the oversight M&E by the Bank Task Team. 5 17. Field level day-to-day monitoring of implementation was adequate. The contractor had not been required to produce an explicit quality assurance plan. However, the contract documents and the inspection process followed by the supervision consultant were deemed sufficient by the PMU and the supervision consultant for quality assurance. The contract documents included: (a) the contract conditions; (b) technical specifications, and (c) the design specifications. The specifications set out testing requirements for the base and asphalt materials, thickness and compaction at specified intervals and maintenance of records of the tests. The single third party inspection during Phase I (see paragraph 23 and 2.4.2) found the quality satisfactory. The supervision consultant submitted monthly and quarterly reports covering physical progress and environmental and social safeguards compliance to the PkHA and the Bank. 18. The Bank Task Team’s oversight M&E of project progress and quality posed a challenge and was informal and supportive but with limited record keeping. The EPP recognized that the law and order situation imposed constraints on the Bank Task Team’s supervision of the project. Task Team members could not visit the project site until mid-2015 and in the early stages visits to Peshawar, the provincial capital and headquarter of PkHA, were restricted by UN Order. The EPP also set out the role of third party monitoring and stated that inclusion of innovative ITC based approaches in the monitoring system would be considered given the restrictions on direct BTT supervision. 19. The M&E plan set out in the EPP included six monthly supervision missions either at the Bank’s Islamabad Office or at the PkHA headquarters in Peshawar. 3 The Bank Task Team received the supervision consultant’s monthly and quarterly reports covering physical progress and safeguards compliance. It was reported that the frequency of supervision and M&E missions specified in the EPP (every six months) was comfortably exceeded and complemented by more informal meetings and phone conversations. The informal process of interaction between the Task Team Leader (TTL) and the Task Team members and the PMU and supervision consultant contributed to monitoring of physical progress and other aspects. The ISRs reported on progress and indicated satisfactory ratings on implementation progress, project management and progress towards achieving the PDO and “Moderate” risk. However, beyond the ISRs, there was no documentary evidence of these meetings nor on M&E implementation by the Bank. There are no Aides Memoire. There is no evidence of a Mid- term Review (MTR), which was a requirement of the Grant Agreement and would have been invaluable, considering the difficult environment and the two additional financings. The effectiveness of Bank oversight monitoring could have been reinforced and enhanced by better formal recording and other measures as specified in the EPP (see the next paragraph on the other measures). The key M&E related points are summarised in the table that follows. 20. The risks arising from the absence of effective physical supervision could have been better addressed. Third party monitoring of progress and quality of works was limited to one ad hoc inspection and audit during Phase I. The Task Team could have tried innovative approaches to mitigate such risks, as proposed in the EPP. These could have included the use of technology in the supervision of physical works. For instance, in Afghanistan smart phones and tablets have been used successfully on Bank projects in the supervision of roadworks in insurgency prone areas. Such measures could not only have helped in the physical supervision of roadworks, but also with the monitoring of safeguards compliance. 3 The participants were to include the Bank’s Task Team, PkHA officials, project manager of the civil works contractor, resident engineer of the construction supervision consultant and two representatives of the Project Affected Persons. 6 Overall Key Points ☼- Information - Accomplishments X - Concerns Rating Moderately ☼ M&E had to be managed within the constraints of the security situation posing risks satisfactory to effectiveness. Task teams were not permitted to visit the project site until very late in the project (mid-2015).  Competent monitoring by supervision consultant onsite and regular reporting of progress and safeguards compliance.  Commitment of the PkHA to a high priority project, a well-functioning PMU and frequent meetings between PMU staff and Task Team members. x Limited evidence of formal M&E by the Task Team. x No mid-term review. x Only one third-party monitoring event during Phase I. x Innovative M&E measures proposed in the EPP not used. 2.4 Safeguard and Fiduciary Compliance 2.4.1 Safeguard Compliance 21. Given the conditions, overall safeguards compliance worked reasonably well. This was largely because of clear guidelines included in the MDTF Environmental and Social Screening and Assessment Framework (ESSAF) which followed OP8.4, the commitment of PMU staff, and the support provided by the Task Team. However, this is not to say there were no problems at all. There were issues that arose during implementation, often as a result of the environmentalissue Anenvironmental An requiringattention issuerequiring attention necessity of “remote” monitoring by the Task Team and lack of sufficient alternative monitoring mechanisms (third-party and ICT-based). The key safeguards compliance points are summarised in the following table below and elaborated in the following paragraphs. Overall Key Points ☼- Information - Accomplishments X - Concerns Rating Environmental Moderately ☼ OP 8.4 guided the ESSAF for the MDTF portfolio which as a whole safeguards satisfactory was placed in environmental impact category “B”.  Disclosure requirements had been fulfilled.  Environmental Management Plan (EMP) prepared as a part of the Environmental Impact Assessment.  Supervision consultants were contractually bound to implement EMP.  Independent monitoring and third party validation consultants engaged.  Reporting (quarterly) by client was regular.  The Bank environmental specialist on the Task Team was in frequent contact with the client and supervision consultant to provide guidance.  Bank environmental team arranged training for MDTF staff  Dedicated PMU Deputy Director (Environmental & Resettlement) or DD (E & R). x The consultant could not station full-time staff responsible for environmental compliance at the project site. 7 Overall Key Points ☼- Information - Accomplishments X - Concerns Rating x The security situation did not allow Bank staff to visit work sites until late in the project. More regular project staff training was needed. x There were some deficiencies in environmental compliance. Social Satisfactory  Efforts made to minimize property acquisition. Safeguards  Virtually all persons who lost property were compensated.  Well functioning grievance redressal process.  Work sites were visited by a Bank appointed consultant.  Independent monitoring and third party validation consultants engaged. x The Deputy Director (Environmental & Resettlement) was withdrawn by the PkHA from the PMU for five months. x Delays in compensation payment at times. x Two persons filed complaints to a court, decisions still pending. x Client’s social safeguards capacity was low. Environmental safeguards 22. In spite of the difficult context, the environmental safeguards compliance was relatively satisfactory. The MDTF ESSAF Environmental awareness training prepared in accordance with OP 8.0 for emergency operations, was applicable for the project. EIAs (Environmental Impact Assessments) / EMPs (Environmental Management Plans) were prepared by the supervision consultant and submitted to the Bank by PkHA for each Phase and broadly complied with. The Task Team environmental specialist conducted training for the relevant staff of the contractor, supervision consultant and the PMU. 23. The project is built upon the previous route using the existing right of way as far as possible to minimize environmental and social sensitivity issues. Hence it falls in the Bank’s Category B for environmental impact and under schedule II of the Pakistan Environmental Protection Act (PEPA) 1997. 24. The main potentially negative environmental impacts were related to the construction stage. They include air quality, erosion and sediment control, noise, rehabilitation and landscaping plans, surface and ground water quality management and public and occupational health and safety aspects. They were considered to be addressable by sound and well implemented mitigation measures. The Deputy Director (Environmental & Resettlement), DD (E&R), conducted frequent monitoring visits and used checklists based on the EMP to assess compliance. The PkHA engaged a monitor independent of the supervision consultant and an auditor to provide further checks and controls, though the monitor and auditor cannot be considered fully independent because of their contractual relationship with the client. 25. There were two additional issues which affected environmental safeguard compliance . The first was that the supervision consultant’s team member responsible for environmental safeguards was not based fulltime on the site. The second was the inability of the Task Team environmental specialist 8 to visit the site until mid-2015. The visit identified gaps in EMP implementation and a number of issues requiring attention, including the need for higher awareness of environmental and safety issues on site, compliance with safety requirements and conditions in workers’ living conditions on site. The evidence from the Bank environmental specialists’ visit indicates the importance of Bank oversight on compliance with environmental safeguards which could have been better provided earlier in the project with the use of ITC based approaches when site visits were not possible. Social safeguards 26. The Pakhtunkhwa Highway Authority’s and supervision consultant’s plan and practice with regard to social safeguards were compliant with the MDTF Environmental and Social Screening and Assessment Framework (ESSAF), Government of Pakistan and Bank requirements. The main aspects to be addressed were the minimization of loss of agricultural land and other assets in the rural areas and the loss of land and structures in the urban areas. There were some issues related to existing structures in built up-areas which were addressed by limiting the right of way to the minimum required for the design. 27. Grant funds were used for providing financial compensation for resettlement and loss of property arising from the acquisition of land for road construction. Such funds cannot normally be used for these purposes. Approval of the RVP was sought and given to provide financial compensation from the Grant funds for the loss of assets and lost production (loss of crops and trees). Since the road largely followed the existing alignment, the amounts of land acquisition and loss of assets were low and the compensation payments below the provisions made in the project papers leading to savings which were used to construct additional road lengths (see Section 1.7). For Phase I, the project paper estimated that 498 persons would be directly affected, about 3.75 acres of land would have to be acquired, for another 1.75 acres where farmers had encroached on the right of way with crops, compensation would have to be paid for crop losses, 1000 trees would be lost, and 158 structures would be affected. The actual number of persons affected and losses requiring compensation were lower than estimated (114 persons affected, 1.6 acres of land in all either acquired or reclaimed with compensation for lost crops and loss of 74 trees and 24 structures. 28. In Phase II there were 88 affected persons, crop losses over 4.5 acres and 731 trees and 23 structures were lost. In Phase III there were 12 affected persons with very small losses overall. In addition, payments were made in all phases to the utility companies to move electricity and telephone pylons and poles. 29. Effective efforts were made to identify the parties to be compensated with 98 per cent of affected parties compensated to date. The remainder are absent and have not responded to efforts to reach them. The grievance redressal mechanism, operated through Grievance Redressal Committees constituted by affected persons, has functioned well in dealing with disputes on land ownership, compensation and the delays in payment. Digital recording of the assets and land before acquisition and construction could have reduced disputes. 30. PkHA engaged auditors to review the procedures followed and the outcome for all three phases of the project. The auditor’s reports confirmed that a high degree of compliance was achieved and recommended the setting up of an environmental and resettlement unit within PkHA to manage these aspects on all its projects. 31. One issue which affected the social and environmental safeguard compliance adversely was the withdrawal by PkHA of the DD (E&R) from the PMU for five months. During this period there were delays in dealing with redressal issues and payment of compensation to affected persons. GoKP appears reluctant to devote staff and resources to social and environmental compliance. This reluctance is a barrier to strengthening the PkHA capacities in these areas with wider and longer term benefits of 9 reduced environmental impacts of road investment during and after the works and improved relations with citizens. 2.4.2 Fiduciary Compliance Procurement 32. Procurement compliance was sound, especially given the circumstances of post-conflict reconstruction. National Competitive Bidding (NCB) for Phase I was followed by well negotiated Phases II and III contracts to combine cost effectiveness with timely and effective implementation. Procurement of the civil works contractor and the construction supervisor for Phase I was by NCB as specified in the EPP. There were three bids: one from a contractor based in Peshawar, and at the time, working on part of the Madyan to Kanju section of S3-B; and two from contractors from outside Khyber Pakhtunkhwa. The PkHA Bid Evaluation Committee recommended acceptance of the Peshawar-based contractor’s bid as technically competent and lowest cost. The recommendation was reviewed by the Task Team, found to be acceptable and the no objection letter was issued on behalf of the Bank. 33. For the reconstruction of additional lengths under Phases II and III and the link roads, the contracts were extended by variation orders to the original contract. The variations were in accordance with Annex-D of BP-11 with technical endorsement of the Task Team based on the information provided by the PMU. A total of eight variation orders, including the two additional financings and the two link roads were ratified. The contract negotiations for the variations appeared to be satisfactory4 and the whole contract was completed within a reasonable period. The advantages of extending the contracts were that: (a) the contractor and supervision consultant had demonstrated their capacities to implement the project and were familiar with the local conditions, and (b) the contractor’s equipment and plant were in place. The bidding process and the preparatory work, familiarization with the project, local conditions and operational procedures compliant with Bank practice for another contractor and consultant could have added several months to the project. Local civil works and supervisory capacity were also considered to be limited. 34. There were several risks associated with the substantial extensions of contracts without NCB. They included: (a) single suppliers (contractor for public works and supervision consultant) quoting higher costs and inferior quality specification of works and services, and (b) poor quality implementation because of complacency that there are no alternative competent providers. The mitigating measures against the risks of extending contracts were: (a) an ad hoc third party audit of Phase 1 works and costs in response to concerns about quality and cost raised by an EU field mission which concluded that the costs were reasonable and the quality satisfactory and in accordance with the applicable technical specifications, and (b) basing the Phases II and III contracts on the 2011 National Highway Authority Composite Schedule of Rates (CSR). As a consequence there would have been savings in Phases II and III contracts because of exclusion of upward revision of CSRs between September 2011 and end of 2013 when the Phase II and III contracts were agreed. The table below summarises the key points on procurement. 4 Paragraphs 37 to 39 provide further explanation. 10 Rating Key Points ☼- Information - Accomplishments X – Concerns Satisfactory ☼ The Bank issued “no-objections” to the initial contract award and the subsequent variations.  Phase I contract was competitively procured using NCB.  The contactor demonstrated capacity to perform in the particular context.  Project extensions awarded to the Phase I contractor based on 2011 CSRs reduced the time required to complete the project while controlling costs.  Third-party audit had found that Phase 1 costs were reasonable and the quality of works was satisfactory x There were possible risks associated with large extensions under additional financing without competitive bidding. Financial Management (FM) 35. The financial management system worked well in complying with the Bank’s requirement for financial reporting and accounting for proper expenditure of Project funds in spite of the client’s initial lack of familiarity with Bank procedures. PMU submitted the required financial reports (Interim Financial Reports, IFRs) and forecasts regularly. Funds were disbursed based on the reports and cash forecasts for the next two quarters. These arrangements enabled timely implementation of the project. 36. The Bank’s FM Team regularly undertook FM supervision, provided training and support to the project FM team and found the client’s FM performance to be satisfactory throughout the project. The supervision of project FM by the Bank involved the Bank FM Team visiting the client’s office and reviewing FM documents with the project FM team. Six such visits linked to FM supervision were undertaken by the Bank Team (the last one in August 2015). The Bank FM Supervision Report of August 2015 identified no issues of concern and rated the FM performance as “satisfactory”. 37. The financial closure of the project and grant have been effectively completed. With the project/grant closure imminent on October 31, 2015, the Bank FM Team: (a) conducted Financial Management Training in May, 2015 for the project FM team on project closure procedures and requirements, and (b) reviewed during the August 2015 supervision matters which needed to be resolved before project/grant closure. The objective of the training was to build the capacity of project staff to efficiently complete the project closing and ensure: (a) timely refund to the Bank of unspent amounts in the assignment account, and (b) timely submission of the final audit reports. The Bank’s FM Team reported in August 2015 that there was an unspent balance of US$276,780 because of exchange rate fluctuations. Refund of this amount had been requested and was expected shortly following the formality of approval by the GOP’s Economic Affairs Division (EAD). 38. Internal and external auditing functions were undertaken by the PkHA internal auditors and the Auditor General’s office, respectively. The internal audit conducted by a consultant (KPMG) raised some issues related to procedures, timely payments, payment from correct budget lines and tax compliance. There are some other issues which either require further explanation (e.g. explanation to Government of Khyber-Pakhtunkhwa (GoKP) Audit Committee that cost escalations are normal parts of construction contracts) or exceptions (e.g. accounting for additional works out of project savings which are not shown in the Annual Development Plan). Most of the issues required further explanation and accommodating exceptions to internal government procedure. None of them were of concern for compliance with the Bank’s requirements. 11 Rating Key Points ☼- Information - Accomplishments X - Concerns Satisfactory ☼ Internal and external auditing functions were conducted by the client’s internal auditor and the Auditor General’s Office.  Reporting from the PMU to the Bank has been timely and demonstrates proper use of funds.  Disbursements by the Bank have been timely and facilitated implementation.  There has been credible internal PkHA / GoKP scrutiny of use of funds.  Bank FM Team undertook regular supervision activities. x The internal audit process has raised some issues which PkHA needs to address. 2.5 Post-completion Operation/Next Phase 39. There are serious concerns on the sustainability of the investment. Phase I was completed in October 2013 (well over two years ago) but no maintenance has yet been undertaken. Pakhtunkhwa Highway Authority (PkHA) insisted that the road was in excellent condition and, therefore, no maintenance would be required for a number of years. PkHA prepares an annual road maintenance plan. Nonetheless, the annual maintenance allocation is less than one-third of the estimated requirement. The plan does not have routine maintenance provisions and there is no routine maintenance system in place. A “forced” maintenance regime, under which the available budget is allocated for repair or rehabilitation of the most deteriorated roads, is in place. While this practice is not uncommon in the region, including Pakistan, it raises questions about the continuing efficacy and sustainability of investments. PkHA is aware of the issues it faces because of limited maintenance resources and of the value of a roads asset management system. However, the Government of Khyber- Pakhtunkhwa (GoKP) does not consider such a system a priority. Rating Key Points ☼- Information - Accomplishments X - Concerns Unsatisfactory  PkHA is aware of the need for maintenance and a roads asset management strategy.  PkHA has a maintenance strategy within the available resources. x Current resources allocated to PkHA for maintenance are inadequate. x GoKP does not see provision of adequate resources for maintenance and a road assets management system as priorities. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation (Rating: High) (a) Relevance of objectives (Rating: High) 40. The PDO to enable the population along the Project corridor to benefit from year-round improved access and mobility is highly relevant in the post-conflict context and in line with GOP’s Damage and Needs Assessment (DNA) and Post Crisis Needs Assessment (PCNA) (see paragraphs 2 and 69). The reconstruction, in particular after the economic and social disruptions during the insurgency and subsequent military action, is relevant for improving income levels from increased sales of farm produce and employment opportunities and improved access to health and education facilities. This assessment is for the whole 40.9km section of highway S3-B reconstructed in 3 phases financed by the initial grant and two additional financings (see 1.7 for details). The 9.4km of link roads reconstructed from project cost savings extended the improvement of access and mobility to more people and improved access from road S3-B to Shamozai Bridge over Swat River. The sections of road S3-B reconstructed were in a badly deteriorated condition preventing year round access and restricting mobility prior to the project for two reasons: (a) lack of maintenance over a number of years because the area was controlled by insurgents preventing highway authority access, and (b) extensive damage caused by the military action. 12 (b) Relevance of design (Rating: High) 41. The technical design of the project was standard and appropriate for the function and is still relevant. The reconstruction widened the road from a mostly 3.5 metre wide single lane to a 7.3 metre wide 2-lane single carriageway asphalt road with appropriate side and cross drainage structures including two bridges. In addition to the widening, the alignment was modified in places to reduce sharp bends and steep gradients. The objectives in widening the road and the other improvements were to better accommodate higher speeds with safety and future growth in traffic. 42. Lessons from projects in other post-conflict and fragile situations have been applied in designing the project. These include incorporating a simple design, limiting the scope of the project to a single road within local implementation capacity, and selecting a project which has high priority for the recipient and obvious benefits for the community to be served. In addition, the grant financing was structured to align incentives and outcomes to ensure effective implementation and safeguard compliance. These attributes have facilitated the quick implementation of the civil works by a local contractor. Key points on relevance, design and implementation are presented in the following table. The “High” rating for relevance overall follows from the high ratings relevance of objectives and design. Rating Key Points ☼- Information - Accomplishments X - Concerns High  Clear objective focused on improving access and mobility.  High GoKP priority and commitment and urgent need for the project to restore a key damaged asset.  Standard technical design capable of being implemented by local contractors and supervisors.  Reconstruction to a higher specification to accommodate higher speeds with safety and future traffic growth. 3.2 Achievement of Project Development Objective (Efficacy) (Rating: Substantial) 43. The development objective of improving access and mobility for the population served by this 40.9 km section of S3-B has been achieved.5 In addition, the incorporation of the 9.4km of link roads has extended the improvement to more beneficiaries. Along with reconstruction of the Madyan to Kanju section of S3-B with USAID assistance, the project has completed almost the whole Madyan to Chakdara road (about 88km). 44. Assessment of the improvement of access and mobility are based on a number the key items. They include PDO indicators utilizing pre-construction estimates of VOCs and vehicle speed, as well as post-construction surveys and estimates of these measures, the Customer Satisfaction Survey (CSS), effectively the beneficiary survey, and evidence from the client’s Project Completion Report. The targets for PDO indicator 1 were 34%, 26% and 5% decreases in VOCs for 2- and 3-axle trucks and passenger buses, respectively. The target for PDO indicator 2 was a 50% increase in operating speeds of commercial vehicles (2-axle and 3-axle trucks and passenger buses). These percent reduction targets were set for the initial10.5 km section constructed under the initial grant and were retained for Additional Financings (AF) 1 and 2. 45. There was an issue regarding the original baseline and target for the first road section for PDO indicators 1 and 2. The absolute values of pre-construction VOCs and speed and their estimated post-construction values for the original 10.5km project cannot be used for the whole 40.9km road section. Further comparable post-construction values are not available 5 As noted in 3.1, the achievement of the PDO considered here is for the whole 40.9 km section of highway S3-B reconstructed in 3 phases as described in 1.7. 13 separately for this 10.5km section. However, evidence to assess achievement for the two PDO indicators for the whole 40.9km section of S3-B is available. The project supervision consultant assessed baseline VOCs and speeds for the whole 40.9km section for the appraisal of road sections to be reconstructed under AFs 1 and 2 in 2013.6 In addition to this, in order to provide more conclusive evidence and clarification on this issue, the ICR team requested the client to commission a post-construction VOC, speed and traffic volume study for the whole reconstructed 40.9km section. The study was conducted in February 2016 by the supervision consultant on a basis comparable with the study carried out in 2013.7 Comparison of the post- construction VOCs and speed with the 2013 baseline show that the first two PDO indicator targets (reduction in VOCs for commercial vehicles and increase in vehicle speed 8) have been overachieved (see Table 2). Table 2: Achievement of PDO Indicators 1 & 2 PDO results indicator 1 (decrease in VOCs, %) 2-axle trucks 3-axle trucks Passenger buses Reduction Target for VOC 28% 24% 9% Achievement (%) 34% 34% 23% PDO results indicator 2 (50% increase in operating speeds of commercial traffic) Achievement (% increase in operating speed) 83% 67% 63% 46. Regarding PDO indicator 3, no definitive conclusion could be drawn from the available information on customer satisfaction on a 1 to 4 scale. The Customer Satisfaction Survey 9 is in effect the beneficiary satisfaction study. The term customer satisfaction is retained here because the PDO Indicator 3 and the title of the study refer to customer satisfaction. The study was conducted during September-October 2015 and included a sample survey of 211 households along the road and two focus groups of road users. Although the CSS shows that on balance local users have a positive view of the road post-construction the study does not provide evidence which can be used to assess road users’ satisfaction levels which can be compared with the PDO results indicator 3 target.10 Nevertheless, in the focus group discussions conducted as a part of the customer satisfaction study, participants reported significant savings on transporting fruit and other local farm produce. Other economic impacts such as more transport service providers, lower transport costs, more new roadside businesses and increase in land prices and rents were also identified. The client’s Completion Report also provided similar evidence from local consultations. 47. There is concern about continuing efficacy in delivering the project objective over the design life of the road. This is because of lack of an effective maintenance strategy and the nature of traffic now plying on the improved road. Efficacy has been achieved at project completion but an adequate maintenance regime required for the continuation of efficacy during the design life of the project is not in place because of insufficient provision of resources. The importance of maintenance for continuing efficacy is all the more important because of the projected traffic growth of 5 per cent per 6 Precise date not known to the ICR team. 7 VOC savings were adjusted for inflation for comparability. 8 Since the target values are closely related to the assumptions used in the financial and economic appraisal, further details are provided in 3.3 below and in Annex 3. 9 The term “survey” is used in the title of the report. The broader term study is also used because it included a survey and focus group discussions. 10 See 3.6 below for further explanation. 14 year to 2032 and the post-construction traffic count showing significant presence of heavy trucks. The key points are summarised in the table below. Rating Key Points ☼- Information - Accomplishments X - Concerns Moderately ☼ The main sources of evidence are the pre-phases II and III construction and post- satisfactory construction studies of traffic volume and speed and changes in VOCs and the Customer Satisfaction Study.  Restoration of service provided by a provincial highway to above pre-conflict levels.  The project has achieved two PDO indicator targets.  Customer satisfaction with the improvement in access and mobility and resulting benefits. x There is concern about continuing efficacy over the longer term because of the absence of a maintenance regime. x The CSS evidence is not aligned with the frame of the third PDO and is therefore not conclusive. 3.3 Efficiency (Rating: Substantial) 48. The efficiency criteria of achieving the Economic Rates of Return (EIRRs) estimated at appraisals of the three phases and cost efficiency have been met. This assessment is based on the following observations:11 (a) ex-post traffic volumes are close to the projections at appraisal; (b) the VOC savings and traffic speed improvements exceed the results indicator targets and assumptions at appraisal, and (c) the construction cost per km is 19 per cent lower than assumed at initial appraisal. On costs, Pre-construction deteriorated road (lack NCB for Phase I and using the 2011 CSRs for of maintenance?) Phases II and III addressed cost effectiveness. Furthermore, there have been cost savings during project implementation (exchange rate movements, lower compensation for affected persons, a cost de-escalation clause in the contract and efficiency savings) resulting in 50.3km length constructed instead of the planned 40.9km from the budget of US$ 26.1 million. The implied reduction in cost per km is 18.7 per cent. The rating is “Substantial” rather than “High” because the maintenance provision is insufficient to ensure preservation of the road in good condition to continue delivering the benefits throughout its life to achieve the estimated EIRR (see paragraphs 44 and 52). 11 Assessment based on evidence from the post-construction study of traffic volumes and speeds and VOCs and project cost information. See Annex 3 for further details. 15 Rating Key Points ☼- Information - Accomplishments X - Concerns Substantial ☼ The key parameters determining project benefits are traffic volumes, VOC savings, traffic speed improvements and constructions costs.  Project has managed to control construction costs and per km cost was lower than the estimate made during appraisal.  Improvements in VOCs and traffic speed exceed assumptions at appraisal. x Risks of lower future benefits and efficiency remain because of deterioration resulting from lack of maintenance. 3.4 Justification of Overall Outcome (Rating: Moderately Satisfactory) 49. There are some accomplishments as well as concerns which on balance justify a “moderately satisfactory” rating. While the individual ratings for relevance, efficacy and efficiency would generally average to an overall rating of Satisfactory, this assessment has factored in other concerns. The key accomplishments are the achievements as measured by the first two PDO indicators, as well as the relevance of the objectives, and efficiency. The concerns include; (a) most importantly, the risks related to lower than projected future benefits because of potential rapid deterioration of the road because of current inadequate maintenance arrangements; (b) to a lesser extent, the inconclusive evidence on achievement of PDO indicator 3, and (c) a small level of ambiguity regarding the baselines and targets for the first segment financed under the original grant. A rating of “moderately unsatisfactory” was considered for efficacy, because of the inadequate maintenance arrangements which have the potential of jeopardising continuing efficacy and the achievement over the project life of the PDO “to enable the population along the Project corridor to benefit from year round improved access and mobility through reconstruction of priority damaged roads and bridges in the conflict hit areas”. Efficiency could also be at risk since the estimated EIRR is based on the quantifiable benefits of the project based on VOC savings and improved traffic speeds (PDO indicators 1 and 2 respectively) being maintained during the life of the project. Hence the importance of maintenance is implicit in the PDO. However, the option of the “moderately unsatisfactory” rating was rejected because the maintenance requirement is not explicitly included in the project papers, the works were successfully carried out, and the Pakhtunkhwa Highway Authority has demonstrated evidence of awareness of the need to improve maintenance. Further, on PDO indicator 3, the deficiency is linked to the lack of suitable data collected by the Customer Satisfaction Survey rather than lack of beneficiaries’ satisfaction with the reconstructed road. Rating Key Points ☼- Information - Accomplishments X – Concerns Moderately  The PDO is highly relevant in the post-conflict context and design and satisfactory implementation were effective and appropriate.  Two PDO indicator targets have been overachieved and efficiency is substantive. x No assessment could be made of achievement of a PDO indicator (customer satisfaction). x Risks of lower future efficacy remain from the potential rapid deterioration from lack of maintenance. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 50. The lower transport costs and improved access are likely to have a positive poverty reduction impact. The Customer Satisfaction Survey Report (CSSR) noted that the general standard of living in the area served by the road was “slightly above the poverty line” and reliant on farming and remittances. While the project is not specifically poverty impact oriented, evidence from studies around the world indicate that lowering transport costs and improving access for disadvantaged communities 16 contributes to poverty reduction. The project does not overtly address gender aspects but the CSSR shows that access to health facilities, markets and schools are the most important non-economic reasons for using the road and these are of particular value for women. (b) Institutional Change/Strengthening 51. Government of Khyber-Pakhtunkhwa and the Pakhtunkhwa Highway Authority have demonstrated that they can implement substantial road projects effectively and can work with external agencies in managing such projects. The project has improved the client’s capacities and confidence. There are more specific lessons from the project (e.g. establishment of an environmental and resettlement unit in PkHA) which if adopted would contribute to institutional strengthening. (c) Other Unintended Outcomes and Impacts 52. The project has contributed to two pillars of the MDTF financing strategy in addition to Pillar I, the intended primary focus. The project was designed to support Pillar I of the MDTF strategy, restoration of damaged infrastructure and disrupted services. In addition, the project contributes to Pillar II (improving governance and service delivery) and Pillar IV (institutional change/strengthening referred to in the previous paragraph). By demonstrating the ability to deliver the project and compliance with social, environmental and fiduciary safeguards, the project contributes to Pillar II. 53. The project may have increased the risk of accidents from increased traffic, aggressive driving and over speeding. Risks associated with Although no empirical evidence to support this statement is the location of an available, the Customer Satisfaction Survey (CSS) responses have identified this risk as one of the unintended consequences of the electricity pylon reconstructed road. CSS respondents indicated that the risk was exacerbated by inadequate signage and positioning of signs and electricity and telephone poles on the road shoulders in built up areas. The danger to pedestrians was perceived to be substantially higher than that for motorists. According to the respondents road safety could be improved by better installation of road signs, controlling over speeding and creating awareness of traffic rules and road safety. 3.6 Summary of Findings of Beneficiary Survey (Customer Satisfaction survey) and/or Stakeholder Workshops 54. This section reports on the customer satisfaction survey, which was in effect intended to be the beneficiary survey, and the stakeholder workshops organised during the ICR team’s mission to collect information and gain insights on project operations and achievements. (a) Beneficiary (or customer satisfaction) survey 55. The implementation consultants, on behalf of the Pakhtunkhwa Highway Authority (PkHA), conducted a customer satisfaction study during September-October 2015, after the completion of construction work on the Kanju to Chakdara section of highway S3-B. A sample of 211 households along the road section was surveyed to obtain information on their economic and non- economic uses of the road and their level of satisfaction with selected features related to the road. Two focus groups provided more insights and depth. 56. The most important non-economic uses of the road are access to health facilities (major purpose for 32 per cent), to markets (major purpose for 30 per cent) and to schools (major purpose for 22 per cent). These are the findings from the survey. The economic reasons for road use identified by the survey were commuting and migration. Although there is heavy reliance on farming 17 including fruit growing in the area, the survey did not inquire about the value of the road for this economic activity. However, in the focus group discussions (FGDs) participants reported significant savings on transporting fruit and other farm produce. Other economic impacts such as more transport service providers, lower transport costs, more new roadside businesses and increase in land prices and rents were also identified. 57. The Customer Satisfaction survey inquired about satisfaction levels for 10 amenities. They are: tea/coffee shops, public call office, public toilets, petrol pumps, service stations, patrolling, ambulance service, helpline, health centers and other facilities. The overall satisfaction level with these amenities is high (77 per cent rating in the 3 or higher category on a 1 to 4 rating scale). However, the satisfaction ratings for these amenities do not relate to PDO indicator 3 which refers to satisfaction with “benefit from year round improved access and mobility”. (b) The ICR team’s fact finding and consultation workshops12 58. The ICR team was not permitted to visit PkHA headquarters in Peshawar and the road site because of the security situation. Two fact finding and consultation workshops were held at the Bank’s Islamabad Office. The workshops were guided by a framework for obtaining the information for the ICR.13 The first workshop was a fact finding and review of experience meeting with the PMU staff and representatives of the supervision consultant. The second was a brainstorming session. Participants included the MDTF staff, Task Team members, 14 PMU staff, representatives of the supervision consultant and the construction contractor. The aim was for the ICR team to report their interim findings and the sources used as a basis for reflecting on project performance and wider lessons to be learnt and identifying gaps in the information required for completing the ICR. 59. The project completion report presented by the PkHA at the first workshop was considered to be the client’s ICR. The report was particularly helpful in clarifying details of procurement procedures, implementation arrangements and safeguards compliance. The commitment and competence of the PMU staff and the close working relationship with the supervision consultant were evident. The PMU staff also emphasised the flexibility, support and guidance of the Task Team as a key factor in the successful expansion and completion of the project. 4. Assessment of Risk to Development Outcome (Rating: Significant) 60. There is a significant risk that project benefits may not be sustained in the longer-term. The desired outcome has been achieved in the short-term. The rehabilitated road provides improved access and mobility to residents living in the road corridor. However, the risk of reduced benefit streams in the longer-term, associated with effectively no maintenance, high traffic growth and the presence of heavy trucks after the improvement of the road is substantial. Rating Key Points ☼- Information - Accomplishments X - Concerns Significant  Desired outcome achieved in the short-term. x The benefit streams may not be sustained in the longer-term due to inadequate maintenance and/or too heavy traffic. 12 These were not stakeholders’ workshops in the strict sense because the beneficiaries from the reconstructed road, the key stakeholders, were not represented at the workshops. 13 See Annex 5 for details. 14 All members were invited. 18 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (Rating: Satisfactory) 61. The Bank team has shown responsiveness and flexibility and provided support that enabled the rapid initiation of the project. Furthermore, this was done in the context of a client unfamiliar with Bank procedures and requirements. The Emergency Project Paper (EPP) was comprehensive and addressed important aspects to be included in a post-conflict context. The table below shows the key points linked to Bank performance. Rating Key Points ☼- Information - Accomplishments X - Concerns Satisfactory ☼ Need for rapid response and sensitivity to the post-conflict fragile situation.  Responsive, flexible and supportive during project preparation and initiation.  Comprehensive EPP as a basis for implementation. (b) Quality of Supervision (including of fiduciary and safeguards policies) (Rating: Moderately satisfactory) 62. The Bank Task Team’s supervision was effective in the circumstances though more could have been done to adapt it to the fragile security situation. There were deficiencies in records of monitoring and evaluation of physical progress and performance. Innovative ICT based monitoring methods specified in the EPP could have been used along with formal independent monitoring of physical works. The Mid-term Review (MTR), one of the Grant Agreement requirements, was not conducted and there were no Aides Memoire as records of decision and action points. Nevertheless, the oversight supervision and guidance on progress, safeguards and fiduciary aspects enabled efficient implementation and compliance. Rating Key Points ☼- Information - Accomplishments X - Concerns Moderately ☼ Supervision approaches had to be adapted to the post-conflict fragile situation. satisfactory  Supportive approach to provide guidance on implementation and compliance.  Maintenance of a sound working relationship with the client. x Lacking formal records of communications and monitoring. x Lacking third party and innovative monitoring approaches. x No MTR and no Aides Memoires. (c) Justification of Rating for Overall Bank Performance (Rating: Moderately satisfactory) 63. Taking account of the ratings on ensuring quality at entry and supervision and the related reasoning, a rating of “moderately satisfactory” is justified (see the following table). In spite of some shortcomings in supervision and related formal records which introduced some potential risks, there is evidence that the Task Team Leader (TTL) and the Task Team managed supervision well to ensure implementation and safeguards compliance of sufficient quality. Rating Key Points ☼- Information - Accomplishments X - Concerns Moderately  Satisfactory rating on quality at entry. satisfactory x Moderately satisfactory rating on quality of supervision 5.2 Borrower Performance (a) Government Performance (Rating: Moderately satisfactory) 64. Government of Pakistan (GOP) and Government of Khyber-Pakhtunkhwa (GoKP) had a high commitment to the MDTF and this project. The high commitment was justified by the acute need to restore a seriously damaged asset in the aftermath of the insurgency and military operations. 19 GOP collaborated with multilateral and bilateral agencies to prepare a Damage Needs Assessment (DNA) and a Post-conflict Needs Assessment (PCNA). GoKP supported Pakhtunkhwa Highway Authority’s preparation for the project and setting up a PMU with the required establishment. A concern is the low priority given to the preservation of the assets created by the project. Rating Key Points ☼- Information - Accomplishments X - Concerns Moderately  High commitment and resources for project preparation and establishment of PMU. satisfactory x Limited commitment to maintenance and adoption of lessons on safeguards. (b) Implementing Agency or Agencies Performance (Rating: Moderately satisfactory) 65. Pakhtunkhwa Highway Authority’s initiatives, commissioning a survey from own resources in advance, responding rapidly to Bank project preparation requirements and setting up a PMU with capable staff are positives. The PMU has worked competently leading to timely implementation and cost savings. It has also followed Bank guidance on safeguards. PkHA is aware of the need for more resources for an adequate maintenance regime but is constrained by resource limitations. This raises a major concern about long term sustainability of the investment. Furthermore, the authority appears to be reluctant to adopt lessons from the project on institutionalising environmental and social safeguard practices. Rating Key Points ☼- Information - Accomplishments X - Concerns Moderately  High commitment and effective management driven by the former PMU director satisfactory and continued by the PkHA and PMU management.  Good performance on compliance with safeguards. x Limited commitment to maintenance and adoption of lessons on safeguards. (c) Justification of Rating for Overall Borrower Performance 66. The combination of moderately satisfactory ratings for government and agencies performance leads to an overall moderately satisfactory borrower performance rating. Overall both the governments (GOP and GoKP) and the highway authority have performed satisfactorily on preparation and enabling implementation. The concerns which lead to the overall moderately satisfactory rating are the risk to sustainability of the benefits from the asset created and institutional strengthening. Rating Key Points ☼- Information - Accomplishments X - Concerns Moderately ☼ Balance between Government and implementation agency performances. satisfactory  Strong commitment and relatively strong performance on implementation and safeguards compliance. x Concern about resources and commitments to sustainability and social and environmental safeguards. 6. Lessons Learned 67. Lesson 1: With a high level of commitment of the client it is possible to design and implement a project quickly in a post-conflict context, with reasonable quality and fulfilling safeguard and fiduciary responsibilities. The Task Team also ably supported the client in fulfilling its responsibilities through quick decision making and informing the client about the decisions informally. 68. Lesson 2: It is possible to implement competitive bidding and achieve transparency in the procurement process even in a post-conflict situation. Pakhtunkhwa Highway Authority procured the initial contract using a competitive process. Although the subsequent variation orders may be questioned on the grounds of lack of transparency and procurement related fiduciary risk, there were two mitigating measures: (a) a third party audit of Phase I which concluded that the costs were 20 reasonable and the outputs were of satisfactory quality; (b) Phases II and III variations were based on the approved Schedule of Rates. 69. Lesson 3: A more innovative project monitoring mechanism is warranted in a situation which restricts the adoption of conventional monitoring. One of the shortcomings of the KP-ERRP monitoring by the Bank was the inability of the Task Team to put in place monitoring mechanisms to substitute for physical supervision of works by the Team. Examples exist in a number of countries that have successfully used ICT methods in the supervision of roadworks in insurgency prone areas. 70. Lesson 4: Informal coordination and review arrangements are not a substitute for formal reviews and appropriate record keeping. There were shortcomings in the Task Team’s formal review processes, absence of a formal structure and records including absence of a Mid-term Review. 71. Lesson 5: Low priority given to safeguards and maintenance is a common problem. A lesson is to include capacity building and institutionalising commitment to safeguards and maintenance into projects. The project experience demonstrates reluctance on the part of the Pakhtunkhwa Highway Authority and the Government of Khyber-Pakhtunkhwa to devote staff and resources to social and environmental compliance beyond fulfilling Bank and other project requirements. Lack of resources and institutional resistance also explain the absence of a sound maintenance strategy. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 72. The ICR team had extensive discussions on a range of topics with PMU staff representing Pakhtunkhwa Highway Authority. The following observations based on these discussions are of particular note: a. The PMU team gave a high level of credit for the success of the project to the TTL and the Task Team, for the speed with which financing was approved, the disbursement arrangements, the support and guidance, and flexibility in accommodating changes. b. They also paid tribute to the previous PMU Director (who passed away two years ago) for his management of the project and drive to ensure timely implementation and securing the project extensions. c. The speed and effectiveness of the KP-ERRP project compared very favorably with the slower implementation of work on the USAID and GOP financed Madyan to Kanju section. 21 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million equivalent) Actual/Latest Estimate Appraisal Estimates (Apr 2016) Appraisal Appraisal Appraisal Category Estimate Estimate Estimate % of Total Amount (EPP July (AF-1, (AF-2, Appraisal 2011) Dec 2013) Apr 2014) (1) (2) (3) (4)=(1)+(2)+(3) (5) (6)=(5)/(4) Component 1: Infrastructure Rebuilding Widening and 5.142 7.670 8.450 21.262 24.23 114% reconstruction of sections of provincial highway S- 3B Relocation of utilities, 1.275 0.630 0.250 2.155 0.38 18% land acquisition and resettlement Sub-Total 6.417 8.300 8.700 23.417 24.61 105% Component 2: Project Management Contract administration & 0.188 0.200 0.200 0.588 0.44 75% construction supervision consultant services Environmental & social 0.053 0.100 0.050 0.203 0.18 89% safeguards consultant services Other project management 0.373 0.500 0.050 0.923 0.39 42% activities through the financing of incremental operating costs Sub-Total 0.614 0.800 0.300 1.714 1.01 59% Total Baseline cost 7.031 25.131 25.62 102% Physical & Price 0.969 0.969 Contingencies Total Project Costs 8.000 9.100 9.000 26.100 25.62 98% Annex-P-1 (b) Financing (in US$ Million equivalent) Actual/Latest Appraisal Estimates Estimate (April 2016) Appraisal Appraisal Appraisal Category Estimate Estimate Estimate % of Total Amount (EPP July (AF-1, (AF-2, Apr Appraisal 2011) Dec 2013) 2014) (1) (2) (3) (4)=(1)+(2)+(3) (5) (6)=(5)/(4) Borrower MDTF 8.00 9.10 9.00 26.10 25.62 98% Total 8.00 9.10 9.00 26.10 25.62 98% Annex-P-2 Annex 2. Outputs by Component Component/Activity Output Component 1: Infrastructure Rebuilding Widening and reconstruction of sections of Reconstruction of total of 50.3km of roads. provincial highway S-3B and 2 link roads. Reconstruction and widening of about 40.9km of highway (7.3 meter wide asphalt concrete 2-lane single carriageway with 1.0 meter wide shoulders) and related structures (including reconstruction of 2 bridges) from Chakdara to Kanju. Reconstruction and widening of 9.4 km of link roads at Shamozai (7.3 meter wide asphalt concrete 2-lane single carriageway with 1.0 meter wide shoulders as far as possible but adjusted where necessary to eliminate the need to acquire assets and remove structures). Relocation of utilities, land acquisition and These activities associated with widening and resettlement reconstruction of sections of highway S3-B. Component 2: Project Management Contract administration & construction These activities associated with widening and supervision consultant services reconstruction of sections of highway S3-B. Environmental & social safeguards consultant These activities associated with widening and services reconstruction of sections of highway S3-B. Other project management activities through These activities associated with widening and the financing of incremental operating costs reconstruction of sections of highway S3-B. Project supervision and management capacity development in the Pakhtunkhwa Highway Authority and Government of Khyber Pakhtunkhwa. Annex-P-3 Annex 3. Economic and Financial Analysis 1. The economic appraisal of the project using the conventional discounted cash flow approach was carried out to support the Phase 1 proposal. Financial costs were adjusted for economic price distortions resulting from market imperfections and trade tariffs. A discount rate consistent with GOP recommendation was used. For Phase I a traffic study including estimates of average traffic speeds and VOCs, classified traffic counts and origin-destination surveys was carried out and a traffic growth projection was made. The appraisal was updated with new 2013 baselines for traffic speed and VOCs for the whole 40.9km Kanju to Chakdara section to support the Phase II and III (AF-1 and AF-2) proposals. Table A3-1 summarizes the results of the appraisals. Table A3-1: Economic Indicators at 12% Discount Rate Phase I Phases II & III (AF-1 & 2) EIRR (%) 29.9 28.0 B/C Ratio 3.1 : 1 2.9 : 1 NPV (Million Rs.) 1,197 1,139 2. Sensitivity analyses indicated that the results were robust to a combination of 10 per cent reduction in benefits and 10 per cent increase in costs and other adverse assumptions with the EIRR remaining well above 20 per cent. 3. The principal determinants of the tangible benefits of the road in the appraisal were: a. Traffic growth (estimated to be just over 5 per cent per year to 2032); b. Increase in traffic speed as a consequence of the improved road leading to savings in time for road users and the costs of operating vehicles, and c. Lower VOCs as a consequence of higher traffic speeds and improved road surface. These tangible benefits combined with the costs determine its net benefits, EIRR, benefit-cost ratio and NPV. To form a post-construction judgment on whether the project is on track to deliver the projected benefits, each of these determinants is reviewed below, starting with VOCs and traffic speed. Tables A3-2 and A3-3 adapted from the Phase I EPP summarize the VOCs and traffic speeds with and without the project for the main types of commercial vehicles. Table A3-2: Reduction in VOCs post-construction (Phase I baseline) VOCs (Rs per km) % reduction Vehicle type after re- construction Pre- Post- construction construction (2010) (2015) 2-axle trucks 23.89 15.84 34 3-axle trucks 24.76 18.33 26 Passenger buses 12.09 11.47 5 Annex-P-4 Table A3-3: Increase in average vehicle operating speeds of commercial traffic post- construction (Phase I baseline) Speed before re- After re-construction Projected % Vehicle type construction speed projected at increase post- (kph, 2010) appraisal (kph, 2015) reconstruction 2-axle Trucks 20 35 75 3-axle Trucks 20 30 50 Passenger Buses 30 50 67 4. Evidently, Table 3-2 shows that the PDO results indicator 1 (34%, 26% and 5% decrease in vehicle operating costs of 2-axle truck, 3-axle truck and passenger buses respectively) is based on the estimates of pre-construction and post-construction VOCs used in the economic appraisal for Phase I. Table 3-3 shows that the projected increases in speeds for commercial traffic (2- and 3- axle trucks and passenger buses) are 50% or higher. PDO results indicator 2 (50% increase in operating speeds of commercial traffic) appears to be based on the estimates of pre-construction and post-construction speeds. The PDO indicators for Phase I were retained for Phases II and III. In combination with information on traffic growth, the PDO indicators 1 and 2, which are based on the principle assumptions underlying the economic benefits, provide a basis for assessing whether the project is on track to deliver the projected benefits. 5. Following the ICR mission in February 2016, the project supervision consultant conducted a VOCs and traffic study which estimated traffic volume and speed and VOC savings. However it did not use the same baseline speed and VOCs as the appraisal for Phase I for which the traffic and VOC data were based on the 10.5km Kanju to Sharif Abad section. The baseline for the 2016 study was pre-construction estimates of VOCs and speeds for the whole Kanju to Chakdara section made in 2013 (see Tables A3-4 and A3-5). The 2013 baseline VOCs and speeds are clearly not comparable with the 2010 baseline (compare Tables A3-2 and A3-4 and Tables A3-3 and A3-5) because the latter relates to just a part of the whole section completed under the project and for VOCs, adjustments for inflation would be needed. The comparison between the 2013 baseline and the 2016 estimates, both for the whole 40.9 road section, shows (see Tables A3-4 and A3-5) that the percentage targets for the PDO results indicators 1 and 2 have been matched (VOC saving for 2-axle trucks) or exceeded. Table A3-4: Reduction in VOCs post-construction (All phases, 2013 baseline) VOCs (Rs per km) % reduction PDO 1 target % Vehicle type post- reduction construction Post- Pre-construction construction (2013 baseline) (2016) 2-axle Trucks 30.91 20.48 34 34 3-axle Trucks 31.75 21.06 34 26 Passenger 37.32 28.76 23 5 Buses Annex-P-5 Table A3-5: Increase in average vehicle operating speeds post-construction (All phases, 2013 baseline) (PDO indicator 2: 50% increase in operating speeds of commercial traffic) Before re- Post-construction % increase post- Vehicle type construction (2013 (2016) (km/hour) construction baseline) (km/hour) 2-axle Trucks 30 55 83 3-axle Trucks 30 50 67 Passenger buses 40 65 63 Average weighted by 38 63 64 2016 volumes 6. The improvements in VOCs and vehicle speeds are consistent with the assumptions made for the pre-appraisal net economic benefit estimates for Phases I, II and III. The projected growth in traffic is another key factor in determining the net benefit. The 2016 VOC and traffic study compared the traffic forecasts for 2015 and 2016 prepared for the Phase I appraisal1 with the post-construction traffic estimate based on traffic counts at the end of February 2016 (see Table A3-6). 7. The traffic forecasts for 2015 and 2016 (Annual Average Daily Traffic, AADT) are close to the 2016 post-construction estimate based on actual traffic counts. The total February 2016 traffic estimate is 3.8 per cent above the projection at appraisal for 2015 and 1.3 per cent below the projection at appraisal for 2016. These are small differences indicating that the traffic projections made at appraisal were sound. There are apparently larger variations for some vehicle types, notably many more passenger buses, fewer cars / jeeps / vans and 2-axle trucks but more 3-axle and 4-axle trucks. More passenger buses and fewer smaller vehicles is most probably explained by mini-buses included in the passenger buses category. The apparent switch to larger trucks is probably because of the improved capacity of the road. 8. Cost savings during project implementation (because of a combination of exchange rate movements, lower compensation for affected persons, a cost de-escalation clause in the contract and efficiency savings) also have implications for the soundness of the appraisal. The appraisal for Phase 1 was based on the assumption that the cost of US$ 8.0 million was for the re- construction of a 10.5km section (Kanju to Sharifa Abad). The total road length re-constructed within this budget was 15.5km (Kanju to Dadahara). Savings in Phases II and III made it possible to add further road lengths (9.4km of roads linking with S3-B at Shamozai). The net effect is a cost saving per km of reconstruction of 18.7 per cent.2 9. In summary, given that the post-construction traffic volume is close to the projection at appraisal, the VOC savings and traffic speed improvements exceed the results indicator targets and assumptions at appraisal and the construction cost per km is 19 per cent lower than assumed at 1 There were no new traffic forecasts for the appraisal of Phases II and III. 2 The estimated construction cost of US$ 26.1 million for 40.9km is approximately US$ 638,000 per km. Construction of 50.3km with this budget brings it down to approximately US$ 519,000 per km, 18.7 per cent reduction. Strictly speaking the saving on original appraisals is greater since this calculation does not take account of the additional 5km constructed under Phase I. If this is adjusted for, the saving is almost 29 per cent (US$ 727,000 per km for re-constructing 35.9km in the appraisals) against the actual cost of US$ 519,000 per km. Annex-P-6 appraisals, the EIRRs and benefit-cost ratios estimated at the appraisals of Phases I, II and III are valid. Table A3-6: Comparison of 2016 traffic volume projected at Phase I appraisal and estimated post-construction (AADT) Vehicle types 2015 traffic 2016 traffic Estimate Differences Differences projection at projection at based on between between Phase I Phase I February 2015 2016 appraisal appraisal 2016 traffic projection projection count and and February February 2016 count 2016 count estimate estimate Motorcycles / Rickshaws 2,406 2,520 3,192 786 672 Cars / Jeeps / 7,863 8,236 5,614 -2,249 -2622 Vans Passenger buses 1,980 2,074 4,429 2,449 2355 Tractor Trolleys 486 520 201 -285 -319 2–axle Trucks 1,084 1,161 325 -759 -836 3–axle Trucks 461 493 638 177 145 4–axle Trucks 100 107 503 403 396 4–axle Truck Trailers 30 32 63 33 31 5–axle Trucks 28 30 17 -11 -13 6–axle Trucks 15 16 14 -1 -2 Total 14,453 15,189 14,996 543 -193 Per cent differences 3.8 -1.3 Annex-P-7 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Zafar Iqbal Raja Sr. Highway Engineer GTI06 Team Leader Rehan Hyder Sr. Procurement Specialist GGO06 Procurement Syed Waseem Abbas Kazmi Sr. Financial Management Specialist GGO24 Financial Management Environmental Javaid Afzal Sr. Environmental Specialist GEN03 Safeguards Samina Mussarat Islam Social Development Consultant CRKI3 Social Safeguards Supervision/ICR Zafar Iqbal Raja Sr. Highway Engineer GTI06 Team Leader Rehan Hyder Sr. Procurement Specialist GGO06 Procurement Syed Waseem Abbas Kazmi Sr. Financial Management Specialist GGO24 Financial Management Environmental Javaid Afzal Sr. Environmental Specialist GEN03 Safeguards Samina Mussarat Islam Social Development Consultant CRKI3 Social Safeguards A.K. Farhad Ahmed Sr. Transport Specialist GT106 Team Leader (ICR) Ross S. Pavis Senior Operations Officer GTIGE Operations Kiritkumar Gunvantrai Vaidya Consultant ICR Ghulam Farid Program Assistant SACPK Team Support Comfort O. Olatunji Program Assistant GT106 Team Support Tema A. Kio-Michael Program Assistant GT106 Team Support Annex-P-8 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ Thousands (including No. of staff weeks travel and consultant costs) Lending (TF098976) FY11 14.03 31.40 FY12 3.95 19.94 FY13 0.00 0.00 FY14 0.00 0.00 FY15 0.00 0.00 Total: 17.98 51.74 Supervision/ICR (BB/TF098976/TF016353) FY11 0.00 0.00 FY12 24.13 67.90 FY13 34.94 100.20 FY14 12.20 30.04 FY15 19.67 53.01 FY16 27.67 109.68 Total: 118.61 360.83 Annex-P-9 Annex 5. Fact Finding and Consultation Workshops3 Report and Results 1. The ICR team was not permitted to visit the PkHA headquarters and the project road because of the security situation. Therefore two workshops were held at the Bank’s Islamabad Office. The participating stakeholders included Bank staff, project PMU staff, representatives of the supervision consultant and the contractor. Because of the restriction on travel to PK, the ICR team was not able to engage with people residing near the road, road users, transport service providers and local businesses, the key stakeholders. Therefore the workshops are referred to as fact finding and consultation and not stakeholder workshops. The workshops were guided by a framework prepared by the ICR team for obtaining information for the ICR on: (a) the overall project experience; (b) design and quality at entry and implementation; (c) M&E; (d) safeguards compliance; (e) fiduciary arrangements and compliance; (f) sustainability; (g) PDO achievement; (h) project efficiency, and (i) other outcomes/impacts and lessons learnt. 2. The first workshop on 22nd February lasting about two and half hours was attended by the PMU staff and representatives of the project supervision consultant. The two main aims of this workshop were: (a) fact finding and elaboration, and (b) reviewing the project management and implementation experience from the perspective of PkHA. 3. The fact finding and elaboration were intended to obtain a detailed understanding of: (a) the cycle of project initiation, implementation and completion; (b) the roles of the PMU staff members in project management; (c) the roles of the supervision consultant in project management and supervision of implementation and safeguard compliance, and (d) how project management and monitoring arrangements with the Task Team functioned given the restrictions on their visits to the site and Peshawar. 4. It was particularly important to obtain a clear understanding of the two AFs and the work under them and the difference in the procurement procedure between Phase I and Phases II and III. The project completion report prepared by the supervision consultant on behalf of the PkHA and the clarifications and elaborations at the meeting were particularly helpful in clarifying the procurement procedures, implementation arrangements and safeguards compliance. The information obtained was also discussed at the second brainstorming workshop. To avoid repetition, the summary of the discussion at the first workshop is not included here but incorporated in the summary of the discussion of the brainstorming session. 5. The participants were also asked to reflect on what had gone well and why and where there had been problems. The main points which emerged from the reflection and related discussion were: a. The high priority given to the reconstruction of this road by GoKP and PkHA backed by the surveys and design for the road commissioned by PkHA from its own resources. b. PkHA and the PMU did not have previous experience of working with the Bank and therefore there was a steep learning curve with respect to compliance with the Bank M&E, fiduciary and safeguards rules and procedures. The support from the TTL and Task Team 3 These are referred to as fact finding and consultation rather than stakeholder workshops because as noted in Section 3.6 the beneficiaries from the reconstructed road, the key stakeholders, were not represented at the workshops because the ICR team was not permitted to visit the road and it was not possible to arrange for representatives of beneficiaries to attend the workshops in the Islamabad. Annex-P-10 complemented by the commitment of the first director of the PMU and staff made it possible to establish the project quickly and complete work within time and budget. c. Once the PMU became familiar with Bank procedures they were easy to comply with and the system for disbursement of funds enabled implementation without delays. d. Progress on the project compared very favourably with that on the part of SB-3 financed from other sources. e. The commitment and competence of the PMU staff and the close working relationship with the supervision consultant were evident. 6. The second workshop was a brainstorming session on 23rd February again lasting about two and half hours. The participants were the MDTF manager and social appraisal specialist, the TTL, some members of the Task Team,4 PMU staff, representatives of the supervision consultant and the construction contractor. The aim was for the ICR team to briefly report their interim findings and the sources used for the interim findings as a basis for reflecting on project performance and identifying gaps in the information required for completing the ICR. The main points which emerged on each principle area covered have been briefly stated below. 7. The main areas of discussion at the brainstorming session are summarized below. a. Overall project experience. (Functioned well in a post-conflict situation. Demonstration of timely and within budget implementation of Phase I with extended length led to the project extensions to complete the whole project. Contributing factors were flexibility and support on the part of MDTF and the Task Team, effective PMU management and supervision and competent project supervision and implementation by the supervision consultant and contractor.) b. Project preparation, design and quality at entry and implementation. (High GoKP and PkHA priority and preparedness were key for identifying this project for implementing under MDTF. The Bank appraisal and approval processes were swift. Prompt start of implementation was possible because the contractor was working in the locality and had plant and machinery in place. The supervision consultant had designed the project and had supervision experience in the area.) c. M&E. (The project documents provided detailed specifications for the works and quality and materials testing. These specifications formed the basis for monitoring by the supervision consultant who provided monthly and quarterly reports of progress to PkHA and the Bank via PkHA. Monitoring by the TTL and the Task Team was through these reports and six monthly meetings. In practice communications between the TTL and Task Team and PMU and the supervision consultant were more frequent than six-monthly. Lack of formal records of TTL and Task Team monitoring was highlighted as an issue.) d. Safeguards compliance. (On environmental safeguards, the ESSAF framework was followed by the supervision consultant 5 to prepare EIAs / EMPs for each phase. The supervision consultant monitored compliance, provided training on site and was responsible for ensuring compliance. PkHA engaged an independent auditor and a monitor to provide reports independent of the supervision consultant. Compliance was reasonable overall though the monitor and the Task Team expert identified non- 4 All members were invited. 5 The supervision consultant is responsible for supervising implementation and for safeguards compliance. Annex-P-11 compliance on some aspects including workers’ welfare and health and safety. The social safeguards, principally concerned with compensation for acquisition of land, worked well overall though highly dependent on a dedicated person in PMU in place to deal with safeguards.) e. Fiduciary compliance. (The two areas discussed here were procurement and financial management. On procurement, following NCB for Phase I, extending the Phase I contractor’s and supervision consultant’s contracts to include Phases II and III without competition was discussed and on balance considered acceptable under the circumstances. The main considerations were the need to reduce the time for completing the project and negotiation of contract conditions to control costs. There were no issues of concern on fiduciary compliance as far as reporting to the Bank was concerned though there were some internal audit issues which needed clarification and resolution.) f. Post completion operations and sustainability. (The establishment of a maintenance regime to preserve the road and the benefits it generates was raised as a matter of concern. PkHA has insufficient funds to set up a maintenance system for its road network. As a consequence a “forced” maintenance regime is in place with the Project road likely to receive attention several years on when it is sufficiently deteriorated. The possibility of support for PkHA to put in place a road assets management system was discussed. This could be related to funding for future projects.) g. Attainment of the PDO and PDO results targets. (At a broad level, there is evidence that the objective “to enable the population along the Project corridor to benefit from year round improved access and mobility” has been met. However, the evidence to determine whether the PDO results indicator targets were met was not available. It was agreed that the supervision consultant would conduct a VOC and traffic study to enable assessment of performance on results indicators 1 and 2 and the MDTF social appraisal specialist would conclude on the results indicator 3.) h. Other adverse or favourable impacts of the project. (The direct adverse effect of the improved road raised at the workshop was associated with the added risk of accidents because of higher speeds. The favourable wider impacts were institutional strengthening and improving governance and service delivery through the experience of implementing the project effectively and ensuring compliance with safeguards. A recommendation of the supervision consultant, which found favour with the PMU staff, was the establishment of an environmental and social safeguards unit within PkHA. However it was thought that the PkHA management would not give a high priority to such an initiative. 8. An overarching issue discussed during the session was the challenge of “remote” monitoring by the Task Team, to what extent this affected programme implementation and performance and whether ICT based “remote” monitoring methods could have improved the project. The general view was that ICT based monitoring should be included in future projects but in the case of this project the lack of such monitoring might not have been such a serious issue. Annex-P-12 Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR I SALIENT FEATURES OF THE PROJECT Project Title Project ID P125584 Borrower Country Pakistan Executing Agency: PkHA Project Implementation Start Date: October 17, 2005 End Date: October 31, 2015 Period Financing Source (US$M) Original Finance Additional Total Actual Finance US$ Million US$ M US$ M US$ M MDTF 8.0 18.1 26.1 26.1 Physical Achievement Target in Original Target in EPP Target in EPP Provincial highway S3-B phase (EPP July AF-1 (December AF-2 (April Achievement 2011) 2013) 2014) Reconstruction and widening 10.5 km 30.2 40.9 50.3km Associated relocation of utilities, land acquisition and resettlement Annex-P-13 II Borrower’s completion report The PkHA submitted its Project Completion Report prepared on its behalf by the supervision consultant. In some areas, such as the three phases, contract completion, expenditure by categories, comparison between planned and actual project progress and the plant and personnel used, the report is informative. It also shows design drawings of road cross sections, the level of expenditure broken down into categories, a brief summary report on compliance with environmental and social safeguards and lessons learnt by the PkHA on safeguards compliance. Nevertheless there are some gaps. While evidence is provided on the total expenditure by work types and the lengths of road constructed, some important quantities are missing, notably the number of bridges and culverts constructed. Evidence for assessment of achievement on PDOs is also missing. The summary in this annex uses the information provided in the client’s Project Completion Report and additional information obtained from project management unit staff and the supervision consultant at the stakeholder workshops. The tabular form is used in the summary to list the borrower’s completion report requirements from the ICR Guidelines in the first column with the relevant information for this project in the second column Borrower’s ICR requirement (from Borrower’s ICR information the “ICR Report Guidelines”) Assessment of the operation’s The objective was to re-construct about 40.9km of objective, design, implementation, provincial highway S3-B which was severely damaged and operational experience. during the military action against insurgents in 2009 to restore year round accessibility and mobility to the local population of some 300,000 people on the right bank of Swat river. The client’s Completion Report shows that the output in the form of length of road constructed has been achieved. As noted above it does not provide full details of the output (quantity of works and number of bridges and culverts). The report refers to the road serving 300,000 people on the right bank of the Swat river but does not provide information on the proportion of the population served and the evidence needed for assessing achievement on the PDO indicators. Assessment of the outcome of the The outcome is assessed as satisfactory since the road has operation against the agreed been completed and is serving the local population. It also objectives. provides an alternative route to the parallel National Highway N-45. As noted above the report does not provide evidence to assess outcome on PDO indicators related to improved accessibility and mobility. However, in response to the ICR team’s request the client commissioned a study to provide evidence on two of the PDO indicators, VOC savings and commercial vehicle speed. Evaluation of the borrower’s own The PkHA has not explicitly included an evaluation of its performance during the preparation performance in its Completion Report. However it can be Annex-P-14 and implementation of the operation, inferred from the completion report and other information with special emphasis on lessons that PkHA finds its performance satisfactory. It identified learned that may be helpful in the restoration of this road as high priority, commissioned future. survey and design of the project, sought and secured funding and implemented the project to completion within time and budget with savings which enabled construction of additional road sections. The client’s Completion Report includes lessons learnt on safeguards: (a) ensuring that the capacity of executing agencies to deal with environmental and social safeguards should be developed through training before project implementation; (b) a safeguards compliance unit should be considered for PkHA, and (c) there should be better coordination with other departments and agencies to reduce delays in, for example, relocation of utility poles and payment of compensation for land and property). Evaluation of the performance of the The completion report did not include an evaluation of the Bank or of other partners during the Bank’s performance. At the first stakeholder workshop the preparation and implementation of PMU staff expressed a high level of satisfaction with the the operation, including the support and guidance provided by the TTL and Task Team effectiveness of their relationships, and the flexibility and promptness of responses. with special emphasis on lessons learned. Description of the proposed PkHA’s completion report did not include a statement on arrangements for future operation of the proposed arrangements for future operation. This issue the project. was raised with PMU staff at the first stakeholder workshop. Because of limited resources for maintenance and the road having just been constructed, no maintenance requirement was anticipated for several years. The road would be repaired / rehabilitated on the principle of “forced” maintenance when it is sufficiently deteriorated. Assisting the preparation of the PkHA and the supervision consultant assisted the ICR team Bank’s ICR. in three ways. 1. By providing a detailed completion report. 2. By engaging in candid discussions on the matters raised and providing further elaboration. 3. By commissioning the supervision consultant to conduct the VOCs and traffic study to enable the team to assess achievement on PDO indicators 1 and 2. Annex-P-15 Annex 7. List of Supporting Documents Project Documents  KP / FATA / Balochistan Multi-Donor Trust Fund Grant Agreement (Khyber-Pakhtunkhwa Emergency Roads Recovery Project) between Islamic Republic of Pakistan and International Bank For Reconstruction And Development / International Development Association acting as administrator of the KP / FATA / Balochistan Multi-Donor Trust Fund, 23rd August 2011.  KP / FATA / Balochistan Multi-Donor Trust Fund Project Agreement (Khyber-Pakhtunkhwa Emergency Roads Recovery Project) between Islamic Republic of Pakistan and International Bank For Reconstruction And Development / International Development Association acting as administrator of the KP / FATA / Balochistan Multi-Donor Trust Fund, 23rd August 2011.  World Bank (2011) MDTF KP Emergency Roads Recovery Project: Integrated Safeguards Datasheet, Appraisal Stage, June (Report No: AC6350).  World Bank (2011) Emergency Project paper on a proposed grant in the amount of US$ 8.0 million under the KP/FATA/Balochistan Multi Donor Trust Fund to the Islamic Republic of Pakistan for a Khyber-Pakhtunkhwa Emergency Roads Recovery Project (KP ERRP), July, (Report No: 61615- PK).  World Bank (2013) MDTF KP-ERRP: Project Information Document (PID) Concept Stage – Additional Financing, October (Report No: 81649).  World Bank (2013) MDTF KP-ERRP: Integrated Safeguards Datasheet – Additional Financing, October (Report No: 81648).  World Bank (2013) Emergency Project paper on proposed additional financing grant in the amount of US$ 9.1 million under the KP/FATA/Balochistan Multi Donor Trust Fund to the Islamic Republic of Pakistan for a Khyber-Pakhtunkhwa Emergency Roads Recovery Project (KP ERRP), December (Report No: 81123-PK).  World Bank (2014) Emergency Project paper on a proposed grant in the amount of US$ 9.0 million under the KP/FATA/Balochistan Multi Donor Trust Fund to the Islamic Republic of Pakistan for a Khyber-Pakhtunkhwa Emergency Roads Recovery Project (KP ERRP), April (Report No: 84700- PK).  World Bank (2015) Project Brief KP-ERRP, May.  World Bank (2015) Restructuring Paper on a Proposed Project Restructuring of KP Emergency Roads Recovery Project US$ 26.1 Million Grant Board Approval Date August 23, 2012 to the Government of Islamic Republic of Pakistan (Date of Restructuring: June 18, 2015, Report No: RES19981).  World Bank (2015) P125584/P14931: Emergency Roads Recovery Project (ERRP) –KP Financial Management Supervision Report, August.  World Bank (2015) P125584 – MDTF KP-ERRP Works Procurement Activities (Document No: 102037)  World Bank Implementation Status & Results (ISR) Reports for P125840 MDTF KR-ERRP: o 2011-12-10, ISR5032 o 2012-6-15, ISR7323 o 2012-12-11, ISR8684 o 2013-5-16, ISR10288 o 2013-12-2, ISR12633 o 2014-6-17, ISR15080 o 2015-1-5 o 2015-7-20 Annex-P-16 Related Reports  Associated Consulting Engineers–ACE (2011) Emergency Roads Recovery Project (ERRP) for Khyber Pakhtunkhwa: Environmental Impact Assessment (EIA) Report.  Associated Consulting Engineers–ACE (2014) Emergency Roads Recovery Project (ERRP) for Khyber Pakhtunkhwa: Monthly Progress Report, April.  Associated Consulting Engineers–ACE (2014) Rehabilitation and Upgradation of Shamozai - Dadahara Section and Chakdara - Shamozai Section (Package -II & III) under KP-ERRP: EMP Quarterly Progress Report (Apr - Jun 2014).  Associated Consulting Engineers–ACE (2014) Rehabilitation and Upgradation of Shamozai - Dadahara Section and Chakdara - Shamozai Section (Package -II & III) under KP-ERRP: EMP Quarterly Progress Report (Oct - Dec 2014).  Associated Consulting Engineers–ACE (2015) Chakdara – Kanju section: Customer satisfaction survey report, November.  Associated Consulting Engineers–ACE (2016) Upgradation / Rehabilitation of Road From Chakdara to Madyan on the Right Bank Of River Swat: Chakdara to Kanju Section (40.924 km) - Contract Completion Report, February.  Associated Consulting Engineers–ACE (2016) Chakdara – Kanju section: Monitoring of post construction results, March.  Frontiers Highway Authority (2011) Upgradation / Rehabilitation of Sharifabad to Kanju Road: Social Impact Assessment and Resettlement Action Plan, August.  Government of Pakistan (2010) Khyber Pakhtunkhwa & Federally Administered Tribal Areas: Post Crisis Needs Assessment (PCNA) with support from ADB, EU, UNDP and World Bank.  MDTF (2010) Environmental and Social Screening and Assessment Framework (ESSAF) for Multi- Donor Trust Fund (MDTF), Islamic Republic of Pakistan and World Bank.  Minutes of MDTF KP Portfolio Review Meeting, November 13, 2014.  Minutes of the MDTF KP Portfolio Review Meeting at the Civil Secretariat, Peshawar on May 18, 2015.  PkHA (2015) Emergency Roads Recovery Project: Draft Internal Audit Report: for the six months period ended 30th June 2015.  Project Director (MDTF KP) (2011) Financial / Technical Bids for Chakdara – Madyan Road Project (Sharif Abad to Kanju Section): Bid Evaluation Report (BER), Frontier Highways Authority, Peshawar.  SMEC (2012) Presentation on 3rd Party Assessment of Sharifabad – Kanju Road Section Funded by MDTF Administered by the World Bank, September. Other Bank Documents  IEG (2014) Guidelines for Reviewing World Bank Implementation Completion and Results Reports: A Manual for Evaluators, World Bank.  OPCS (2006) Implementation completion and results report: Guidelines (last updated 07/22/2014), World Bank.  OPSPQ (2013) Results Framework and M&E Guidance Note, April, World Bank.  OPSPQ (2013) Investment project financing: Additional financing guidance note, World Bank.  World Bank Operational Manual – Procedures BP10.00 (2013).  World Bank (2013) OP 4.01 - Environmental Assessment, 1999 (revised April 2013). Annex-P-17 MAP – KP-ERRP Annex-P-18