Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) Report Number : ICRR0021095 1. Project Data Project ID Project Name P096019 IN: HP State Roads Project Country Practice Area(Lead) Additional Financing India Transport & Digital Development P130616 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-48600,IBRD-81990 30-Jun-2013 303,430,000.00 Bank Approval Date Closing Date (Actual) 05-Jun-2007 30-Jun-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 220,000,000.00 0.00 Revised Commitment 266,698,008.44 0.00 Actual 266,698,008.44 0.00 Prepared by Reviewed by ICR Review Coordinator Group Victoria Alexeeva Peter Nigel Freeman Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The project development objective was to reduce transport costs and to improve traffic flows on priority segments of the core road network of the State of Himachal Pradesh (HP) (Loan Agreement, page 5; Project Appraisal Document, page 3). b. Were the project objectives/key associated outcome targets revised during implementation? No Page 1 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components 1. Core Network Improvement (appraisal US$235.25 million; AF US$61.6 million; actual US$320.6 million). This component consisted of road upgrading works, including widening, realignment, new structures and pavement strengthening, on approximately 447 km of state roads in the core network, plus associated supervision, land acquisition, and application of social and environment safeguard measures. 2. Core Network Maintenance and Management (appraisal US$66.02 million; actual US$41.6 million). The component included (a) periodic maintenance (overlays and reseals) and minor rehabilitation (replacement and addition of base materials and structures) of about 2,000 km of the core road network; (b) piloting 11 performance-based maintenance (PBM) contracts; (c) accident black spot improvements; (d) pre-investment studies for road network improvement and maintenance; and (e) capacity enhancement in road maintenance, financing, and management. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total actual project cost was US$365.2 million that increased by 20% form the estimated US$303.43 million due cost overruns of the road network improvement works. Financing. The project was financed through the original IBRD loan of US$220 million that was supplemented by an additional financing (AF) loan of US$61.7 million to total US$281.7 million. The loans were disbursed at US$266.7 million. Borrower contribution. The Borrower committed US$83.43 million at appraisal and US$35.5 million at the time of additional financing. At closure, the actual contribution was slightly lower than the total commitment of US$118.9 million and amounted to US$98.5 million. Dates. The project was approved in 2007 for an implementation period of six years. The project closing date was extended by an additional four years from June 30, 2013 to June 30, 2017. The first extension of three years was at the time of additional financing approved in 2012 to finance cost overruns in upgrading works, afforestation, and land acquisition under component 1 (AF Restructuring Paper 2012). The second extension was for a further year in 2016 due to delays in completion of two large road upgrading contracts. The third project restructuring was in 2017 to cancel US$15 million from the AF Loan because (a) the Indian Rupee depreciated against the US Dollar, producing a gain in the available amount of local currency, in which most contracts were established and (b) one of the road upgrading contracts would not be fully completed by the revised closing date, and the corresponding part of the contract amount would not be used. 3. Relevance of Objectives Rationale Himachal Pradesh (HP), with its largely mountainous topography and significant forest cover, presents special Page 2 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) challenges to the roads sector. The difficult terrain, seasonally poor weather conditions, scarcity of construction materials, and dispersion of the population cause the unit costs of road development and maintenance to be significantly higher (50–100 percent) than equivalent works in states with mainly flat topography. The combined effects of the monsoon season during July-September and the snows in winter limit the construction season to about six months of the year. At the time of project appraisal, over 90 percent of the HP State’s population lived in rural areas and over two thirds of employment was generated in the agricultural sector. There were considerable disparities in access to services across the state. The main issues facing the road sector in HP at appraisal were: (a) lack of past investment; (b) a poorly funded and implemented maintenance regime; (c) monolithic and traditional management by the Public Works Department (PWD) with a large in-house force account staff of 44,000 laborers; (d) outdated business practices; (e) absence of a coherent financial strategy; and (f) limited application of modem planning, design and construction methods. The project development objectives were relevant to the main challenges in the road sector of the HP state. There is, however, no road sector policy of the state government, and the HP state's broad development priorities focus on hydro-power generation, horticulture, industrial investment, and tourism (Economic Survey 2016-17). These sectors, as described by the ICR (page 11), would require a robust and sustainable road transport network. The project remained consistent with the World Bank Group's Country Partnership Strategy (CPS) FY13-17 in directly supporting the outcome of “improved transport connectivity” under the first of three engagement areas, integration, whose focus was on improved physical connectivity and strengthening of market mechanisms. The project also contributed to another CPS’s engagement area- inclusion, through improving access to services for marginalized segments of population. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To reduce transport costs on priority segments of the core road network of the state of Himachal Pradesh. Rationale Outputs • 355 km of the HP State's core road network were upgraded from single to two- lane roads against the target of 447 km (about 80% of the target). About 12 km were cancelled from several packages due to land acquisition issues. The remaining 80 km were under implementation at project closure with financing through the State's own resources. • Periodic maintenance was carried out on 1,831 km of roads against a target of 2,000 km (1,485 km of conventional contracts and 346 km of performance-based contracts). The remaining 169 km were Page 3 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) declared as National Highways by the Government of India and so were cancelled from the project. • 25 blackspots were improved on the core network, exceeding the target of 20. • A road sector finance study was carried out to assess fund requirements for the state road network for future 10 years. As the ICR (page 18) notes, the consideration of the Road Fund Bill was broken off due to a change of the state government. The draft bill would have financed the Road Fund from a state fees on fuels, revenues from commercial use of road space, an incremental fee on cement bags, fees to PWD for implementing other departments’ works, and a share of the State Marketing Board’s collections. • A Road Accident Data Base Management System (RADMS) was developed and had been implemented in all districts as of July 2015, and the Police established an Accident Data Management Cell. The application wiould enable all concerned stakeholders to identify and address road safety challenges on a common platform. At the project’s closing, close to 100 percent of accident data were being captured by the Police Department through the RADMS system. • Technical assistance was provided for RIDC to help achieve ISO 9001:2008 Quality Management and ISO 14001:2004 Environmental Management. Certifications were awarded in 2011 in nine units of RIDC and the National Highways wing of the Public Works Department (PWD). • Upgrading of PWD’s existing road maintenance management system to a web-based system as well as the computerization of PWD to integrate core activities of the department by way of automated business processes and work flow systems using an electronic information database were not completed at project closure and are expected to be delivered in 2018. Outcomes • No outcome indicator was identified at appraisal to measure the reduction in transport costs. A proxy indicator—vehicle operating costs (VOCs)—was used at closure. The baseline average VOC in 2006 for the 10 roads concerned was 14.39 Rs/km based on pre-project prices and road conditions (as measured by the International Roughness Index (IRI), which was 9.7 for the roads as a whole). The post-upgrading VOC derived in 2006 was an average of 10.01 Rs/km. The final average VOC at closure was 9.78 Rs/km for the project-financed roads, based on 2017 prices and post-upgrading road conditions in 2017. This represents an overall reduction of 32 percent. • The baseline value for 2014 was 0.28 deaths involved in traffic accidents per 1,000 vehicles. This figure steadily declined each year subsequently, reaching 0.21 in 2017. The ICR (page 15) notes that this preliminary trend may be attributed to the project, considering that, during the period, there were no other significant road infrastructure interventions, legal reforms, enforcement improvements, or awareness campaigns related to road safety in HP State. At the same time, safety enhancements were incorporated in all the upgrading works (e.g., safer alignments and intersections, guard rails, markings, signage). • The share of labor costs for the in-house force account staff in total spending on routine and emergency maintenance was reduced from 75 percent in 2007 to 62 percent in 2017, not reaching the targeted 50 percent. • A new road financing mechanism for the core network was not put in place as planned under the project. Page 4 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To improve traffic flows on priority segments of the core road network of the State of Himachal Pradesh. Rationale Outputs are the same as under Objective 1. Outcomes • Average speeds on the upgraded roads rose from an average of 29 km/hr before the project to 40 km/hr after completion of the works, an increase of 38 percent, surpassing the targeted increase of 25 percent. • The project achieved the target for reduction of the share of the core network in poor condition from 40 percent to 10 percent. The length of the total core network as of mid-2017 was 3,684 km, while the percentage of core network roads in poor condition at project closure was 10.4 percent (383/3,684). The length of the state core network was reduced from 4,400 in 2006 to 3,684 in 2017 due to reclassifications of State Highways to National Highways. Using the 4,000 km figure, the result achieved by 2017 would be 9.6 percent. • The overall road user satisfaction index score for all roads in 10 of the state’s 12 districts went up from an average of 2.5 in 2007 to 3.3 in 2016. The ICR (page 12) notes that it is difficult to attribute these higher scores to the project, as there were other road improvements implemented in HP during 2007- 2017, including the Rural Road Project (2004-2010), the Prime Minister's Rural Road Scheme (RRP/PMGSY) (2010-2107), and other carried out and financed by GoI and HP State resources. However, the road user satisfaction survey of 2016 also collected data for eight of the project- upgraded roads, and found that the average score for the users of the project’s upgraded roads reached 3.619, surpassing the target of 3.0. It was not possible to obtain the source of the pre-project satisfaction level of 1.5 cited in the project appraisal document. The satisfaction scores are not higher because traffic volumes increased substantially over the period, leading to traffic congestion and other concerns such as inadequate parking and street lighting. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale The overall efficacy rating is substantial as both objectives were substantially achieved, with shortcomings in the implementation of capacity enhancement activities in road maintenance, financing, and management. Page 5 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic Analysis. An economic analysis was performed separately for two categories of road works financed under the project— upgrading works (Component 1) and periodic maintenance (Component 2). For upgrading, the economic analysis was performed for 435 km of roads, which were upgraded between 2008 to 2017 from single lane highway in poor condition (weighted average roughness of 9.7 m/km) to two lane highway in good/excellent condition (weighted average roughness in 2017 is 3.7 m/km). These roads were divided into 10 packages/segments. The HDM4 model was used to perform the economic analysis for these roads. For the roads that received periodic maintenance, analysis was performed for a sample of 140 km of roads divided into seven packages/segments. In this, case the Roads Economic Decision model was used. The economic benefits of both categories of road works were estimated from: (a) time savings benefits (Value of Time/VoT) and (b) reduction in VOC. For the road upgrading works, the overall EIRRs at appraisal and completion (based on weighted average) were 26.8 percent and 25.9 percent, respectively. Gains from higher than expected traffic volumes were mostly offset by the cost escalations. For periodic maintenance, the comparison to results at appraisal was not possible, due to little overlap between the roads evaluated at appraisal and those evaluated at the ICR stage. Despite this, the overall EIRR at appraisal was 26.9 percent, which is similar to the estimated EIRR at completion (27 percent) for a different set of roads. Administrative/Operational efficiency. Although the results of the ex post economic analysis are satisfactory, the project’s significant cost and time overruns reduced its overall efficiency. The costs of the road upgrading contracts, which made up a majority of total project costs, were 31 percent higher than anticipated. The main reasons for cost overruns were (i) slow procurement and difficulties in acquiring all the land needed for rights-of-way, obtaining forest clearances, and implementing utility shifting, leading to cost escalation of materials, especially bitumen; (ii) deficiencies of detailed designs, attributed to complexities of topography and underlying geology in mountainous terrain that were not thoroughly investigated beforehand and failure to carefully review the quality and accuracy of the detailed designs before incorporating them in bid documents; and (iii) the use of officially set unit costs (“schedules of rates”) for estimating costs for the detailed designs, which did not reflect the real costs of specific works in particular locations (ICR, page 20). In addition, the four extra years of project implementation substantially increased the project’s management costs, and several activities were not completed by the closing date. *Table (a) below presents the EIRRs for the road upgrading works. Page 6 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 78.00 Appraisal  27.00 Not Applicable 88.00 ICR Estimate  26.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is rated substantial as the project objectives were relevant to the road sector challenges of the HP State; the government, however, did not have a strategy or policy that set its development priorities in the road sector. Through upgrade works on the 355 km of the HP State's core road network, the project achieved substantially its two objectives of reducing transport costs and improving traffic flows on priority segments of the core road network of the State of Himachal Pradesh. There were shortcomings in the implementation of capacity enhancement activities in road maintenance, financing, and management. Despite satisfactory results of the ex post economic analysis, the project’s significant cost and time overruns reduced its overall efficiency, which is rated modest. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Financial. There is a risk of inadequate funding to properly maintain the assets created under the project. The recommendations of the 2006 Road Sector Finance Study, which had initially been received favorably by the GoHP, were not implemented. Institutional. There is a need for institutional strengthening to ensure maintenance of the assets. The Maintenance Action Plan and Institutional Strengthening Action Plan were adopted by the State as part of the Bank-financed rural road project but the results were limited to the development of simple maintenance management systems and the first steps toward allocating maintenance funds according to the priorities established under the annual plans. Page 7 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) 8. Assessment of Bank Performance a. Quality-at-Entry The project design was generally straightforward, however the scope of the civil works and institutional changes proved to be too ambitious for the capacity of the Road and Other Infrastructure Development Corporation (RIDC) of the Public Works Department (PWD). As part of the project preparation, PWD undertook a study on road sector finance, which projected a significant funding gap for road upgrading and maintenance if the state continued to rely on the traditional sources and recommended establishing a dedicated road funding mechanism through the combination of road user charges and public-private partnerships. The project design also included a number of challenging elements, namely the establishment of a new road maintenance system and reform of the institutional arrangements for road asset management; these changes were not achieved in the end. The Bank overestimated the willingness of the GoHP to implement major policy and institutional reforms in the roads sector, as it relied on the high level of support expressed by the GoHP leadership in place at the time of project preparation. Several other risks were underestimated that included risks of high staff turnover and inadequate skills in RIDC, inadequate supply of contractors and consultants, construction delays and cost overruns. At the same time, at preparation, GoHP and the Bank made efforts to ensure that the project met the implementation readiness steps for appraisal, to address the pervasive implementation delays in road projects in India. These included: (a) adequate staff; (b) an Operations Manual; (c) advanced start of procurement of works and key supervision services; (d) advanced action on land acquisition, regulatory clearances and utility relocation; (e) signing of MOU between the implementing entity and utility companies; (f) early mobilization of an NGO to support resettlement and rehabilitation of project affected people; and (g) training to the implementing agency and auditing staff on application of FIDIC (International Federation of Consulting Engineers) conditions of contract (ICR, page 17). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Bank supervision was regular; there were a total of 18 well-staffed supervision missions. The Bank’s efforts focused on the civil works, including oversight of construction, procurement and safeguards. The team consistently made field visits to obtain firsthand information and provide advice on the road works and associated issues. The ICR (page 24) points out that the Bank did not give enough importance to institutional development issues during project implementation. The team’s primary focus on the upgrading and maintenance works resulted in delayed implementation of the web-based RADMS and the computerization of PWD. There were important issues— notably performance of a contract supervision consultant, road maintenance financing, and PWD/RIDC structural reforms—which were not resolved, and were only monitored and commented on. No timely measures were undertaken to address extreme delays under the PBM contracts and blackspot improvement contracts. The team was not proactive on the 'soft' issues and focuses largely on "business as usual' engineering. Page 8 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) The systemic issue that RIDC was to take over management of the state’s core road network was agreed, but not acted upon. Contractors’ performance highlighted in the first supervision meeting in 2007 was not properly addressed and did not improve much. There was a major financing gap due to cost overruns discussed at mid-term review that led to the request for AF. It is not clear why there was no special workshop to discuss some of these matters, and why these problems were not reported to higher management for more drastic steps to be taken. In addition, the Bank team did not correct shortcomings in the original results framework, and there were some outstanding issues in social safeguards that were expected to be completed after project closure. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The two objectives of reducing transport costs and improving traffic flows were clear and linked to the rehabilitation works on the core network. These, however, were limited in scope and did not reflect the institutional focus of several project activities. There was no indicator identified for transport costs reduction. No key outcome indicators were related to Component 2, Core Network Maintenance and Management, which accounted for nearly US$60 million or about 22 % of total project costs. The original baseline and target values were found to have been incorrect, as they were taken from the National Crime Records Bureau and pertained to the entire State road network, not specifically to the State Highways. Prior to the project, a computerized data base of PWD’s road and structural assets was being developed, including data on road condition and traffic for about 75 percent of the PWD network. Responsibility for M&E was vested with RIDC. However, the data collection and tabulation activities for M&E were to be contracted out to consultants. There was no M&E focal point in the RIDC project staff at appraisal. b. M&E Implementation RIDC monitored and updated the outcome and intermediate indicators periodically during the life of the project. Four intermediate indicators (awareness by construction workers and road users of HIV/AIDS prevention, setting up quality assurance system, development of accounting policies and procedures, and implementation of the training plan) were not monitored or reported in ISRs. Page 9 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) c. M&E Utilization According to the ICR (page 22), M&E data were regularly used to assess project implementation and progress toward the project’s outcomes. At the time of mid-term review and the AF, the data were taken into account in making decisions on contract management, cost overruns, and project timing. M&E Quality Rating Modest 10. Other Issues a. Safeguards This was a safeguard category "A" project. According to the PAD (page 15), the five safeguard policies triggered were Environmental Assessment (OP/BP 4.01), Natural Habitats (OPBP 4.04), Physical Cultural Resources (OP/BP 4.11), Involuntary Resettlement (OP/BP 4.12), and Forests (OP/BP 4.36). Environmental. The ICR (page 22) reports that Environmental Management Plans (EMPs) were largely prepared on time, were of acceptable quality, and were implemented adequately.The Bank focused implementation support on bioengineering, and it maintained a strong emphasis on this throughout the project’s life. This helped to scale up the initial pilot to a practice of applying bioengineering to all the upgrading works. The project achieved the intermediate indicator target for the survival rate of trees planted; the target was 50 percent, and the actual was 69.75 percent (out of a total of 432,692 planted). The Bank team identified some instances of contractor negligence and non-compliance that caused environmental problems and that were subsequently corrected. During several months starting in late 2014, environmental performance was rated moderately unsatisfactory due to serious problems in contracts 5-I, 5-II, 6-I and 6-II with poor debris management, protection of forest lands, dust pollution, road safety, and workplace safety. Also, delays in obtaining forest clearances caused works to lag behind schedule. Social. Land acquisition and resettlement, which often was slow, primarily due to inadequate coordination between RIDC and the office of the Land Acquisition Officer (LAO), but also to other issues including inaccurate information about land holdings in Resettlement Action Plans (RAPs), mismatches between road design drawings and revenue maps, changes of alignments, delays in negotiations and disbursements, and other aspects of the complex R&R process. The ICR reports that the Bank team provided detailed due diligence and follow-up through field visits on resettlement and compensation issues, as well as other social safeguards matters as they arose. Of the total compensation due, 96.2% had been paid by the project closing date. Residual amounts unpaid on packages 3, 4, 5, 6 and 8 were due to factors beyond the control of RIDC: (a) demise of owners, with legal heirs still to be brought on record, (b) court cases or stays granted, and (c) disputes among co-owners. The ICR (page 23) informs that it is expected that these cases would be resolved. Of the total R&R assistance due, 93.5 percent had been paid by the closing date. Residual R&R amounts unpaid on packages 1, 2 and 6 were due to factors beyond the control of RIDC: (a) land owners not turning up to Page 10 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) receive payment despite notices, (b) court cases, and (c) beneficiaries not being traceable. b. Fiduciary Compliance Financial management. According to the ICR (page 22), FM performance was generally satisfactory during the life of the project, with some minor shortcomings. The Bank’s Financial Management Specialists provided thorough due diligence and follow-up, to help ensure that, inter alia, staffing gaps were filled, counterpart funding was regularly allocated, the FM Manual was updated, payment processing was expedited, financial reports and audits were prepared and submitted on time, and small irregularities highlighted in audits were addressed. Procurement. Procurement of works was generally slow during the first three years of project implementation. Procurement delays were due partly to RIDC’s capacity constraints, which were reflected in deficient bid packages and bid evaluations. The Bank did not do sufficient diligence early on regarding the “right sizing” of bid packages, as noted by the ICR (page 22). Procurement performance improved in 2010, as RIDC’s procurement capacity improved. In some instances, for example in the periodic maintenance contracts, delays were caused by poor bid response. The Bank assisted RIDC in addressing this by breaking down the packages into smaller ones. c. Unintended impacts (Positive or Negative) --- d. Other The project piloted and mainstreamed bioengineering as a cost-effective and environmentally friendly slope stabilization measure. Training and capacity building were provided in these techniques to HPPWD staff and laborers in various divisions. In addition to applying bioengineering for slope stabilization throughout the upgraded roads, the project also rehabilitated about 80 debris disposal sites using various bioengineering techniques. Bioengineering is now being applied to other HPPWD road projects. Work is ongoing to analyze and document each bioengineering technique and quantify its cost-effectiveness. Specifications for bioengineering in slope stabilization and protection were prepared and published in 2015, with copies distributed to all of PWD’s field Divisions (ICR, page 16). 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Moderately Moderately Satisfactory --- Page 11 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) Satisfactory Moderately Bank Performance Moderately Satisfactory --- Satisfactory Shortcomings in M&E included a missing outcome indicator for the objective of reducing transport costs, and Quality of M&E Substantial Modest lack of indicators at the outcome and intermediate level to capture the results under Component 2 as well as institutional activities. Quality of ICR Substantial --- 12. Lessons IEG has selected four lessons from the ICR, with some adaptation of the language: • To achieve systemic policy and institutional improvements in road financing and asset management, it is essential to engage proactively with governments on policy and institutional reform in an intensive and sustained manner. The project period spanned 10 years, however it largely failed in bringing meaningful reforms and structural changes in road financing and road asset management in HP. The project team focused on “business as usual” engineering approach and did not provide sufficient attention to institutional issues. The project is seen as a missed opportunity to focus on institutional reforms as the key element of the Bank’s value proposition. • When a project involves a substantial amount of road works, it is important to first focus on strengthening in-house capacity for technical quality assurance and contract management. Under this project, inadequate capacity in these areas led the implementing agency to rely unduly on the contract supervision consultant and the dispute resolution/arbitration mechanisms, which proved to be adversarial and ineffective. Flaws in detailed designs leading to major variation orders and cost escalations can best be prevented by the employer’s careful ex-ante quality review. The implementing agency needs to assess the key areas of competence required at each level of management and design a capacity building and technical assistance program in contract management and quality assurance. • The implementing agency and its head should be vested with appropriate authority for most management decisions. Under this project, the lack of independent authority of the chief engineer/project director in the implementing agency was a significant contributor to implementation delays. Most decision making was centralized at higher levels, including a board which did not convene often. Page 12 of 13 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: HP State Roads Project (P096019) • For road investments, a market study of local contractors should be carried out during project preparation to guide the packaging of works so as to maximize the participation of contractors and obtain the best value for money. The Bank’s Project Procurement Strategy for Development (PPSD) should in the future increase the likelihood of attracting well-performing bidders for road improvement contracts. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is concise, candid, and outcome- oriented. It provides a thorough and insightful analysis of factors that affected the project implementation. Its candidness is particularly notable with regard to the Bank performance. The quality of evidence is adequate. Lessons are based on the project experience and useful for Bank engagement in the country as well as other similar operations. The ICR should have stated what safeguard policies were triggered at appraisal and AF, and reported on the project compliance with Bank safeguards. a. Quality of ICR Rating Substantial Page 13 of 13