ICRR 14100 Report Number : ICRR14100 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/05/2014 Country : India Project ID : P077856 Appraisal Actual Project Name : Lucknow-muzaffarpur US$M ): Project Costs (US$M): US$851.24 US$805.97 National Highway Project L/C Number : L4764 Loan/ US$M): Loan /Credit (US$M): US$ 620.00 US$620.00 Sector Board : Transport Cofinancing (US$M): US$M ): Cofinanciers : Board Approval Date : 12/21/2004 Closing Date : 06/30/2010 06/30/2012 Sector (s): Roads and highways (96%); Other social services (3%); Forestry (1%) Theme (s): Infrastructure services for private sector development (50% - P); Injuries and non-communicable diseases (25% - S); HIV/AIDS (25% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : William B. Herbert Peter Nigel Freeman Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: According to the Project Appraisal Document (PAD) and consistent with the Loan Agreement (p.15), the Project Development Objective (PDO) was "for road users to benefit from an improved journey between Lucknow and Muzaffarput." However, as a result of poor performance the project was formally restructured in May 2010, At that time the PDO was revised "to improve the journey on NH-28 between Lucknow and the Uttar Pradesh (UP)-Bihar Boarder, and to support National Highways Authority of India (NHAI) in improving safety and sustainability of the project corridor." Subsequently there was a second restructuring in June 2012. However, there was no change to the PDO. Rather the second restructuring cancelled two poorly performing components . b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 05/04/2010 c. Components: The original project had only one component; Highway Upgrading (US$ 802.87 million at appraisal and US $800.12 million at completion). This component was aimed at removing the capacity constraints on the National Highway network by upgrading 483 km section of the national highway between Lucknow and Muzaffarpur . The component also included construction supervision, delivery of entitlements under a Resettlement Action Plan, and ancillary activities such as afforestation, monitoring and evaluation and road safety and HIV /AIDS awareness programs. In May 2010 the project was formally restructured . The Highway Upgrading component was reduced in scope by a third from 483 km to 328 km and from US$ 802.87 million to US$ 785.39 million. However, as part of the May 2010 restructuring, three additional components were added : 1) Strengthen the Road Safety Cell (US$ 2.75 million at restructuring and US$ 2.75 actual cost). This component addressed the deterioration of safety in construction zones by enhanced safety of designs, construction practices, network operations, and enforcement; 2)Develop Safer-Greener Highways Pilot (US$ 26 million at restructuring and US$ 0.0 actual cost). This component was designed to enhance safety and environmental measures designed to retrofit 55 km of existing six lane section of the Delhi-Panipat highway; and 3) Implement a new Enterprise Resource Planning (ERP) system (US$ 34 million at restructuring and US$ 0.0 actual cost). This component was designed to overcome shortcomings in the existing financial accounting and reporting systems by strengthening fiduciary controls through enhanced management and accounting systems . Prior to the project closing a second restructuring was carried out . Due to delays in procurement, the Safer -Greener Highways Pilot (US$ 26 million) and the Implementation of a new ERP system (US$ 34 million) were dropped from the project. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost: At appraisal the project cost was estimated to be US$ 805.97 million, but in the May 2010 restructuring three additional components were added and the estimate was thus increased to US$ 851.24 million. However, these components were not carried out and so in the June 2012 restructuring the cost reverted to the US$ 805.97 million. Financing: An IBRD loan of US $602.00 million (77% of the original project cost) was approved in December 2004. Borrower Contribution: At appraisal the borrower contribution was estimated at US$ 185.97 million (23% of project cost), but in the May 2010 restructuring this amount was revised to US$ 231.24 million. However, since the additional components were not implemented in June 2012 it was again reduced to US$185.97 million. Dates: . Due to procurement delays in a number of large civil works contracts and the restructuring of the project, the closing date was extended from 06/30/2010 to 06/30/2012. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Modest (original objective).. The original project objective was for "road users to benefit from an improved journey between Lucknow and Muzaffarpur ." At the time of appraisal the objective was in line with the Bank's Country Strategy and remained relevant throughout the project . For example, according to the 2013 Country Partnership Strategy, India foresees "a massive increase in infrastructure investment " over the next five years. However, as pointed out in the ICR (p.15), the Project Development Objective had a "singular focus on civil works " This single focus amounted to a missed opportunity to address the "foremost development priority in the transport sector ", namely, the institutional challenges facing the National Highways Development Project . These challenges included building up weak management and reversing a lack of skill sets in the areas of quality control, finance, procurement, and technical knowledge. The ICR notes that four previous projects had similar development objectives and implementation procedures, but, (as shown in Annex 13 of the ICR), two of those projects showed a large number of "moderately unsatisfactory" ratings. Given the past history with this sector, greater attention should have been given to defining the objectives to ensure they were achievable .. Substantial (revised objective) In May 2010, the original Project Development Objective was modified to "improve the journey on NH-28 between Lucknow and the Uttar Pradesh /Bihar Boarder, and to support National Highway Authority of India (NHAI) in improving safety and sustainability of the project corridor ." The road upgrading aspect remained valid and in addition the objective addressed the important issues of improved safety and sustainability . b. Relevance of Design: . The original design was narrowly aimed at removing capacity constraints on the Substantial (original objective). National Highway (NH) network by upgrading to a 4-lane divided carriageway standard 483 km of NH 28 between Lucknow and Muzaffarput. The project also funded the construction supervision teams; entitlements under a Resettlement Action Plan; and ancillary activities such as afforestation, monitoring and evaluation activities and road safety. These activities were straight forward, and sequenced correctly to achieve the limited project objective . Modest (revised objective) In May 2010, five years after approval, the project was restructured . The length of the project corridor was reduced from 483 km to 328 km. reflecting the realities of weak performance . Three new activities were added to the project design: i) Safer-Greener Highway Pilot; ii) Fiduciary Controls; and iii) Strengthen Road Safety Cell. Although the road upgrading aspect remained valid and some attempt was made to improve safety and sustainability, the fundamental institutional weaknesses were not addressed . 4. Achievement of Objectives (Efficacy): Because the project was formally restructured efficacy is assessed against both the original and revised project objectives. The original project objective was “to benefit road users by improving the journey between Lucknow and Muzaffarpur.� The revised PDO in May 2010 was to "improve the journey on NH-28 between Lucknow and the Uttar Pradesh/Bihar Border, and to support National Highway Authority of India (NHAI) in improving safety and sustainability of the project corridor ." Like for the original PDO there was no planned outcome - this was an output driven project. Under the restructured project, the scope of physical works was reduced from 483 km to 327 km. In addition, three new components were added to support the NHAI to improve safety and sustainability of the project corridor . These components included i) The Safer Greener Highways Pilot; ii) Strengthening the Road Safety Cell, and iii ) Implementation of a new Enterprise Resources Planning System . The project achieved its revised roadwork's output target (reflecting what was achievable ) by upgrading 328 km of highway to a four-lane standard with improved riding quality . The project was also able to achieve some of its targets in terms of reducing vehicle travel time, operating costs, and the number of fatalities and accidents, but for a reduced road length. For example, the target to reduce travel time by 20% was achieved by both cars and trucks (23% for cars and 25% for trucks). A 10% reduction in vehicle operating cost was achieved for cars, but not for trucks . In the case of fatal and non-fatal accidents the target of a 10% reduction was achieved in most, but not all, sections of the project corridor. The unqualified audit output target was also met . The overall efficacy at the end of the project against the original objective was modestly achieved , while against the revised objective it was substantially achieved for the highway upgrading although the three additional activities were not completed. 5. Efficiency: Modest The PAD (p.10) indicates an economic internal rate of return for the "most preferred alternative base cases " varied from 16% to 34% with an overall average rate of return of 26% and a net present value of US$ 1,018 million. As indicated in the ICR (p.18) the combined EIRR for the project corridor was 26%, (Annex 3.) While the EIRR and NPV shows that the civil works component after restructuring still gave a positive return, this disguises serious shortcomings in efficiency which include that 156 km. of highway were dropped from the project, there were significant cost and time overruns in the completion of the civil works, and the additional components added at restructuring were dropped . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re-estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal Yes 26% 88% ICR estimate Yes 26% 99% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of the project's original objective was modest, but reflected a missed opportunity to address the institutional weaknesses, while the relevance of design (to this limited objective) was substantial. The relevance of the project's revised objective was substantial, as it also addressed the issue of sustainability, and implicitly capacity . However the relevance of design to meet this objective was modest, because the fundamental institutional weaknesses were still not addressed . Having reduced the project corridor target by 156 km, the project managed to upgrade a little over 300 km of highways to meet the revised target . However, little if any progress was made in achieving other outputs, especially those linked to addressing institutional challenges . Efficacy was rated modest under the original objective and substantial under the revised objective . Efficiency was rated modest. Based on the percentage of disbursements made prior to the formal restructuring of the project, a relative value of 75% is given to the achievement of the original Project Development Objective (PDO), while the remaining 25% is applied to the revised PDO. a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: To sustain the improvement of the project corridor will require NHAI to further address its institutional weaknesses . This includes addressing weak contract management, monitoring, and developing a sustainable maintenance program. Although NHAI has funding for two year O&M contracts and has already initiated a procurement process to let them, it is uncertain whether NHAI will be in a position to maintain the upgrades through "Operate, Maintain and Toll concessions." Past experience under previous projects does not indicate the likelihood that NHAI will be successful. For example, the Third National Highways Project and the Grand Trunk Road Improvement Project both indicate that borrower performance was only "moderately satisfactory". In other areas, such as road safety, there is a risk that improvements gained under the project may not be sustained . In meeting some of the institutional challenges at NHAI, it is too early to determine if the separate Technical Assistance Project, designed to strengthen NHAI, will achieve its objectives. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The framework (Annex 3 of the PAD) included adequate outcome and intermediate results indicators for achieving the original PDO. For example, targets to reduce travel time, vehicle operating costs, and fatal accidents, were appropriate and established a causal link to project objectives . The PAD claimed that the Bank had taken into account lessons learned from previous projects in the transport sector in India . However, it appears that those lessons were not taken seriously . For example, the PAD (p.5) indicates that previous projects suffered from slow implementation due to delayed completion of pre -construction activities. The PAD also indicated that no further technical assistance was needed to support NHAI, but there was a need to emphasize the importance of improving capacity for better project management arrangements . However, it is clear that this was given insufficient thought and the project design was too ambitious for the capacity that existed . As indicated in the ICR (p.7) "in the initial 3-4 years of implementation several problems faced the project including prolonged delays in pre-construction activities, and delays due to weak contract management by NHAI ". At the strategic level, the project did not address any of the larger sector development priorities and according to the ICR "this diminished the project's strategic relevance and approach ." at -Entry Rating : Quality -at- Unsatisfactory b. Quality of supervision: The Bank's supervision team identified implementation problems early on and discussed them with NHAI . The team monitored progress and took some measures to keep the project on track . As a result, two thirds of road originally planned was completed and provided a positive return . Nevertheless, despite the team's efforts, according to the ICR, (p.22) "the Bank's supervision remained largely ineffective " in getting NHAI to resolve the operational issues. This may be related in part to the fact that while the Bank identified issues and made recommendations, there is little evidence to show that there would be consequences if action were not taken . Indeed, the only action taken was restructuring the project to enable NHAI to achieve less with the same level of funding. The restructuring did not require any serious commitment on the part of NHAI to improve its implementation practices. Had the restructuring taken place sooner it may have been possible to have achieved a better result, but by the time it happened 75% of the funds had been disbursed even though not one of the 12 packages had been completed . Arguably there was also fruitless expenditure since work had continued on some packages that were then cancelled . Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: In 1998 the GOI initiated a National Highway Development Project (NHDP) to strengthen and widen to four lane standard a core national highway network of about 14,300 km in a phased manner over 10 years. The total cost for the network was estimated to be US$ 14 billion. To date, over US$ 5 billion has been disbursed. Funding for this network is from domestic resources, tolls, private sector, and from domestic and international development banks. GOI continues to be committed to achieving the objectives of NHDP . The Lucknow-Muzaffarput National Highway Project represented one section of NHDP and GOI played an important part in the preparation of the project . According to the ICR the Borrower "extended a reasonable level of implementation support to the project ." However, GOI intervention could have been more forceful in ensuring NHAI addressed the operational issues that resulted in having to restructure the project twice . Based on the track record of previous IBRD supported projects, GOI should have done a better job in assessing the implementation arrangements and capacity of the implementing agency . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: NHAI was active in the preparation of the project but did not address many of the implementation issues nor address internal management issues such as weak contract management and failure to provide adequate staffing at the PIU. NHAI capitalized on restructuring the project in terms of reducing the length of the project corridor and completed work on 328 km. of highway with a positive return . However, NHAI failed to implement two components of the restructured project . Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: According to the PAD (p.7), NHAI already operated a satisfactory Management Information System which produced physical and financial progress reports . For the project the main data collection would be with the supervision consultants . While NHAI had in place a MIS, it is not clear that they had a good M&E system to monitor the PDO and related outcome indicators . Indicators were sufficient, given the limited scope of the project, however it is not clear how specific targets were identified . It is also not clear why monitoring of the indicators was only to be carried out through a survey at the end of the project with no intermediate checks that could have triggered course corrections. b. M&E Implementation: The civil works component was monitored by four Project Implementing Units, which were often understaffed . The ICR does not explain how the other components were monitored other than work carried out by the Bank's supervision team. c. M&E Utilization: Other than recognizing that a number of key contracts were behind schedule and performing poorly, there is no indication that data collected for other components were utilized . M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: The environmental safeguard category of the project is "A". According to the PAD (p.15), a full assessment was carried out and a suitable management instrument prepared . Safeguard Policies Triggered by the Project : Environmental Assessment (OP 4.01): An Environmental Assessment carried out during project preparation determined the potential adverse impact of the proposed road widening . An Environmental Management Plan was developed and implemented. However, compliance in the initial years of the project remained patchy and varied across the contract packages . At one point the Bank's supervision team rated compliance as "unsatisfactory" but this was later rated as "satisfactory" after remedial action. Cultural Property (OPN 11.03): Mitigation measures were established to safeguard Kushinagar, a World Heritage Site. The ICR does not specifically comment on this cultural property, but does indicate that by the time the project closed the rating for social safeguards, including 259 religious sites was "satisfactory". Involuntary Resettlement (OB/BP 4.12): Acquisition, displacement and resettlement impacts were high in the project : 442 ha of private land; 14,976 titleholders and 8,418 non titleholders as well as 259 religious properties were affected. During implementation there were mixed results with frequent changes in the level of compliance, ranging from satisfactory to unsatisfactory and back to satisfactory . According to the ICR (p.12) by the closure of the project all compensation issues had been resolved with a few exceptions which are now in the court system . However, it should be noted that the consultants hired to evaluate the implementation of the Resettlement and Rehabilitation program were not hired until the closing date . As such, the evaluation and survey is on -going and will continue to be monitored by the Bank until the final report is submitted . The ICR provides a range of ratings from past supervision reports for each safeguard . These ratings range from "satisfactory" to "unsatisfactory". The ICR also indicates (p.12) that in respect to land acquisition and resettlement, "all affected families received compensation and main entitlements, except for a few cases that were in court due to ownership disputes." However, the ICR does not state that all safeguards were met and were in compliance with Bank requirements. b. Fiduciary Compliance: Financial management performance of the project was mixed with unsatisfactory ratings in the initial phase of the project, although showing an improvement during the last two years . SOE audit reports were generally timely and unqualified. NHAI audit reports also improved over the course of the project . According to the ICR, despite such improvements, the Bank noted that there was still a significant financial risk . c. Unintended Impacts (positive or negative): No other unintended outcomes /impacts were identified. d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Unsatisfactory The relevance of the project's original Unsatisfactory objective was modest, while the relevance of design (to this limited objective) was substantial. The relevance of the project's revised objective was substantial, but the relevance of design to meet this objective was modest. Efficacy was rated modest under the original objective and substantial under the revised objective. Efficiency was modest. Based on the percentage of disbursements made prior to the formal restructuring of the project, a relative value of 75% is given to the achievement of the original Project Development Objective (PDO), while the remaining 25% is applied to the revised PDO. Risk to Development Significant Significant Outcome : Bank Performance : Moderately Unsatisfactory At appraisal the Bank should have Unsatisfactory focused more on implementation capacity and on project design . Given its knowledge of weaknesses in the borrower's institutional capacity, the project was too ambitious. During implementation, the task team could have been more forceful in respect to consequences for poor performance . Restructuring should have taken place earlier. ICR guidelines also require downgrading the overall rating if one component - in this case QAE - is rated "unsatisfactory". Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR offers a number of useful and important lessons . One particularly important lesson is the need to ensure institutional strengthening is accorded full attention, especially if the support for capacity building is funded from a separate operation. The Bank should ensure a proper appraisal of institutional needs and ensure there is appropriate funding for the necessary technical support . This part of the appraisal should be documented and recorded in the PAD and if funding is from a separate operation there should be an exchange of management letters with the Borrower to agree upon the use of funds and to ensure they will be monitored . During supervision missions and in the follow up management letter, it is important that supervision teams make clear that inaction or the failure on the part of the borrower /implementing agency to resolve issues will have consequences. Repeating the same issues with no follow up action does little to change the outcome . An action program that can be monitored needs to be agree upon and consequences for inaction need to be clear . When restructuring a project, the task team and borrower need to ensure any systemic issues that created the need to restructure are addressed . While this project was restructured twice, little attention was given to the causes for delays and poor performance . Restructuring should not be a tool to reward poor performance . Components should not be dropped from a project one month prior to the closing date . A Simple design does not ensure smooth or effective implementation . Because this project had only one objective, the design of the project was straight forward, but relied too much on "standard implementation arrangements that had worked satisfactorily for other ongoing Bank funded projects will largely continue for this project ." Unfortunately, this was an "easy out" for the Bank's task team. The Bank needs to take seriously the implementation arrangements in the design of all projects . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is well written and provides a sound basis for rating the overall performance of the project . The TTL has a good understanding of ICR guidelines . For example, in dealing with a project that was restructured twice, the TTL was able to come to a fair assessment of the project by using the "weighting system." The ICR does a good job in taking the reader through what starts as a simple /straight forward project to a poorly executed project and explains the reasons for poor performance . The one shortfall is that the ICR does not specifically state that all safeguards were met and were in compliance with Bank activities, although this is implied . a.Quality of ICR Rating : Satisfactory