Document of The World Bank Report No.: 83251-PL. PROJECT PERFORMANCE ASSESSMENT REPORT POLAND ROAD MAINTENANCE AND REHABILITATION PROJECT (LOAN NO. 72230 POL) SECOND ROAD MAINTENANCE AND REHABILITATION PROJECT (LOAN NO. 72820 POL ) ROAD MAINTENANCE AND REHABILITATION III PROJECT (LOAN NO. 73840 POL ) December 18, 2013 IEG Public Sector Evaluation Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Polish Zloty (PLN) 1995 US$1.00 PLN 2.5 1996 US$1.00 PLN 2.8 1997 US$1.00 PLN 3.5 1998 US$1.00 PLN 3.5 1999 US$1.00 PLN 4.2 2000 US$1.00 PLN 4.3 2001 US$1.00 PLN 4.0 2002 US$1.00 PLN 3.9 2003 US$1.00 PLN 3.8 2004 US$1.00 PLN 3.1 2005 US$1.00 PLN 3.3 2006 US$1.00 PLN 2.9 2007 US$1.00 PLN 2.5 2008 US$1.00 PLN 3.0 2009 US$1.00 PLN 2.8 2010 US$1.00 PLN 3.0 2011 US$1.00 PLN 3.4 Abbreviations and Acronyms EBRD European Bank for Reconstruction IEG Independent Evaluation Group and Development IBRD International Bank for EIB European Investment Bank Reconstruction and Development ERR Economic Rate of Return PAD Project Appraisal Document EU European Union PPAR Project Performance Assessment GDDKiA General Directorate for National Report Roads and Motorways PLN Polish Zloty HDM-4 Highway Development and SIL Sector Investment Loan Maintenance SWAp Sector-Wide Approach ICR Implementation Completion and Results Report Fiscal Year Government: January 1 to December 31 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Public Sector Evaluation : Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation : Ms. Marie Gaarder Task Manager : Ms. Midori Makino iii Contents Principal Ratings ................................................................................................................. v Key Staff Responsible........................................................................................................ vi Preface................................................................................................................................ ix Summary ............................................................................................................................ xi 1. Background and Context................................................................................................. 1 2. Road Maintenance and Rehabilitation Project (2004-2009) ........................................... 6 Objectives, Design, and Relevance ..................................................................................... 6 Implementation ................................................................................................................... 9 Achievement of the Objectives ......................................................................................... 11 Efficiency .......................................................................................................................... 14 Ratings .............................................................................................................................. 14 3. Second Road Maintenance and Rehabilitation Project (2005-2009) ............................ 16 Objectives, Design, and Relevance ................................................................................... 16 Implementation ................................................................................................................. 18 Achievement of the Objectives ......................................................................................... 19 Efficiency .......................................................................................................................... 20 Ratings .............................................................................................................................. 20 4. Third Road Maintenance and Rehabilitation Project (2006-2011) ............................... 22 Objectives, Design, and Relevance ................................................................................... 22 Implementation ................................................................................................................. 25 Achievement of the Objectives ......................................................................................... 27 Efficiency .......................................................................................................................... 30 Ratings .............................................................................................................................. 30 5. Lessons .......................................................................................................................... 32 References ......................................................................................................................... 35 Annex A. Basic Data Sheet ............................................................................................... 37 Annex B. National Roads Expenditures, Financing, Outputs and Condition 2003-2010. 46 Annex C. Trends in Road Traffic Fatalities in EU Countries 2001-2011 ........................ 47 Annex D. Use of HDM-4 under Budget Constraints ........................................................ 48 Annex E. List of Persons Met ........................................................................................... 50 This report was prepared by Midori Makino and George Tharakan (consultant), who assessed the project in February 2012. The report was peer reviewed by Isabel Chatterton and panel reviewed by Peter Freeman. Romayne Pereira provided administrative support. iv Boxes Box 1: Poland – Integrated Road Safety Program GAMBIT 2005 .................................... 5 Tables Table 1: Prior Projects in Poland Road Sector Since the 1990s ......................................... 2 Table 2: Project Components, Costs and Sub-components ................................................ 6 Table 3: Planned and Actual Project Costs by Component (US$ Millions) ..................... 11 Table 4: Project Components, Costs and Sub-components .............................................. 16 Table 5: Planned and Actual Project Costs by Component (US$ Millions) ..................... 19 Table 6: Project Components, Costs and Sub-components .............................................. 23 Table 7: Planned and Actual Project Costs by Component (US$ Millions) ..................... 27 v Principal Ratings Road Maintenance and Rehabilitation Project ICR* ICR Review* PPAR Outcome Moderately Moderately Satisfactory Moderately Satisfactory Unsatisfactory Risk to Development Moderate Moderate Significant Outcome Bank Performance Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Moderately Moderately Moderately Satisfactory Performance Unsatisfactory Unsatisfactory Second Road Maintenance and Rehabilitation Project ICR* ICR Review* PPAR Outcome Moderately Moderately Satisfactory Moderately Satisfactory Unsatisfactory Risk to Development Moderate Moderate Significant Outcome Bank Performance Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Moderately Moderately Moderately Satisfactory Performance Unsatisfactory Unsatisfactory Road Maintenance and Rehabilitation III Project ICR* ICR Review* PPAR Outcome Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Risk to Development Significant Significant Significant Outcome Bank Performance Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Performance * The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. vi Key Staff Responsible Road Maintenance and Rehabilitation Project Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Michel Audige Motoo Konishi Roger Grawe Completion Radoslaw Czapski Henry G.R. Kerali Theodore O. Ahlers Second Road Maintenance and Rehabilitation Project Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Anca C. Dumitrescu Motoo Konishi Roger Grawe Completion Radoslaw Czapski Henry G.R. Kerali Theodore O. Ahlers Road Maintenance and Rehabilitation III Project Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Anca C. Dumitrescu Motoo Konishi Daniela Gressani Completion Radoslaw Czapski Henry G.R. Kerali Peter C. Harrold vii IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. ix Preface This is the Project Performance Assessment Report prepared by the Independent Evaluation Group (IEG) for a sequence of three road sector projects in Poland: the Road Maintenance and Rehabilitation Project (Loan 72230), the Second Road Maintenance and Rehabilitation Project (Loan 72820), and the Road Maintenance and Rehabilitation Project III (Loan 73840). The three loans supported rehabilitation of Poland’s national road network and the strengthening of road sector institutions at the time of Poland’s accession to the European Union. These three International Bank for Reconstruction and Development (IBRD) loans to the Government of Poland were approved by the Board of Directors on March 30, 2004, March 29, 2005, and June 6, 2006, with loan amounts of USD 126.0 million, USD 130.5 million, and USD 180.2 million, respectively, and became effective within three months of approval. The three loans were designed as hybrid operations; the physical road works were financed using a Sector Wide Approach (SWAp) with Bank funds pooled with funds from the national budget, and contracts were procured using the Government’s own systems, while the institutional development components were financed as Specific Investment Loans (SIL) using Bank standard procurement and financial management arrangements. At appraisal, their total project costs were estimated to be USD 175 million, USD 210.5 million, and USD 376.2 million, respectively. The third loan was co- financed by the European Investment Bank (EIB) with a loan of USD166.0 million. For the first two Loans (72230 and 72820), the closing dates were extended by 18 months to December 31, 2008 and June 30, 2009. The third loan closed as planned on September 15, 2011. These three projects were selected by IEG for review as input into the IEG evaluation, “Improving institutional capability and financial viability to sustain transport”. They also represent a rather innovative approach, with fast disbursing physical works completed in about a year from loan approval, and much slower institutional development components which took longer and overlapped the other loans during implementation. IEG prepared this report based on an examination of the Project Appraisal Documents, the Implementation Completion and Results Reports, the Legal Agreements, project files and archives, as well as other relevant reports, memoranda and working papers. Discussions were also held with Bank staff in Washington, DC and in the resident mission in Warsaw. An IEG field mission visited Poland in January/February, 2012 to review the results on the ground and to hold discussions with relevant government officials, and other sector stakeholders. The mission appreciates all support and attention given by the borrowers and all concerned parties in Poland as well as in Washington, DC. Following IEG practice, a copy of the draft Report was sent to government officials and borrowing agencies for their review but no comments have been received. xi Summary This Project Performance Assessment Report (PPAR) assesses the development effectiveness and sustainability of a sequence of three loans that supported maintenance and rehabilitation of Poland’s national road network. The first loan was approved in 2004 and the last loan closed in 2011. Over this eight year period, these loans supported Poland’s national roads agency, GDDKiA, which is responsible for the approximately 14,000 km national roads network. They also supported the National Road Safety Council in a comprehensive program to improve road safety and reduce road accident fatalities. Combined, these loans provided about USD 422 million in financing for a total project cost of about USD 760 million. European Investment Bank co-financed the last loan with another USD 166 million equivalent of funding for the rehabilitation program. Preparation of the Road Maintenance and Rehabilitation program started on the eve of Poland’s accession to the European Union (EU) in 2004. The program anticipated the large infusions of EU funding for road improvements and the consequent need to strengthen GDDKiA implementation capacity. It also funded rehabilitation expenses that would not be eligible for EU funding, in order to reduce the large back-log of roads in poor condition. At the start of the program only 37 percent of the national road network was in good condition and 30 percent was in poor condition; by the end of the program the target of 60 percent in good condition had essentially been achieved, but about 18 percent of the network remained in poor condition. The first loan, for the Road Maintenance and Rehabilitation Project (Loan 72230), was approved in 2004, and financed the rehabilitation of 917 km of national roads, development and implementation of a Modernization Action Plan for the national roads agency GDDKiA, a Road Safety Improvement Program, and Technical Assistance to Increase Public-Private Partnerships in the road sector. The Second Road Maintenance and Rehabilitation Project (Loan 72820), approved in 2005, financed the rehabilitation of 749 km of national roads and the development of a Management Information System for GDDKiA. The Road Maintenance and Rehabilitation Project III (Loan 73840), approved in 2006, financed the rehabilitation of 1,103 km of national roads, implementation of the Management Information System, Road Safety and technical assistance to Ministry of Infrastructure (formerly Ministry of Transport and Construction). Together these three loans rehabilitated about 2,700 km of national roads, or about 17 percent of the national roads network, over an 8 year period. The loans were designed as hybrid operations; the physical road works were financed using a Sector Wide Approach (SWAp) with Bank funds pooled with funds from the national budget and contracts procured using the Government’s own systems, while the institutional development components were financed as a Specific Investment Loan (SIL) using Bank standard procurement and financial management arrangements. All three projects disbursed well on the civil works components, with the majority of the disbursements completed within a year of loan effectiveness. These components were implemented using Poland’s own systems for procurement and financial management, with the only restriction being that individual contracts be valued under EUR 6 million. This resulted in numerous small contracts with the average contract being for a road xii section of about 5 km valued at EUR 2.4 million. While these contracts would be considered small for international contractors they appear to have suited the capabilities of the local construction industry. Technical audits done after contract completion found the works to be of good quality. Restricting the value of individual contracts adopted for the civil works may therefore be considered to have contributed to the quality of the roads. Road safety had been a long standing concern, and the Government had initiated a comprehensive program, GAMBIT, to tackle the problem. In 2003, just before EU accession, road safety in Poland was among the worst in the EU 25 countries. At the time annual fatalities from road accidents exceeded 5,500 and rapid increases in the vehicle fleet, following EU accession, threatened to push these numbers much higher. By 2010, though the vehicle fleet had grown by about 60 percent, the number of fatalities had declined to about 3,900 and annual fatality rates per 10,000 vehicles had declined from 3.20 to around 2.45. Despite this progress, Poland’s road traffic fatality rate still remains one of the highest among the EU countries in 2011. GDDKiA’s institutional development was also a key objective of the Road Rehabilitation and Maintenance program. Essentially this aimed at improving institutional capacity to plan and implement large volumes of road works efficiently with adequate internal controls. To this end the program supported the introduction of a modern Management Information System and the planning tools needed to plan and evaluate programs for the rehabilitation of the network and for effective management of new construction works. The program also developed plans to restructure organizational arrangements to achieve higher efficiencies in working procedures. This involved decentralization of some responsibilities to GDDKiA’s regional branches and equipping them with the necessary systems to undertake these responsibilities. The Management Information System development was delayed and it is only now being rolled out to the various branches of GDDKiA. The entire system is to be connected over a wide area network. Likewise the planning and evaluation capabilities at headquarters and at the branch level have been enhanced by the introduction of tools such as the Highway Development and Maintenance (HDM-4) model which allows the evaluation of road rehabilitation plans to ensure their cost effectiveness. A review using HDM-4 found that median Economic Rate of Return (ERR) for rehabilitation projects was 37 percent. However, use of HDM-4 to ensure the efficiency of the rehabilitation program was only partially effective. While it is important to ensure that individual rehabilitation projects have an adequate ERR, it is also important to clear the maintenance backlog in a reasonable period of time within available budgets. This requires that maintenance budget constraints be factored into the choice of maintenance interventions. The roads in poor condition at the start of the program in 2003 totaled about 30 percent of the national roads network. In 2011, about 18 percent of the network was still in poor condition. This means that over an 8 year period the program had not managed to eliminate the maintenance backlog. This shortcoming relates principally to the shortfalls in maintenance budgets, and could have been avoided if the HDM-4 analysis had incorporated budget constraints and chosen rehabilitation alternatives that eliminated the maintenance back-log within the available budget envelope, even if this meant sacrificing some pavement life. xiii Another important objective of the program was to establish a reliable and stable source of funding for maintenance of the national road network. At the time of preparation of the Road Rehabilitation and Maintenance program, public expenditure on the national road network was expected to grow from US$2 billion in 2005 to US$6 billion a year by 2010, with the bulk of the funding coming from the EU. For the most part, however, the EU funds were not available for rehabilitation of the existing network. What was also not expected was that the counterpart funds needed to utilize the EU funds would strain the Government’s financial resources for the road sector. The National Road Fund which was to have provided a stable source of maintenance funding was restricted to funding new construction only in order to support the EU funded program. Consequently the Road Rehabilitation and Maintenance program did not succeed in creating a reliable mechanism for funding road maintenance from the Road Fund. The road networks were expected to be maintained with financial allocation from the state budget which often fell short of the required level. This remains a deficiency in the achievement of a sustainable system for the maintenance of the national road network. Ratings Outcomes for both the first and second project were rated “Moderately Unsatisfactory” due primarily to their deficiency in the achievement of objectives. Both projects had stable and reliable funding of maintenance as an objective, and this was not achieved when the National Road Fund was diverted to funding new construction only. The third project had substantially achieved its objectives, and demonstrated economic efficiency. Consequently the outcome rating is moderately satisfactory. Risk to development outcome is rated significant for all three projects largely on account of the uncertainty in ensuring a reliable and stable source of funding for maintenance of the network. Reductions in maintenance funding for the years 2010 and 2011 have already resulted in the proportion of the network in good condition dropping from 59.6 percent in 2009 to 58.7 percent in 2011 although it went back up in 2011 to 62.7 percent after the government decided to fund the overdue periodic maintenance. Continuation of such ad hoc implementation of periodic maintenance not only risks losing the gains in network condition achieved so far, but also risks dissipating the institutional capacity for maintenance developed under the Road Rehabilitation and Maintenance program. Lessons The key lessons derived, based on the findings of this PPAR, are the following: i. When planning sequential operations that involve sector reforms, it is important to ensure that sufficient disbursement leverage is retained to ensure the satisfactory implementation of the institutional development components, as well as a realistic time-frame for the operation. In this case, the implementation of three Road Rehabilitation and Maintenance projects were phased over three consecutive years. The disbursements were fast but the Bank experienced difficulty leveraging over the policy actions and institutional development programs which took longer to mature. xiv ii. The use of country systems, combined with adequate ex-post audits can contribute to fast disbursement of quality works. The Sector Wide Approach adopted for the implementation of civil works using the borrower’s own systems worked well in a country where there is already a functioning system. Fiduciary and safeguards oversight based on ex-post audits of the efficacy of internal controls are particularly useful as they also help the government improve the internal controls of their own managed programs. Quality control based on third party technical audits also proved adequate and efficient. iii. It is crucial to take into account the budget constraints when using HDM-4 analysis. Using HDM-4 to analyze only the rates of return on rehabilitation projects ignores the financial viability aspect of the program and pays little attention to reducing the roads in poor condition. The rehabilitation program should be designed to deliver a maintainable network within a reasonable time frame, and this requires taking into consideration a realistic budget envelope over the program period. iv. Objectives are more likely to be achieved when they have comprehensive monitoring indicators. While proper indicators were set for most objectives, there were no indicators linked to the objective of achieving reliable and stable funding of road maintenance or financial viability. This may have contributed to the program’s difficulty in achieving significant progress on this sub-objective. Caroline Heider Director-General Evaluation 1 1. Background and Context 1.1 Poland has experienced high economic growth in recent years, relative to other European economies, and is expected to rapidly emerge as a leading member of the European Union. With a population of just under 40 million, a Gross National Income per capita of USD 6,800 which is well above the Upper Middle Income Country average, and strategically located between the European Union (EU) and Russia, Poland is positioned to become one of Europe’s major economies. Since the year 2000, Poland experienced 4 percent annual average gross domestic product growth, as compared with the EU-27 average of negative 3.5 percent. Over the same period, cargo and passenger transport grew even faster at around 6 percent and 4 percent per annum, respectively. This pace of growth placed increasing demands on Poland’s already deficient transport systems. Added to this was the “quality” demands placed on its transport infrastructure upon becoming a member of the EU. 1.2 Accession to the EU in May 2004, added some urgency for Poland’s road sector to meet the higher standards of performance expected of EU member countries. These have included modernizing its road administration systems to enhance the quality and physical condition of the road network, integrating key trans-European corridors in Poland with the EU network, for instance by enhancing the axle load capacity to the 11.5 tons/axle EU standard, improving road user perceptions of the quality of infrastructure services, and improving road safety performance. The objectives of the three Bank financed projects evaluated in this report, the Road Maintenance and Rehabilitation project, the Second Road Maintenance and Rehabilitation project, and the Third Road Maintenance and Rehabilitation project, which were initiated in 2004, 2005 and 2006, respectively, mirrored these challenges. 1.3 Furthermore, as a candidate to the Euro Zone, Poland needed to meet the convergence criteria of the European Commission Treaty, which added constraints on public spending and investment, including for transport. Government finances of Euro Zone countries are evaluated on two criteria: (i) the annual government deficit - the ratio of annual government budget deficit to the Gross Domestic Product which may not exceed 3 percent; and (ii) government debt – the ratio of gross government debt to Gross Domestic Product, which should not exceed 60 percent. These requirements limit the potential for expanding public spending, even in critical areas such as badly needed maintenance of the road network. 1.4 While significant amounts of EU structure and cohesion funds were available as grants to develop those parts of Poland’s national road network that integrate with major Trans-European corridors, the lack of institutional capacity to implement such large programs of works was a major constraint. The Bank in its assistance strategy therefore focused on strengthening the institutional capacity of the national road agency GDDKiA to implement a large-scale investment program funded by the EU, to maintain and improve the condition of the rest of the 14,000 km national roads network that fell outside of the EU programs, and to undertake programs to improve road safety, which had become an increasing concern. Many of the systems and organizational arrangements in the road sector 2 were legacies from the previous centrally managed era and suffered inefficiencies such as expensive force account maintenance, which also needed to be addressed. Prior Support by the World Bank and Other Multi-lateral Development Banks 1.5 Since the beginning of the 1990s, the World Bank and the European Union have been active in providing support to the Polish road sector. As shown in Table 1, the World Bank has assisted through three projects during this period, prior to the three loans reviewed in this PPAR. European institutions, i.e. the European Investment Bank (EIB) and the European Bank for Reconstruction and Development (EBRD), also provided support to the sector through three operations which mainly focused on motorway development. Table 1: Prior Projects in Poland Road Sector Since the 1990s Organization Projects World Bank First Transport Project (1990) – Sector Restructuring and Management Reform Roads Project (1992) – Roads Network Management, Road Safety and Combined Transport Second Roads Project (1998) – Road Network Development and Modernization European Investment Highways Development – Highway Project Bank Motorways Development – A-2 Project and A-4 Extension European Bank for Motorway Development Project Reconstruction and Development Source: Road Maintenance and Rehabilitation Project Appraisal Document 1.6 Despite this earlier support, in 2003 a major proportion of Poland’s national road network was in an unsatisfactory condition (with 30 percent of the network in “poor” condition), and institutional capacity to implement works programs was weak. During the 2004-2006 period, the National Roads Agency (GDDKiA) was only able to implement 16 percent of the budget allocated by the EU for road works. 1 Fully utilizing the funds available from the EU for road works would require that Poland augment its implementation capacity. Likewise for road safety, Poland faced a major challenge. In 2003, its road traffic injury fatality rate was double the EU average. The situation was exacerbated by rapidly increasing motorization, following EU accession, which placed heavy demands on the road network, as integration with the EU and introduction of key reforms caused rapid increases in both freight and passenger traffic. 1.7 It was recognized even during the EU accession process, that institutions other than the Bank such as the EU, EIB and EBRD, were likely to be the longer term partners for 1 Poland: Transport Policy Note – Toward a Sustainable Land Transport Sector, World Bank, Washington D.C. 2010. 3 financing the large investment needs of Poland’s road network. 2 However, their envisaged large-scale assistance was likely to be constrained by Poland’s weak institutional capacity in the road sector. Consequently, the World Bank’s widely recognized expertise and experience in road sector institutional development was requested as a useful precursor to the assistance planned by the various European institutions. National Roads Network Improvement 1.8 In 2001, the proportion of national road network that was in poor or unsatisfactory condition totaled 71.5 percent of the network - that is, only 28.5 percent of the network was in good condition at that time. By 2003, the latter figure had improved to 40 percent, while roads in poor condition stood at about 30 percent. It was also foreseen that upon accession to the EU and with the infusion of EU funds to support road investments, there would be heavy demands on the road sector budget to provide counterpart funds for the EU investment program, which would squeeze maintenance budgets. This was due to EU funds being available only for network improvements and not for rehabilitation. It was in this context that Government and the Bank initiated preparation of a new Road Maintenance and Rehabilitation program for Poland. It was agreed that the Bank’s financial support to the Government’s medium-term roads program would focus on rehabilitation, and would be provided through three sequential loans scheduled one year apart. These are the three projects reviewed in this PPAR. The third project, approved in June 2006, closed in September 2011. 1.9 In preparing these projects, three approaches were considered by the Bank team and the government for this sequence of investments: a Private Sector Adjustment Loan, a Sector Adjustment Loan, or a Sector-Wide Approach (SWAp)/Specific Investment Loan (SIL) hybrid. The first two approaches were found to be unworkable due to difficulties with Poland’s procurement law and financial management systems which were not acceptable under the Bank’s policy guidelines. Under the hybrid approach, which was eventually adopted, funds for road maintenance and rehabilitation works would be pooled with those from the national budget. The works would be implemented using Poland’s own country systems for procurement and financial management with individual contracts limited to EUR 6 million, while the institutional development components would be financed under standard Bank implementation arrangements. This made the loan a hybrid: using country systems for the physical works, which allowed quicker implementation; and Bank standard procurement and financial management for the technical assistance which allowed closer supervision of these more complex institutional development components. Fiduciary Arrangements 1.10 Because the Bank would rely substantially on Borrower systems, which in turn were being adapted to conform with relevant EU directives, project preparation included an independent review of the Public Procurement Law with a view to harmonizing it with EU directives and Bank policies. While EU directives are generally based on concepts from accepted international procurement practices, their overriding objective is market integration. Consequently, the amended Polish Procurement Law had to be adapted under this project in 2 Poland: Country Partnership Strategy, World Bank, 2003. 4 four areas: (i) no merit point based evaluation would be carried out; (ii) no domestic preference will apply under National Competitive Bidding; (iii) selection of consultants will be carried out as per World Bank guidelines; and (iv) the “best offer” will be defined as the “lowest evaluated bid 1.11 The financial management arrangements included clearly specified arrangements for funds flow and controls for three types of project expenditures: (i) the GDDKiA Modernization Action Plan which was a technical assistance component managed by GDDKiA; (ii) the Road Maintenance and Rehabilitation component which was managed by the respective GDDKiA branches, of which there are 16; and (iii) the Road Safety Improvement components managed by the National Road Safety Council. The pooling of funds provided by the national budget, the Bank, and other donors had a slight drawback in that it is not possible to determine precisely which assets were financed by the Bank. This is the natural and expected consequence of the pooled funds which is, however, justified by the benefits of using the Borrower’s own well established systems and processes, with minor adjustments as needed to conform to Bank policies and guidelines. Third party technical audits have provided the necessary assurances of the quality of the works produced. The Bank’s reliance on the Borrower’s controls over quality of works appear to have been justified and the pooled approach validated. Road Safety 1.12 As noted earlier, Poland faced a major challenge in the area of road safety. An assessment in 2003 placed Poland’s road safety record 22nd among 25 EU member countries (GAMBIT, 2005). The problem of road safety was recognized by the Government in the early 1990s, and a National Road Safety Council was established in 1993 by an Act of Parliament. Box 1 below presents the evolution of the Government’s Integrated Road Safety Program (GAMBIT), which was launched in 2001, and the role played by the World Bank in this effort. Support for GAMBIT was therefore a key element of the road maintenance and rehabilitation program, which supported the latest version of the program, referred to as GAMBIT 2005. 1.13 While Poland managed to reduce fatality rate per 10,000 vehicles from 3.20 in 2005 to 2.77 in 2009 and 2.45 in 2010 (the quality of new vehicle fleets and other factors may have contributed to increased safety), and annual fatalities due to road accidents from around 5,500 in 2003 to 3,907 in 2010, but Poland’s road fatality rate remains to be one of the highest among the EU countries in 2012. In interpreting these figures, it may be noted that road fatalities in Poland are subject to significant annual variations due to factors such as winter weather conditions and the level of economic activity. 5 Box 1: Poland – Integrated Road Safety Program GAMBIT 2005 In 1992 a group of World Bank experts produced a report on road safety in Poland that claimed that road safety could be improved if systemic actions were undertaken by the Government (Gerondeau, C. et al. “Road Safety in Poland”, 1992). The Polish government acknowledged the importance of the report, and the National Road Safety Council was established in 1993. Soon thereafter, the Transport Minister commissioned the Scientific Research Committee of the Government to develop a research project, the Integrated Road Safety Program (GAMBIT). Subsequently, GAMBIT 2000 was adopted by the Council of Ministers in 2001 with the strategic objective of reducing annual road accident fatalities from an estimated number of over 6,500 in 2000 to less than 4,000 by 2010. In Poland road safety was thereafter made the direct responsibility of the Council of Ministers. When Poland joined the European Union in 2004, it ranked 22nd among the EU-25 member countries for road safety. GAMBIT 2005 was launched with the ambitious target of halving the number of road accident deaths within a decade. The program aimed to save more than 10,000 lives and protect several hundred thousand people from disability caused by road accidents. It was recognized, that the key to success would be a partnership involving central departments, government bodies, regional and local governments, as well as insurers, non-governmental organizations, educational institutions and others. The National Road Safety Council, as the high- level coordinating body, has been the instrument for making this partnership work. The objectives and priorities of GAMBIT 2005 are summarized as follows: • Build a basis for an effective and long-term policy – road safety organization, road safety management, sectoral actions (police, health, schools, etc.) • Develop safe road behavior – Speed limits, seat belts, alcohol related concerns • Protect pedestrians, children and cyclists • Build and maintain a safe roads infrastructure – evaluation, safe networks of roads and streets, traffic management and enforcement • Reduce accident severity and consequences – vehicle safety, forgiving roads (trees, poles), rescue of victims, emergency medical services. It was estimated that 40 percent of the overall goals would be achieved through infrastructure schemes (road construction, black spots improvement, and maintenance) and 60percent through other schemes (public education, driver training and licensing, traffic law enforcement and emergency services). Source: GAMBIT 2005 6 2. Road Maintenance and Rehabilitation Project (2004- 2009) Objectives, Design, and Relevance OBJECTIVES 2.1 The objective for the first project was stated in the Loan Agreement (LA) and Project Appraisal Document (PAD) as: “To improve the effectiveness of Poland’s national road rehabilitation and maintenance systems by: (i) significantly increasing the percentage of national roads in good condition; (ii) establishing reliable and stable funding for the national road rehabilitation and maintenance network and for road safety; and (iii) improving the capacity within GDDKiA to operate efficiently and effectively and to reflect the views of road users in developing its programs.” Thus, the project had an overall objective and three sub-objectives pointing to intermediate outcomes. COMPONENTS 2.2 The components and sub-components are presented in Table 2 together with their estimated and actual costs. There were four components: (1) GDDKiA Modernization Action Plan; (2) Road Maintenance and Rehabilitation Program; (3) Road Safety Improvement Program; and (4) Technical Assistance to Increase Public Private Partnerships. The physical works (component two) accounted for about 90 percent of the loan. Table 2: Project Components, Costs and Sub-components Components Sub-components (1) GDDKiA Modernization (i) Diagnose the current situation at GDDKiA, implement modernization Action Plan measures at headquarters, and develop procedures for preparation and Appraisal: US$2.86 million implementation of projects; (ii) Complete reorganization at the central and Actual: US$1.60 million regional levels, and finalize procedures for preparation and implementation of road projects; (iii) Establish a proper system of strategic planning; and (iv) Evaluate the efficiency of implemented measures, identify gaps and review the potential for further improvement. (2) Road Maintenance and The funds were utilized for rehabilitation of the network to address the Rehabilitation Program maintenance backlog, and maintain the balance between maintenance / Appraisal: US$131.60 million rehabilitation versus strengthening / new construction for which substantial Actual: US$133.31million funds were being provided from the EU. National budget funds for periodic maintenance and rehabilitation were pooled with resources provided by WB and EIB loans. (3) Road Safety Improvement Black spots remediation, traffic calming measures, equipment for the road Program traffic police, the fire brigade and ambulance services, and support for the Appraisal: US$6.44 million Road Transport Inspectorate. Program coordination by the National Road Actual: US$6.57 million Safety Council. (4) Technical Assistance to Design and procure long-term pilot contracts for performance – based Increase Public Private management and maintenance of roads. Partnerships Appraisal: US$0.71 million Actual: US$0.84 million Source: Project Appraisal Documents and Implementation Completion Reports 7 2.3 The GDDKiA Modernization Action Plan – the first component - had four stages. The first was to prepare a detailed diagnosis of the GDDKiA organization, with particular attention to the deficiencies causing the non-execution of planned programs during 2002 and 2003. A detailed re-organization plan was also prepared together with procedures for investment project preparation and execution. The second stage was the re-organization of GDDKiA. In the third stage, a system of Strategic Planning including a time-table for critical projects up to 2013, plans for motorway concessions, and a plan for strengthening selected sections of the national roads network to meet the 11.5 ton European axle load standard were prepared. The final stage reviewed the changes implemented and the need for further changes to increase the effectiveness of the GDDKiA organization. 2.4 The Road Maintenance and Rehabilitation component was designed to enable a large number of rehabilitation contracts to be implemented quickly using Poland’s own systems for fiduciary safeguards. Funds provided through the loan were pooled with loan funds from the EIB and allocations for road maintenance and rehabilitation in the national budget. World Bank funds were disbursed as a percentage (67 percent) of an agreed program of rehabilitation works, using local procurement procedures with all contracts being below EUR 6 million. This was designed as a fast-disbursing component to meet the urgent rehabilitation needs of the network. 2.5 The Road Safety Improvement Program was a broad ranging effort to tackle the high rate of road accidents in Poland, which had been estimated to cost the country as much as two percent of GDP annually. It supported GAMBIT, the Government program described earlier, which was coordinated by the National Road Safety Council, and involved in addition to the Ministry of Transport, the Ministry of the Interior, the National Police, Ministries of Health and Education, and others. It addressed accident black spots, provision of enforcement equipment for the traffic police, education of drivers and pedestrians, improved signals, public awareness, and other critical aspects of road safety. 2.6 The Public-Private Partnership component focused on developing a policy framework for out-sourcing road maintenance using performance-based management and maintenance of roads contracts and developing the needed contract documents. RELEVANCE 2.7 The relevance of the project objectives was Substantial. The Country Assistance Strategy of 2002 prepared by the World Bank at the time of Poland’s accession to the EU, recognized the importance of strengthening road sector institutions to enable absorption of the vastly expanded resources to be provided by the EU and other European institutions. The Strategy also noted that the incidence of unsafe and congested roads and high transport costs added to regional disparities and made the country less attractive to private investment. The project objectives reflected most of these challenges that country was facing at that time and established an appropriate role for the Bank’s comparative strengths. The Country Partnership Strategy of 2009 continued to note that while the EU and other European institutions, notably the EIB were the main partners and external sources of financing for 8 Poland's road sector, the Bank could continue to play an important role in helping to build institutions and in complementary investments. The first two projects, however, did not have an objective related to the much needed improvement in road safety as an EU member, but instead only included the funding for road safety to be part of reliable and stable funding objective. 2.8 The relevance of the project design with regard to its approach and the components financed to achieve the objectives was Modest. The hybrid approach used in the project design was an innovative response to an unusual situation. At that juncture, Poland urgently needed access to quick-disbursing road maintenance and rehabilitation funds to prevent a deterioration of the remainder of its road network not benefitting from EU funding. Since the requirements of the EU program were draining resources from the rest of the network, the Bank’s approach to pooled funding using the country systems was critical to stabilizing the situation. On the other hand, the institutional strengthening activities required a longer gestation and closer supervision of the technical assistance for which the SIL approach was more suited. The modernization program for GDDKiA also came just in time to meet the requirements of the EU program. There were, however inconsistencies between the objectives and the components and indicators. The project had a component on road safety and monitored related indicators although it did not have a road safety objective, and the project’s design did not address the sub-objective of achieving stable and reliable funding for road maintenance. This was critical to the sustainability of the road sector, and its absence from the project design proved to be a shortcoming. MONITORING AND EVALUATION 2.9 Monitoring and Evaluation (M&E) Design: The M&E framework for the project consisted of the following: (i) clear monitorable milestones for the Modernization Action Plan; (ii) effective and functioning data collection system that supports the use of HDM-4 by GDDKiA to track road condition and plan its maintenance activities; (iii) data collection on road traffic fatalities each year; and (iv) regular road user satisfaction surveys conducted by GDDKiA. The Modernization Action Plan did not include clear and specific milestones which could be readily monitored, but the other three activities had clearer frameworks and milestones for M&E. 2.10 These systems and tools were used to monitor the key performance indicators which were established at appraisal stage: (i) adoption and continued use of improved road maintenance and rehabilitation practices within GDDKiA; (ii) evidence of use of a coherent 6-year rolling roads expenditure program together with an adequately balanced maintenance/investment program; (iii) significant increase in the percentage of the road network in good condition when compared with the pre-project condition; and (iv) evidence of high road user satisfaction following rehabilitation or maintenance operations when users are polled, both about the quality of the completed sections and the inconveniences associated with the rehabilitation or maintenance process. 2.11 M&E relied on two arrangements. With regard to the condition of the road network, it relied on the regular data collection carried out by GDDKiA’s Department of Studies. This office maintains an excellent database of current road conditions, and is responsible for 9 carrying out analytical work using the Bank’s HDM-4 Model to evaluate maintenance and rehabilitation programs. It relies on the various regional branches to supply data that are directly integrated with the HDM-4 input software. The other aspect of M&E is the road safety statistics on road accidents and fatality rates. These data are collected by the National Road Safety Council, which maintains detailed data on accidents, injuries, fatalities, and the vehicle fleet to support the M&E design of the project. 2.12 An aspect of M&E which was missed and later proved to be important was any indicator in the M&E framework on ensuring a stable and reliable source of maintenance funding. This was a critical oversight, as the shortfalls in maintenance funding may jeopardize the sustainability of the improvements made under the road maintenance and rehabilitation program. 2.13 M&E Implementation: The data collection and application of the HDM-4 methodology has been successfully undertaken at the central level as well as at 13 of 16 regional branch offices. While some improvement in the methodology adopted for HDM-4 in planning maintenance is possible it is clear that implementation of the data collection system for road asset management has been reasonably widespread among GDDKiA branch offices. Likewise, the road accident/fatality data are well maintained by National Road Safety Council and provide a solid basis for M&E of the road safety aspects. 2.14 M&E Utilization: Both systems established for monitoring and evaluation, the HDM based road data and the collection of road safety statistics, performed satisfactorily, and the data produced has assisted the Government in judging the effectiveness of both the road rehabilitation program and the road safety program. However, the absence of any system to monitor the adequacy of road maintenance funding proved to be a major deficiency. 2.15 Overall quality of the project’s M&E is rated as Modest. Implementation 2.16 The project was approved on March 30, 2004 and closed on December 31, 2008 with 19 months extension. Due to the change in government in 2005, various proposals were considered for restructuring GDDKiA, including eliminating the entity and decentralizing responsibilities to the regional branches. GDDKiA was ultimately retained, but the intervening delay resulted in an extension of the closing dates for the project. IMPLEMENTATION EXPERIENCE 2.17 Implementation of the maintenance and rehabilitation component alongside GDDKiA’s own maintenance program, with the funds pooled and disbursed in specified ratio of Bank to Government resources, and utilizing country systems for all procurement/financial management requirements, led to effective and speedy implementation of the works. Financial management report based disbursements also proved effective in establishing a smooth and timely flow of funds. Discussions with Branch level Directors indicated that the physical works proceeded without any delays caused by the implementation and disbursement arrangements of the Bank loan. To the contrary, they stated that apart from 10 regular review meetings with Bank supervision teams and site visits, works implementation was rarely if ever affected by Bank processes. Disbursement of loan funds allocated to physical works was largely completed within a year or so of loan effectiveness (see Annex B). 2.18 The intention of providing a quick disbursing loan for an activity that was an urgent national priority, was therefore well served. By providing this assistance through a sequence of loans, the use of country systems could be tested in the first loan, and adjustments made when subsequent loans were approved. The technical assistance components, however, took much longer to implement, and the loan had to be extended by 18 months on this account. 2.19 GDDKiA Modernization Action Plan: The Action Plan sought to formalize contract- type business plans governing the Ministry’s mandate to GDDKiA, and GDDKiA’s mandate to its regional branches. The target was to agree among the parties and then to implement the necessary institutional changes. It later emerged that GDDKiA was less in tune with the changes proposed, which had primarily been championed by the Ministry. The consultant’s report was delivered in December 2004, but a change of Government in 2005 caused a long hiatus in the implementation. Whereas the previous Minister favored a wide-ranging decentralization of responsibilities to the regional branches, this was reversed by the new Government. It was not until May 2008 - seven months before project closing - when new top management was appointed at GDDKiA, that these plans were taken up. The key changes were: (i) decentralization of some functions from headquarters to branch offices, such as investment planning, preparation and implementation; (ii) re-engineering of key business processes; and (iii) creating new units at Headquarters for functions such as environmental analyses and information technology that had not existed previously. 2.20 Public Private Partnerships: This component involved the preparation of performance-based road maintenance contracts. This initiative experienced delays from the very beginning of the project. The contract for the preparatory studies was awarded only in March 2007 and there was a serious lack of understanding between the consultants and the Government counterparts about the purpose of the studies. In 2008, the Ministry decided to cancel the pilot sub-component, and the funding was diverted to civil works. The reason for the failure of the pilot was that, while the concept was supported by the Ministry and by top GDDKiA management, strong resistance came from GDDKiA operating staff. Deficiencies in the quality of the consultant’s output and frequent changes of top GDDKiA management during 2006 - 2008 also contributed. 2.21 Implementation of the Road Safety component was through the National Road Safety Council, and this inter-ministerial body reporting to the Council of Ministers was tasked to tap various entities at the national level needed to address the multi-faceted challenges of road safety. The implementation arrangements, whereby various concerned agencies were given independent budgets annually, were designed to create a degree of competition in completing tasks in order to preserve the allocations for the following year. Funding for road safety initiatives was supposed to have been provided through the Road Fund. However, most of the activities were eventually funded by the project. 11 PROJECT COSTS 2.22 Actual project cost was $143.04 million which was exactly the planned total cost. However, there were some reallocations among components as shown in Table 3 below. Table 3: Planned and Actual Project Costs by Component (US$ Millions) Components Appraisal Actual Percentage of Estimate* Cost* Appraisal A. GDDKiA Modernization Action Plan 2.86 1.60 56% B. Road Maint & Rehabilitation Program 131.60 133.31 101% C. Road Safety Improvement Program 6.44 6.57 102% D. Design and Procure long-term pilot 0.71 0.84 118% performance-based management and maintenance contract Total Project Cost 141.61 142.32 100.5%** Front end fee 1.43 0.72 50% Total Financing Required 143.04 143.04 100% *Exchange rate 1 EUR=1.4304 USD 29 Oct 2007; ** Additional 0.5% from decrease in the front end fee Source: Road Maintenance and Rehabilitation Project, Implementation Completion Report dated June 2009 SAFEGUARDS AND FIDUCIARY COMPLIANCE 2.23 The project was rated as a ‘financial intermediary’ for purposes of safeguards, and the civil works triggered only minor environmental and social impacts since these were small- scale rehabilitation works within the existing roads’ rights of way and did not require any land acquisition or entail any displacement of people. The Polish environmental safeguards upon which the project relied had been reviewed and found to be adequate to the Bank’s requirements. 2.24 The project clearly specified financial management arrangement for funds flow and controls for three types of project expenditures for the technical assistance, physical investments, and road safety components, and these arrangements appear to have been effective. At one stage in 2006, financial management was rated moderately unsatisfactory because release payments to contractors was taking excessively long and the Ministry was failing to submit satisfactory financial management reports. These problems were overcome by training to GDDKiA branch level and headquarters units on disbursement procedures. Poland’s procurement law was amended to align with EU directives, and this arrangement also appears to have worked during implementation because the project did not encounter serious procurement issues. Achievement of the Objectives 2.25 The overarching objective was articulated in the PAD as “improve the effectiveness of Poland’s national road rehabilitation and maintenance systems”, with three sub-objectives as follows. 12 SIGNIFICANTLY INCREASE THE PERCENTAGE OF NATIONAL ROADS IN GOOD CONDITION 2.26 Altogether, the project financed contracts for the rehabilitation of 917 km of national roads. The works were packaged into small contracts worth less than EUR 6 million in order to conform with the procurement arrangements of the loan for using the Borrower’s own systems. The average contract was about 5 km in length and cost about USD2.4 million, and was well suited to the capabilities of the local construction industry. The quality of the works were subjected to a technical audit and found to be of good quality. 2.27 Between 2003 and 2005, the percentage of national roads in good condition increased from 37 percent in 2003 (baseline) to 49 percent, whereas the target established for 2005 was 43 percent. It further improved to 53 percent in 2008, whereas the target established was 55 percent by 2006. 2.28 The road maintenance and rehabilitation program, collectively funded by the Bank, Borrower, and other financiers including EIB and EU, was the major program being implemented by the government and therefore it is likely that the program has contributed to the achievement of this sub-objective, and therefore it is rated as Substantial. ESTABLISH RELIABLE AND STABLE FUNDING FOR THE NATIONAL ROAD REHABILITATION AND MAINTENANCE NETWORK AND FOR ROAD SAFETY 2.29 At the time of preparation of the road maintenance and rehabilitation program, Poland was expected to have the resources for adequate road maintenance and rehabilitation. Public expenditure on the national road network was to grow from US$2 billion in 2005 to US$6 billion a year from 2006 to 2010, with the bulk of the funding coming from the EU. What was not expected was that the counterpart financing requirement for utilizing the EU funds would strain the Government’s financial resources for the road sector. The severity and the length of the financial crisis that has impacted the Government’s finances were also unexpected. 2.30 Following the launch of the rehabilitation program using the project loan, EIB loans in the amount of EUR 300 million were used for rehabilitation during 2008-2010 and EUR350 million in 2009-2011. However, the program did not succeed in creating a stable and reliable mechanism for the funding of road maintenance because in 2009, the National Road Fund which was to have provided this stable source of maintenance funding, was restructured and was specifically required to fund only new construction activities with minor exceptions made for rehabilitation works that can be justified on road safety grounds. This prohibition of the use of the National Road Fund for maintenance is due to the government’s desire to fully utilize the EU funding available for new construction, and for which the provision of sufficient counterpart funds is critical and cannot be delayed. The road networks were expected to be maintained with financial allocation from the state budget but it often fell short of the required PLN2.5-3 billion/year. Similarly, road safety programs did not receive funding from the Road Fund, and whatever has been accomplished relied on funding provided under the Bank loan. The legislation passed in 2013 is now directing all revenues from speed management program to the National Road Fund for road safety specific rehabilitation of national roads starting in 2014. 13 2.31 Achievement on of this objective is consequently rated as Modest. IMPROVE THE CAPACITY WITHIN GDDKIA TO OPERATE EFFICIENTLY AND EFFECTIVELY AND REFLECT VIEWS OF ROAD USERS IN DEVELOPING ITS PROGRAMS 2.32 In meetings with the Studies Department of GDDKiA, it was apparent that GDDKiA had developed an extensive capacity to collect data and analyze road maintenance requirements using the HDM-4 model. In addition to the headquarters level, 13 of the 16 regional GDDKiA branches now routinely utilize the HDM-4 methodology to track the condition of their networks and to evaluate proposed rehabilitation programs. This is a significant achievement. Their data collection systems are now directly linked to the HDM-4 software. However, GDDKiA is not yet utilizing the full capabilities of the HDM-4 approach such as for the evaluation of alternatives, and savings in the initial rehabilitation costs may be possible. But the current practice essentially meets the requirement that the road maintenance and rehabilitation program undertake economic evaluations for all rehabilitation projects. At the time of project closure it was expected that GDDKiA would eventually extend this capability to evaluate the economics of alternative interventions, such as reconstruction versus strengthening or overlays. Such a globally optimal assessment of alternatives would be especially useful when budget constraints require that available resources be stretched optimally to protect the network as a whole, which is precisely the situation Poland now faces (see Chapter 4). 2.33 GDDKiA conducted extensive road user surveys in 2004, 2005, with the help of a professional polling firm based in Poland. Subsequent user surveys done under the second and third projects (2009 and 2011) showed a good awareness of GDDKiA programs and confirmation that road users noticed significant improvements in the national road network. Results of these subsequent surveys are elaborated in Chapters 3 and 4. 2.34 The Performance-based Management and Maintenance of Roads pilot program did not get implemented. Restrictions on expansion of sovereign liabilities under the EU accession rules are said to have posed difficulties for this pilot program concept. Also, bids received for the pilot program were found to be high and were considered unaffordable. Consequently, GDDKiA did not award a pilot program contract. However, it subsequently developed an alternative called the “Sustained Standards” approach for outsourcing routine maintenance in early 2011, partially based on the pilot program contracts developed under the road maintenance and rehabilitation program (see Chapter 4). 2.35 Some elements of the road maintenance and rehabilitation program have made useful contributions to GDDKiA’s effectiveness. GDDKiA’s introduction of HDM-4 analysis, implementation of a modern management information system using SAP, and conducting of road user satisfaction surveys, have been effective, but there was limited evidence that demonstrated the improved capacity of GDDKiA to operate efficiently and effectively. The overall effectiveness of the program is therefore rated as Modest. 14 Efficiency 2.36 An ex-post economic evaluation was conducted after the first two projects, but for a few examples from Bialystok region, based on a sample review. The calculation done at appraisal showed between 27 and 50 percent rate of return, and after the works were completed (i.e. using actual construction cost and actual opening year traffic) the rate of return ranged between 42 and 87 percent. However, the methodology used for these two calculations are not fully comparable and since the analysis was only done for one region it is not representative of the national road network. According to an economic analysis for all co- financed rehabilitation tasks implemented by GDDKiA in 2005-2007 (first two projects) which was carried out as part of a quality review of the World Bank in 2009, the weighted average ERR was 27.1 percent. 2.37 The efficiency of the technical assistance components need to be qualitatively assessed and the delayed and incomplete implementation of the Modernization Action Plan would only yield a modest rating on efficiency. Overall efficiency is hence rated as Modest. Ratings OUTCOME 2.38 Relevance of objective is Substantial, relevance of design is Modest, and efficiency is rated Modest. Of the three development sub-objectives, achievement has been rated Substantial for one and Modest for the other two. The effectiveness of the road program is undermined by the slow progress in improving the sector’s institutional capability and funding for road maintenance and therefore achievement of the overarching objective is rated modest. Consequently, overall outcome is rated Moderately Unsatisfactory. RISK TO DEVELOPMENT OUTCOME 2.39 Risk to development outcome is rated Significant. Maintaining a balance between rehabilitation needs of the existing national roads network and new construction and improvement works was an important consideration in the Bank’s support for the program. Government commitment to rehabilitation of the network, and to maintaining the network in a stable condition at around 60 percent in good condition, appears to have been sacrificed in recent years in order to support EU funded new construction programs. If adequate funding for maintenance is not secured, it is likely to affect the sustainability of Poland’s road network. BANK PERFORMANCE 2.40 Quality at Entry: The project concept with regard to objectives and the design of the components was certainly of high relevance to Poland’s needs at the time and continue to be so even now. The strong focus on road safety, systematic road maintenance project evaluation using HDM-4, and reform of road sector institutions indicate a high degree of Bank staff commitment to road sector development. However, in the design of the project and in its monitoring and evaluation framework, insufficient attention has been given to 15 ensuring least cost solutions and stable funding for maintenance of the network. The availability of stable maintenance funding lies at the core of effective road network management, and not including a monitoring indicator on the funding mechanism probably contributed to its not being given the needed emphasis. Rating: Moderately Satisfactory 2.41 Quality of Supervision: The number and staffing of supervision missions was adequate. The team did commendable work in supporting the Government’s efforts on a number of fronts, particularly on road safety. On maintenance funding for the national road network, the team should have taken action to address the concerns raised about the reductions in funding caused by the priority given to the EU funded programs. Rating: Moderately Satisfactory. 2.42 Overall rating of Bank Performance: Inadequate funding of maintenance coupled with the higher/ excessive axle loads of Trans-European traffic, is putting the national roads network in Poland at heightened risk of premature failure. The Bank as the external agency most focused on maintenance of the national road network, should have taken a more pro- active stance vis-à-vis the Government to alert them to the perverse effect of these policies. Rating: Moderately Satisfactory. BORROWER PERFORMANCE 2.43 Government Performance: The Ministry of Infrastructure/Transport has performed satisfactorily with regard to rapid compliance with conditions of Board presentation and effectiveness, as well as in complying with its obligations regarding financial management and reporting. Counterpart funding was substantially higher than the amount planned at appraisal. However, it did not pay sufficient attention to securing maintenance funding through establishing reliable and stable funding for national road networks. With regards to road safety, the National Road Safety Council was not given sufficient resources or accountability to ensure smooth implementation of activities and sound coordination between and among the stakeholders. Rating: Moderately Unsatisfactory 2.44 Implementing Agencies Performance: GDDKiA processed, awarded and implemented a large number of rehabilitation contracts within a short period of time. Both GDDKiA and the National Road Safety Council have also performed well on their responsibilities under the technical assistance components of the project, though there were some delays in implementation of the Modernization Action Plan by GDDKiA. Rating: Moderately Satisfactory. 2.45 Overall Borrower Performance: The performance of the borrower overall has been pro-active and has resulted in a project that disbursed well and where a majority of the components were implemented but there were significant issues related to securing reliable funding for the purpose of maintenance. On balance, overall Borrower performance is rated as Moderately Unsatisfactory. 16 3. Second Road Maintenance and Rehabilitation Project (2005-2009) Objectives, Design, and Relevance OBJECTIVES 3.1 The second road maintenance and rehabilitation project had similar objective and sub-objectives as the first project both in the Loan Agreement and in the PAD: “To continue improving the effectiveness of Poland’s national road rehabilitation and maintenance systems by: (i) further increasing the percentage of national roads in good condition; (ii) establishing reliable and stable funding for the national road rehabilitation and maintenance network; and (iii) improving the capacity within GDDKiA to operate efficiently and effectively.” 3.2 The Monitoring Indicators for the second project differed from those for the first project in some respects: preparation of 6-year rolling plans was eliminated, and targets for reduced road fatalities and percentage of roads in good condition were enhanced. A new indicator for length of network strengthened for 11.5 ton axle loads was introduced for the second project. COMPONENTS 3.3 The project had two components: (1) Road Maintenance and Rehabilitation Program; and (2) GDDKiA Management Information System. Additional details on these components and their sub-components are in Table 5. The description of the Road Maintenance and Rehabilitation component is identical to that for the first project. The Management Information System component is described below. Table 4: Project Components, Costs and Sub-components Components Sub-components (1) Second Road Maintenance The funds were utilized for rehabilitation of the network to address the and Rehabilitation Program maintenance backlog, and maintain the balance between maintenance / Appraisal: US$137.32 million rehabilitation versus strengthening / new construction for which Actual: US$137.32 million substantial funds were being provided from the EU. National budget funds for periodic maintenance and rehabilitation were pooled with resources provided by WB and EIB loans. (2) Management Information Establish a modern Management Information System for System GDDKiA which integrates financial-accounting information with Appraisal: US$5.72 million contract data, including planning information and physical Actual US$5.72 million measures of works progress and implementation efficiency. Source: Project Appraisal Documents and Implementation Completion Reports 3.4 GDDKiA Management Information System: The need for a modern Management Information System for GDDKiA was identified during the implementation of the first project. Based on a review of available systems it was decided to use SAP to develop an information system for GDDKiA. The intention was to standardize across the GDDKiA organization, including both headquarters and regional branch offices, accounting practices 17 and internal control procedures as well as the flow of documents and approvals. It included establishment of a wide area network connecting headquarters with all the 16 branch offices. RELEVANCE 3.5 The relevance of the project objectives was Substantial. As described in the previous Chapter, the Country Assistance Strategy of 2002 prepared by the World Bank at the time of Poland’s accession to the EU, recognized the importance of strengthening road sector institutions to enable absorption of the vastly expanded resources to be provided by the EU and other European institutions. The Strategy also noted that the incidence of unsafe and congested roads and high transport costs added to regional disparities and made the country less attractive to private investment. The most of the project objectives reflected these challenges that country was facing at that time and established an appropriate role for the Bank’s comparative strengths. The Country Partnership Strategy of 2009 continued to note that while the EU and other European institutions, notably the EIB were the main partners and external sources of financing for Poland's road sector, the Bank could continue to play an important role in helping to build institutions and in complementary investments. The first two projects, however, did not have an objective related to the much needed improvement in road safety as an EU member, but instead only included the funding for road safety to be part of reliable and stable funding objective. 3.6 The relevance of the project design with regard to its approach and the components financed to achieve the objectives was Modest. As in the first project, the hybrid approach used in the project design for quick disbursement of funds. Since the counterpart funding requirements of the EU program were draining resources from the rest of the network, the Bank’s fast disbursing loan was critical to stabilizing the situation. On the other hand, the technical assistance component which included the implementation of management information system required a longer gestation and closer supervision for which the SIL approach was more suited. While the component on road maintenance and rehabilitation program was well suited to achieve the first objective of further increasing the percentage of national roads in good condition, the project’s intention to improve GDDKiA’s capacity to operate efficiently and effectively only through a component to establish a modern management information system seemed too ambitious. However, as in the first project, there were inconsistencies between the objectives and project design. The project had a component on road safety and the related monitoring indicators although it did not have a road safety objective, and the project’s design did not address the sub-objective of achieving stable and reliable funding for road maintenance. This was critical to the sustainability of the road sector, and its absence from the project design and monitoring indicators proved to be a major shortcoming. MONITORING AND EVALUATION 3.7 M&E Design: There is a lack of internal consistency in the M&E design. The PAD states “The progress reports will focus on results rather than providing process related information”, but then goes on to state that the reports would list all activities carried out, including road maintenance and rehabilitation contracts executed during this period and progress made in the implementation of the management information system. 18 3.8 M&E Implementation: M&E was implemented using a system of annual progress reports and there is a lack of internal consistency between the monitoring indicators set out for the project and the contents of the progress reports. The M&E system for the project essentially retained the arrangements made for the first project. With regard to the condition of the road network, it relied on the regular data collection carried out by GDDKiA’s Department of Studies. While road safety was not an explicit objective or a component under this project, the M&E continued to include the road safety statistics on road accidents and fatality rates. 3.9 M&E Utilization: Both systems established for M&E, the HDM-4 based road condition data and the collection of road safety statistics, performed well. The data produced has assisted the Government in judging the effectiveness of the road rehabilitation and the road safety programs. 3.10 However, inconsistencies between the monitoring indicators, the projects components and the progress reporting systems of this project, and the absence of any system to monitor the adequacy of maintenance budgets are deficiencies, and the overall quality of the M&E framework for the project is therefore rated as Modest. Implementation 3.11 The project was approved on March 29, 2005 and it closed on June 30, 2009 with 18 months extension. IMPLEMENTATION EXPERIENCE 3.12 Discussion of the Road Maintenance and Rehabilitation component of second project is covered in Chapter 2 as this component is simply a continuation of the one in first project. This section focuses on the implementation experience of the Management Information System which was a new component introduced in the second project. 3.13 GDDKiA Management Information System: A Strategy for a Comprehensive Information Technology System had been developed in early 2004 as a part of the Modernization Action Plan for GDDKiA. During 2006 it was decided to proceed with development of an Management Information system under three main contracts – a service contract to establish a wide area network, design of a Enterprise Resource Planning system to be developed based on the SAP software system, and a consulting contract to computerize workflow processing. The system’s contract was signed in January 2007, however the workflow processing contract was cancelled from the loan and postponed until the contract implementation had advanced sufficiently to allow better coordination. The pilot start-up of the key Management Information System modules, all financial management and accounting modules, took place in January 2009, right at the end of the project. In April 2011, the main financial, human resources, administrative and budgeting modules went online, and now full roll-out of these new systems to all branches is underway and expected to be completed by 2013. The long delay in setting up the SAP system is largely attributed to the need for a high degree of customization of the software. 19 PROJECT COSTS 3.14 Actual project cost was 143.04 million which was exactly the planned total cost as shown in Table 5 below. Table 5: Planned and Actual Project Costs by Component (US$ Millions) Components Appraisal Actual Percentage of Estimate Cost Appraisal A. Road Maint. & Rehabilitation Program 137.32 137.32 100% B. Goods Under Management Inf. System 5.29 5.29 100% C. Consultant services including auditing 0.43 0.43 100% services for TA and Management Information System Total Project Cost 143.04 143.04 100% Source: ICR dated June 2009 SAFEGUARDS AND FIDUCIARY COMPLIANCE 3.15 The project was rated as ‘financial intermediaries’ for purposes of safeguards, and the civil works triggered only minor environmental and social impacts since these were small scale rehabilitation works within the existing road’s right of way and did not require any land acquisition or entail any displacement of people. The Polish environmental safeguards upon which the project relied had been reviewed and found to be adequate to the Bank’s requirements. At one stage in 2006 financial management was rated moderately unsatisfactory because release of payments to contractors was taking excessively long and the Ministry was failing to submit satisfactory financial management reports. These problems were overcome by Bank supplied training to staff of the Ministry. Achievement of the Objectives 3.16 The overarching objective was articulated as “to continue improving the effectiveness of Poland’s national road rehabilitation and maintenance systems”, with three sub-objectives as follows. FURTHER INCREASE THE PERCENTAGE OF NATIONAL ROADS IN GOOD CONDITION 3.17 The percentage of national roads in good condition increased from 46 percent in 2004 (Baseline for the second project) to 53 percent in 2008, whereas the target established was 55 percent by 2006. Hence achievement on this sub-objective fell a little short of the target and is therefore rated as Substantial. ESTABLISH RELIABLE AND STABLE FUNDING FOR THE NATIONAL ROAD REHABILITATION AND MAINTENANCE NETWORK 3.18 As noted previously, the road maintenance and rehabilitation program did not succeed in creating a stable and reliable mechanism for the funding especially for the purpose of periodic road maintenance. The National Road Fund which was to have provided this stable source of maintenance funding was restricted to fund only new construction activities with 20 minor exceptions made for rehabilitation works that can be justified on road safety grounds. The road networks were expected to be maintained with financial allocation from the state budget but it often fell short of the required PLN2.5-3 billion/year. It should be noted that at the time of preparation of the second project, the maintenance funding issue and the restrictions on the use of the National Road Fund for maintenance had not emerged and could not be anticipated. With regards to the funding for road safety, the legislation passed in 2013 is now directing all revenues from speed management program to the National Road Fund for road safety specific rehabilitation of national roads starting in 2014. Achievement on Sub- objective 2 is rated as Modest. IMPROVE THE CAPACITY WITHIN GDDKIA TO OPERATE EFFICIENTLY AND EFFECTIVELY 3.19 The Management Information System developed for GDDKiA under the second project went online at headquarters in April 2011 two years after the project closed. As of April 2012, it was being rolled out to various branch offices. While the full SAP capability available in various modules was not yet functional, the key modules such as financial management, human resources, budgeting, and administrative modules, have been made available. The roll-out to all branches is expected to be completed by end-2013. Implementation of the SAP started in 2007, and has taken longer than expected due to customization requirements which proved difficult but this is not unusual for new implementations of SAP. However, much of this could not be completed under the second project and had to be taken up under the third project. The contribution of the second project to achieving this sub-objective is rated as Modest. Efficiency 3.20 As described under the Efficiency section in the previous Chapter, an economic rate of return was calculated after the first two projects were completed, but for a few sample examples in only one region, and the calculations were not comparable to the rate of return calculations done at project appraisal. Despite GDDKiA’s assessments that point to 75 percent of contracts having an ex-ante ERR above 20 percent, the absence of the ex-post analysis in a comparative manner it difficult to generalize this result. 3.21 The efficiency of the technical assistance components need to be qualitatively assessed and the delayed and incomplete implementation of the Modernization Action Plan would only yield a modest rating on efficiency. Overall efficiency is hence rated as Modest. Ratings OUTCOME 3.22 Relevance of objective is rated Substantial, relevance of design Modest, and the efficiency is rated Modest. Of the three development sub-objectives, achievement has been rated Substantial for one, and Modest for the other two. The overall objective of continuing improving the effectiveness of Poland’s program is rated Modest. The overall project outcome is therefore rated Moderately Unsatisfactory. 21 RISK TO DEVELOPMENT OUTCOME 3.23 Risk to development outcome is rated Significant. As described in the previous Chapter, Government commitment to maintain the network in a stable condition appears to have been sacrificed in recent years in order to support EU funded new construction programs. Depriving road maintenance funding of its allocations from the Road Fund is one of the more unfortunate consequences of this shift in policy. If adequate funding for maintenance is not provided in a reliable and stable manner, it is likely to affect the sustainability of Poland’s road network. BANK PERFORMANCE 3.24 Quality at Entry: The project concept with regard to objectives and the design of the components was certainly of substantial relevance to Poland’s needs at the time and continue to be so even now. The identification of the critical need to improve GDDKiA management information systems during implementation of first project speaks to the strong relevance of the design of the second project. However, there were no project activities to support the second objective. As a result, in the execution of the project and in its monitoring and evaluation framework, insufficient attention has been given to ensuring stable funding for maintenance of the network. The availability of stable funding lies at the core of effective road network management, and not including a monitoring indicator on the funding mechanism probably contributed to its not being given the needed emphasis. Rating: Moderately Satisfactory 3.25 Quality of Supervision: The number and staffing of supervision missions was adequate. The team did commendable work in supporting the Government’s efforts on a number of fronts, particularly on road safety. On maintenance funding for the national road network uninvolved in EU programs, the team should have taken action to address the concerns raised about reductions in funding caused by the higher priority given to providing counterpart funding for the EU programs. Rating: Moderately Satisfactory. 3.26 Overall rating of Bank Performance: Inadequate funding of maintenance coupled with the higher/ excessive axle loads of Trans-European traffic, is putting the national roads network in Poland at heightened risk of premature failure. The Bank as the external agency most involved in the maintenance of the network could have taken a pro-active stance to alert the Government of the perverse effect of these policies. Rating: Moderately Satisfactory. BORROWER PERFORMANCE 3.27 Government Performance: The Ministry of Infrastructure/Transport performed satisfactorily with regard to rapid compliance with conditions of Board presentation and effectiveness, as well as in complying with its obligations regarding financial management and reporting. Counterpart funding at completion was about three times the amount planned at appraisal. However, it did not pay sufficient attention to securing maintenance funding through establishing reliable and stable funding for national road networks. With regards to road safety, the National Road Safety Council was not given sufficient resources or 22 accountability to ensure smooth implementation of activities and sound coordination between and among the stakeholders. Rating: Moderately Unsatisfactory. 3.28 Implementing Agencies Performance: GDDKiA processed, awarded and implemented a large number of rehabilitation contracts within a short period of time. GDDKiA performed well on its responsibilities under the technical assistance components of the project, though there were some delays in implementation of the Modernization Action Plan. Rating: Moderately Satisfactory. 3.29 Overall rating of Borrower Performance: The performance of the borrower overall has been pro-active and has resulted in a project that disbursed well and where a majority of the sub-components were implemented. Overall Borrower performance is therefore rated as Moderately Unsatisfactory. 4. Third Road Maintenance and Rehabilitation Project (2006-2011) Objectives, Design, and Relevance OBJECTIVES 4.1 The project objective, as stated in the Loan Agreement was to “assist the Borrower in continuing improving the effectiveness of the country’s national road rehabilitation and maintenance systems, with emphasis on quality, efficiency, financial viability, and road user satisfaction”. The objective is similar in the PAD but it also includes road safety as an added dimension for emphasis and relevant components and indicators are included in the project design. Therefore the PAD objectives are used for the purpose of this assessment. COMPONENTS 4.2 The project had four components: (1) Road Maintenance and Rehabilitation Program; (2) GDDKiA Management Information System implementation; (3) Road Safety; and (4) Technical Assistance to Ministry of Transport and Construction. Additional details on these components and their sub-components are in Table 8. Some of the components, e.g. for road maintenance and rehabilitation, were modified slightly in this third project from previous approaches adopted in first two projects. The following describes the modified or new components introduced into the program in the third project. 23 Table 6: Project Components, Costs and Sub-components Components Sub-components (1) Road Maintenance and Same as for component (2) of the first project, and at least one Rehabilitation Program pilot contract for performance based management and Appraisal: US$167.0 million maintenance of roads, and a public awareness campaign Actual: US$181.92 million (2) Implementation of Complete the implementation of GDDKiA’s management Management Information information system. The design of the system and its initial System implementation was financed under the second project. The Appraisal: US$3.0 million system will be completed through implementation of two Actual: US$ 4.4 million packages including Wide Area Network services and a security system. (3) Road Safety This component supported road safety campaigns on themes such Appraisal: US$6.6 million as “alcohol and driving”, “speeding”, “safety of children” and Actual: US$6.85 million “pedestrian safety”, as well as technical assistance to the secretariat of the National Road Safety Council. (4) Technical Assistance This component supported (i) an Intelligent Transport System Appraisal: US$ 3.6 million (ITS) study; (ii) Technical assistance to help better and faster Actual: US$ 1.28 million absorption of EU funds; (iii) assistance to improve road technical standards and traffic management; and (iv) audit of the road maintenance and rehabilitation projects Source: Project Appraisal Documents and Implementation Completion Reports 4.3 The road maintenance and rehabilitation component consisted of three sub- components: (i) physical works for periodic maintenance and rehabilitation covering about 165 contracts spread throughout the national roads network; (ii) at least one pilot contract for performance-based management and maintenance of roads, and (iii) a public awareness campaign in support of the Government’s road construction and rehabilitation program, based on the communications strategy developed by GDDKiA in 2005. The Bank would contribute a reduced 48 percent share of this program, in view of the EIB contribution to the financing of the civil works. 4.4 Implementation of the GDDKiA management information system, which had been developed and partially implemented under the second project, was taken up for completion under this project. This required implementation of a wide area network to connect regional branches to Headquarters and a security system for the Management Information System. This component was fully financed by the Bank. 4.5 Road Safety Component: Under this project the road safety component focused on public awareness campaigns based on themes such as “alcohol and driving”, “speeding”, “safety of children” and “pedestrian safety”. The component also funded Technical Assistance to the National Road Safety Council. This component was fully financed by the Bank. 4.6 Technical Assistance to the Ministry of Transport and Construction: The component supported (i) an Intelligent Transport System (ITS) study; (ii) technical assistance to help better and faster absorption of EU regional funds for the ongoing (2004-2006) and future (2007-2013) EU transport operational program in Poland; (iii) assistance to improve 24 technical standards and traffic management programs; and (iv) audit for the road maintenance and rehabilitation projects. The Bank financed 78 percent of this component. RELEVANCE 4.7 The relevance of the project objectives was High. As described in the previous Chapter, the Country Assistance Strategy of 2002 prepared by the World Bank at the time of Poland’s accession to the EU, recognized the importance of strengthening road sector institutions to enable absorption of the vastly expanded resources to be provided by the EU and other European institutions. The Strategy also noted that the incidence of unsafe and congested roads and high transport costs added to regional disparities and made the country less attractive to private investment. The project objectives reflected these challenges that country was facing at that time and established an appropriate role for the Bank’s comparative strengths. The Country Partnership Strategy of 2009 continued to note that while the EU and other European institutions, notably the EIB were the main partners and external sources of financing for Poland's road sector, the Bank could continue to play an important role in helping to build institutions and in complementary investments. 4.8 The relevance of the project design with regard to its approach and the components financed to achieve the objectives was Modest. The hybrid approach used in the project design was the same as that used for the previous road maintenance and rehabilitation projects. The approach remained as relevant as before: Poland urgently needed access to quick disbursing road maintenance and rehabilitation funds since the counterpart funding requirement of the EU program was draining resources from the road sector. Therefore, it was critical to use an approach, such as the SWAp that was designed to facilitate fast disbursement of project funds to stabilize g the situation. The road maintenance and rehabilitation component that supported the civil works was appropriate for achieving the quality of Poland’s national road network and consequently road user satisfaction, and the technical assistance would contribute to the efficiency of the systems. Poland also needed to continue improving its road safety record as an EU member, to reduce road accident fatality rates, and the Bank funded road safety component was critical to this effort. The project objective as specified in the financing agreement, however omitted the reference to road safety while it was included in the PAD objective, component, and indicators. While the project’s design was appropriate and relevant in these respects, it did not address the sub-objective of achieving financial viability. This was critical to the sustainability of the project and the road sector, and its absence from the project design and monitoring indicators proved to be a major shortcoming. MONITORING AND EVALUATION 4.9 M&E Design: The M&E system designed for the project essentially retained the arrangements made for the first and second projects. The indicators covered most of the key objectives – quality, efficiency, safety and road user satisfaction, the one exception being financial viability. With regard to the condition of the road network, it relied on the regular data collection carried out by GDDKiA’s Department of Studies. The other aspect of M&E is the road safety statistics on road accidents and fatality rates. These data were collected by 25 the National Road Safety Council and they have maintained detailed data on accidents, injuries, fatalities, and vehicle fleet as necessary to support the M&E design of the project. 4.10 Failure to address the sub-objective of ensuring the financial viability of the road sector, in particular for the funding of maintenance, was a critical oversight. The shortfalls in maintenance funding now appear to have jeopardized the sustainability of the improvements made under the program. 4.11 M&E Implementation: GDDKiA has a comprehensive data collection system for road conditions and traffic levels as needed to support the monitoring indicators in these aspects. The National Road Safety Council maintains up-to-date information on traffic and accidents statistics. Therefore data on road pavement conditions and safety statistics were well provided for in the M&E framework. Road user perceptions were the subject of four surveys undertaken in 2005, 2006, 2009 and 2011 and these were useful in recording improvements in user perceptions of the road network. As noted earlier, the main deficiency in the M&E framework was the absence of any indicators or data on the financial and budget situation for the maintenance of the road network. Presently, in addition to the fuel levies being deposited in the Road Fund, there are toll revenues being generated from the new motorway developments. However, the M&E framework for the project does not produce a consolidated view of these resources or their availability to fund maintenance. 4.12 M&E Utilization: Both systems established for monitoring and evaluation, the HDM 4 based road data and the collection of road safety statistics, continued to perform well, and the data produced has assisted the Government in judging the effectiveness of both the road rehabilitation and the road safety programs. 4.13 Overall quality of the M&E system for the project is rated as Modest. Implementation 4.14 The project was approved on June 6, 2006 and closed on September 15, 2011 according to the original schedule. IMPLEMENTATION EXPERIENCE 4.15 Road Maintenance and Rehabilitation Program: Road rehabilitation works proceeded very rapidly which is indicative of an enhanced capability of GDDKiA. Within less than a year almost 80 percent of the road works had been completed with 810 km of roads rehabilitated. As a result, more road works were completed than originally planned. At project closing, road works had been implemented on 218 contracts with a total length of 1,103 km. The performance based management and maintenance of roads initiative however experienced delays, and the contract for preparatory studies was awarded only in March 2007 due to a lack of understanding by GDDKiA as to the purpose of the study. In 2008 this sub- component was canceled by the Ministry, after bids received were found to be high, and the allocated funds were used for civil works. The Ministry eventually developed an alternative approach to outsourcing maintenance which did not involve rehabilitation. This approach named the Sustained Standards approach, is for routine maintenance (only) of a rehabilitated 26 or newly constructed road. A four year maintenance contract using this approach was awarded for an 84 km section of the A3 highway. Subsequently, GDDKiA also decided to outsource all rehabilitation works because they found the cost to be nearly a third of in-house rehabilitation works. While some force account maintenance work continues, its role has been significantly reduced. It may be concluded that the performance based management and maintenance of roads initiative has borne fruit, though not directly as anticipated – this may be considered a spill-over effect of the Bank’s initiative on outsourcing road rehabilitation and maintenance. The public awareness campaign was successfully completed and also helped develop media awareness and training, including media campaigns on road traffic safety. 4.16 GDDKiA Management Information System: A Strategy for a Comprehensive Information Technology System had been developed in early 2004 as a part of the Modernization Action Plan for GDDKiA. During 2006, under the second project, it was decided to proceed with development of an Management Information System under three main contracts – a service contract to establish a wide area network, a contract for the design of an Enterprise Resource Planning system based on the SAP software system, and a consulting contract to computerize workflow processing. The contract was signed in January 2007, however the workflow processing contract was cancelled from the loan and postponed until the contract implementation had advanced sufficiently to allow better coordination between the two. The pilot start-up of the key Management Information System modules, and all financial management and accounting modules, took place in January 2009 under the second project and in April 2011, the main financial, human resources, administrative and budgeting modules went online, and now full roll-out of these new systems to all branches is underway and expected to be completed by 2013. The long delay in setting up the SAP system is attributed to the need for a high degree of customization of the software. 4.17 Technical Assistance to Ministry of Transport and Construction: This Technical Assistance took the form of a study on Intelligent Transport Systems. Unfortunately, the expectations of this study which involved diverse disciplines, shifted and the eventual outputs of the consultants work did not match the expectations of the client. Consequently, the study was eventually dropped, and the funds were utilized for a variety of minor technical assistance activities such as a strategic environment assessment for road transport development. PROJECT COSTS 4.18 Actual project cost was $194.45 million versus the planned total cost of $180.20 million as shown in Table 7 below. With the exception of the technical assistance to Ministry of Transport and Construction, which disbursed only about a third of the amount planned, all of the other components exceeded the planned budget. The increase was due to the appreciation of the Euro, in which the loan was denominated, versus the USD. In the case of the physical works the cost increases were also due to reallocation of funds to road works from other components and an expansion of the works undertaken. 27 Table 7: Planned and Actual Project Costs by Component (US$ Millions) Components Appraisal Actual Percentage of Estimate Cost Appraisal A. Road Maintenance and Rehabilitation 167.0 181.92 109% Program B. Implementation of Management 3.0 4.40 147% Information System C. Road Safety 6.6 6.85 104% D. Technical Assistance 3.6 1.28 36% Total Project Cost 180.20 194.45 108% Source: ICR dated June 2009 SAFEGUARDS AND FIDUCIARY COMPLIANCE 4.19 The project’s works triggered only minor environmental impacts since these were small scale rehabilitation works within the existing road’s right of way and did not require any land acquisition or entail any displacement of people. The Polish environmental safeguards upon which the project relied had been reviewed and found to be adequate to the Bank’s requirements. 4.20 Annual procurement post reviews confirmed that procurement activities related to the works component were implemented using country procurement systems, and were carried out satisfactorily. Financial management was rated “moderately satisfactory” by Bank supervision missions principally because the implementation of the SAP financial systems was delayed. In general Financial Management/Disbursement processes have been satisfactory with clean audit opinions. The Audit reports and the Financial Management Reports were generally received on time. Achievement of the Objectives 4.21 The overarching objective of the program was stated to "continue improving the effectiveness of Poland’s national road rehabilitation and maintenance systems, with emphasis on quality, efficiency, financial viability, safety and road user satisfaction'. Assessment of each of the five sub-objectives; quality, efficiency, financial viability, safety, and road user satisfaction are presented below. QUALITY 4.22 The project implemented 218 contracts with a total length of 1,103 km which corresponds to six percent of the national road network. As a result, between 2003 and 2009, the percentage of national roads in good condition increased from 37 percent in 2003 (baseline of the first project) to 59.6 percent in 2009, essentially achieving the project target of 60 percent in good condition. According to public information, this had improved to 62.7 percent by year 2012. The load bearing capacity of the road network has improved significantly beyond the target of 3,300 km from the baseline of 2,191 km to 5,897 km. The IEG mission which was undertaken after project closure learnt that the works undertaken has 28 been subjected to technical audits and found to be of good quality. The achievement of the quality objective has therefore been rated as High. EFFICIENCY 4.23 The pilot for the performance based maintenance and management of roads was not implemented as scheduled because of the strong resistance from GDDKiA operating staff, deficiencies in the quality of the consultant, and frequent changes of GDDKiA management. However, towards the end of the project GDDKiA did procure a four year maintenance contract termed "sustained standards" contract, similar to what was originally prepared under the project for 84 km of the expressway and planned to use this contract for all newly constructed highways and upgraded roads. In addition some routine maintenance works were outsourced. As of 2013, the government has rolled out this new form of performance based contract in at least six road networks. The sustained standards” contracts implemented by GDDKiA found that using the private contractor, the same maintenance work could be carried out with only 37 percent of the costs, saving 63 percent of the maintenance budget. This approach has proved beneficial as it also allows outsourcing of maintenance during a construction contract’s defects- liability period (typically 5 years from completion) and thereby improves contract enforcement. This should in the long-run allow GDDKiA to gradually phase out its force account routine maintenance operations, and therefore may be considered a significant advance over current arrangements. The sustained standards approach is a simpler form of performance based maintenance out-sourcing, as it relates only to routine maintenance, and administering such contracts would be less complex and therefore more predictable. The reduced risks should also allow the contractors to bid lower, more competitive prices. The government continues to approach the World Bank to seek for more guidance in handling performance based contracts. Albeit with delays, the project at least partly contributed to the government's plan to use performance based road maintenance and management contracts. These performance based contracts are medium term in nature (first contracts were four years and the newest ones are seven years), they are likely to stabilize funding for the part of the network covered by those contracts as well. GDDKiA has also managed to contain its number of employees despite the increasing budget and this has resulted in significant productivity increase for the agency. 4.24 With regards to the modernization action plan, the project supported the reorganization of GDDiKA into three departments; (i) planning; (ii) investment; and (iii) maintenance and establishment of units for traffic management and quality control of construction. Functional reclassification of road system however was not implemented as scheduled because of political resistance to reallocate the budget. Some progress has been made to revise the procurement rule, and to modernize human resources, information and data, and financial management. However, the use of the system is limited to accounting and financial management and there was limited evidence on the effectiveness of the implemented activities in building the capacity of GDDiKA to operate efficiently. 4.25 The enabling environment to use the HDM-4 system was created through system installation and capacity building of GDDKiA staff, and the system is being used for the purpose of assessing the returns of each investment by all sixteen regions. In at least two to three regions, the system has also been used for optimizing the allocation of resources at the 29 network level under budget constraints. Some regions are experiencing difficulties using the system for the network analysis and prioritization purpose because of the complexity in collecting the necessary data (see Annex D for detailed description of the use of HDM-4 under budget constraints). 4.26 On balance, the efficiency objective of the project is rated as Substantial. FINANCIAL VIABILITY 4.27 The National Road Fund which would have provided stable source of maintenance funding, has been specifically instructed to fund only new construction with minor exceptions for works justified on road safety grounds. Counterpart financing requirement for accessing EU funds therefore strained the Government’s financial resources available for the road sector. The severity and length of the financial crisis that has impacted the Government’s finances since 2009 has exacerbated the situation. As a result, the funding for rehabilitation and maintenance was reduced in 2010 and 2011, after the closing of the project. In 2012, the government increased its budget for periodic maintenance but it is unclear whether this increase in allocation of maintenance funding will be sustained during the next few years. While the routine road maintenance funding is consistently being funded, securing sufficient amount of periodic maintenance funding in a reliable manner continues to be a challenge. Therefore the achievement of the financial viability objective is rated as Modest. ROAD SAFETY 4.28 Outputs related to road safety included; (i) individual and institutional consultancy services to the National Road Safety Council; (ii) road safety studies and analysis, (iii) soft measures in the form of public relations activities and awareness campaigns; and (iv) support for investments and implementation of safety equipment and facilities. Large road safety campaigns were launched to modify behavior to promote road safety, including topics such as “driving vehicles under alcoholic influence”, “driving vehicles at unreasonable speed”, and”usage of seatbelts and child restraint”. Blackspot elimination program on regional roads were also implemented. 4.29 The number of road accident fatalities in Poland declined from 5,500 (Baseline of the first project) in 2003 to 3,907 in 2010 which was well below the project target of <4700 fatalities. However, Poland failed to achieve its road safety program's Vision Zero target of reducing the fatalities by 50 percent between 2003 and 2013, and continued to have the highest fatality rate in the EU at 110 per million population, compared to the EU average of 60 in 2012. According to a report produced by Global Road Safety Facility in 2013, EU average in reducing road deaths over the period 2001-2011 was 45%, compared to 24% in Poland (Annex C). Achievement of the road safety objective is therefore rated as Modest. ROAD USER SATISFACTION 4.30 After the baseline survey conducted in 2004, GDDKiA conducted surveys in 2005, 2009 and most recently in 2011 with the help of a professional polling firm based in Poland. The last two surveys were carried out during this third project implementation period. The 30 latest survey found significant improvements in road user perceptions of both GDDKiA as an institution as well as of the physical condition of the national roads network. It showed a good awareness of GDDKiA programs and confirmed that road users noticed significant improvements in the national road network. In 2005 only 2-3 percent of users were found to have a positive opinion of the national road network, whereas by 2009 this proportion had increased to 30-35 percent, and in 2011 it was at 38 percent. The users’ enhanced awareness was noticed also for the increasing enforcement of road safety measures and improvements in road geometrics with elimination of accident black spots. The surveys have also sensitized GDDKiA to road user perceptions of GDDKiA own operations. While the results of the surveys show the desired improving trend, their frequency and regularity needs improvement. On balance, the achievement of this sub-objective is rated as Substantial. Efficiency 4.31 95 percent of the 2005-2006 contracts and 17 percent of the 2008 and 2011 contracts had ex-ante economic rate of return (ERR), using the HDM-4 model. The overall ex-ante ERR was 37 percent and the median ERR for periodic maintenance for overlay of 30-50mm was 43 percent, for rehabilitation it was 38 percent, and for widening of roads to 7.0 meters it was 28 percent. improvement An ex-post economic evaluation was conducted for the Implementation Completion and Results Report based on the actual traffic growth rate (3.4 percent p.a. versus 5.2 percent p.a. assumed at appraisal) and the actual costs (which were 2 percent above estimated costs), and the rate of return for a sample project was 33.4 percent, indicating a small reduction but still yielding an acceptable ERR. 4.32 GDDKiA’s assessments point to 75 percent of contracts having an ex-ante ERR above 20 percent. While the ex-post analysis was not done for all roads, the result of the sample project analysis indicates that it is likely that most roads had a rate of return above the benchmark of 12 percent and close to the appraisal value of 37 percent. Efficiency for this project is therefore rated as Substantial. Ratings OUTCOME 4.33 Relevance of objective was High, relevance of design was Modest, and efficiency was Substantial. Achievement of the quality objective was High, efficiency and road user satisfaction were Substantial, and financial viability and road safety were Negligible. Based on the above assessments the PPAR outcome rating is Moderately Satisfactory. RISK TO DEVELOPMENT OUTCOME 4.34 Risk to development outcome is rated Significant. Maintaining a balance between rehabilitation needs of the existing national roads network and new construction and improvement works, was an important consideration in the Bank’s support for Poland’s road maintenance and rehabilitation program. Government commitment to rehabilitation of the network, and to maintaining the network in a stable condition at around 60 percent in good condition, appears to have been sacrificed in recent years in order to support EU funded new 31 construction programs. Depriving road maintenance funding of its allocations from the Road Fund is one of the more unfortunate consequences of this shift in policy. The fiscal constraints facing the Government since the financial crisis in 2009 may have contributed to an inadequate appreciation of the damage done both to the physical network itself, and to the institutional capability for maintenance, by the frequent neglect of periodic maintenance. In the years 2011-2013 the allocations for current maintenance and rehabilitation reached around PLN2.5 billion and tolling of the newer expressways now under construction may provide additional revenues to the Road Fund. However, the Government needs to give a high priority to restoring the reliable and stable allocations for periodic maintenance. Road safety improvement program is expected to be funded in a stable manner following the legislation adopted in 2013 that directs all revenues from speed management program to the National Road Fund for road safety specific rehabilitation of national roads. BANK PERFORMANCE 4.35 Quality at Entry: The project concept with regard to objectives and the design of the components was certainly of high relevance to Poland’s needs at the time and continue to be so even now. The strong focus on road safety, systematic road maintenance planning using HDM-4, and reforms of road sector institutions indicate a high degree of Bank staff commitment to road sector development. However, in the execution of the project and in its monitoring and evaluation framework, insufficient attention appears to have been given to the financial viability aspects. In addition, the project monitoring indicators related to road safety were not in line with the higher ambitions of the national road safety program which was to reduce fatalities by 50 percent between 2003 and 2013. Rating: Moderately Satisfactory 4.36 Quality of Supervision: While the number and staffing of supervision missions was adequate, their effectiveness could have been greater. The Bank could have reacted more forcefully on the Government’s refusal to utilize the National Road Fund for maintenance and rehabilitation of the network. It is understandable that owing to the fast disbursing nature of the three operations, the Bank had little leverage over Government’s actions subsequent to the financial crisis in 2008/9. It would have been worthwhile to take a strong position, by for instance suspending disbursements or other such action, to indicate to Government the seriousness with which the Bank views such developments. Despite the shortcomings the PPAR has found that the Bank support has contributed significantly to the road maintenance and rehabilitation program in Poland, as evidenced in their continued request for the Bank’s technical assistance even after project closure. Rating: Moderately Satisfactory. 4.37 Overall rating of Bank Performance: Frequent maintenance funding shortfalls coupled with the higher/ excessive axle loads of Trans-European traffic, is putting the national roads network in Poland at heightened risk of premature failure. This adverse process has already begun with reducing proportions of the network in good condition in 2010 and 2011. The Bank as the agency most focused on maintenance and rehabilitation of the national road network, should have taken a more pro-active stance vis-à-vis the Government and the EU to alert them to the perverse effect of their funding priorities. Rating: Moderately Satisfactory. 32 BORROWER PERFORMANCE 4.38 Government Performance: The Ministry of Infrastructure/Transport has performed satisfactorily with regard to rapid compliance with conditions of Board presentation and effectiveness, as well as complying with its obligations regarding financial management and reporting. However, the Government’s commitment to the objective of improving the condition of the national road network appears to have been weakened by the fiscal crisis faced by Poland since 2008/9. In 2012 the road conditions improved when the Government invested in the long overdue periodic maintenance activities, but the prohibition on the use of the National Road Fund for maintenance expenditures, which should correctly be its only valid use, was caused by a shift in the government’s short term priorities during the fiscal crisis, and now that the situation has improved the prohibition could be lifted to restore funding of the maintenance and rehabilitation program. With regards to road safety, the National Road Safety Council and other agencies were not given clear leadership, accountability, and responsibility to ensure sound coordination between and among the stakeholders. Rating: Moderately Unsatisfactory 4.39 Implementing Agencies Performance: Both GDDKiA and the National Road Safety Council have performed well on their responsibilities under the project with a few shortcomings. GDDKiA implemented the road maintenance and rehabilitation component effectively, disbursing a bulk of the funds within a little more than a year, and the works produced have been found to be of good quality. The target to be achieved for road network condition was essentially met. Though the performance-based management and maintenance of roads pilot contract was unsuccessful, based on this experience GDDKiA did succeed in introducing a modified form of performance based maintenance contract (Sustained Standards contract) that was better suited to conditions in Poland and creates the possibility of progressive outsourcing of road maintenance works. Implementation of the Management Information Systems technical assistance components did suffer long delays and was only partially successful. The Enterprise Resource Planning system in particular encountered significant problems and some modules had to be dropped. Likewise the Intelligent Transport System Study was dropped due to lack of coordination between GDDKiA and the consultant on the objectives of the study. National Road Safety Council performed well on its campaigns to boost public awareness in a number of areas such as seat belt use, alcohol and driving, speeding, child safety, and pedestrian safety. Rating: Moderately Satisfactory 4.40 Overall Borrower Performance: Notwithstanding the above noted deficiency in the Government’s performance, the performance of the borrower overall has been pro-active and has resulted in a project that disbursed well and where a majority of the components were implemented efficiently. Overall Borrower performance is therefore rated as Moderately Satisfactory. 5. Lessons 5.1 The assessment of three consecutive projects supporting Poland’s road maintenance and rehabilitation projects have found that when planning sequential operations that involve 33 sector reforms, it is important to ensure that sufficient disbursement leverage is retained to ensure the satisfactory implementation of the institutional development components, as well as a realistic time-frame for the operation. In retrospect, a critical issue raised by the experience of Poland’s Road Maintenance and Rehabilitation Program is the appropriateness of a strategy of funding fast disbursing physical works, coupled with an institutional development program which required a much longer period for implementation. A majority of the funding for physical works, which constituted about 90 percent of the loan amounts, had been disbursed in the first three years of the program, 2004 through 2006. When subsequent issues arose, most particularly with respect to maintenance funding and the availability of the National Road Fund for maintenance and rehabilitation of the network, the Bank did not have effective recourse to the usual remedies of suspension of loan disbursements. While principled persuasion is always possible, the added leverage of withholding funding is often needed to get the attention of senior policy makers. 5.2 The second lesson relates to the use of country systems which worked when combined with adequate ex-post audits, as it contributed to fast disbursement of quality works. The Sector Wide Approach adopted for the implementation of civil works using the borrower’s own systems worked well in a country where there is already a functioning system. Rehabilitation works using country systems for small contracts (each worth less than EUR 6 million) led to a cost effective program with high rates of return. Fiduciary and safeguards oversight can be based on ex-post audits of the efficacy of internal controls. Quality control based on third party technical audits also proved adequate and efficient. This modality for implementation is not used often in Bank projects, and the experience here suggests that for future Bank funded operations, upfront assessment should be made systematically to see whether the country systems can be used. Packaging of the works into smaller contracts also appears to have benefitted the local construction industry and allowed faster implementation of the rehabilitation program. 5.3 The third lesson regards the use of HDM-4 to ensure the efficiency of the rehabilitation program. While it is important to ensure that individual rehabilitation projects have an adequate ERR, it is also important to clear the maintenance backlog in a reasonable period of time within available budgets. This requires that maintenance budget constraints be factored into the choice of maintenance interventions. The roads in poor condition at the start of the program in 2003 totaled about 30 percent of the national roads network. In 2011, there were still about18 percent of the network in poor condition. This means the program over a 8 year period had not managed to eliminate the maintenance backlog of roads in poor condition. This failure relates principally to the shortfalls in maintenance budgets, and could probably have been avoided if the HDM-4 analysis had considered the budget constraints and chosen rehabilitation alternatives that fitted within the budget envelope. 5.4 Finally, it is important that all objectives set for the program be reflected in the monitoring indicators for the program. Absent this the objective is unlikely to receive the attention needed. The absence of clearly monitorable indicators for maintenance funding may have contributed to the inadequate attention given to establishing a stable and reliable source of funding for maintenance of the national roads network. Road safety, on the other hand, was identified as a critical issue for the country to address at project design stage, and consequently components and monitoring indicators were included but lacked a specific 34 objective in the first two projects. While the third project explicitly included a road safety objective, the related indicators were under ambitious compared to the country’s overall goal. 35 References World Bank. 1988. “Road Deterioration in Developing Countries – Causes and Remedies”. Washington, D.C. ______. 2002. Poland Country Assistance Strategy, Washington, D.C. ______. 2004. Road Maintenance and Rehabilitation Project, Project Appraisal Document, Report No. 27577 – POL. Washington D.C. ______. 2005a. Poland Country Partnership Strategy, Washington, D.C. ______. 2005b. Second Road Maintenance and Rehabilitation Project, Project Appraisal Document, Report No. 31324-PL. Washington D.C. ______. 2006. Road Maintenance and Rehabilitation Project III, Project Appraisal Document, Report No. 35324-PL. Washington D.C. ______. 2009a. Poland Country Partnership Strategy, Report No. 48666-PL. Washington, D.C. ______. 2009b. Road Maintenance and Rehabilitation Project, Implementation Completion and Results Report, Report No. ICR0000540. Washington, D.C. ______. 2009c. Second Road Maintenance and Rehabilitation Project, Implementation Completion and Results Report, Report No. ICR0000541. Washington, D.C. ______. 2010. Transport Policy Note – Toward a Sustainable Land Transport Sector, GAMBIT 2005, National Road Safety Council, Poland, 2005 ______. 2012. Road Maintenance and Rehabilitation Project III, Implementation Completion and Results Report, Report No. ICR2141. Washington, D.C. ______. 2013. Global Road Safety Facility. “Country Report on Poland Road Safety Management Capacity Review 37 ANNEX A Annex A. Basic Data Sheet ROAD MAINTENANCE AND REHABILITATION PROJECT (LOAN NO. 72230-POL) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 195.2 324.3 166 Loan amount 124.7 122.0 98 Cancellation - 0.74 - Cumulative Estimated and Actual Disbursements FY04 FY05 FY06 FY07 FY08 FY09 Appraisal estimate (US$M) 120.0 123.0 126.0 126.0 126.0 126.0 Actual (US$M) 1.23 114.6 120.3 120.3 122.0 122.0 Actual as % of appraisal 1.0 93 95 95 96 96 Date of final disbursement: June 2009 Project Dates Original Actual Negotiations 02/17/2004 02/17/2004 Board approval 03/30/2004 03/30/2004 Signing 04/07/2004 04/07/2004 Effectiveness 05/03/2004 06/10/2004 Closing date 06/30/2006 12/31/2008 Staff Inputs (staff weeks) Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff weeks USD Thousands (including travel and consultant costs Lending FY03 23.9 62.8 FY04 31.8 101.1 Total 56.3 164.7 ANNEX A 38 Supervision/ICR FY04 0.7 0.85 FY05 21.1 58.6 FY06 18.1 52.6 FY07 9.6 30.0 FY08 12.3 31.8 FY09 17.0 51.6 Total 78.9 226.0 Task Team Members Responsibility/ Names Title Unit Specialty Lending Michel Audigė Lead Transport Specialist ECSSD Team Leader Anca Dumitrescu Transport Specialist ECSSD Team Leader Andreas Schliessler Senior Transport Specialist ECSSD Peer Reviewer Barbara Letachowicz Environmental Specialist ECSSD Environment William Paterson Consultant EASTR Peer Reviewer Chiyo Kanda Quality Reviewer OPCIL Quality Chiyo Kanda Senior Operations Officer OPCIL Peer Reviewer Claudia Pardinas-Ocana Senior Counsel LEGEC Legal Daria Goldstein Counsel LEGEC Legal Elena Kastlerova Transport Specialist ECSIE Transport Elzbieta Siemenska Procurement Specialist ECSPS Procurement Iwona Warzecha Senior Financial Management ECSPS Financial Specialist Management Marie Laygo Program Assistant ECSSD Team Support Supervision/ICR Michel Audigė Lead Transport Specialist ECSSD Team Leader Anca Dumitrescu Transport Specialist ECSSD Team Leader Radoslaw Czapski Operations Officer ECSIE Team Leader Claudia Pardinas-Ocana Senior Counsel LEGEC Legal Daria Goldstein Counsel LEGEC Legal Iwona Warzecha Sr. Financial Mgmt Specialist ECSPS Financial Mgmt Karina Mostipan Sr. Procurement Specialist ECSPS Procurement Magdalena Wasik Program Assistant ECCU7 Team Support Malgorzata Michnowska Program Assistant ECCU7 Team Support Marcin Jan Sasin Economist ECSPE Economic Analysis Marie Laygo Program Assistant ECSIE Team Support Piotr Krzyzanowski Senior Environmental Specialist ECSSD Safeguards Robert Kietlinski Senior Operations Officer ECSIE Operations Zoe Kolovou Senior Counsel LEGEC Legal Ross Pavis Operations Officer ECSSD ICR Team Leader Graham Smith Consultant ECSSD ICR Author 39 ANNEX A Other Project Data Borrower/Executing Agency: Ministry of Finance, Republic of Poland/General Directorate for National Roads and Motorways (GDDKiA) Follow-on Operations Operation Loan no. Amount Board date (US$ million) Second Road Maintenance and Rehabilitation IBRD- 130.5 March 29, 2005 Project 72820 ANNEX A 40 SECOND ROAD MAINTENANCE AND REHABILITATION PROJECT (LOAN NO. 72820- POL) Key Project Data (amounts in US$ million) Actual as % of Appraisal Actual or appraisal estimate current estimate estimate Total project costs 208.0 371.3 178 Loan amount 130.5 121.3 93 Cancellation - 0.79 - Co-financing 13 N.A.* N.A.* *Actual contribution by the Borrower including through external sources, such as EU grants or EIB loans is estimated to amount to $250 million. Cumulative Estimated and Actual Disbursements FY05 FY06 FY07 FY08 FY09 FY10 Appraisal estimate (US$M) 70.0 128.0 130.5 130.5 130.5 130.5 Actual (US$M) 0.0 115.6 116.6 117.3 121.3 121.3 Actual as % of appraisal 70 Date of final disbursement: December 2009 Project Dates Original Actual Negotiations 02/17/2005 02/17/2005 Board approval 03/30/2004 03/29/2005 Signing 04/26/2005 04/26/2005 Effectiveness 05/02/2005 06/16/2005 Closing date 12/31/2007 06/30/2009 Staff Inputs (staff weeks) Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs Total 78.9 226.0 Lending FY05 35.7 112.9 Total 35.7 112.9 Supervision/ICR FY05 0.2 0.1 41 ANNEX A FY06 30.2 77.1 FY07 15.6 54.2 FY08 16.7 58.6 FY09 19.4 58.2 Total 82.1 248.2 Task Team Members Responsibility/ Names Title Unit Specialty Lending Michel Audigė Lead Transport Specialist ECSSD Team Leader Anca Dumitrescu Transport Specialist ECSSD Team Leader Andreas Schliessler Senior Transport Specialist ECSSD Peer Reviewer Barbara Letachowicz Environmental Specialist ECSSD Environment William Paterson Consultant EASTR Peer Reviewer Chiyo Kanda Quality Reviewer OPCIL Quality Chiyo Kanda Senior Operations Officer OPCIL Peer Reviewer Claudia Pardinas-Ocana Senior Counsel LEGEC Legal Daria Goldstein Counsel LEGEC Legal Elena Kastlerova Transport Specialist ECSIE Transport Elzbieta Siemenska Procurement Specialist ECSPS Procurement Iwona Warzecha Senior Financial Management ECSPS Financial Specialist Management Marie Laygo Program Assistant ECSSD Team Support Supervision/ICR Michel Audigė Lead Transport Specialist ECSSD Team Leader Anca Dumitrescu Transport Specialist ECSSD Team Leader Radoslaw Czapski Operations Officer ECSIE Team Leader Claudia Pardinas-Ocana Senior Counsel LEGEC Legal Daria Goldstein Counsel LEGEC Legal Iwona Warzecha Sr. Financial Management ECSPS Financial Specialist Management Karina Mostipan Sr. Procurement Specialist ECSPS Procurement Magdalena Wasik Program Assistant ECCU7 Team Support Malgorzata Michnowska Program Assistant ECCU7 Team Support Marcin Jan Sasin Economist ECSPE Economic Analysis Marie Laygo Program Assistant ECSIE Team Support Piotr Krzyzanowski Senior Environmental Specialist ECSSD Safeguards Robert Kietlinski Senior Operations Officer ECSIE Operations ANNEX A 42 Zoe Kolovou Senior Counsel LEGEC Legal Ross Pavis Operations Officer ECSSD ICR Team Leader Graham Smith Consultant ECSSD ICR Author Other Project Data Borrower/Executing Agency: Borrower/Executing Agency: Ministry of Finance, Republic of Poland/General Directorate for National Roads and Motorways (GDDKiA) Follow-on Operations Operation Loan no. Amount Board date (US$ million) Third Road Maintenance and IBRD- 180.20 June 6, 2006 Rehabilitation Project 73840 43 ANNEX A THIRD ROAD MAINTENANCE AND REHABILITATION PROJECT (LOAN 73840-POL ) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 376.20 453.39 120.52 Loan amount 180.20 195.45 108.46 Co-financing 166.00 202.77 122.15 Cancellation - 2.67 - Cumulative Estimated and Actual Disbursements FY07 FY08 FY09 FY10 FY11 Appraisal estimate 144.00 155.00 164.00 172.00 180.20 (US$M) Actual (US$M) 151.80 153.80 179.80 186.70 190.90 Actual as % of appraisal 105 99 110 109 106 Date of final disbursement: Project Dates Original Actual Appraisal - 03/08/2006 Board approval 06/06/2006 06/06/2006 Signing 06/23/2006 06/23/2006 Effectiveness 05/07/2006 07/05/2006 Closing date 09/15/2011 09/15/2011 Staff Inputs (staff weeks) Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of staff weeks US Dollar (including travel and consultant costs) Lending FY06 33.52 119.81 FY07 - 134.94 Total 33.52 254.75 Supervision/ICR FY07 28.53 79.35 FY08 23.28 98.22 ANNEX A 44 FY09 23.23 92.03 FY10 26.57 92.27 FY11 47.91 109.93 FY12 13.83 67.60 Total 163.35 539.40 Task Team members Responsibility/ Names Title Unit Specialty Lending Michel Audige Sector Manager, Transport ECSS5 Program Team Leader Radoslaw Czapski Sr. Infrastructure Specialist ECSS5 Transport and Operational Support Anea Cristina Dumitrescu Sr. Transport Spec. AFTTR TTL for Preparation Daria Goldstein Sr. Counsel EEGAF Lawyer Marie Antoinette Laygo Program Assistant ECSSD Team Assistant Barbara Letachowicz Operations Officer ECSS3 Environment and Safeguards Karina Mostipan Sr. Procurement Specialist ECS02 Social Safeguards Procurement Jan Pakulski Sr. Social Development & ECSS4 Social Safeguards Civil Iwona Warzecha Sr. Financial Management ECS03 Financial Management Specialist Supervision/ICR Michel Audige Sector Manager, Transport SASDT Program TL Radoslaw Czapski Sr. Infrastructure Specialist ECSS5 TTL May 2008 Andreas Schliessler Lead Transport Spec. ECSS5 TTL 2006-May 2008 Jaroslaw Giemza Consultant ECSS5 Road Safety Expert Galina S. Kuznetsova Sr. Financial management ECS03 Financial Management Specialist Marie Antoinette Laygo Program Assistant ECSSD Team Assistant Barbara Letachowicz Operations Officer ECSS3 Environment and Safeguards Malgorzata Michnowska Program Assistant ECCPL Local Team Assistant Karina Mostipan Sr. Procurement Specialist ECS02 Procurement Audit Jan Pakulski Sr. Social Development & ECSS4 Social Safeguards and Civil Communications Expert Ireneusz M. Smolewski Senior Procurement ECS02 Procurement Specialist 45 ANNEX A Antti P. Talvitie Consultant MNSTR Road Sector Specialist Iwona Warzecha Sr. Financial Management ECS03 Financial Management Specialist Barbara Ziolkowska Procurement Analust ECS02 Procurement Coral Bird Program Assistant ECSS5 Team Assistant ANNEX B 46 Annex B. National Roads Expenditures, Financing, Outputs and Condition 2003- 2010 Calendar Year 2003 2004 2005 2006 2007 2008 2009 2010 2011 Road Maintenance and Rehabilitation Projects I, II & III Implementation Periods: Road Maintenance and Rehabilitation Project X X X X X Second Road Maintenance and Rehabilitation Project X X X X X Third Road Maintenance and Rehabilitation Project X X X X X X National Road Maintenance & Rehab Expenditure (m PLN): 1776* 1744* 1615+ 2608+ 2290+ 3053+ 1521+ WB Roads II Project (m PLN) 62 19 WB Road Maintenance and Rehabilitation Project (m PLN) 399 2 WB Second Project (m PLN) 383 WB Third Project (m PLN) 454 55 14 EIB Loans (m PLN) 157 547 National Roads Treated under WB Funding (km)** 917 749 1053 ~40 ~10 National Roads Network Condition (% Good)++ 37.0% 45.5% 48.9% 53.2% 54.9% 53.6% 59.6% 59.1% 58.7% * Third Project Appraisal Document + Transport Policy Note 2011, World Bank ** Implementation Completion Reports of the three projects ++ GDDKiA Dept of Studies (HDM Unit) 47 ANNEX C Annex C. Trends in Road Traffic Fatalities in EU Countries 2001-2011 Source: Country Report on Poland Road Safety Management Capacity Review Report Number 7819-PL, June 2013 ANNEX D 48 Annex D. Use of HDM-4 under Budget Constraints The national road network in Poland comprises 580 km of motorways and 13,600 km of national roads. In addition there are about 190,000 km of secondary and local roads. The Central Roads Agency GDDKiA administers the motorways and national roads. At the time of Poland’s accession to the European Union (May 2004) the road network was much less extensive than the average for the EU-15 countries: excluding local roads, the road density, in km per 100 sq km, was 43 km in the EU-15 countries versus 12 km in Poland. The network was also in poor shape, with about 30 percent (or 4000 km) of the national roads in poor condition and requiring rehabilitation. Furthermore, budgetary constraints affected maintenance, and could cause the network to deteriorate rapidly. Increasing trans- European truck traffic which has higher axle loads, could also cause extensive damage. The project financed rehabilitation of 1103 km of national roads at a cost of $438 million (at 3.45 PLN/USD), which is an average cost of $400,000 per km. At this unit cost per km clearing the backlog of national roads in poor condition would cost about $1.6 billion. Unit costs for different types of rehabilitation of such roads in Poland range between $200,000 per km for an overlay to $610,000 per km for full reconstruction. The program financed by the project is shown to the left in the Table below. In choosing the type of rehabilitation, alternatives that could have reduced the initial cost were not evaluated. An illustrative “budget constrained” strategy with lower initial costs, is shown in the Table on the right. While the latter may entail an increase in vehicle operating costs and shorter pavement life, it produces a 40 percent saving in the up-front rehabilitation cost. Such an approach stretches available resources in order to restore the network quickly, thereby slowing or halting further deterioration of the network. Project Rehabilitation Costs* Costs Reduced Under Budget Constraint Road Works Type Avg $/km Length Cost Road Works Type Avg $/km Length Cost (000) ( %) ($m) (000) ( %) ($m) Perdc Maint 30-50 mm 135 12 25 Perdc Maint 30-50 mm 135 12 25 Strengthng 80/270 mm 200-470 61 240 Strengthng 80-100 mm 200-250 84 208 Full Reconstruction 610 24 144 Full Reconstruction 610 0 Widening to 7m 810 4 30 Widening to 7m 810 4 30 Total Cost 100 439 Total Cost 100 263 The analyses required to identify and evaluate such alternatives is possible using the Bank’s HDM-4 model. In the case of Poland, HDM 4 was only used to assess Economic Rates of Return (ERR) for a specified rehabilitation option, as required under the loan agreement. There was no least-cost optimization using budget constraints. The ERRs determined are in the table below, and are generally well above what is required. But they give no indication of the large cost savings possible in the initial cost if a slight increase in the lifecycle cost is accepted. 49 ANNEX D This analysis shows that focusing on just the ERR misses an important consideration – reducing investment cost is especially important when budgets are constrained as in the case of Poland as this could affect the sustainability of the overall rehabilitation program. The large discontinuity that typically exists in the unit costs of a full reconstruction versus a thick overlay (80 - 100 mm) makes it important that the HDM 4 analysis globally assesses both the ERRs and viable lower cost solutions to optimize the overall rehabilitation program. Road Works Type Avg $/km Average annual Average daily traffic (vpd) (000) ERR (%) Thin Overlay 30-50 mm 135 7229 43 Strengthng 80/270 mm 200-470 10,056 38 Full Reconstruction 610 9,752 19 Widening to 7m 810 6,966 28 50 ANNEX E Annex E. List of Persons Met Government 1. Mr. Jarosław Waszkiewicz, Roads and Motorways Department, Ministry of Infrastructure 2. Mr. Rafał Wójcik, Roads and Motorways Department, Ministry of Infrastructure 3. Mr. Michał Perliński, Roads and Motorways Department, Ministry of Infrastructure 4. Mr. Tomasz Michalski – Director, Transport Policy Department, Ministry of Infrastructure Implementing Agencies 5. Ms. Katarzyna Turska - Director, Paulina Karbowy - Project Coordinator, National Road Safety Council 6. Mr. Andrzej Maciejewski – Director, Road Maintenance Department, GDDKiA 7. Ms. Joanna Nurkiewicz - Deputy General Director, Road Maintenance Department, GDDKiA 8. Ms. Katarzyna Gnyp - Chief Accountant, GDDKiA 9. Mr. P.Skociński - Director, GDDKiA 10. Ms. Urszula Nelken – Spokesperson, GDDKiA 11. Ms. Anna Bołtryk - PR Team Member, GDDKiA 12. Mr. Marek Rolla - Director, Department of Studies GDDKiA 13. Mr. Gronkiewicz, Gubała, Byczewski, Warsaw Branch, GDDKiA Financiers 14. Robert Kietliński - Transport Sector Engineer, European Investment Bank 15. Messrs. Radek Czapski, Senior Infrastructure Specialist, World Bank 16. Mr. Jarek Giemza, Consultant, World Bank 17. Mr. Xavier Devictor, Country Manager for Poland, World Bank