ICRR 14234 Report Number : ICRR14234 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/25/2014 Country : Jordan Project ID : P081505 Appraisal Actual Project Name : Jo - Amman US$M ): Project Costs (US$M): 161.0 456.24 Development Corridor L/C Number : L7228; L7634 Loan/ US$M ): Loan /Credit (US$M): 38.0 71.0 Sector Board : Transport US$M): Cofinancing (US$M ): 71.9 161.03 Cofinanciers : Arab Fund for Board Approval Date : 06/15/2004 Economic and Social Closing Date : 06/30/2009 12/31/2012 Development (AFESD), European Investment Bank (EIB) Sector (s): Roads and highways (76%); Central government administration (16%); General transportation sector (8%) Theme (s): Trade facilitation and market access (40% - P); Infrastructure services for private sector development (40% - P); Regional integration (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Maha J. Armaly Peter Nigel Freeman Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: According to the Loan Agreement, the objectives of the project were “to assist the Borrower in: (i) supporting more efficient transport and logistics services by removing key infrastructure transport bottlenecks; and (ii) providing access to affordable land for productive investment and urban development purposes .� (Schedule 2, p.13). The objectives as stated in the Project Appraisal Document (PAD) were identical (p. 11). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: A. Amman Ring Road (ARR), Phase 1 ($106.91 million planned, $279.87 million actual): This component included construction of four sections of the ring road totaling 40 km financed by three different donors : an 18.5 km segment from the Desert Highway 7, which is about 7 km north of Queen Alia International Airport northeast to about 1 km south of the Madounah Road Intersection, including two interchanges and bidirectional service roads (financed by the Arab Fund for Economic and Social Development - AFESD); a 14.5 km section of 2-lane, dual carriageway connecting the first section and the Zarqa Highway, including the Madounah road Interchange and access to the site of the new Customs Depot and Inland Port facilities, the Fork Interchange with the Zarqa Eastern Bypass, and some service and access roads (financed by IBRD); and two segments financed by the European Investment Bank (EIB)– (a) the Zarqa Eastern Bypass connecting the Phase 1 Ring Road with the Zarqa/Syrian Border Highway (4.6 km), including an interchange) and (b) the last 2.4 km of the Ring Road called the Zarqa Through Link to the city of Zarqa, including an intersection with the Amman Highway and a viaduct . B. Inland Port and Trucking Industry ($8.06 million planned, $3.01 million actual): This component was to support development of an inland port at the Madounah Interchange by constructing access infrastructure (roads and fencing) and extending trunk and network infrastructure for utility services . The Inland Port was to include a Logistics Service Center (a customs-bonded area with transport and logistics -related activities), a new Customs Depot, and a Business Zone surrounding the Center . It was to finance relocating the Amman Customs Depot to the Logistics Service Center and to support ongoing reforms to (a) consolidate the trucking industry by reducing the number of operators; (b) create competition to improve efficiency and service quality; and (c) renew the trucking fleet. Assistance was to be provided to select a private operator to develop and operate the Inland Port, including preparing studies to identify and develop an appropriate regulatory structure for the Inland Port, tender and transaction documents for selecting the developer /operator, and an operating contract /agreement with specified standards. C. Project Management and Amman Development Corridor Planning and Promotion ($4.71 million planned, $14.66 million actual): The component was to fund: (a) technical support for urban planning initiatives, including a master plan for the Amman Development Corridor and a technical unit in charge of promoting and guiding development within the Corridor; (b) transport planning and traffic engineering support to improve the coordination nd rd and integration of transport, urban development, and land use initiatives; (c) master planning for the 2 and 3 phases of the Ring Road (the western and northern sections ); and (d) project management and specialized skills for monitoring and evaluation, environmental management and resettlement, urban and transport planning, traffic engineering, and logistics. Land Acquisition and Compensation Costs ($40.9 million planned, $158.43 actual). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Costs : Due to unanticipated increases in the cost of key inputs and technical modifications to the design during construction, actual total project costs ($456.24 million) were 2.8 times the appraisal estimates ($160.98 million). Financing : The three financiers provided additional financing of a total of $ 295.26 million in 2008. The Bank provided a total loan of US$71 million; The EIB provided US$82.87 million (including a grant of US$4.92 million for the Master Plan and Ring Road Phase 2 and 3 studies); and the AFESD provided of US$78.16 million. The ICR, and this review, assesses the full project, including the portions /sub-components with parallel financing from the EIB and the AFESD. Borrower Contribution : The borrower ultimately contributed $ 194.63 million, nearly 4 times the appraisal estimate of $51.08 million. Of the total government contribution, $ 158 million was in land acquisition. Dates : The project closing date was extended twice, first for 30 months until 12/31/2011 and then for 12 more months until 12/31/2012. A number of outputs were not yet delivered by project closing . 3. Relevance of Objectives & Design: a. Relevance of Objectives: Modest . The Project's objective to support efficient transport and logistics services by removing key infrastructure services is in line with the Country Partnership Strategy (CPS- FY12-15) which aims to support sustainable growth through investments in core Infrastructure . Transport accounts for about 11 percent of GDP of Jordan and earns foreign exchange through the provision of transshipment services . Jordan has a developed highway and roads network, but certain links in the trunk road system are missing . Together with the EIB, the Government is preparing a National Highway Master Plan that will serve as the basis for all major inter -urban highway sector developments in Jordan in 2010-30. The Government’s new Executive Development Program (EDP 2011-2013) builds on the National Agenda and serves as an action plan for the Government for the next stage . The aim of the EDP in the transport sector is to provide safe, effective and economic transportation services through efficient transport and logistics services supported by removing key infrastructure bottlenecks . However, while improving competitiveness and attracting investments is a strategic goal in the CPS, access to affordable land was not identified as a bottleneck to this objective, and urban development was not a priority in this latest CPS. b. Relevance of Design: Modest . The design of the Project fairly addresses the first stated objective of the Project to support efficient transport and logistics services by removing key infrastructure transport bottlenecks . The construction of the Amman ring road aimed in part to increase the efficiency of transport between the international airport in Amman and Jordan ’s main industrial city of Zarqa, and to relieve congestion in Amman by rerouting traffic away from the city center . The development of an inland port with appropriate logistics aimed to build the necessary infrastructure to facilitate transport of trade and to remove existing bottlenecks . Transport sector studies and master plans were to provide further guidance to take appropriate reform actions . The objective to provide access to affordable land for productive investment and urban development was ambitious for the short time period and for the inputs provided through the Project . The master plans and promotional activities envisaged under the Project would only provide seeds for further action plans and activities that would begin to address the objective proposed . The ICR states (p. 9), that indicators did not include measurable and attributable indicators for the land development objective. Appropriate intermediate indicators were necessary to track costs and traffic volumes . 4. Achievement of Objectives (Efficacy): Objective 1: Supporting more efficient transport and logistics services (Modest ) Outputs : — The ring road Phase 1 is 84% completed at almost 3 times the cost. Sections 2 and 3 (financed by IBRD and EIB respectively) are 100% completed. Section 1, financed by AFSED, is 60% complete. — Preliminary work (8 Km of access roads, 27 hectares of parking, drainage, water pipes, electric cables, land clearing) conducted, but incomplete. — Customs depot (EIB financing), inland port and logistics framework not undertaken . — Report on trucking sector review funded by the European Union (EU). Some reform actions reported. Additional broader actions need to be accomplished . — Amman Metro Area Transport planning completed . — Feasibility studies for Phase 2 & 3 and for mass transit were completed and financed by EIB . PPP study to construct Phase 2 and 3 financed by IFC. Outcomes — Cost of transporting 1 TEU did not reach objective largely due to 250% increase in oil prices — Travel time from Airport to Zarqa reduced from 60 to 48 minutes, but did not reach target of 40 minutes — Traffic diverted from urban streets (8% vs. target of 10%) — Cargo traffic increased, but not due to the Project . — The share of over-age heavy trucks dropped from two -thirds in 2003 to less than a third by 2009, but excess in heavy long-distance freight vehicles remains a problem . Stronger enforcement of existing regulations needed . Overall the Project fell short of achieving this objective with major shortcomings in achieving efficiency in the logistics services. At completion, the project is not finished and final costs are not determined . At least one outcome listed (increase in cargo volume) is achieved but not due to the Project – as also mentioned in the ICR (cargo depot not completed). Reform outcome (in terms of trucking sector restructuring ) is mixed, and is difficult to associate directly with the Project study. There is minor shortcoming in the achievement of efficiency in travel time . Objective 2: Providing access to affordable land for productive investment and urban development purposes (Negligible ) Outputs : — In principle, the ring road provides access to newly cleared lands with minimal and limited services – but these are far from ready for productive investments and urban development . — Master Planning for ADC area completed (financed by EIB) but at about 4 times the appraisal and revised estimated costs. No promotional activities reported . Outcomes : — The completed Master Plan to develop 300 sq. Km bounded by the ring road lays the foundation for local governments to coordinate long term actions for developments around the ring road . 5. Efficiency: At appraisal, the ERR for the road component was calculated at 17.3%. It was recalculated in 2008 at 12.73%. Using the same methodology, the ERR was calculated at 12.71% in the ICR prepared in May 2013. For the road component, costs went from $106.92 million at appraisal to $278.87 million at project end. The increases were attributed to fuel costs, material costs, inflation, and lack of success in initial bidding . Government fiscal tension, translated into sector expenditure ceilings which impacted donor funded projects, and contributed to the slow down in the implementation of the project. Unit costs/km built under the Project varied among the sections of the road (roughly US$5-6 million for sections 1 and 2, and 12 million for section 3), and are higher than the current average cost (US$4-5 million- source ICR team). Sections of the road are not complete, and final costs may be higher . With respect to the second component (inland port, customs depot and studies ), no ERR was calculated at appraisal . Except for the trucking industry, and some infrastructure planning and access, this component had been practically dropped and expenditures were well below appraisal and revised budgets . Land acquisition costs were much higher than estimated at appraisal, in part because owners were fully compensated rather than being paid 75% of valuations as provided by law as well as due to an increase in the number of impacted persons. Expenditures under the third component were 24% above the revised budget and about 3 times the original budget. In addition, ICR footnote on p. 22 indicates that more financing (about US$29.58 million) is yet to be obtained. Efficiency is rated modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 17.3% 100% ICR estimate Yes 12.71% 90% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of objectives and design is modest . The objective of supporting more efficient transport and logistics services is modestly achieved, while the achievement of the objective of providing access to affordable land for productive investment and urban development purposes is negligible . Efficiency is modest. These ratings amount to significant shortcomings that render an unsatisfactory outcome rating . a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR indicates major decisions that are necessary to ensure the sustainability of the Project remain unresolved. Maintenance of the ring road is likely to take place, but there are signs (theft of copper wires) that show that more concrete action is needed . The ICR team indicated that the Government usually pays more attention to rehabilitation than maintenance. Attention is needed to ensure non -encroachment on the land, particularly by temporary transient traders. The debate continues whether to employ private sector contractors for operations and maintenance. The decision on whether to build the custom depot and how to manage the associated facilities also remain unresolved with the Government wavering on what to do with the now cleared land and other facilities that were built. Further delays remain a possibility, nevertheless, with regards to the completion of Section 1 of the road, the ICR team confirmed that financing is now secured from the United Arab Emirates . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: The Bank identified the project based on borrower priorities . According to the ICR team, The Bank, with Borrower agreement realized that a project focusing on the ring road only was not in line with the latest thinking and practices in the transport sector, and therefore development around the ring road area was also envisaged – though with insufficient preparation for the stated objective . The Bank was successful in working with the Government and several donors in putting together the necessary financing package and investment plan . The large investments and associated land acquisition was at a ready stage . Land acquisition framework was well prepared. Safeguard and fiduciary arrangements were in place . The M&E system was weak for measuring outcomes of the second objective as stated . Studies included in the project lay the foundation for reforms and better articulated planning of the Amman Metropolitan Area in the future . Fiduciary aspects were also ready . With this background, the Project was prepared in a surprisingly short time (2 months). The large cost overruns due to optimistic estimates and missing technical information, point to insufficient technical preparation of the Project . According to the team, the project costs were not revised from their initial costs in 2000 to 2004 prices. Similarly, the procurement schedule was optimistic and did not anticipate the time required for prequalification. The mismatch between the design of the Project components to deliver the second objective weakens the quality at entry . at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: Considerable attention is given to the social and environmental safeguards which were properly undertaken . The Bank considered fiscal constraints of the Government, provided and mobilized additional financing from participating donors. Economic analysis was undertaken at mid -term and for project restructuring to ensure the economic viability of the ring road remains valid despite the increased costs . The Bank coordinated the implementation of the large project financed by several donors . However, the Bank team focused attention mainly on the road component at the expense of other components . Restructuring was not comprehensive and did not address the issue of the second objective (urban development, access to productive land ). M&E indicators were changed during restructuring and implementation, and were not consistently utilized. The second component remained intact (additional financing provided) despite signs of Government wavering on its decisions as early as 2006 (para 21). The Bank did not follow up on institutional arrangements agreed during preparation (steering committees and technical units ) that were to be established to advance the second component in particular, including the promotional aspects of urban development . No alternative arrangements were proposed . Skills in urban and commercial development of a new zone were not utilized appropriately to address the problems that arose during implementation . The project was rated satisfactory throughout its lifetime despite major delays, cost overruns, changes in government preferences, and dropping components, which indicates that the team did not well appreciate the severity of the implementation problems. Quality of Supervision Rating : Unsatisfactory Overall Bank Performance Rating : Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government provided, in a timely manner, the financial resources required for the Project including for the increased costs of land acquisition as well as for the project ’s investments. Nonetheless, the inconsistency of the Government’s actions, reversals of earlier decisions without explanation and failure to deliver decisions promised on outstanding issues, have contributed to the weak and delayed project achievements . New government imposed budgetary constraints and the abolition of fuel subsidies due to the fiscal crisis impacted the performance of the project in respect of cost and delays . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: The Ministry of Public Works and Housing (MoPWH) devoted much thought and effort to the project at all stages, particularly basic conception, coordination with other government bodies, selection and management of effective consultants and contractors . Greater attention to the adequacy of cost estimates when the original tenders were considered would have probably avoided rebidding . Other than initial delays in the bidding process, the Ministry supervised the work and worked collaboratively with the contractors . Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: M&E design at appraisal included measurable outcome and intermediate targets with baseline for the first objective. Traffic statistics (travel time and volume of traffic diverted from urban areas ) were to be measured through annual surveys, while the costs and volume of cargo transport were to be taken from statistics from the Ministries of Transport and Finance. Intermediate indicators were to be measured through annual surveys on licensed trucking firms, ADC development, and traffic counts . As to improvements in the consolidation of the trucking industry, numbers of licensed firms and general cargo volumes were to be measured annually . Physical progress was to be measured from consultants and contractors reports . Performance measures for the second objective of the project were not adequate. b. M&E Implementation: During restructuring, outcome indicators were consolidated . The indicator for development within ADC zone was dropped, but the related second objective did not change . Intermediate indicators were also consolidated . The ICR clarified that the increase in cargo volumes at the existing port had increased but not due to the project (since the inland port and associated services were not built ). The ICR team confirmed that surveys were undertaken as planned. There is no reporting on type of urban and industrial transactions as this component was in effect dropped . The ICR team indicated these data were reported consistently by the implementing agency, but because they were intermediate indicators, they were not used consistently . c. M&E Utilization: The ICR (p.9) indicates that the M&E indicators were not well utilized to track progress of the project towards its objective by either the Bank or the MoPWH . The MoPWH, as the implementing agency, used M&E data to track the physical development of the Project in terms of monitoring the environmental aspects of the construction and road operations (p. 9). A local research institution used data on the land development dimensions of the project and focused on region-wide aggregation of data. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: According to the PAD (pp. 20-21), the project is classified under Category A for Environmental Assessment . The Project triggers the Environment Safeguard (OP/BP 4.01); Cultural Property (OP4.11); and Involuntary Resettlement (OP/BP4.12). Safeguard issues were comprehensively addressed at all stages of the Project . Identifying impacts were initiated in 1999 and updated in 2003. Environment : An Environmental Management Plan (EMP) was completed before the start of the project . Contractors were required to adhere to environmental and safety standards . According to the ICR (p. 10) the PMT, with the help of consultants developed a thorough scheme to closely monitor contractor ’s performance and made a positive difference to the quality of environmental performance . Cultural Heritage : A Cultural Resources Management Plan (CRMP – PAD Annex 10A) was part of the EMP. The ICR does not elaborate on the plan ’s implementation. Involuntary Resettlement . Land Acquisition and Resettlement Plan (LARP – PAD Annex 10B) was completed before the project could go ahead. As to resettlement, the initial survey indicated that the road project would impact about 500 plots, with about 1,900 identified owners (cost $40 million), the very large majority being private parties . Special attention was given to vulnerable groups including women . Follow up during implementation included independent monitor reviews. The final estimate was that 3,028 people were affected and the final cost of land acquisition amounted to US$158 million (para 28) – a 2.48 times increase in per person cost . b. Fiduciary Compliance: Procurement and financial management arrangements were well articulated in the PAD and were in accordance with Bank guidelines and procedures . Procurement faced major delays due to prequalification and lack of sufficient response from bidders. The ICR does not report on receipt of audit reports . The project team confirmed that all audit reports were received, were unqualified, and there were no financial management issues . c. Unintended Impacts (positive or negative): d. Other: 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Unsatisfactory IEG rates the relevance, efficacy and Unsatisfactory efficiency as modest putting the overall outcome in the Unsatisfactory category. Risk to Development High High Outcome : Bank Performance : Moderately Unsatisfactory IEG rates the Bank's performance at Unsatisfactory entry as moderately unsatisfactory largely because the Bank (a) did not adequately review the costs of the Project at appraisal, which affected the outcome of the Project; and (b) included a not highly relevant objective in the Project's PDO. IEG rates the Bank performance at supervision as unsatisfactory because (a) the Bank did not take appropriate action to review the objectives of the Project, and (b) the project was rated satisfactory throughout implementation indicating problems were not recognized or admitted. Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR poses questions and alternatives about the design and outcome of the Project as lessons learned : (a) government agencies to be involved : the ICR recommends that the Ministry of Municipalities and Greater Amman Municipality were important partners who could have influenced the outcome particularly on the master plans and urban planning fronts; (b) appropriateness of Bank departments and skills involved and difficulty of coordinating multi -sectoral projects leading to questioning whether a series of projects would have been more appropriate; (c) appropriate procurement actions including (i) double checking cost estimates, (ii) adequacy of sequencing and packaging of contracts in a multi -donor financed project (d) importance of appropriate indicators and their consequent monitoring; (e) inadequacy of spending caps imposed by the Government; and (f) importance of good land acquisition practices and good environmental monitoring and replication of good practices across the country . IEG derives the following lessons : (a) While the ring road physical component was less complex to implement, the multiplicity of donors and contractors increases the need for coordination and agreement to avoid delays . (b) Complex components such as the inland port and the development around the ring road area requires even more coordination among the donors and the government to ensure rigorous background analysis, consistency of approach and agreement on outcome throughout project implementation . (c) Training of government officials and promotional activities targeting the private sector for commercial development is important to ensure success . (d) Appropriate skills from the development partners and from the government sides within a strong, committed and active institutional framework are essential to support new urban and commercial development investments. 14. Assessment Recommended? Yes No Why? A review of the costs of the project, including land acquisition costs, is recommended as the costs were about three times higher than appraisal estimates . The review should assess what was finally delivered and the final costs when the last road section is completed . Timing and scheduling of land acquisition may be an important element to avoid increased costs, a study of which can be useful in other projects . 15. Comments on Quality of ICR: While overall IEG finds the ICR was well-written and provided a candid assessment of the considerable limitations of the project, there were five key shortcomings that resulted in an unsatisfactory rating for the ICR. They are as follows: 1. The main problem with the report was the disconnect between the identification of the implementation and post-completion issues and the overly optimistic references and possible long term outcomes of the project . Despite strong indications of a lack of commitment from government to make the project work, the report still presents a positive illustration of the long term benefits . This inconsistency makes it difficult to get an accurate performance story on the viability of the project and on future prospects . ICRs are intended to be a useful tool for future investments and confusing messages make the value of the report limited . 2. The 'relevance of design and efficacy' are poorly handled in the report and there is a lack of evidence to support the statements made against this criteria . Without an adequate evidence base to support the comments, the argument does not provide sufficient insight on the issues confronted, nor on the viability and realism of the project in the preparatory phase . Where a project has so significantly under -estimated costs, additional explanation of these issues would be expected . 3. There is a considerable disconnect between the narrative and the ratings included in the document . In the section on the justification for a moderately unsatisfactory outcome (p. 12), the report uses the following wording: "... objective 1 has not yet been fully achieved ... and objective 2 was not achieved and is unlikely to be fully achieved...the project remains uncertain of reaching the operational stage even now when nearly twice the expected construction time has passed . Supporting pieces of the project, such as Customs Depot, inland port and strong support to local governments towards land development have gradually been dropped without significant explanation or evident rationale ." While IEG recognizes and supports the ICR's candid discussion in this section, the narrative strongly infers an unsatisfactory outcome and yet it was only rated as MU . 4. The analysis of the reasons for the high cost overruns is incomplete and weak . Jordan’s economic problems in 2010 were used as a reason for the delays and increased costs (ICR para 19). But the oil price increases were erratic during the period 2004-2012 (see: www.infomine.com) and do not solely explain the increased costs (including the monitoring indicator on cargo transport costs ). The inflation rate in Jordan has remained well below the 6% mark except for a hike to 14.9% in 2008 (See CSP) and therefore the story here is incomplete . 5. The section on 'lessons learned' does not adequately deal with the considerable limitations and problems encountered in this project. It provides a very generic set of issues that could be applied to a large number of projects in the Bank and offers little on the major contextual and political economy issues that made this a poor investment. IEG expects the lessons learnt section to be a helpful guide to future projects in the country, region and sector. Therefore, additional details and analysis of what went wrong, where the process failed and how we might ensure it doesn't happen in the future was expected . It is helpful that the ICR highlighted the projects limitations, but having more insight on why these issues arose and what to avoid in the future is expected in these instances if we are to adhere to appropriate measures of accountability . Given these shortcomings, IEG finds the report did not meet the minimum requirements of an ICR and did not adequately articulate the many issues that led to the unsatisfactory project outcome rating. a.Quality of ICR Rating : Unsatisfactory