Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) Report Number : ICRR0020953 1. Project Data Project ID Project Name UG-Transport Sector Development P092837 Project Country Practice Area(Lead) Additional Financing Uganda Transport & ICT P121097 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-46790,IDA-49490,TF-11094 30-Jun-2014 196,146,000.00 Bank Approval Date Closing Date (Actual) 10-Dec-2009 31-Jan-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 190,000,000.00 6,146,000.00 Revised Commitment 181,220,763.26 6,146,000.00 Actual 159,078,449.73 6,146,000.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan John R. Eriksson Christopher David Nelson IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components a. Objectives The Project Development Objective (PDO) as stated in the Financing Agreement (Schedule 1, page 5) and in the Project Appraisal Document (PAD) was: "To improve the connectivity and efficiency of the Recipient's transport sector through: (i) improved condition of national road network: (ii) improved capacity for road safety management. and, (iii) improved transport sector and national road management." Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components There were five components. One. Upgrading and Rehabilitation of National Roads. (Appraisal estimate US$162.10 million. estimate with Additional Financing US235.10 million. Actual cost at closure US$140.70 million). This component aimed at upgrading parts of the national road network. Activities included: (i) paving about 160 kilometers (km) of the Gulu to Atiak and Vurra-Arua-Oraba roads: (ii) preparing design and bidding documents for reconstructing the Tororo to Soroti (151 km) and the Lira-Kamdini-Gulu roads (148 km). The scope of this component was expanded to cover the cost of upgrading 66km of the Kamwenge-Fort Portal road with Additional Financing (AF) for the project. With this, the total length of the road network to be rehabilitated under the project was about 226 km. Two. Enhanced Road Safety. (Appraisal estimate US$4.50 million. Actual cost at closure US$2.10 million). This component aimed at establishing the National Road Safety Authority (NRSA). Activities included: (i) preparing a draft road safety policy, strategy and a draft law for creating the NRSA: and, (ii) operationalizing the police crash data base. Three. Preparation of Kampala Urban Transport Project. (Appraisal estimate US$4.50 million. Actual cost at closure US$4.10 million). This component aimed at preparing a pre-feasibility study for establishing a Bus Rapid Transit (BRT) system in the Greater Kampala Metropolitan Area (GKMA). Activities included: (i) preparation of design and bidding documents for the BRT in the selected corridor, drafting bidding documents for the bus operators, fare collectors, fund managers, preparing system financial models, central business district traffic management and parking studies, a bicycle path master plan and drafting legislation for the Metropolitan Area Transport Authority (MATA). Four. Institutional Support to the Ministry of Works and Transport (MoWT). (Appraisal estimate US$10.90 million. Actual cost at closure US$5.80 million). This component aimed at providing technical assistance to the MoWT for performing its core functions (namely policy making, strategic planning, sector oversight and monitoring) and outsourcing its executive and regulatory functions to newly-created state agencies. Activities included: (i) strengthening the MoWT's policy and planning division through updating sector policy, reviewing the sector legal framework, monitoring and evaluation (M&E) and introducing a Transport Sector Data Management System (TSDMS): (ii) creating the proposed Multi- sector Transport Regulatory Authority (MTRA): (iii) assistance for transforming the management of district, urban and community access roads (DUCAR) into an agency: and, (iv) other support through technical assistance, financing equipment and operating costs. Five. Institutional Support to the Uganda National Roads Authority (UNRA). (Appraisal estimate US$16.00 million. Revised estimate following restructuring US$18.00 million. Actual cost at closure US$12.50 million). This component aimed at strengthening the UNRA's capacity for project implementation. Activities included: (i) Refurbishment of the regional offices of UNRA: (ii) providing technical assistance: (iii) financing studies by UNRA for improving performance: and, (iii) training, financing office equipment and supervision vehicles and operating costs. The scope of this component was Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) expanded to include activities associated with strengthening the internal audit functions of UNRA to undertake technical audits of road projects, with the AF for the project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. (Appraisal estimate US$198.00 million. estimate following restructuring US$273.00 million. Actual cost at closure US$165.20 million). Actual cost was lower than estimated due to the cancellation of grants (discussed below). Project Financing. The project was financed by an IDA grant and by a United Kingdom Department for International Development (DFID) Trust Fund. (Appraisal estimate US$190.00 million. AF of US$75.00 was approved. With this the total financing for the project was US$265.00 million. Amount disbursed at closure US$159.10 million). The amount disbursed was much lower than estimated due to the Borrower's non-compliance with its obligation to implement the project in conformity with environmental and social standards and practices. This led to the suspension of disbursements on October 22,2015 and eventual cancellation of the project (discussed below). There was parallel financing for complementary activities associated with institutional strengthening from the Danish International Development Agency (DANIDA) and the European Commission (EC). Borrower Contribution. None was planned. Dates. AF was approved on 06/16/2011 to expand the scope of project activities. With AF, the project closing date was extended by one year to 01/31/2016. The project was restructured on 05/31/2013 through a Level 2 restructuring for reallocating amounts between the Credit and Grant Agreements. On October 22, 2015, the Bank suspended disbursements due to the Borrower's non-compliance with environmental and social safeguards and on December 21, 2015, the Bank cancelled the credit (discussed in Sections 3b and 11). 3. Relevance of Objectives & Design a. Relevance of Objectives The PDO is relevant to the government strategy. Roads account for over 90% of passenger and freight traffic in Uganda and roads are the only transport mode for the rural population. Uganda's road infrastructure is important in the regional context, as Uganda's roads link the landlocked countries of Rwanda, Burundi, South Sudan and eastern parts of the Democratic Republic of Congo (DRC) to the ports of Mombasa in Kenya and Dar es Salaam in Tanzania. The PDO was relevant to the Government's Poverty Reduction Action Plans (PRAPs) at appraisal. The third version of the plan for the 2008-09 period underscored the need for shifting policy from recovery to sustainable growth and structural transformation through enhancing competitiveness. The National Transport Strategy (NTPS) adopted in 2002 to be implemented in phases, highlighted the need for promoting less costly, efficient and reliable transport services. The phase 3 of the strategy for the 2009/10 to 2018/19 period highlighted the need for rehabilitating the major road corridors for meeting the growing transport demand, providing support for economic development and expanding regional trade. The National Development Plan for the 2010/11 -2014/15 period identified the development of the transport sector and the road sector in particular as priorities due to the strong Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) links between agricultural production and poverty reduction. The Uganda 2040 launched in April 2013 highlighted the importance of the transport sector for Uganda. The PDO was relevant to the Bank strategy for Uganda. At appraisal, the Uganda Joint Assistance Strategy (UJAS) for the 2005-2009 period highlighted the need for promoting less costly, efficient and reliable transport services for promoting agricultural and industrial production, trade, tourism, social and administrative services. The strategic objective of the Country Assistance Strategy (CAS) for the 2011-2015 period was "enhancing public infrastructure". For the road sector the objectives included strengthening the roads budget, facilitating trade with neighboring countries and improving governance arrangements. Rating Substantial b. Relevance of Design The statement of the PDO was clear. Project activities and their outputs were likely to produce the specified outcomes which were measurable. Activities such as upgrading parts of the road network were aimed at improving their condition. Activities such as preparing the road safety strategy and creating the National Road Safety Authority (NRSA) were aimed at improving road safety. Technical assistance for establishing a Bus Rapid Transit (BRT) system for the Greater Kampala Metropolitan Area (GKMA), to the Ministry of Works and Transport (MoWT) for performing its core functions and to the Uganda National Roads Authority (UNRA) for improving its performance were all aimed at improving transport sector management. The combination of these activities could be expected to improving the connectivity and efficiency of Uganda's transport sector. There were however several shortcomings in design. • The project design specified five institutional components, including establishing four new state agencies (the Multi-Sectoral Regulatory Agency, the Metropolitan Area Transport Authority, the National Road Safety Agency and the District, Urban and Community Road Agency). This design, together with the existing Uganda National Roads Authority (UNRA) was overly complex, given the weak organizational structure of UNRA. • The design which included both road infrastructure investment and transport sector reforms was unrealistic with respect to the project timelines. This is particularly so given that the experience with prior Bank financed projects in Uganda had shown that sector reforms take more time than anticipated. • It is not clear if there was due diligence regarding safeguards compliance, with respect to the activities that were added following AF for the project, given that the road section that was to be rehabilitated was initially not identified as challenging from a safeguards point of view. This was despite the fact that the road to be rehabilitated was in an area with considerable influx of labor to a poor rural area with systemic social issues (such as, sexual violence against children by road workers, increased child labor and school dropouts, increased number of accidents on the road as a result of the stone quarry, a rise in crime in the community, poor compensation practices and unclear redress mechanisms, among others). The failure to incorporate appropriate mechanisms resulted in a complaint to the Inspection Panel on December 19, 2014. (The Inspection Panel is a three-member body created in 1993 by the World Bank's Board of Executive Directors to provide an independent forum for Project-Affected Persons), followed, by the Bank Management Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) acknowledging serious weakness in the preparation of this project, suspension of disbursements on October 21, 2015 and eventual cancellation of the credit on December 21, 2015 (discussed in Section 11). Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve the connectivity and efficiency of the transport sector through improved condition of the National Road Network. Rationale Outputs. • 166 km of roads were rehabilitated as compared to the original target of 159 km. However, of the 66km of roads that were to be rehabilitated following the approval of AF, only 44% or 29 km was completed when the credit was cancelled. The team has clarified that the balance of this road network has to date been completed with government funding. • The feasibility studies, environmental and social assessments and detailed engineering designs for future road upgrades were completed as targeted. Outcomes. • The average vehicle operating cost (US$ per vehicle km) on the Gulu-Atiak and Vurra-Oraba road segments were reduced by a third from US$0.35 to US$0.22 at project closure as targeted. The vehicle operating costs on the Kamwenge-Fort Portal road (the road that was added with the approval of AF) were not calculated, as these roads were not completed at project closure (according to the project team, the Kamwenge-Fort Portal road has recently been completed) . These roads are however expected to be completed with government funding. • According to the data based on survey by the Uganda National Road Agency (UNRA), the travel time on the Gulu-Atiak and Vurra-Oraba roads was reduced from two hours at the baseline to one hour as targeted. Travel time on the not completed Kamwenge-Fort Portal road was not calculated at project closure. • National roads (defined as roads that connect districts and cities with one another and with neighboring countries) in poor condition reduced from 36% at the baseline to 22%. This was short of the target of 15%. • 84% of the rural population had access to all season roads in the target area at project closure as compared to 64% at the baseline. This was short of the target of 90%. Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To improve the connectivity and efficiency of the transport sector through improved capacity for road safety management. Rationale Outputs. • The National Road Safety Authority (NRSA) - envisioned to be an autonomous, self-accounting central government authority for coordinating road safety activities and initiatives- was not created as targeted. • The Road Crash database was established as targeted. The ICR however provides no evidence on how the database was used. • The activity associated with updating the Traffic and Road Safety Act of 1998 was not completed and the final report produced by the Consultant was deemed to be unsatisfactory by the Ministry's technical team. Outcomes. • The death rate per 10,000 vehicles declined from 44 in 2013 to 26 by project closure. However, the decline in road accident fatalities could not be attributed to project activities, given that NRSA was not created and the decline may have been due to external factors (such as the traffic police introducing dedicated enforcement teams targeting major causes of accidents such as speeding, drunk driving and incompetent drivers). Although the decline in death rate from 44 to 26 by project closure was close to the World Health Organization (WHO) estimate of 27.8 (Source: World Health Organization, 2015. Road Safety Report, Geneva), the death rate per 100,000 vehicles in Uganda still remained one of the highest rates in Africa at project closure. Rating Negligible PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) To improve the connectivity and efficiency of the transport sector through improved transport sector and national road management. Rationale Outputs. • Design and bidding documents for phase 1 of the Bus Rapid Transit (BRT) were completed and approved by a technical committee and stakeholders at a workshop at project closure. • The UNRA's share of administrative costs as part of overall budget at project closure was less than five percent as targeted. • The UNRA Management Information System (MIS) was established. • About 10,000 km of district roads were reclassified as part of the national road network and transferred to UNRA's management. The data collection (including mapping, an inventory of road assets, condition assessment and traffic census) was completed. • Annual customer satisfaction survey was carried out by the Uganda Road Fund. • The Transport Sector Data Management System (TSDMS) was operational at project closure and produces an Annual Sector Performance Review. • The Framework for Multi-Sectoral Transport Regulatory Authority (MTRA) was completed but not yet approved by Cabinet at project closure. • The transport policy was updated but not yet approved by Cabinet at project closure. • The Traffic and Road Safety Legal Framework was drafted but not legislated at project closure. • The Agency for District, Urban and Community Access Roads (DUCAR) was not yet established at project closure. • Capacity of UNRA staff to carry out technical audits increased from six at the baseline to seven at project closure. This was short of the target of 14. Outcomes. • The indicators were output-oriented. The ICR however provides no information on the uses of the outputs produced under the auspices of this project. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Efficiency Economic Analysis. An economic analysis was conducted both at appraisal and at closure for activities associated with upgrading of the two segments of the national road network of the original product (upgrading Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) of the Gul-Atiak road and the Vurra-Orabha road), using the fourth version of the Highway Development and Management Model (HDM-4). These components accounted for 81% and 85% respectively of the project cost at appraisal and at closure. The potential project benefits were assumed to come from savings in Vehicle Operating Costs (VOSs), time savings, reduction in accident costs and induced agricultural production. The Net Present Value (NPV) at 12% discount rate at closure for the Gul-Aitak road was US$65.09 million as compared to the NPV of US$25.68 million at appraisal and the NPV at closure for the Vurra_Orabha road was US$22.51 million as compared to the NPV of US$53.96 million at appraisal. The NPV for the Gul-Aitak road segment had a better return due to the higher than anticipated traffic on this road. The ICR does not provide information on the substantially lower NPV of the Vurra-Orabha road segment at project closure. The ex post Economic Internal Rate of Return (EIRR) for the two segments was 28% and 18% was as compared to the ex ante EIRR of 18% and 21% respectively. The road segment that was added following the approval of AF (the Kamwenge-For Portal Road) and which represented a third of the road investment cost, had a NPV of US$19 million and an EIRR of 17% at appraisal. No further economic valuation was made for this road segment at closure as the Credit was cancelled and the road remained unfinished at project closure. Operational and Administrative Issues. There were delays during implementation. Although the Trust Fund Grant Agreement was approved on April 6, 2010, it was only signed 25 months later on May 27, 2012. By this time the value of the grant had declined by 30% due to exchange rate fluctuations. The team clarified that by the signing date, there were very few procured activities, all of which were financed under the credit. At the time of the approval of AF, there were procurement delays (While the project envisaged strengthening the Uganda National Road Authority's (UNRA's) Internal Audit Unit to carry out technical audits, the selected contractor was mobilized only in August 2013, some 23 months after Board approval). Despite steps to improve procurement capacity, most of the activities associated with transport sector reforms (such as the establishment of the National Road Safety, the Metropolitan Transport Authority, the agency for District, Urban and Community Access Roads and the Multi-Sector Transport Regulatory Agency), were not completed. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 81.00 Appraisal  20.00 Not Applicable 85.00 ICR Estimate  22.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) The relevance of the PDO for the government strategy and the Bank strategy for Uganda is rated as Substantial. Relevance of design is rated as Modest in view of the overly complex design and inadequate mechanisms for compliance with safeguards. Efficacy of two sub-objectives - To improve the connectivity and efficiency of the transport sector through improved condition of the National Road Network and to improve national road management - were rated as Modest. Efficacy of the third sub-objective -to improve road safety management - was rated as Negligible. Efficiency was rated as Modest. Although the two road segments that had been rehabilitated were economically viable, there were administrative and operational inefficiencies with several activities including sector reforms not yet completed at project closure. Given the major shortcomings in the project's achievements of its objectives, in its efficiency and design, the overall outcome rating is Unsatisfactory. a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating Government Commitment/Ownership. Although appropriate institutional arrangements were made for maintenance of the two road segments that were rehabilitated under the auspices of this project, there is Substantial risk to development outcome. It is not clear that adequate resources would be continue to be made available for financing the maintenance of the rehabilitated roads. Given that the Uganda Road Fund (URD) remains underfunded, there is already a maintenance backlog and not all the Project Affected Peoples (PAPs) had been compensated at project closure. Government's commitment to sector reforms associated with the Kampala Urban Project, the Metropolitan Area Transport Authority (MATA) and updating the National Transport Policy and Strategy (NTPS) was also unclear given that the government had still not enacted legislation regarding these activities. Institutional Risk. Although the National Road Safety Policy was prepared under the aegis of this project, it is not clear whether the government will effectively implement the policy with the necessary legal, regulatory and institutional measures. The team clarified that government remains committed to the implementation of the National Transport Policy and Strategy and that the government was getting support from the Korea Green Growth Trust Fund to support Uganda in developing a comprehensive outcome based logistics policy that incentivizes green practices and technologies aimed at minimizing greenhouse gas emissions (GHG) from the sector. a. Risk to Development Outcome Rating Substantial 8. Assessment of Bank Performance Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) a. Quality-at-Entry This project was prepared based on the lessons from prior Bank-financed road transport projects in Uganda (Road Development Program- phases one, two and three). The lessons incorporated in the design of this project included, specification of measurable outputs within a time frame, technical assistance for building local capacity and specifying benchmarks for effective policy and institutional reforms. The preparation team was diligent in securing funding through the United Kingdom Department for International Development (DFID) for project activities associated with road safety and institutional support. Appropriate arrangements were made at appraisal for safeguards and fiduciary compliance (discussed in section 11). • As discussed in section 3b, the project with five components and aimed at creating four new state agencies was overly complex and unrealistic, given the staffing, budgetary and political implications of establishing these entities. Eventually the Ministry of Works and Transport (MoWT) regarded only two of these agencies as priorities (National Roads Safety Agency and the Metropolitan Area Transport Authority) and even these two agencies were not created at project closure. • Although prior experience with Bank-financed projects had shown that sector reforms take more time, there was no plan to mitigate this risk. • Although at preparation the team developed a project specific Governance and Accountability Action Plan (GAAP) which aimed at developing a system for promoting transparency and accountability and which was considered best practice, the project underestimated the risk associated with governance. Complaints from the public relating to the award of contracts, mismanagement, abuse of office and corruption in the Uganda National Roads Authority (UNRA) led to the appointment of a new Executive Director on April 27, 2015 during project implementation. On June 8, 2015, the President of Uganda appointed a Commission of Inquiry to into the affairs of the UNRA and in September, nearly 900 employees of UNRA were dismissed and a process was launched to screen former employees who reapplied for their jobs. Some former employees were not reappointed and some were recommended for prosecution. This major disruption in UNRA constrained its operational capacity during implementation. • There were shortcomings in M&E design (discussed in Section 10). Quality-at-Entry Rating Unsatisfactory b. Quality of supervision Twelve Implementation Status Reports (ISRs) were filed over a seven -year period. The supervision team included a transport economist, highway engineers, a social safeguard specialist and financial management and procurement specialists. Supervision of the reform program was intensive with a full time Bank Staff member hired in the country office, working with the Ministry on a daily basis in addition to support provided by DFID. However, there were several shortcomings in supervision. • The team underestimated the time required to sign the agreement with DFID which entailed a series of procedural requirements such as including gaining support from stakeholders, preparing a Bill to set up a new authority and obtaining the Certificate of Financial Clearance from the Ministry of Finance, Planning and Economic Development (MoFPED). The Grant Agreement was signed 23 months later than Bank Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) approval on May 27, 2012. This contributed to delays in the initial years of project implementation. • It was apparent that the AF proposal was not adequately prepared, regarding measures aimed at ensuring safeguards compliance. Although in the updated draft of the 2011 Environmental and Social Impact Assessment (ESIA) for the Kamwenge-Fort Portal Road, the project was proposed as a "Category A" project signifying potentially significant adverse impacts, the project was downgraded to a "category B" project in the AF paper. As a "Category B" project, the project was not subject to the kind of increased internal scrutiny required for a "Category A" project. The ESIA did not identify in sufficient detail the key risks arising from the influx of a large number of construction workers and therefore the potential impacts on the affected poor rural communities in the project intervened area. The mitigation measures that were incorporated for protecting forests and natural habitats, identifying Project Affected Persons and social concerns raised by the communities (such as sexual violence against children by road workers, increased child labor and schools and increased number of accidents on the road as a result of the stone quarry) were inadequate and not properly addressed in the Environmental and Social Management Plan. It is not clear if the skill set of the social safeguard specialist was adequate for identifying gender based violence issues in particular. • There may have been lack of due diligence regarding the number of project affected people. The Bank relied on figures for affected people that were out of date and based on verification sampling rather than full census. This was despite the recognized weakness of the original census under the Resettlement Action Plan (RAP) commissioned by the African Development Bank (AfDB). The vulnerability assessment of the updated RAP was inadequate and did not properly identify the necessary assistance programs targeting vulnerable groups. The RAP did not include a grievance redress mechanism. • The appraisal documents for the AF did not include the required analysis of risks to women and children caused by labor influx, in particular those risks related to sex with minors, teenage pregnancies, sexual harassment, child labor and child labor. It is not apparent if the supervision team had people with the required expertise to address issues related to gender-based violence and child protection. Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance There was a disconnect between what had been articulated in Government policy and commitment to implement the steps necessary to achieve the policy goals. Although the Government policy aimed at restructuring the Ministry of Works and Transport to focus on key functions (such as formulating policies, setting standards, strategic planning, sector oversight and monitoring) and outsourcing the activities to newly created agencies such as the National Road Strategy Agency (NRSA), this decision was turned down by the Cabinet. The ICR provides no information as to why the decision was turned down by the Cabinet. This contributed to slowing the reform process. Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) Although the Government supported the implementation of a Governance and Accountability Plan aimed at establishing a culture for promoting transparency and accountability in the road sector, governance remained a major concern. As indicated in section 9b, in the wake of complaints from the public relating to corruption in Uganda National Roads Authority (UNRA), the President of Uganda appointed a Commission of Inquiry to probe into the affairs of the UNRA. The subsequent dismissal and re-appointment of selected staff (after a thorough screening) caused a major disruption to the operational capacity of UNRA and this in turn hampered the implementation of this project. The failure to provide a commensurate increase in staff for UNRA, following the addition of 10,000 km of district roads that were designated as national roads, further adversely impacted the project implementation. Government Performance Rating Unsatisfactory b. Implementing Agency Performance The two implementing agencies were the Ministry of Works and Transport (MoWT) and the Uganda National Roads Authority (UNRA)- an autonomous body tasked with the responsibility of managing the national road network. The project coordinators within MoWT and UNRA were responsible for day-to-day management of the project. MoWT. There were procurement delays associated with recruiting additional staff which contributed to implementation delays. This was further exacerbated by the failure to secure Cabinet approval for the creation of the four new agencies. Although the design of the proposal of the Kampala Bus Rapid Transit (BRT) was completed, planning could not commence without cabinet approval of the Kampala Metropolitan Area Transport Authority (MATA). UNRA. The significant staffing shortage at UNRA due to continuing budget constraints and a cap on hiring contributed to the implementation delays. UNRA was unable to hold the contractor responsible for undertaking the 36 remedial actions required for safeguards compliance. Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design There were five key outcome indicators. Of these three indicators - savings in transport cost and travel time and increase in the percentage of rural population with access to an all season road were appropriate for monitoring performance with respect to the condition of the road network. There were Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) however no key outcome indicators for monitoring performance with respect to activities associated with improved road sector management. It is not clear if indicators pertaining to changes in the annual rate of road accidents were either appropriate or could be attributed to the project given that the project only supported activities aimed at establishing the National Road Safety Authority (NRSA) and operationalizing the police crash data base. b. M&E Implementation Following the approval of the AF, key indicators were added with respect to the project activities. These included savings in transport cost and time for the road segment that was added and a new indicator associated with monitoring the capacity of the Uganda National Roads Authority (UNRA) to carry out responsibilities pertaining to technical audits. The Data Base Section (DBS) of the Policy and Planning Division of the Ministry of Works and Transport (MoWT) was overall in charge for monitoring and reporting on the performance of the transport sector and the Uganda National Roads Agency (UNRA) was in charge of day-to-day monitoring and reporting. During implementation, the Ministry of Works and Transport (MoWT) was established and the staff capacity to utilize the database was strengthened. c. M&E Utilization The indicators were utilized for monitoring project performance. The database set in the MoWT was to date being used in the annual performance review for assisting decision making for all transport modes. M&E Quality Rating Substantial 11. Other Issues a. Safeguards The project was classified as a Category 'B' project for environmental purposes. Other than environmental assessment (OP/BP 4.01), two safeguards were triggered: Physical Cultural Resources (OP/BP4.11) and Involuntary Resettlement (OP/BP 4.12). Original Project. An Environmental and Social Impact Assessment (ESIA) was conducted at appraisal and Environment and Social Management Plans (ESMP) were prepared and publicly-disclosed at appraisal (PAD, page 85). Land acquisition was anticipated for project activities associated with expansion and re- alignment of roads. About 2044 People Affected Peoples (PAPs) were expected to be affected by the project activities. Resettlement Action Plans (RAPs) were prepared and publicly-disclosed at appraisal (PAD, page 86). The mitigation measures to ensure compliance with road safety measures and health and safety of workers were adhered to during implementation with respect to the two road segments that were Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) rehabilitated. The physical cultural resources were identified at appraisal were not affected, except in one case of a sacred area in the Koboko district. In this case, compensation was paid to the PAPs. For the Gulu- Atiak road 1,884 out of 2,027 PAPs were compensated and for the Varruba-Oraba road, 3,276 out of 3,935 PAPs received compensation. The reason for the shortfall was that while additional PAPs were identified, no funds were allocated for land acquisition in the financial year because the compensation was supposed to have been concluded. The ICR (page 14) notes that special arrangements have been made by the government to clear the backlog. Project following AF. In addition to the safeguards triggered at appraisal (that is Environmental Assessment (OP/BP 4.01), Physical Cultural Resources (OP/BP 4.11) and Involuntary Resettlement (OP/BP 4.12), two additional social safeguards were triggered at AF: Natural Habitats (OP/BP 4.04) and Forests (OP/BP 4.36). Although the Environmental and Social Impact Assessment (ESIA) proposed that it be classified as a "Category A" project with potentially significant adverse impacts, the AF paper erroneously downgraded this to a Category B project. There was subsequent non-compliance with safeguards. On December 19, 2014, the Inspection Panel received a request for inspection from community members from project intervened areas. The request raised issues pertaining to lack of community participation, sexual violence against children by road workers, increased child labor and school dropouts, increased number of accidents on the road and a result of the stone quarry, a rise in crime in the community, poor compensation practices and unclear redress mechanism. A Non-Government Organization (NGO), "Joy for Children, Uganda" represented the communities. As these concerns had not been raised previously with Bank management, the Panel gave management the opportunity to address the problems. Several missions involving Bank management, specialists and the task team reviewed the shortcomings in implementation and discussions were held with Uganda National Roads Authority (UNRA) and the government. On September 11, 2015, a second request from the same entities was received reinstating earlier concerns and noting that actions to address these problems were in their view unsatisfactory as many mitigation measures had not been implemented. Following a request by the Panel to the Bank management, a Notice to Correct was issued to the contractor, requiring the contractor to undertake 36 remedial actions, together with specification of a deadline for each remedial action. Eventually on October 21, 2015 the Bank suspended disbursements under the project due to the Borrower's non-compliance with the Bank's environmental and social standards and practices. One year after it was issued, the number of remedial actions taken by the contractors had increased to 32, but the contractor remained out of compliance in respect of four: work permits for foreign personnel, blasting operations at the quarry, workplace accidents, compensation and grievance redress. All the actions were eventually complied with by the end of April 2017 (ICR, page 12). Forests and Natural Habitats. A section of the road to be rehabilitated under the project traversed the Kibale National Park, protecting a large area of forest previously managed as a logged forest reserve. The section of the road that was to be rehabilitated was originally proposed not to be widened to eliminate impacts associated with land take and destruction of trees and their canopies. However On December 22,2014 the Uganda Wildlife Authority approved the wider section of the road through Kibale National Park, on condition that only a limited number compensation of trees would be removed during the road upgrading and that some additional speed reduction devices (speed humps) were installed. This condition was adhered to. Physical and Cultural Resources. There was one cultural site in the project intervened area which was protected under the Historical Monuments Act. The ICR (page 14) notes that the Museum Authorities inspected the site and it as Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) determined that it would be unaffected by the project. Involuntary Resettlement. Originally the number of project affected persons were expected to be 2,430 and an additional 414 Project Affected Peoples (PAPs) were added as eligible for compensation. As of March 27, 2017, 94% of PAPs were compensated. b. Fiduciary Compliance Financial Management. Financial management arrangements for this project were based on those used for past IDA financed projects in Uganda. An assessment of the financial management arrangements was conducted at appraisal. The assessment concluded that the arrangements were satisfactory and the financial management risk was rated as Moderate (PAD, page 12). Although there were financial management issues during implementation such as lack of supporting documentation for overseas trips and not up-to-date reconciliations of project accounts with Bank reconciliations, these were rectified and there were no ineligible payments and irregularities (ICR, page 15). The ICR provides no information on the quality of audits and whether they were unqualified.. Procurement. An assessment of the capacity of the Implementing Agency to address procurement issues was conducted at appraisal (PAD, page 72). The procurement risk was rated as Substantial primarily due to the inadequate capacity in the Procurement and Disposal Unit (PDU) and shortage of staff in PDU. An Action Plan to mitigate procurement risk was prepared at appraisal (PAD, page 13). The ICR (page 15) notes that procurement commenced late in several cases such as purchase of equipment and review of engineering designs for future projects. The measures incorporated for improving procurement capacity following the approval of AF included, outsourcing independent procurement evaluation to validate the Uganda National Roads Authority (UNRAs) procurement actions and strengthening of the capacity of UNRA's Internal Audit Unit to undertake technical audits. There were delays associated with hiring procurement specialists. The ICR does not indicate any procurement issues. c. Unintended impacts (Positive or Negative) --- d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) Risk to Development Substantial Substantial --- Outcome Bank Performance Unsatisfactory Unsatisfactory --- Borrower Performance Unsatisfactory Unsatisfactory --- Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons Of the nine lessons listed in the ICR from the experience of implementing this project, the following seven lessons are the most important, with some adaptation of language. (1) New transport agencies should only be established when there s sound understanding of the financial, legal and political implications of such initiatives. There needs to be sufficient capacity in place to ensure the necessary stakeholder buy in, financial clearance from the appropriate financial ministry and suitable legislation. Envisioning establishment of four new agencies for covering road safety, metropolitan transport, multi-sector transport regulation and district urban and community access roads as done in the case of this project was unrealistic particularly given the project time lines and the absorptive capacity of the Ministry of Works and Transport to implement such initiatives. (2). Reform initiatives and capacity building are as important as physical investment components in transport sector projects. In the case of this project, although project activities aimed at sector reforms, the focus of the project was more on physical construction progress once contracts had been awarded and not so much on the softer components of the projected. This contributed to the undermining of these aspects of project activities. (3) When capacity of the implementing agency is low, it would be prudent to focus on fewer but more focused activities. In the case of this project, there were simply too many activities, particularly given the capacity of the implementing agencies. (4) In complex projects, a thorough assessment of all aspects relating to the implementing agency's capacity including safeguards is essential, with credible measures to address the identified weakness. In large infrastructure projects, the Bank needs to ensure that sufficient contract capacity is in place before implementation commences and that qualified environmental safeguards staff are assigned to the project, or if this is not likely, that alternative arrangements are made for greater implementation oversight. (5). When the scope of a project is expanded substantially, a thorough review of safeguards implementation of the new components needs to be undertaken. In some cases, the revised project should not necessarily be given the same environmental classification as the parent project. In this project, the circumstances associated with the added road were much more complex than the ones in the original project and an environmental category "A" recognizing potentially significant adverse impacts, should have been designated. (6) Failure to put in place robust community engagement processes can weaken the Bank's ability to anticipate potential social impacts and respond appropriately when problems arise during Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review UG-Transport Sector Development Project (P092837) implementation. In this project, no grievance redress mechanism was in place or partnerships with Non Governmental Organizations (NGOs) that could assist in understanding and resolving sensitive local issues. (7). The World Bank staff skills and capacity need to match the risk and complexity of the anticipated operations. In the case of this project, the Bank over-relied on newly hired safeguard staff who had received limited support from others in the region and from headquarters. Until after the Inspection Panel compliant was received, there was no deployment of staff with the right skills to address systemic social risks or to engage the communities. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR provides a detailed overview of the project and is well-written. The narrative supports the ratings and available evidence. The ICR also provides a detailed and a candid description of the safeguard issues that arose following the approval of AF. The ICR follows the ICR guidelines and the quality of evidence and analysis is aligned to the messages outlined in the ICR. The ICR could have provided more details on the quality of financial audits. a. Quality of ICR Rating Substantial Page 17 of 17