Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Report Number : ICRR0021557 1. Project Data Project ID Project Name P092537 DRC-Multi-Modal Transp Country Practice Area(Lead) Additional Financing Congo, Democratic Republic of Transport P129594 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H5950,IDA-H8590 31-Dec-2015 379,310,884.72 Bank Approval Date Closing Date (Actual) 29-Jun-2010 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 255,000,000.00 0.00 Revised Commitment 411,532,816.40 0.00 Actual 385,554,992.94 0.00 Prepared by Reviewed by ICR Review Coordinator Group Kavita Mathur Fernando Manibog Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The Project Development Objectives (PDOs) as stated in the Financing Agreement (dated July 14, 2010, page 6) were: (i) to improve transport connectivity in the Recipient’s territory to support national economic integration, (ii) to restore Société Nationale des Chemins de Fer du Congo (National Railway Company of DRC - SNCC’s) financial and operational viability, and Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) (iii) to implement a sector wide governance plan and strengthen transport state owned enterprises (SOEs) operational performances. Revised objectives: The PDOs were revised on May 15, 2013 (level 1 restructuring). The sub-objective (i) “to improve transport connectivity in the Recipient’s territory to support national economic integration” was revised as follows: “to improve transport connectivity in the Recipient’s territory”. The sub-objective (ii) “restore SNCC’s financial and operational viability” was not modified. The sub-objective (iii) “to implement a sector wide governance plan and strengthen transport state owned enterprises’ operational performances”, was revised as follows: “strengthen transport state owned enterprises operational performances”. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 15-May-2013 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components Component 1. Société Nationale des Chemins de Fer du Congo (National Railway Company of DRC - SNCC’s) recovery plan (cost at appraisal US$218.85 million; actual cost US$365.54 million). This component was expected to support the reform of SNCC management and operations through: (i) the signature of a five-year management contract with an internationally selected private operator that would replace the existing private operator hired under a stabilization contract; (ii) payments of up to 1,675 voluntary retirees' departing indemnities and social security pension contributions; (iii) financing of track and rolling stock rehabilitation and upgrade; (iv) reimbursement of eligible SNCC operational costs (mainly fuel, electricity and water); (v) training of key new and existing personnel; Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) (vi) monitoring of the project’s impact; (vii) funding of a study to review SNCC's relationship with mining operators and development of a transparent tariff policy in support of mobilizing financing from mining companies; and (viii) funding of financial and procurements audits. Component 2. Operational performance strengthening and improved governance of the sector (cost at appraisal US$25.45 million; actual cost US$23.1 million). This component was expected to finance: (i) the acquisition of urgently needed equipment for selected transport state owned enterprises (SOEs) (the National Marine Ways Management Agency (RVM), the National Airways Management Agency (RVA), the National Transport Office (ONATRA) and the National Waterways Management Agency (RVF)) to improve their overall performance while allowing them to devote limited internal resources towards restructuring; (ii) pay for the retirement indemnities of 77 RVF agents using an approach similar to that of SNCC; (iii) pay for an internal diagnostic of the Ministry of Transport (MoT) to identify possible reorganizational scenarios; (iv) finance MoT agents' training and MoT equipment; (v) finance the annual audits of SOEs procurement and financial management; and (vi) allow the development of a sector wide governance plan which will then be tailored for adoption by each individual SOE. Component 3. International trade procedures simplification (cost at appraisal US$2 million; actual cost US$1.89 million). The component would (i) support the development of an international trade procedures simplification strategy and the associated action plan, including materials, equipment (based on the ongoing study on the facilitation of international trade financed by Trade Facilitation Facility Trust Fund); and (ii) assist in the implementation of international trade agreements. Component 4. Project management (cost at appraisal US$8.70 million; actual cost US$12.41 million). This was expected to fund the cost of the Project Management Entity (PME) located within the MoT. The PME included two Project Units - one located in Lubumbashi within the SNCC to manage Component 1 while the other would be based in Kinshasa to manage Components 2 and 3. Revised Components (ICR para 17) At appraisal, there was a large financing gap that was expected to be parallel-financed by an official credit line of US$200 million from the Government of China. However, this did not materialize. The Additional Financing (AF) of 2013 was designed to close a financing gap. The AF increased the scope of Component 1 and Component 2. Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Component 1: Additional improvement of rail track and infrastructure (US$64.8 million); acquisition of new locomotives (US$23.1 million); rehabilitation of freight wagons and passenger coaches (US$1.8 million); acquisition and installation of logistics and workshop equipment (US$3.8 million); improvement in telecommunications (US$3.5 million); improvement of facilities and acquisition of miscellaneous equipment (US$2.4 million); studies and technical support; additional operational support and institutional development (US$47.2 million) including the funding of SNCC eligible operational expenses (mainly fuel, electricity and water bills, locomotive rentals, and materials and spare parts), a management contract with a private operator, and the training of key new and existing personnel; and the expansion of the SNCC social plan to 400 more retirees eligible for retirement in 2013 (US$7.6 million). Component 2. For river transport, the AF added hydrographic and hydrological surveys on Kwilu, Sankuru, Ruki, Busira, Tshuapa, Lukenie and Mfimi rivers and the construction and installation of navigation aids on the Lua, Mongala and Lualaba rivers (US$ 5 million for RVF), equipment for RVF workshop in Kinshasa; as well as consultants services for Directorate of Maritime Affairs and Waterways, Ministry of Transport and Communication (DMVN) (US$ 5 million), including a revision of DRC’s river navigation code, capacity building and training, provision of supplies and equipment for the Directorate and office rehabilitation. For air transport, the AF added the Civil Aviation Authority (Autorité de l’Aviation Civile- AAC) as a new beneficiary. New activities (US$ 5 million) included consultants’ services for the recertification of DRC’s passenger airlines crews and aircrafts; twinning of the AAC with a foreign civil aviation for a three-year period; technical assistance and training for the implementation of a five-year technical capacity strengthening plan for AAC; and office rehabilitation and provision of IT and office equipment. The second Restructuring (Level 2) dated July 9, 2014, cancelled the activities benefiting DMVN (office rehabilitation, technical supplies, IT and office equipment under component 2). Component 3 was refocused on the implementation of the recommendation of the studies on international trade facilitation funded under the project and on the physical improvements on the Beach Ngobila passenger terminal in the Congo river. The component 3 “International trade procedures simplification” was renamed “International trade procedures simplification and infrastructure investments”. The third Restructuring (Level 2) dated November 29, 2016 increased the scope of component 1 by focusing the remaining US$27 million towards infrastructure works (rehabilitating and maintaining railway tracks). The total length of rehabilitated track was increased from 600 km (at the time of AF) to 1,300 km. Additional track maintenance of about 1,000 km was added. The scope of component 2 was reduced due to the withdrawal of the Company of Transports and Ports (SCTP) as project beneficiary because of noncompliance with legal conditions. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Project Cost. The total actual project cost was US$393.94 million compared to the original estimated of US$831 million and revised estimate of US$435 million. Financing: The amount of the original grant was US$255 million. After the withdrawal of one key financer (a bilateral loan from China of US$ 200 million), an Additional Financing grant of US$180 million was approved in 2013. The total combined Bank funding was thus US$435 million. However, due to the exchange rate fluctuation between the SDR and the US$, the total funding for the project in US$ was reduced by about US$ 22 million. On June 2018, US$23.18 million were cancelled as part of undisbursed proceeds (ICR page 50). The actual disbursement was US$385.55 million. Borrower Contribution: At appraisal, the Borrower had committed US$376 million. The actual Borrower contribution was US$111.59 million (ICR para 41). Dates: The June 2013 additional financing extended the closing date to allow the completion of activities. The project closed on June 30, 2018, 2.5 years after the original closing date of December 31, 2015. 3. Relevance of Objectives Rationale Alignment with the Bank’s strategy. The project objectives are aligned with the first and second strategic objectives of the Bank’s latest Country Assistance Strategy (CAS FY2013-2016). The strategic objective 1 aims at “increasing state effectiveness and improving good governance” and outcome indicator 1.3 is “increasing the operational performance of SOEs (SNCC and RVA).” The strategic objective 2 aims at boosting competitiveness to accelerate private-sector-led growth and job creation and outcome indicator 2.2 is “improved connectivity and access to transport infrastructure.” The project objectives are also aligned with the Bank’s Strategic Country Diagnostic (March 2018) that identifies transport connectivity, transport SOE governance and performance as key development issues. Alignment with country priorities. The project objectives were aligned with the government’s SOE reform program which aimed at: (a) transforming each SOE into a commercial entity to allow necessary labor, financial and operational reform that the preexisting public enterprise status did not permit; and (b) bringing in outside private expertise to manage each SOE during the transition phase leading to actual public-private partnership deals under contracts named “stabilization contracts” (ICR para 5). Realism and previous sector experience were not addressed in the ICR’s section on Relevance of Objectives, which entirely focuses only on the alignment of the PDOs with strategy documents. In the Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) absence of this discussion, the base of evidence is therefore partial, hence the rating is substantial, rather than high. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve transport connectivity in the Recipient’s territory to support national economic integration. Rationale Theory of Change. The reform and simplification of international trade and transit procedures was expected to facilitate trade and transit flows through the Democratic Republic of Congo (DRC) borders. This, in turn, would be causally linked to increased trade volumes and the promotion of greater economic integration in Central Africa. Moreover, stronger institutional and SOE performance would also be expected to directly and causally support economic integration. The achievement of both objective 2 and 3 - the improved financial and operational and performance of SNC and strengthening of transport SOEs operational performance was expected to improve transport connectivity. Outputs • A Trade Facilitation Committee was created but without a clear road map or budget. • A study for trade facilitation were carried out. However, the recommendations were not taken into consideration by the government (ICR para 44). • The Permanent National Committee for Trade Facilitation was not created. Outcomes The indicator “average transit time of mining exports by rail between the Copperbelt region and the Zambian border (days)” was dropped. Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Rating Negligible PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective To improve transport connectivity in the Recipient’s territory. Revised Rationale Outputs See outputs listed under objectives 2 and 3. Outcomes This objective was measured against the indicator “number of direct beneficiary” defined as the number of passengers on SNCC network. The target of 100,000 passengers was not met, as by project closing the total number of SNCC passengers was only 10,120. There was no other indicator to measure transport connectivity. Revised Rating Negligible PHEFFICACYTBL Objective 2 Objective To restore Société Nationale des Chemins de Fer du Congo (National Railway Company of DRC - SNCC’s) financial and operational viability. Rationale Theory of Change. The provision of technical assistance (TA) and equipment SNCC was expected to lead to improvements in its financial and operational and performance. The implementation of TA (through a management contract with a private operator and the training of key new and existing personnel) would also be causally linked to improvements in governance and management by refocusing the company on its core business of railway transport (and not other functions such as port operations, river transportation, schools and hospital operation). Physical investments in rolling stock and track rehabilitation would support the foregoing institutional strengthening by allowing SNCC to improve its services, increase freight and passengers, and reduce its debt. In a causal chain, the series of institutional TA and physical activities was expected to lead to achieving the goal of restoring SNCC’s financial and operational viability. Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Outputs • The private operator was hired under a performance contract management. However, the role was reduced to technical assistance. At the start of the project, the team had 13 experts with decision level functions. This arrangement could not be sustained beyond 2015, since the government did not commit to structural railway reforms. The private operator team was reduced to 5 experts by the end of the project (ICR para 66). • Technical Assistance was implemented by hiring experts in the fields of operation, renewal and rehabilitation of the track, rehabilitation of rolling stock, finance management, cost control, and project monitoring. • The non-core activities were not separated from SNCC. • The staff retrenchment plan for 4,354 SNCC employees was implemented. This represented a third of the permanent staff of SNCC. • The rehabilitation of 1,262 km of tracks was implemented, against a target of 1,300 km. • The rehabilitation of 732 freight wagons (against a target of 600), 10 passengers’ wagons (against a target of 50), and 52 wagons for ballast (against a target of 60) was implemented. • The rehabilitation of 8 locomotives and 5 switch locomotives and the purchase of 18 new locomotives (in 2015 and put into service in 2017) was implemented. In addition, the project financed the leasing of 20 locomotives between 2012 and 2016. •The project financed the purchase of fuel and lubricants (costing US$45 million) during the period September 2011 - July 2015. • The project also financed the acquisition of spare parts, weigh bridges, two weighing systems for Kalemie and Kananga (which at project closure were still in workshops in Lubumbashi), 8 dump-trucks, 18 vehicles (cars and ambulances), 3 backhoe loaders, and 7 vans. Outcome The implementation of the staff retrenchment plan improved the staff productivity. At project completion (June 2018), the staff productivity (Traffic Unit, or TU/staff/year) was 28,147 against the revised target of Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) 16,000 (original target was 70,000) and baseline of 17,700. The ICR does not discuss why the revised target (16,000) was lower than the baseline of 17,700. SNCC’s cash flow from operations, i.e. Earnings Before Interest, Taxes and Amortization (EBITA per year) did not improve but stabilized around US$(-)100 million in 2015 (achieving the revised target of US$(-)100 million). The original target was US$15 million, and baseline was US$(-)35 million. The hiring of a private operator did not improve SNCC’s internal processes and managerial practices as the role of the private operator was reduced from a performance contract management to technical assistance. The private operator was unable to implement the cost-saving measures agreed with the Bank (ICR para 66). SNCC is even further away from being on a sustainable financial and operational path than it was in 2010. The company accumulates additional debt at a pace of about US$ 30 million per year. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To implement a sector wide governance plan and strengthen transport state owned enterprises (SOEs) operational performances. Rationale Theory of Change. The preparation and adoption of governance plans by transport SOEs was expected to strengthen their operational performance. Outputs The Steering Committee for State-Owned Enterprises Reform (COPIREP) prepared the governance plans for SOEs. Outcome The Boards of SNCC, RVA, RVF, CVM, SCTP adopted the governance plans. However, the scope of the governance plans was narrow and included procedures related to internal audit, operations manual, management dashboard, and ethics. The plans could have included other important topics such as: investments and markets; salaries of executives; content of the management report; and accounting and financial information. In addition, these plans could have identified governance issues or weaknesses for each SOE and included specific measures with a timetable for implementation (ICR para 25). Four SOEs (RVA, CVM, AAC and RVF) completed annual audits between 2012 and 2017. Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Rating Negligible PHREVDELTBL PHINNERREVISEDTBL Objective 3 Revision 1 Revised Objective Strengthen transport state owned enterprises operational performances Revised Rationale Outputs National Airways Management Agency (RVA). The planned 5 Automated Dependent Surveillance- Broadcast (ADS-B) systems were installed in Kinshasa, Lubumbashi, Kisangani, Mbandaka, and Ilebo. However, two were not functional at project closing due to unreliable access to power. National Waterways Management Agency (RVF). • The rehabilitation of river marking boat Mongala was completed. • A total of 1,172 fixed and 33 floating navigation aids were built for river markings construction. • Navigational aids and markings were installed on 1424 km of waterways against the target of 1,500 km. Outcome Except for the improvement of RVA's operational performance, there is no evidence that the operational performance of other SOEs improved. • The average number of aircraft proximity traffic incidents (AIRPROX) was reduced significantly from 12 incidents reported yearly to 2 (revised target was 5). This was due to improved surveillance of airspace through the use of ADS-B and TopSky system and the availability of precise GPS-derived data for controllers. This outcome cannot be fully attributable to the project as the project financed 5 ADB-S and the African Development Bank financed 16 ADB-S. • The average number of losses of communication (COMMS) was reduced significantly from 78 to 12 per year. The revised target of 15 COMMS was exceeded. Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) Revised Rating Negligible PHOVRLEFFRATTBL Rationale The project did not achieve its original or revised objectives. Overall Efficacy Rating Primary reason Negligible Low achievement PHREVISEDTBL 5. Efficiency At appraisal, the economic analysis of the project focused on Component 1 (the SNCC recovery plan) and the tangible financial and economic benefits of the proposed activities were evaluated. From a financial standpoint, the SNCC recovery plan was expected to result in positive Earnings Before Interest, Taxes and Amortization (EBITA). Also, strong staff productivity gains were estimated to be threefold, and were also expected to strengthen SNCC’s long term financial performance and reduce the risk of a reversal in its financial performance. The economic gains of a rehabilitated SNCC railway network were estimated to be over US$1.2 billion over a 20 year period. The estimated Economic Rate of Return (ERR) was 19.5% The PAD (para 61) further noted that Components 2 and 3 involved activities within several SOEs in different transport subsectors which constituted only a small part of a larger transport system. It was, therefore, not possible to calculate one overall or an individual EIRR for these components. At completion, the ICR did not calculate the ex-post ERR. The ICR reports (para 29) that despite large investments, the project’s achievements are negligible. Therefore, as in the ICR, the efficiency of the project is rated negligible. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) 26.00 Appraisal  19.50 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Outcome under the original objectives is rated highly unsatisfactory, based on Substantial relevance of objectives, negligible Efficacy (negligible achievement of the three objectives), and negligible efficiency. Outcome under the revised objectives is also rated highly unsatisfactory based on Substantial relevance of objectives, negligible Efficacy (negligible achievement of the three objectives), and negligible efficiency. Overall outcome is rated highly unsatisfactory. a. Outcome Rating Highly Unsatisfactory 7. Risk to Development Outcome The risk to development outcome is assessed as high mainly because the current financial condition of the SNCC is unsound. SNCC is bankrupt and therefore it is highly unlikely that the infrastructure provided by the project i.e. railway tracks, buildings, and equipment will be adequately maintained after project closure (ICR para 62). 8. Assessment of Bank Performance a. Quality-at-Entry The Bank had been engaged in SOE reform in DRC since 2003. The project was designed to support this reform program and targeted DRC’s main SOEs. However, the scope of the project was ambitious and (with the exception of roads) covered all transport subsectors: railways, inland waterways, ports, port access channel/maritime transport, airports and aviation; with a focus on railways (85 percent of project funding). Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) The Bank could have been more selective in the number of SOEs and on the nature of support provided to them. Implementation arrangement were also too complex with 25 legal covenants and 15 disbursement categories that created implementation challenges for both the Borrower and the Bank and had to be revised several times (ICR para 53). The highly ambitious institutional objectives could have benefitted from a more robust evaluation of SOEs governance or political economy. The institutional diagnostics were left for implementation phase, deferring the identification of concrete action plans (ICR para 53). The M&E design was weak (see section 9 below). Quality-at-Entry Rating Unsatisfactory b. Quality of supervision The project had high staff turnover, as there were 7 Task Team Leaders (TTLs) during the 8-year implementation period. This staff turnover created uncertainty for the client (ICR para 56). Due to security issues in 2016-2017, a supervision mission to visit the SNCC network in Kasai was cancelled. The field visits by the Bank team could have included additional project areas. For example, the Bank team did not visit sites where river markings were installed (ICR para 57). The high volume of procurement transactions affected the ability of the Bank team to provide timely response to the client. The project had 417 items in the Procurement Plan (211 under PIU based in Lubumbashi, and 206 under PIU based in Kinshasa). The procurement should have been consolidated to achieve economies of scale (ICR para 58). While the project displayed low performance, the Bank continued with the project with the hope that the situation could be turned around instead of going for a more radical mid-course correction (ICR para 59). The Bank did not suspend or cancel the project even though the project did not comply with a number of legal covenants and disbursement conditions. Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E design was weak. The project included output indicators. The indicator to measure transport connectivity and economic integration (the first PDO) was inadequate. It measured direct beneficiaries but covered only passengers transported by the railways sector. It could have considered other transport modes. The other indicator “average transit time of mining exports by rail between the Copperbelt region and the Zambian border (days)” was dropped. The indicators for measuring the third PDO to “implement a sector wide governance plan and strengthen transport state owned enterprises’ operational performances” could have been defined more precisely, with explicit targets and baselines to measure operational performance of SOEs. The ICR reports (para 46) that initial targets were overestimated and 2 out of the 9 initial PDO indicators were dropped and replaced by new indicators; also, 3 out of the 8 initial intermediate indicators were dropped and 4 intermediate indicators had been added. b. M&E Implementation The ICR reports (para 48) that project indicators were monitored by the recipient SOEs and the two full-time M&E officers at Kinshasa and Lubumbashi PIUs. Three indicators were monitored by the PIUs while the 15 others were monitored by beneficiary SOEs (SCTP, SNCC). RVA did not provided any data until a few months before project closing. c. M&E Utilization The indicators were used to inform the 2016 restructuring, which redefined the scope and expected outcome of component 3 and introduced indicators for the river marking activity (ICR para 48). M&E Quality Rating Modest 10. Other Issues a. Safeguards Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) At appraisal, the project was assigned environmental category B. As construction activity was limited to the rehabilitation of existing rail alignments, no significant environmental and social impacts were anticipated. Safeguards policies triggered included: Environmental Assessment (OP 4.01), Indigenous Peoples (OP 4.10); Physical Cultural Resources (OP 4.11), and Involuntary Resettlement (OP 4.12) (PAD para 86). An Environmental Management Framework (EMF) and Resettlement Policy Framework (RPF) were prepared and finalized (disclosed in-country on 12/31/2009, and at the InfoShop on 1/21/2010). These were expected to guide the preparation of appropriate safeguard instruments for specific subprojects, including Environmental Assessments (EAs), Environmental Management Plans (EMPs), Indigenous People Plans (IPP) and Resettlement Action Plans (RAPs) if and when required. The ICR reports (para 50) that overall safeguards performance was rated satisfactory during implementation, but does not provide any details on the actual safeguards implementation. b. Fiduciary Compliance Financial management: The financial management team was composed of 8 staff (4 in Kinshasa and 4 in Lubumbashi). The team included 1 financial management specialist, 1 treasurer, 1 accountant and 1 internal auditor. This arrangement seems to be adequate as the quarterly financial reports were prepared on time and submitted satisfactorily to the Bank. Annual external audits were also processed satisfactorily, and auditors issued unqualified opinions. The ICR rated financial management as satisfactory (para 51). Procurement: The procurement team was composed of 7 staff (3 in Kinshasa and 4 in Lubumbashi). The Kinshasa team included 1 Lead Procurement specialist, 1 Procurement specialist and 1 procurement assistant. The Lubumbashi team was composed of 1 Lead Procurement specialist, 2 Procurement specialists and 1 procurement assistant. The ICR reports (para 52) that because of weak technical capacity of the SOEs, the procurement packages and bidding review processes were adversely affected. Also, there were delays on the Bank side for the clearance of these documents. Two cases of corrupt and fraudulent practices were found by the Bank’s Integrity Service (INT). INT also identified potential integrity risks within the Project. The ICR rated procurement management as moderately satisfactory. c. Unintended impacts (Positive or Negative) --- d. Other --- Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment There is no disagreement as the derivation of project outcome ratings in the ICR is incorrect. According to Appendix H, page 38 of the Bank’s ICR preparation Outcome Unsatisfactory Highly Unsatisfactory guidelines, ratings of substantial for relevance of objectives, negligible for efficacy, and negligible for efficiency correspond to an overall outcome rating of highly unsatisfactory. The quality of entry and Moderately supervision had significant Bank Performance Unsatisfactory Unsatisfactory shortcomings (see section 8 above). Quality of M&E Modest Modest --- Quality of ICR Modest --- 12. Lessons The following lessons are taken from the ICR with some adaptation of language. • In a post-conflict country, characterized by weak implementation capacity and political inertia, it is critical to manage expectations, limit the scope of intervention and leave space to scale-up only once positive initial results are achieved. The scope of the project was ambitious and (with the exception of roads) covered all transport subsectors: railways, inland waterways, ports, port access channel/maritime transport, airports and aviation. • Efforts to improve operational and financial performance of SOEs through investments in assets and private participation need to be accompanied by adequate government commitment to reforms and strong motivation to improve poor management practices in the SOEs, to show results. The project experience shows that the operational and financial performance of the railways could not be improved through investments in rail assets and hiring of private operator alone. The problems ran deeper and involved especially the lack of motivation and poor management practices that were widespread inside the railway company. The government's commitment to the reforms was also low. The absence of support from the Ministry of Transport and of the Ministry of Portfolio based on a roadmap Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DRC-Multi-Modal Transp (P092537) prevented the project from achieving its development objective. The role of the private operator was reduced from a performance contract management to technical assistance. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is results oriented and provides good lessons from project experience. However, there are some significant shortcomings. The overall project outcome rating is derived incorrectly. The ICR does not provide an adequate description of project implementation challenges. It does not provide any details on the implementation of safeguard policies. It is not explained as to why the revised staff productivity target (16,000) was lower than the baseline of 17,700. a. Quality of ICR Rating Modest Page 17 of 17