ICRR 10680 Report Number : ICRR10680 ICR Review Operations Evaluation Department 1. Project Data : Date Posted : 08/02/2000 PROJ ID : P010452 OEDID: OEDID : C2558 Appraisal Actual Project Name : Sindh Special US$M ) Project Costs (US$M) 58.6 49.9 Development Project Country : Pakistan Loan/ US$M ) Loan /Credit (US$M) 46.8 42.07 Sector, Major Sect .: Institutional US$M ) Cofinancing (US$M) Development , Public Sector Management L/C Number : C2558 FY ) Board Approval (FY) 94 Partners involved : Closing Date 12/31/1998 06/30/1999 Prepared by : Reviewed by : Group Manager : Group : Michael R. Lav John H. Johnson Ruben Lamdany OEDCR 2. Project Objectives and Components a. Objectives The project's revised objectives, as approved by the Board on February 25, 1999, included: (a) begin to overcome serious environmental and infrastructure problems in Karachi and interior cities in the Sindh; (b) support Sindh's institutional reform program; and (c) strengthen the capacity of Karachi Municipal Corporation . b. Components The components of the revised project included : 1. Physical Objectives: (a) channelization of the Korangi Nallah, (b) Malir River flood protection works, (c) Lyari River bridge, (d) a railway overpass on the University Road, (e) improvements at 16 intersections in Karachi, (f) road rehabilitation works in Sukkur, (g) bus terminals in Larkana and Mirpurkhas, (h) a truck terminal in Nawabshah, (i) taxi and wagon stands in Jacobabad, and (j) a bus terminal in Sehwan Sharif. 2. Sector Policies/Financial Objectives: (a) increasing the yield of the property tax in Karachi through a combination of increased rates, reduced exemptions, property revaluation, and improved collection, (b) furthering financial management reforms in GOS through containment of low priority recurrent expenditures and subsidies and improving financial systems; improve GOS operations by establishing a three -tiered system to enable collaborative policy formulation among bureaucrats, policy makers and the public, with the three tiers being Working Groups, Secretaries' Committee, and Cabinet Committee, with EXPACO established as the secretariat to the Cabinet Committee to coordinate policy formulation as well as to provide overall coordination for the project activities, (c) developing a digital map for Karachi, and (d) strengthening KMC's land management. 3. KTC component: Almost half of the revised project comprised severance payments for past employees of the Karachi Transport Corporation, which, however, had been defunct since 1997. c. Comments on Project Cost, Financing and Dates The original project for US$46.8 million was approved by the Board on December 16, 1993 and was made effective on September 21, 1994. The revised project comprised US$ 42.07 million as approved by the Board in February, 1999. The project closed on June 30, 1999, about six months later than the original closing date of December 31, 1998. 3. Achievement of Relevant Objectives : 1. Physical Objectives: (a) the channelization of Korangi Nallah, (b) Malir River flood protection works, (c) Lyari River bridge, (d) improvements at 14 intersections in Karachi (of the 16 intended) , (e) road rehabilitation works at Sukkur, (f) bus terminals in Larkana and Mirpurkhas, and (g) a truck terminal at Nawabshah. 2. Sector/Financial management objectives: (a) KMC repaired and restored over 500 land maps and scanned the lease documents of two divisions, which were steps taken towards preparing a digital map of Karachi . 4. Significant Outcomes /Impacts : Flood protection and drainage works at Korangi Nallah and Malir, the road rehabilitation works at Sukkur, bus terminals at Larkhana and Mirpurkhas, and the truck terminal at Nawabshah . 5. Significant Shortcomings (including non -compliance with safeguard policies ): There was poor quality at entry (lack of ownership, excessive complexity ), and the project also lingered for years before it was restructured. There were also significant specific shortcomings in a number of the project's components: 1. Physical Objectives: (a) The University Road overpass was without value, since the railroad had already stopped using the tracks under the overpass . The ICR calculates the ERR of this component at -5% because of the residual value of the investment, but it is not clear what this residual value would really be, nor why the ERR is not much worse the -5% since any residual use of the asset would be far in the future while costs have already been incurred . (b) Improvements at 2 of the 16 intersections were not achieved under the project . More importantly, the works which were completed to improve traffic at 14 intersections in Karachi were not designed as and do not form part of a coherent traffic management system . (c) The taxi and wagon stands in Jacobabad and the bus terminal in Sehwan Sharif were only 80% complete at project closing, and GOS will complete these using its own resources . 2. KTC severance payments were half of revised project but were not relevant to any of the project's objectives . Although the payments provided income to past employees in fulfillment of government obligations, this is essentially equivalent to budget support in a case where there were many problems with budget and fiscal management . 3. Policy/Institutional strengthening. (a) KMC could not complete training for Land Department staff, and could not scan the lease documents of 5 (out of 7) divisions, (b) EXPACO did not play the role envisaged under the project for collaborative policy formulation, (c) the ICR notes that the property tax reform is "far from complete" and (d) the containment of low priority recurrent expenditures or unwarranted subsidies was apparently not achieved, although it is not entire clear from the ICR whether this evaluation is contained in the ICR's comment that the financial management reform could not be completed . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Unsatisfactory Highly Unsatisfactory The ICR rates the project as unsatisfactory. However, OED finds the project to have been highly unsatisfactory because of failures to strengthen policy and institutions, the mispecification of some of the physical components, and the lack of justification/relevance of the KTC severance pay component which comprised nearly half of the revised project's costs (see section 5 above for details). Institutional Dev .: Negligible Negligible Sustainability : Unlikely Unlikely Bank Performance : Deficient Highly Unsatisfactory Both the ICR and the OED review agree that project formulation and supervision were not satisfactory. Restructuring was tardy. Deficient is the lowest rating available for the ICR's rating of Bank performance. Borrower Perf .: Deficient Highly Unsatisfactory The Borrower failed to implement most of the policy/institutional reform measures it committed to support even under the revised project, delayed in appointing key staff, and lagged in implementing many aspects of the physical works . Deficient is the lowest rating available in the ICR, while the Highly Unsatisfactory rating given by OED is based on the negligible achievements under many components and substantial shortfalls in others . Quality of ICR : Satisfactory 7. Lessons of Broad Applicability : (1) The project clearly lacked ownership, and is another unfortunate example of how lack of ownership dooms a project. A systemic approach should be taken whenever a large traffic management program is formulated . (2) Risk analysis should be taken seriously . This holds for the project as a whole, as well as many of its components . For example, better risk analysis might have averted proceeding to construct an overpass over railroad tracks which were only infrequently used at the time of project formulation and which ceased to be used even while the project was being implemented. 8. Audit Recommended? Yes No 9. Comments on Quality of ICR : The ICR goes to considerable lengths to gather and assess new data on sub -project returns, and is frank and incisive on the reasons for the project's failure . It also has some good lessons . However, it might have also looked a bit more at the lack of coherence of the large KTC severance payment component . This Evaluation Summary places more emphasis on the project as revised by the Board, in keeping with OED guidelines, while the ICR evaluates the project essentially on the basis of its original formulation, in keeping with the guidelines for ICRs .