BANGLADESH Rural Transport Improvement Project Phase I Report No. 106729 JUNE 30, 2016 © 2016 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. The World Bank does not guarantee the accuracy of the data included in this work. The Any queries on rights and licenses, including boundaries, colors, denominations, and other subsidiary rights, should be addressed to information shown on any map in this work do World Bank Publications, The World Bank not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 106729 PROJECT PERFORMANCE ASSESSMENT REPORT Bangladesh RURAL TRANSPORT IMPROVEMENT PROJECT (CREDIT IDA-37910, IDA-37911) June 30, 2016 IEG Financial, Private Sector, and Sustainable Development Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Bangladeshi Taka 2003 US$1.00 BDT58.20 2004 US$1.00 BDT58.65 2005 US$1.00 BDT60.98 2006 US$1.00 BDT66.58 2007 US$1.00 BDT69.10 2008 US$1.00 BDT68.55 2009 US$1.00 BDT68.90 2010 US$1.00 BDT68.97 2011 US$1.00 BDT70.82 Abbreviations and Acronyms CAS Country Assistance Strategy CE Chief Engineer DC District Commissioner DSM Design and Supervision Management consulting Firms EMIS Environmental Management Information System EMP Environmental Management Plan ERR Economic Rate of Return FGD Focus Groups Discussion GoB Government of Bangladesh ICB International Competitive Bidding ICR Implementation Completion Report ICRR Implementation Completion Report Review IDA International Development Association IEG Independent Evaluation Group ISAP Institutional Strengthening Action Program LGED Local Government Engineering Department LGI Local Government Institutions LGIP Local Government Improvement Plan NCB National Competitive Bidding NPV Net Present Value ORA Operational Risk Assessment PAD Project Appraisal Document PPAR Project Performance Assessment Report PRSP Poverty Reduction Strategy Paper PIU Project Implementation Unit RTIP Rural Transport Improvement Project RTIP II Rural Transport Improvement Project II iii RRMIMP I Roads and Markets Improvement and Maintenance Project I RRMIMP II Roads and Markets Improvement and Maintenance Project II TTL Task Team Leader UP Union Parishad UZ Upazila Fiscal Year Government: July 1 – June 30 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Financial, Private Sector & Sustainable Development : Mr. Marvin Taylor- Dormond Manager, IEG, Sustainable Development : Ms. Midori Makino Task Manager : Ms. Lauren Kelly v Contents Principal Ratings ................................................................................................................ vi Key Staff Responsible........................................................................................................ vi Preface................................................................................................................................ ix Summary ............................................................................................................................. x 1. Background and Context................................................................................................. 1 2. Objectives, Design, and their Relevance ........................................................................ 2 3. Implementation ............................................................................................................... 4 4. Achievement of the Objectives ....................................................................................... 7 5. Efficiency ...................................................................................................................... 12 6. Ratings .......................................................................................................................... 13 7. Bank Performance ......................................................................................................... 14 8. Borrower Performance .................................................................................................. 15 9. Monitoring and Evaluation ........................................................................................... 16 10. Lessons........................................................................................................................ 17 References ......................................................................................................................... 18 Appendix A. Basic Data Sheet.......................................................................................... 19 Appendix B. List of Persons Met ...................................................................................... 22 Appendix C. Borrower Comments ................................................................................... 23 This report was prepared by Abhinav Kumar Gupta, consultant under the guidance of Lauren Kelly, Senior Evaluation Officer in the Independent Evaluation Group of the World Bank, in collaboration with IEG consultants Kathryn Steingraber, Sonia Sarder. The report was peer reviewed by Pradeep Mitra and panel reviewed by Peter Nigel Freeman. Vibhuti Khanna provided administrative support. vi Principal Ratings ICR* ICR Review* PPAR Outcome Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Risk to Moderate Moderate Moderate Development Outcome Bank Performance Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Satisfactory Moderately Satisfactory Moderately Satisfactory Performance * The Implementation Completion Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEGWB product that seeks to independently verify the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Fabio Galli/Mohi Uz Vincent Gouarne Frederick Thomas Temple Zaman Completion Reefat Sultana Karla Carvajal Salman Zaheer Gonzalez vii IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, and Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. ix Preface This is a Project Performance Assessment of the Rural Transport Improvement Project (RTIP) that became effective on July 30, 2003, that was originally scheduled to close on June 30, 2009, and that ended on the extended closing date of June 30, 2012. The total appraisal costs were estimated at US$250 million including US$204 in IDA contributions. IEG was able to validate the actual IDA contributions of US$228.70 including US$20 million of additional financing. A total of US$ 4.5 million was also recorded as being cancelled. Efforts were made to obtain total cost information at t both the ICR and the PPAR stage, including the borrower contribution, but this information was not made available to the IEG team. The project development objective was to provide rural communities with improved access to social services and economic opportunities, and to enhance the capacity of relevant government institutions to better manage rural transport infrastructure. A second phase of the project – RTIP II – was approved in December 2008 and was under implementation at the time of this review. This assessment learns from the cumulative implementation experience, but only assesses and rates the first phase. This report was prepared by Abhinav Kumar Gupta, consultant, under the guidance of Ms. Lauren Kelly, Senior Evaluation Officer in the Independent Evaluation Group of the World Bank. The assessment was supported by Kathryn Steingraber and Sonia Sarder, and was overseen by Ms. Midori Makino, Manager IEGSD. The team would like to recognize the strong support provided by the World Bank Country Office in Dhaka and the excellent assistance from the Local Government Engineering Department, including the Project Management Unit, regional staff and community facilitators in the state of Sirajganj. This assessment was selected to provide input to a Bangladesh Country Study commissioned as part of IEG’s forthcoming Evaluation of the World Bank Group’s Contribution to the Development of the Rural Non-Farm Economy to Alleviate Poverty (2004-2014). Methodology. This project performance assessment seeks to validate the relevance, efficiency and effectiveness of the reported results of the Rural Transport Improvement Project. In preparation for the mission, a desk review of the documentation was conducted that included a review of the Project Appraisal Document, Implementation Completion and Results Report, legal and project files, the mid-term review and the Social Economic Monitoring Evaluation commissioned by the project. The desk review was supplemented by group interviews of local beneficiaries in the project areas. Following standard IEG procedures, copies of the draft PPAR will be shared with relevant Government officials and agencies for their review and comment. Comments received will be included in Annex C of the report. x Summary This is a Project Performance Assessment of the Rural Transport Improvement Project (RTIP) that became effective on July 30, 2003, that was originally scheduled to close on June 30, 2009, and that ended on the extended closing date of June 30, 2012. The total appraisal costs were estimated at US$250 million including US$204 in IDA contributions. IEG was able to validate the actual IDA contributions of US$228.70 including US$20 million of additional financing. A total of US$ 4.5 million was also recorded as being cancelled. Efforts were made to obtain total cost information at t both the ICR and the PPAR stage, including the borrower contribution, but this information was not made available to the IEG team. . The PDO of the project was to (1) provide rural communities with improved access to social services and economic opportunities, and to (2) enhance the capacity of relevant government institutions to better manage rural transport infrastructure. A second phase of the project – RTIP II – was approved in December 2012 and was under implementation at the time of this review. This assessment learns from the cumulative implementation experience, but only assesses and rates the first phase. The relevance of both objectives are substantial. The objectives are in line with the Country Assistance Strategy goals for rural development in Bangladesh, both at the time of appraisal and at close, including with the specific CAS goal at the time of project close to maintain and extending Bangladesh’s transport network, including the rehabilitation of rural transport. The objectives are have also relevant the Government's Poverty Reduction Strategy which aims to reduce the incidence of rural poverty by improving the infrastructure of the rural economy. The objective statement lacked specificity, however, with regard to the category of rural beneficiaries that it intended to support. The relevance of design is also substantial. The project design had an appropriate balance among the various aspect of the physical works for improving rural roads to enhance access to markets and production centers; rehabilitation and maintenance of roads; improvement of rural markets; and technical assistance and capacity building. The theory of change is well grounded in evidence, but the results framework lacked specific and attributable links between the intermediate outcomes anticipated and the project development objectives. A Socio-Economic survey was designed to measure the PDO, but it fell short of measuring distributional impacts unlike the impact assessments that had been commissioned for the Bank supported predecessor projects in the rural roads sector during the decade prior. The project substantially achieved its objectives of improving access to social and economic activities. The project only modestly achieved its objective of enhancing the capacity of relevant government institutions to better manage rural transport infrastructure. While physical targets were met, and in most cases surpassed, the original road quality metrics introduced during appraisal were not adhered to, posing significant threats to the sustainability of the rural road network, that in turn, poses risk to the aim of increasing access to economic and social opportunities for rural communities. The capacity building objective only partially achieved its training objectives and evidence is lacking that the training provided enhanced the capacity of relevant government institutions to better manage rural xi transport infrastructure, with an emphasis on better management. Metrics used to measure the capacity building aims were either insufficient (reduction of operating costs without a complementary metric associated with work quality) or we not met (resource mobilization). Efficiency is rated substantial. The ex-post economic analysis estimated an ERR for the road improvement component of 19.2% and for the rural markets component of 35.2%. These costs represented about 75% of the project costs. These calculations are in line with the ex- ante ERR estimate of 20.5% for the road improvement and 28.9% for the rural market improvement component. The level of road quality and the management challenges associated with the markets however raises some concern about the higher than anticipated rates of return, especially for the market infrastructure. The project also experienced inefficiencies owing to delays and cost escalations that were encountered during the project period. Regarding the cost escalation and delays, the Mid-Term Review of the project which took place in December 2005 identified the key issues related to delays in implementation, and after follow up discussions, the LGED moved to start cancelling the contracts for non- performance. In total 73 contracts had to be cancelled. Since the additional financing resulted in extended closing date, most of the cancelled contracts were rebid and completed satisfactorily. Although delayed and with some cost escalation, LGED’s prompt contractual decisions helped in the completion of about 70 contracts. Bank performance is moderately satisfactory. Despite having access to lessons learned across several predecessor projects, quality at entry was undermined by a lack of awareness and capacity to implement the Bank’s operational policies, to support better fiduciary management, and to put in place a system that not only supported the completion of physical works, but that also focused on the software, including management capacity and decentralized resource mobilization constraints and opportunities. Though the quality of supervision was timely and recommendations were followed up with actions, it was undermined by ineffective management of social and environmental risks, unclear articulation of specific aims and measurable targets, discrepancies around the metrics for road quality and ineffective International Competitive Bidding (ICB) tool to gain efficiencies for rural construction and rehabilitation. Borrower performance is rated moderately satisfactory. Government commitment was high, including an increase in the budgetary allocation for rural road maintenance, however the borrower deviated from the agreed upon road quality standards set at appraisal. While the Government also took steps to provide LGED with greater autonomy and tools to strengthen its institutional capacity, there was demonstrated commitment towards decentralized management of rural infrastructure, including the need for enhanced resource mobilization to meet local repair and construction needs. The implementation agency effectively oversaw the delivery of the project objectives in line with the agreed upon metrics, but there was inadequate attention and support for the effective implementation of environmental and social safeguards and evidence of non-compliance with the operational policies. There were also irregularities related to mobilization of payments for contracts that were subsequently xii canceled due to contractor performance issues, and some delays in financial reporting which were addressed during supervision missions. Lessons Human capacity development is as important as technical training in projects promoting rural decentralized infrastructure service delivery. In the RTIP, while physical targets were met or exceeded, more effective and in-depth training may have helped to smooth project implementation and to build longer term capacity within the Government, including for environmental management. A decision to rely on a parallel Bank financed project for planned training did not generate the results anticipated, as confirmed by IEG’s review of both this and the parallel project. Training in the parallel project was found to have been inadequate. Civil works contracts should include a transparently negotiated, price escalation provision, even during short implementation periods, but ensuring a more realistic estimate would be important, too. In the case of RTIP, the price of construction materials rose sharply during the short implementation period (eighteen months) which resulted in many contractors abandoning the works. Timing and sequencing are critical considerations when awarding civil works contracts. In the case of RTIP, the implementing agency awarded civil works contracts at the beginning of the rainy season or late in the dry season, resulting in non-activity of contractors for half of the year and delays in contract implementation. All contracts which were awarded in this way took an extra year for completion. On the other hand, contracts awarded at the end of rainy season had two full dry seasons to be completed. Marvin Taylor-Dormond Director, IEG Financial, Private Sector, and Sustainable Development 1 1. Background and Context 1.1 Bangladesh, with a population of about 150 million and a land area of 147,570 square kilometers is amongst the most densely-populated countries in the world. The economy of Bangladesh grew steadily at about 5.6 per annum between 2000 and 2010 and at a rate of about 6 percent per annum since then. Headcount poverty fell steadily from 48.9% in 2000 to 31.5% in 2010 and, given slowing population growth, led to there being 17 million fewer poor people in 2010 compared to 2000. Rural poverty has also fallen significantly from 52.3 percent in 2000 to 35.2 percent in 2010. 1.2 The two main drivers of poverty reduction were: Growth in labor income—higher returns to farm and nonfarm endowments--which contributed nearly two-thirds (64%) to poverty reduction and the demographic transition. Fertility rates have declined from 6.3 births/women in 1975 to 2.3 births/woman in 2011. The associated fall in the child dependency ratio and a rising share of the working age population in the total population contributed to 25% of the poverty reduction. 1.3 From 2000 to 2005, increased wages in the nonfarm sector made a substantial contribution to poverty reduction. Three-quarters of new jobs added during the entire decade were in the non-agricultural sector, with more of this occurring during the first half of the decade. There was a reduction in the proportion of illiterates and an increased share in the workforce of those with completed primary and lower secondary education, as well as large population shifts from rural to urban regions. From 2005 – 2010: poverty reduction was largely owed to rising returns to farm endowments in the form of (1) increased wages of rural unskilled labor,(i.e., not associated with changes in education or occupation) and (2) rising returns to rural agriculture-based households engaged in cultivation of their own farms. 1.4 Entry barriers faced by the rural poor in accessing improved income-generating activities in the rural nonfarm economy include lack of education, poor infrastructure in areas distant from urban markets, and lack of electricity. The Rural Transport Improvement Project (RTIP) was designed to improve rural transport infrastructure, consisting of rural roads, inland water transport and rural markets. The project follows decades of sustained support to the sector, beginning most notably with the Bank supported Rural Infrastructure Strategy Study (1984) followed by a series of IDA funded projects including the first and the second Rural Roads and Markets Improvement and Maintenance Project which helped to facilitate economic and social development in targeted districts by improving rural mobility and access to services. RTIP was designed to scale up the approach while strengthening local governmental capacity and consolidating various donor initiatives underway. 2 2. Objectives, Design, and their Relevance 2.1 Project Development Objective The development objectives of the project were to (1) provide rural communities with improved access to social services and economic opportunity, and to (2) enhance the capacity of relevant government institutions to better manage rural transport infrastructure. The PDO was the same in the PAD (page 2) and Development Credit Agreement (page 13). 2.2 Project Components: The original project contained eight components. While the PAD and ICR provides the appraisal costs by component in US dollars, these were inconsistent between the two documents. And, while the PAD provides the component costs in US dollars, the ICR only provides the actual component costs in the Bangladesh Taka. This assessment uses the appraisal costs included in the PAD, and the costs in Taka reported in the ICR. For the purpose of comparing costs, this assessment converts the US dollar into Bangladesh Taka at the time of project appraisal. (1) Improvement of about 1,100 km of Upazila roads (there are feeder roads known by their acronym UZR), (BDT 5,309.7M/US$91.2M at appraisal; actual costs were BDT 8,638.1M, 162.7% of the cost at appraisal). This component was designed to improve rural access and connectivity to markets and production centers in the 21 project districts by making key UZR passable all year around and by improving their riding quality and carrying capacity. (2) Improvement of about 500 km of Union roads (these are rural roads) (BDT 1,129.5M/US$19.4M at appraisal; actual costs were BDT 2,085.8M, 184.7% of the cost at appraisal). This component was designed to improve rural access and connectivity in the 21 project districts by making key Union roads passable all year around and by improving their riding quality. (3) Periodic maintenance/rehabilitation of about 1,500 km Upazila roads. (BDT 1,874.7M/US$32.2M at appraisal; actual costs were BDT3425.2M, 182.7% of the cost at appraisal). This component was designed to help reduce the periodic maintenance/rehabilitation backlog on the Local Government Engineering Department (LGED) managed portion of the road network by funding the overlaying, resealing and minor rehabilitation of 1,500 km of key UZRs in the 21 project districts. (4) Construction of about 15 km of culverts/bridges on Union Roads (BDT 1,467.1M/US$25.2M at appraisal; actual costs were BDT2640.4M, 180% of the cost at appraisal). This component was designed to make approximately 5,000 km of URs passable year-around in the 21 project districts, and to provide appropriate drainage structure on these roads. 3 (5) Improvement and/or construction of about 150 rural markets and 45 river jetties. (BDT844.2 M/US$14.5M at appraisal; actual costs were BDT330.9M, 39.2% of the cost at appraisal). The aim of this component was to help lower the cost of goods and commodities in the 21 project districts by facilitating their trade. Implementation of these activities did not materialize as anticipated, as reflected by the actual costs, including 35 markets that were dropped due to either technical feasibility or failure to mobilize local contributions as required (which could also reflect a lack of local demand). (6) Land acquisition, implementation of Resettlement Framework, Environmental Management Framework, Resettlement Action Plan, Environmental Management Plan, Indigenous People's Development Plan, utility relocation (BDT675.4M/US$11.6M at appraisal; actual costs were BDT125.1M, 18.5% of the cost at appraisal). Component finance was provided to assess and mitigate the social and environmental impacts of implementing the civil works components. Land acquisition and cash compensation was to be funded entirely by the Government of Bangladesh. This component was severely underfunded, reflective of the overall shortcomings associated with the implementation of the environmental and social risk mitigation measures. (7) Design Supervision Monitoring consultant services, quality, financial and procurement audit services, and other consultant services. (BDT675.4M/US$11.6M at appraisal; actual costs were BDT984.4M, 145.8% of the cost at appraisal). This component was designed to ensure the timely, quality, and cost effective construction, maintenance/rehabilitation of the civil works components under the project. (8) Technical Assistance, training, capacity building of Local Government Engineering Department (LGED) and Local Government Institutions (LGIs), equipment and pre-investment studies. (BDT291.1M/US$5M at appraisal; actual costs were BDT317.3M, 109% of the cost at appraisal).This component was designed to finance a wide variety of activities and investments including enhanced training programs of LGED staff, LGED implementation of improved and new business processes and systems, rural government capacity building initiatives and rural road transport safety. This component was not fully implemented, in part because of the approval and implementation of a parallel Bank project that started to offer similar training. (Issues associated with the implementation of the training activities in the project are further discussed in the efficacy section). 2.3 A component on Flood Rehabilitation was added in February, 2008 following approval of an Additional Financing. While this component was treated separately in the Aide-Memoires, no formal changes in the system were made to add the component in the Project Portal. The additional financing of US$25 million IDA Grant funds (of which only US$20.5 million was utilized) supported recovery efforts of the flood affected population. . The component had two parts. The first part with a financing of US$ 10 million was used to carry out livelihood restoration of communities directly affected by the 2007 Floods in the project and neighboring districts, while the remaining US$15 million were transferred to PKSF to support 2007 Flood restoration activities. PKSF administered the micro-credit program through the Disaster Management Fund (DMF) and its Partner Organizations (POs) which provide the low interest loans to eligible families. 4 Relevance of Objective 2.4 The relevance of the objectives is Substantial. The World Bank’s Country Assistance Strategy goals for rural development in Bangladesh at the time of appraisal time were to: a) support growth in agriculture and non-farm activities; (b) increase opportunities and assets available to the poor; (c) improve rural infrastructure services; and (d) strengthen local institutions. The objectives are also in line with the Country Assistance Strategy goals at the time of project close. The 2011-2014 CAS included a goal of maintaining and extending Bangladesh’s transport network, including the rehabilitation of rural transport. The objectives are also relevant the Government's Poverty Reduction Strategy which aims to reduce the incidence of rural poverty by improving the infrastructure of the rural economy. The objective statement lacked specificity, however with regard to the category of rural beneficiaries that it intended to support. As evidence on distributional impacts will show, this lack of specificity in the objective statement may have missed an opportunity to have had more direct, poverty-related impacts. Relevance of Design 2.5 The relevance of design is Substantial. The project design had an appropriate balance among the various aspect of the physical works for improving rural roads to enhance access to markets and production centers; rehabilitation and maintenance of roads; improvement of rural markets; and technical assistance and capacity building. The theory of change is well grounded in evidence: It is reasonable to assume that improving rural roads would lead to a reduction in travel time and transport costs. It was also reasonable to assume that enhanced access would augment economic opportunities (agricultural and non-farm activities, and household employment) and access to social services (clinics, schools, etc.). The results framework lacked specific links however between the intermediate outcomes anticipated and the project development objectives. A Socio-Economic survey was designed to measure the PDO, but it fell short of measuring distributional impacts unlike impact assessments that were commissioned for Bank supported predecessor projects in the sector. 3. Implementation Project costs 3.1 The project was implemented over a period of nine years, due to the extension of the project to support additional flood rehabilitation work and delays that were mainly attributable to challenges associated with land acquisition and compensation and poor contract management. These delays mainly affected the improvement of Upazila roads while the road maintenance activities progressed satisfactorily (since they did not involve land acquisition). During the project period, there was also a severe price increase of construction materials (especially cement and reinforcement steel) in international markets (2007 to 2008) which resulted in a major setback in the construction of the Upazila roads. Many contractors abandoned the site because of the high price of materials since price escalation was not included as part of their contract. 5 3.2 The Mid-Term Review, which took place in December 2005, identified many of the key issues leading to delays. After the Mid-Term, LGED initiated the cancellation of non- performing contracts. As a result, about 73 contracts were cancelled, while 70 were effectively completed. 3.3 After the mid-term, the project introduced four International Competitive Bidding (ICB) contract packages on a pilot basis - which were awarded in 2008 - with each package comprising four to five roads. The aim was to test the ICB modality against the National Competitive Bidding (NCB). These contracts were characterized by severe delays however, with the explanation provided that the awards covered too large of a geographic space. Two contracts were completed with a one year extension beyond the original contract period of thirty months. The other two contracts were cancelled and re-awarded as NCBs. 3.4 In 2008, the World Bank and LGED initiated an Operational Risk Assessment study to assess fiduciary and operational risks in LGED’s management of projects, assets and other resources, to evaluate the efficacy of external review of decision-making by LGED and the LGD, to identify options for future monitoring of operational risks in LGED and the LGD, and to prioritize options which are realistic and to effectively minimize the major operational risks identified. 3.5 Project Dates. The project became effective on July 30, 2003. The original closing date was June 30, 2009. At the time of approval of the Additional Financing, the closing date was extended to June 30, 2011 to allow completion of the originally planned civil works and to meet needs arising from the 2007 Flood. A further extension of one year to June 30, 2012, was subsequently granted to allow completion of flood rehabilitation contracts which experienced delays in implementation 3.6 Project Costs and Financing. Total actual project costs were US$199.5 million, including additional financing, compared to the appraisal estimate of US$255 million including physical and price contingency costs. Additional Financing of SDR 12.6 million (US$20.0 million equivalent) was approved by the Board on February 7, 2008. The Additional Financing was to help meet the Recipient's increased funding requirement caused by the impact of floods that hit the Recipient's territory in August 2007. 3.7 At appraisal, it was expected that IDA would finance 80.85% (US$206.2 million) with the Borrower financing the remaining 19.15% (US$48.8 million). At project closure about US$4.5 million of the total Credit, including the Additional Financing, remained undisbursed and was cancelled. This amount included US$236,408 that was refunded to IDA as a result of ineligible expenses due to contracting issues. The balance of US$4.5 million resulted from (i) depreciation of the Taka against the dollar; (ii) the decision not to rebid cancelled contracts in 2010; and (iii) changes in the SDR/dollar rate. There was no other external financing for the project. There have been no information on the extent of borrower contribution, including in the financing table in Annex 1 of ICR. 6 Compliance with the World Bank’s Policies 3.8 The project was classified as Category “A” for purposes of Environmental Assessment. The Environmental Impact Assessment found that the unmitigated effects of the project would include impacts on water quality, aquatic ecology, and management of soil and borrow pit areas, but the incidence level was unknown. 3.9 With regard to environmental risk mitigation, the ICR states that "overall monitoring and quality control of environmental management required much attention from LGED. The ICR found weaknesses in the documentation of EMP progress reports and monitoring results as well as the establishment of Environmental Management Information System, which made it difficult for actual data assessment” (ICR. Page 12 Para 4). 3.10 A Category A project requires the retention of independent environmental experts not affiliated with the project. This was not a highly risky project, so it is understood that an independent advisory panel was not involved. However, the ICR notes that the consulting firm that was hired did not manage the [environmental management] activities, so that this work became the direct responsibility of the implementing agency. This assessment also points to the costs, available in Annex A of the ICR, on Safeguards implementation. Against a projected 805 BDT, 125.08 BDT were spent, representing a shortfall of 85 percent of the appraised costs. Overall, IEG notes that ICRs require a clear statement of full compliance, and this is especially the case for ICRs of Category A projects. This statement was not made available to IEG in the ICR nor was evidence of full compliance made available through the comment period. 3.11 According to the PAD (page 21), in addition to Environmental Assessment (OP 4.01), three other safeguards policies were triggered: Cultural Property (OPN 11.03), Indigenous Peoples (OP 4.10) and Involuntary Resettlement (OP 4.12). 3.12 With regard to the social safeguards, specifically Involuntary Resettlement, there was a failure to properly compensate resettled beneficiaries (roughly 8 percent of the identified project affected persons were not compensated) owing to very weak capacity at the District and local level. Land Acquisition and resettlement was a specific component of the project. A total of 173.24 ha were acquired, affecting 15,735 landowners, yielding an average loss per landholder of 0.011 ha. Difficulties in implementing the RAPs (there were three phases of resettlement) are attributed to the high transaction costs associated with managing resettlement of small land holdings. In some cases, the travel time and costs of obtaining compensation outweighed the compensation package (which needed to be obtained at District Headquarters). This includes persons living abroad that would have had to travel home to obtain the compensation. In other cases, landowners lacked the requisite legal records needed to obtain compensation. 3.13 Yet, overall, evidence attests that the compensation process was slow and unwieldly. Sufficient training was not provided, and information, as a result, did not adequately flow between the PMU, to district to the local implementing level. With regard to cultural property, although the Project Appraisal Document notes that no designated cultural property will be impacted, but that a small number of minor community and religious properties are 7 close the civil works, and that all such properties will be carefully protected, there is no reference to the cultural property safeguard in the ICR. 3.14 Weaknesses are associated with the challenges associated with engaging multiple ministries. Acquisitions were carried out by involving, as the ordinance required, officials who belonged to different ministries and had no accountability whatsoever to the project authority. Valuation of acquired lands and other assets required participation of several Government agencies like the Public Works Department, Forestry, Fisheries, etc. 3.15 Indigenous People. An Indigenous Peoples Planning Framework consistent with the Bank’s OP 4.10 was prepared. Reportedly, the ICR indicates that the project caused no adverse impacts on indigenous peoples who were dispersed in small settlements in some of the project districts. 4. Achievement of the Objectives 4.1 The project had two development objectives. The project substantially provided rural communities with improved access to social services and economic opportunities, however it only modestly enhanced the capacity of relevant government institutions to better manage rural transport infrastructure. Evidence against the first objective was mainly obtained by IEG from the Socioeconomic Monitoring and Evaluation Report (SEME). For the second objective, the results indicators and interviews with project staff were used. Objective 1. Provide rural communities with improved access to social services and economic opportunities. Outputs 4.2 The project met or surpassed most of its physical targets.  Against a target of 1600km of UZRs, the project improved 1638 km of UZRs.  Against a target of 15,000 meters of missing bridges/culverts, the project constructed or repaired 15,965 meters of missing bridges/culverts to support the expansion of the passable rural road network;  Against a target of 150 rural markets and 45 river jetties, the project constructed or rehabilitated 123 rural markets and 32 river jetties.  Another metric aimed to generate 24,000 person years of employment. Against this aim, the civil works contracts generated 50,275 person-years of employment in the project area. There is no information available on wages, skills, or the number of persons employed, and no information on job quality. Women were provided opportunities in off- pavement maintenance and tree planting, but these opportunities are less lucrative than pavement maintenance. Destitute women were specifically targeted as part of a tree planting campaign to mitigate environmental risks. Tree planting was carried out along 8 905km of roads; these women were employed under Labor Contracting Societies and were also responsible for tree maintenance. Outcomes 4.3 This assessment points to the discrepancy between the PAD and the ICR concerning the indicator used in the results framework to measure project outcomes, with regard the international road roughness index. At appraisal, the project team identified a total of about 1,100 km of Upazila Roads for improvement under the project. These roads were selected from the list of feeder roads identified through a techno-economic prioritization study using detailed economic analysis. Traffic counts were taken on the roads identified for the first phase program, for example. It then set a target of achieving an IRI score of <4 as an indicator of road improvement. It points to the base case road condition, which varied from IRI 10 to 18. This indicator was not measured. Rather, the PMU used an indicator of an IRI score of <6. The ICR points out this discrepancy. IEG finds that there is no evidence in the project documentation that the indicator was ever formally revised 0 for example at mid- term. Interviews conducted for the assessment and the comments provided indicated that he Government of Bangladesh commonly uses a <6 score. However, the results frame should therefore have been modified accordingly during the implementation period. Comments indicate that the agreement was in place before 2005, and that this may have changed at a point thereafter (See Bank Performance Section). With a baseline of 3,800 km, the result reported against this metric is 0 km constructed to the <4 standard. Rather, against a baseline of 3800 km, the project reports constructing 6,800 km of rural roads to a standard of IRI<6. 4.4 The SEME provides evidence of economic outcomes for the project area as a whole, as compared to a control. It reports that average monthly income and expenditure rose by 73.5% and 55.8% respectively in RTIP project areas versus by 14.8% and 33.9% respectively in control areas. Agricultural and non-agricultural production increased in rural villages as a consequence of traffic improvement. The growth in the number of enterprises per road was higher in villages assisted by RTIP than in controls. 4.5 The Rehabilitation and improvement of markets is reported by the SEME to be associated with the stimulation of trade, (but because the metrics used to compare the project areas versus control for market improvement are different, it is unclear whether this impact is attributable to the project). The project documentation however points to significant risks with regard to market sustainability, and this in turn will have an impact on the ability of the markets to contribute to the economic aims envisioned by the PDO. Project documentation points to concerns about the management of the completed markets and the generally inadequate maintenance. Although Market Management Committees have been given legal status, they require more training, and more clarity on responsibilities, on safety operations, and maintenance. 4.6 The road maintenance and expansion also had an effect on access to finance. In RTIP project areas, the average number of financial institutions increased by 73% for informal institutions (NGOs offering access to finance) and by 8% for formal institutions (national banks, such as Sonali, Janata, Rupali, Pubali, etc.). For the control areas, the average number of informal financial agents (NGOs) increased by 48% and access to formal banking 9 remained the same. The number of new account holders in regular Bank branches rose by twice as much in RTIP project villages as compared to control villages. 4.7 The SEME also provides evidence of social outcomes for the project area as a whole, as compared to a control. Total school enrollment rose by 12.2% in project villages and decreased by 60% in control villages. The number of teachers present in government primary schools increased by 47% on average in project areas (road, including a 19.35% and 27.27% increase in male and female teacher presence respectively) as compared to an increase of 16% on average in control areas, including a change of are -12.50 % and +28 % in the male and female presence respectively. In RTIP project villages, the number of healthcare service recipients per month went up by 32% with a 35% growth in the number of female recipients as compared to a decrease by 20% in the number of healthcare service recipients per month in the control areas and a 3% increase in the number of female recipients. 4.8 But the SEME does not measure distributional impacts. An impact evaluation commissioned of the predecessor rural road projects conducted distributional analysis of per capita expenditures, non-landed assets and agricultural production, where households were divided into four quartiles according to their initial position in the distribution of the outcome of interest. The analysis found that the poorest quartile of households did not share in the benefits of rural roads. Households in the second poorest quartile (between the 25th and 50th percentile) saw positive gains in food expenditure per capita. Significant improvements in per capita expenditure and assets occurred primarily between the 50th and 75th percentiles of the distribution. 4.9 It also does not report on the attenuation effects of the investment over time. The impact assessment of the predecessor project found that, when comparing the impact of the project on households that received the project between 2001 and 2005 and those that received it over a longer period between 1997 and 2005, the effects on per capita expenditure, schooling and transport costs were attenuated for those households that had received the project from the beginning. The employment effects on non-agricultural wage work were also stronger in the long-term. The employment results reflect greater access to non-farm opportunities and reduced dependence on agricultural wage work, although these effects emerge only over the long term. The authors suggest that there is a feedback effect between off-farm work and rural road development, where road improvements foster markets that become increasingly diversified across sectors. The attenuation of the effects on schooling could be explained by the lack of maintenance in roads, a problem that the Bangladesh Systematic Country Diagnostic refers to as a mindset of “build, neglect, rebuild.” 4.10 As the project was under implementation for almost a decade, it is unclear why the program did not implement a similar assessment of the distributional impacts of the RTIP program, so that more light could be shed on who benefited, how, and why. 10 Objective 2. Enhance Capacity of Relevant Government Institutions to Better Manage Rural Transport Infrastructure 4.11 The project implemented two types of capacity building activities, that included project finance valued at US$ 6 million. It financed an:  Institutional Strengthening Action Program (ISAP) for LGED that included a focus on strategy, organizational development, financial management and audit, road maintenance and asset management, rural transport safety, and environmental and social risk management. The various ISAP targets were aimed at enhancing the key elements of LGED's policy framework, planning and operational capabilities.  Local Government Improvement Program (LGIP). The program was designed to help building local government capacity for infrastructure management and related revenue mobilization, with the aim of influencing wider reforms in the rural road sector. It included basic training for Union Parishads in preparation of investment schemes, for conducting public consultations, for records maintenance etc. This program was piloted in 5 districts, and after evaluation, rolled out to the rest of 21 districts. After the basic training was completed, a second phase of Intensive Strengthening was done on a needed basis, as a hands-on training for Union Parishad management. It was planned as a pilot, by selecting one UP in each district with the aim of and then rolling it out. Outputs 4.12 Output targets against this objective were only partially met. These included:  The implementation of an LGED-wide Institutional Strengthening Action Plan (ISAP).  Twenty-three training courses, workshops and seminars that trained a total of 12,859 participants, including 48 LGED engineers who received overseas training.  The implementation of the LGIP program. Basic Training activities were completed, but Intensive Strengthening activities were only partly implemented. A total of 20,324 people from the Union Parishads received training through the provision of 276 grants. However, in 2008-2009, these trainings were seen as less desirable with the arrival of another World Bank project – the Local Government Support Program - that offered training without the RTIP requirement of a 20 percent matching grant. Training was also interrupted by the UP election cycle that diverted attention away from institutional strengthening and associated reforms. A planned resource mobilization study was dropped, as this too was foreseen to be conducted by the Bank’s concurrent Local Government Support Program. Outcomes 4.13 There are several claims made in the RTIP project documentation that are unsubstantiated by evidence. These claims include the assertion that institutional reforms at LGED transformed it into an “agency with vision and strategy” and a “focus on continued 11 modernization”. Documentation also states that the LGIP program built local government capacity - in budgeting, planning and project management – to enable them to better participate with LGED in prioritization of investments. Other statements are limited to outputs. The project reports that the maintenance policy developed under the project was adopted by LGED, for example, and that this alone is a significant milestone. Other claims are problematic since they differ widely from the project evidence. For example, statements about environmental strengthening - facilitated by the project – that led to mainstreaming environmental management into regular operations of LGED. (See section below on Compliance with the World Bank’s Operational Policies). 4.14 The project utilized several metrics to measure the outcomes of the capacity building efforts financed by the project. First, it sought to increase operating efficiency (at LGED) as measured by the annual operating costs as a percentage of the total annual budget. The project helped the LGED to reduce the value of its annual operating costs from 11.7 percent of its operating budget to 4.9 percent. IEG’s mission learned that, as a result of a recommendation of an Institutional Strengthening Action Plan, these costs were adjusted to 6.7 percent of operating costs at the time of the mission. The increased costs reflect a recommendation to increase the diversity of skills and to create more mid-level positions. 4.15 IEG’s assessment finds this metric to have been ill-designed, especially with regard to the many shortfalls that have been observed in managing environment and social risks, contractual management, and the ability to map, monitor, report and respond to issues pertaining to road quality and rural infrastructure maintenance (including markets). A reduction in operating costs during the project period, and the reported increase in efficiency, should have been accompanied by evidence that the ability of the agency to manage and oversee contracts had been enhanced. The project period was plagued by contractual issues, fiduciary management issues, and an inability to manage the environmental and social issues that embody rural road planning and design. 4.16 A second metric associated with the LGIP that was utilized was resource mobilization. The project did not meet its target of a 30% increase in revenue mobilization by participating Local Government Institutions. By project end, the project reported that resource mobilization was at the same level that it had been a decade ago (compared to a baseline of 200,000 taka in 2003, the project reported resources to be 200,000 in 2013). According to project documentation, this end target could not be achieved due to unresolved national level policy issues. 4.17 One of the explanations provided for why the LGIP trainings – and the associated capacity results- were only partially achieved was the existence of another parallel World Bank project, the Local Government Support Project (2006-2011). IEG reviewed the results of this project that aimed to "to develop an accountable system of local governance, capable to provide basic services that meet community priorities, supported by a predictable and transparent fiscal transfer system." A desk review of this project suggests that this project was not a substitute for the RTIP capacity building aims, with regard specifically to the rural road and rural infrastructure constraints that the Ups face. Rather, the two different projects seemed to engender an unhealthy competition between not altogether complementary aims. Further, the parallel project offered only basic training, not intense support to Local 12 Governments. The ICRR, for example, reports that basic training was provided through a 5- day module to nearly 50,000 local public officials in planning, budgeting, public financial management, safeguards compliance, and good governance. It also notes that the design of the training program lacked sufficient ownership from the government, that the approach proved to be too top-down and that it hindered the institutional development of the UPs. Gender Integration 4.18 Gender analysis is one part of IEG’s project performance assessment methodology: this includes an analysis of gender consideration at design, implementation and a review of gender related impacts after project close. The gender analysis is conducted to provide an additional learning lens about what works and what does not work with regard to gender integration across the Bank’s interventions, as well as to help with the assessment of distributional impacts. 4.19 A review of the project documents indicates that while the project objective was not gender specific, the aim of improving rural connectivity in project areas was relevantly in line with both male and female accessibility needs. There were no gender specific indicators, however it is reasonable to assume that improving rural roads would lead to a reduction in travel time and transport costs for women. Gender related results were reported in the Socio- Economic Monitoring and Evaluation (SEME). According to the SEME, in project villages, the number of health care service recipients increased by 32%, with a 35% growth in the number of female recipients; in control villages, the corresponding increases were 20% and 3%. Total school enrolment rose by 12.2% in project villages and decreased by 60% in control villages. However, gender disaggregated data was not collected for productive investments, such as income earned from increase access to rural markets, the river jetties, or the women’s corner. 4.20 Women were provided opportunities in off-pavement maintenance and tree planting, but these opportunities are less lucrative than pavement maintenance. This was recognized by the project and efforts were to be made in the second phase to address this. Destitute women were specifically targeted as part of a tree planting campaign to mitigate environmental risks. Tree planting was carried out along 905km of roads; these women were employed under Labor Contracting Societies and were also responsible for tree maintenance. 5. Efficiency 5.1 Efficiency is rated Substantial. The project at appraisal conducted a cost benefit analysis for components with quantifiable benefits. These included three of the eight components: i) improvement of about 1,000 kms of roads; ii) first year periodic maintenance of about 1,500 roads; and iii) improvement and construction of about 150 rural markets. At appraisal, the three components represented about 72% of the project costs. The ex-ante economic rate of return (ERR) was 20.5% for the road improvement; 52% return for the road maintenance and rehabilitation component; and 28.9% for the rural market improvement component. 13 5.2 The ex-post economic analysis estimated an ERR for the road improvement component of 19.2% and for the rural markets component of 35.2%. These costs represented about 75% of the project cost at closure. For reasons which are unclear, the ERR for the maintenance component was not calculated at closure, but it would almost certainly have had a positive NPV, as the ICR points out. The ICR would have added value if it had calculated the ERR for the maintenance, which had the highest return of the components that were assessed. As discussed in the efficacy section (and further in the risk to development outcome section below), the rate of returns estimated for the rural markets appears high since at least 35 are reported not to have been built, and since according to the ICR and interviews conducted for this assessment, their sustainability is questionable (in terms of management). 5.3 Though the project has been rated substantial, it is important to mention that there were delays and cost escalations that were encountered in the project. Due to the delays, the project was given two extensions until June 30, 2011 and June 30, 2012. Regarding the cost escalation and delays, the Mid-Term Review of the project which took place in December 2005 identified the key issues related to delays in implementation, and after follow up discussions, the LGED moved to start cancelling the contracts for non-performance, which led to cost escalation. In total 73 contracts had to be cancelled. Since the additional financing resulted in extended closing date, most of the cancelled contracts were rebid and completed satisfactorily. Although delayed and with some cost escalation, LGED’s prompt contractual decisions helped in the completion of about 70 contracts. 6. Ratings Outcome 6.1 Overall Outcome of the project is rated Moderately Satisfactory. Relevance of objectives and design are rated Substantial. The project substantially achieved its objectives of improving access to social and economic activities, however environmental and social risks associated with this category “A” project were note effectively mitigated. The project only modestly achieved its objective of enhancing the capacity of relevant government institutions to better manage rural transport infrastructure. Efficiency is rated Substantial, although road quality and the management challenges associated with the markets raises some concern about the higher than anticipated rates of return, especially for the market infrastructure. Risk to Development Outcome 6.2 The risk to Development Outcome Rating is High because of the following outlined risks: a) Political risk: The Government remains committed to the project’s objectives and concept. A number of initiatives started under the project are expected to continue and be further developed under the proposed second Rural Transport Improvement Project. Commitment is high at the district level. 14 b) Institutional risk: Key areas of institutional risk include perceived weak commitment by the Central Government to decentralize management and development of rural infrastructure and to implement planned institutional strengthening measures for LGED, as well as interference by local politicians and businessmen with the leasing of local markets and river jetties. These institutional risks will require further commitment from the Government to continue to fund adequately LGED and its institutional strengthening program. Other risks include the maintenance and management of the rural markets constructed. c) Technical risk: The chief concern here remains the quality of the roads built and the adequacy of the road maintenance effort. Although the maintenance allocation increased significantly under the project, the annual amount actually required for routine and periodic maintenance does not appear to have been systematically determined. While the quality of completed works was considered acceptable, a major challenge is posed by the increased volume of traffic on rehabilitated roads, especially of overloaded trucks. Overloading is a national problem on the Bangladeshi road network. Legislation to address the issue is under preparation. 7. Bank Performance Quality at Entry 7.1 The Quality at Entry Rating is Moderately Satisfactory. The project built on lessons learned from previous IDA funded operations including two Rural Road and Markets Improvement and Maintenance projects. While several upfront actions were taken, for example on procurement, to ensure implementation readiness, the results framework and associated indicators fell short of being able to measure the project development objectives related to improved access to social services and economic opportunities. However, the utilization of an external assessment to measure related impacts is commendable. There is also a lack of clarity in the program documentation on language used to determine whether a road would be in “good and fair condition”; since baseline data was not collected it was somewhat unclear as to how this would be measured. Quality of Supervision 7.2 The Quality of Supervision rating is Moderately Satisfactory. Formal supervision missions were carried out twice a year. The project’s physical targets were met and the team was proactive with respect to contractual issues and delays in financial and audit reporting, as pointed out by the Mid-Term review and the Operational Risk Assessment study. 7.3 There were three Task Team Leaders over the life of the project. Interviews conducted during the comment period for this assessment revealed that there was some discontinuity with regard to the way that the project was planned, the way that it was implemented, and the way that results were measured. Amongst these issues is inconsistency around the metrics used to measure the road standards. At design, and up through 2005, there was clarity with regard to the metric utilized. After extensive consultations with engineers, and using benchmarks set for the region, the team applied an International Road Roughness 15 Index measure of <4. This measure was agreed to by the Government of Bangladesh at the time of the project design and according to the comments received by IEG, any changes that were made were made after 2005. 7.4 example, Supervision was not effective however in helping the counterpart to effectively manage the environmental and social risks associated with road construction. Supervision also did not clearly articulate the specific aims and measurable targets associated with the additional financing made available for flood rehabilitation. More attention should also been paid to resolving the discrepancy around the metrics for road quality. Also, while it was appropriate to package a number of civil works contracts for International Competitive Bidding (ICB), given both the nature of rural roads (size and locations) and the difficult conditions in rural Bangladesh, it was overly optimistic to envision ICB as an effective tool to gain efficiencies for rural road construction and rehabilitation. 7.5 Overall Bank performance is rated Moderately Satisfactory, in line with the IEG ratings criteria. 8. Borrower Performance Government Performance 8.1 Government performance is rated Moderately Satisfactory. Government commitment during preparation and throughout implementation was demonstrated by the provision of timely counterpart funding and an increase in the budgetary allocation for rural road maintenance. Government also took steps to provide LGED with greater autonomy and tools to strengthen its institutional capacity, although there is less demonstrated commitment towards decentralized management of rural infrastructure, including the need for enhanced resource mobilization to meet local repair and construction needs. The Government of Bangladesh, through the multiple agencies responsible for ensuring operational policy compliance, should have ensured the effective implementation of the environmental and social risk mitigation measures associated with the road program. Evidence points to inadequate treatment of all of the social and environmental risks that were triggered by the project at appraisal. Implementation Agency Performance 8.2 The implementation Agency performance rating is rated as Moderately Satisfactory. The implementing agency was the Local Government Engineering Department (LGED). A Project Management Unit was housed at LGED headquarters. The 21 district offices were responsible for implementation of project work in the respective districts including supervision, measurement and payment. Supervision of the civil works was done jointly by the LGED district office and Upazila engineers and by two appointed Design and Supervision Management consulting (DSM) firms. Yet, due to the limited capacity of the DSM in the districts, most of the project supervision was carried out by LGED. This is reflected in the adjust component costs, which show that oversight and management of the project cost 145 percent more than had been anticipated at appraisal. While LGED managed a large volume of small contracts in a timely and transparent way, there were a number of 16 irregularities related to mobilization of payments for contracts that were subsequently canceled due to contractor performance issues. There were also some delays in financial reporting which were addressed during supervision missions. Financial Management and Procurement 8.3 Financial Management. During the early stages of implementation, financial management was weak. This was attributable to the heavily decentralized financial structure of the project, which included mobilizing adequate accounting resources from many field offices and transmitting and consolidating the information at headquarters. However, a new Unified Financial Management System was developed for LGED. A subsequent review of the earlier financial issues indicates that most were resolved and that the new system was generating timely reports by the closing stages of the project. Nevertheless, throughout implementation, LGED had difficulty resolving material issues raised in the audit reports. These issues were related to obtaining timely financial data from sub-offices in rural parts of Bangladesh. The ICR reports (page 11) that “substantial progress” in resolving these issues was made during appraisal of the second Rural Transport Improvement project, and that “all audit objections on the financial statements [of the project under review] until 30 June 2011, that were material to IDA have been satisfactorily resolved.” 8.4 Procurement. Overall, procurement was carried out in accordance with Bank guidelines. Initially, there were a number of procurement issues including award of contracts beyond the initial period of bid validity. There were also some problems associated with two ICB contracts, which subsequently had to be cancelled due to nonperformance. Cancelling these contracts was not related to mismanagement but rather to the contractor’s inability to mobilize in multiple construction sites. There were no reported cases of mis-procurement. 8.5 Overall Borrower Performance is rated Moderately Satisfactory. 9. Monitoring and Evaluation 9.1 Monitoring and Evaluation Design, Implementation and Utilization are rated as Modest. 9.2 M&E Design: The monitoring and evaluation framework, as presented in the project appraisal document, includes mainly output indicators, associated with the components. It does not include adequate PDO indicators. Two indicators can be used as proxies: reduced journey time and reduced travel costs, but these still fall short of providing adequate evidence against the project development objectives. Indicators linked to the capacity building objective also fall short: they include the completion and implementation of an institutional strengthening plan, number of persons trained etc. Another shortcoming of M&E design was the lack of an adequate M&E plan, including training and allocation of roles and responsibilities between the different implementing layers of the project. While the responsibility for M&E was stated to reside with LGED, critically needed data collection activities lied outside of the main management information system – such as data needed to report against the PDO, or against the “do no harm” aspects of this project as reflected by the fact that the safeguard objectives are embedded as a component of the project. 17 9.3 M&E Implementation: LGED assumed responsibility for monitoring the outputs for rural road construction, rehabilitation, and maintenance. LGED was supported by an internationally experienced Design, Supervision and Monitoring (DSM) Consultant. However, due to the Consultant’s weak performance, much of the quality control of the civil works was carried out by LGED's Quality Unit. The socio-economic monitoring and evaluation system was used to collect data on the impact of the project on beneficiaries. Indicators for the flood rehabilitation activities (supported by the Additional Financing) were never added to the results agreement of the project and were not covered by the monitoring activities. 9.4 M&E Utilization: There is no evidence that the M&E system was used during implementation of the project, but the methodology developed for the Socioeconomic Monitoring and Evaluation system has been integrated into LGED. 10. Lessons  Human capacity development is as important as technical training in projects promoting rural decentralized infrastructure service delivery. In the RTIP, while physical targets were met or exceeded, more effective and in-depth training may have helped to smooth project implementation and to build longer term capacity within the Government, including for environmental management. A decision to rely on a parallel Bank financed project for planned training did not generate the results anticipated, as confirmed by IEG’s review of both this and the parallel project. Training in the parallel project was found to have been inadequate.  Civil works contracts should include a transparently negotiated, price escalation provision, even during short implementation periods, but ensuring a more realistic estimate would be important, too. In the case of RTIP, the price of construction materials rose sharply during the short implementation period (eighteen months) which resulted in many contractors abandoning the works.  Timing and sequencing are critical considerations when awarding civil works contracts. In the case of RTIP, the implementing agency awarded civil works contracts at the beginning of the rainy season or late in the dry season, resulting in non-activity of contractors for half of the year and delays in contract implementation. All contracts which were awarded in this way took an extra year for completion. On the other hand, contracts awarded at the end of rainy season had two full dry seasons to be completed. 18 References World Bank. Rural Transport Improvement Project. Project Appraisal Document, May 2003 World Bank. Rural Transport Improvement Project. Credit Agreement, June 2003 World Bank. Rural Transport Improvement Project. Project Paper, December 2007 World Bank. Rural Transport Improvement Project. Financing Agreement, February 2008 World Bank. Rural Transport Improvement Project. Aide-Memoires (2002 – 2012) World Bank. Rural Transport Improvement Project. Back-to-office reports and letters to Government (2002 – 2012) World Bank. Rural Transport Improvement Project. Country Assistance Strategy (2011-2014), September 2010 World Bank. Rural Transport Improvement Project. Implementation Status Reports World Bank. Rural Transport Improvement Project. Borrower’s Completion Report, September 2012 Socio-Economic Monitoring and Evaluation Report of RTIP (Final), June 2010 Assessment of Progressive Impacts of Institutional Strengthening Strategy, October 2012 Final Report on the “TA Services to support implementation of ISAP of LGED” under RTIP. June 2008 LGED Annual Report 2010-2011 19 Appendix A. Basic Data Sheet Rural Transport Improvement Project (credit IDA-37910, IDA-37911) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 250 — — Credit amount 204 204 100 Additional Financing — 20.0 — Cancellation 4.5 4.5 100 Note: — = not available. Cumulative Estimated and Actual Disbursements 2004 2005 2006 2007 2008 2009 Appraisal estimate (US$M) 18.20 57.40 108.10 155.60 181.00 190.00 Actual (US$M) 11.56 31.70 75.32 104.77 128.29 162.60 Actual as % of appraisal 63.51 55.23 69.67 67.33 70.87 85.26 Date of final disbursement: 07/08/2012 Project Dates Original Actual Initiating memorandum 5/15/2001 7/30/2003 Negotiations 5/5/2003 5/12/2003 Board approval 7/15/2002 6/19/2003 Signing 6/30/2003 6/30/2003 Effectiveness 7/30/2003 6/30/2003 Additional Financing 2/7/2008 6/30/2012 Closing date 6/30/2009 6/30/2012 APPENDIX A 20 Staff Time and Cost Stage of Project Cycle USD Thousands (including travel and consultant costs) Lending FY01 19.17 FY02 83.77 FY03 242.14 FY04 0.28 Total: 345.36 Supervision/ICR FY01 0.00 FY02 0.00 FY03 0.00 FY04 95.81 FY05 98.50 FY06 117.84 FY07 69.42 FY08 118.35 FY09 90.82 FY10 153.94 FY11 139.98 FY12 42.22 FY13 49.00 21 APPENDIX A Task Team Members Names Title Unit Responsibility/ Specialty Lending Fabio Galli Lead Transport Specialist AFTTR TTL J. Channe Highway Engineer A. Bansal Transport Planner K.M. Maqsoodul Mannan Consultant SARPS Reidar Kvam Manager CESPQ Mohammad Sayeed Consultant SASSP Ishtiaque Ahmed Transport Specialist SASDT Supervision/ICR Burhanuddin Ahmed Sr Financial Management Specialist SARFM Teen Kari Barua Consultant, Social SASHN Aminur Rahman Chowdhury Consultant SARFM Shakil Ahmed Ferdausi Senior Environmental Specialist SASDI Jean-Noel Guillossou Program Manager AFTTR TTL David C. Hanrahan Consultant SASDI Shamsul M. Hoque Temporary SASFP Md. Tafazzal Hossain Program Assistant SASDO Consultant, Institutional Ernst-August Huning SASDT Strengthening Marghoob Bin Hussein Senior Procurement Specialist SARPS Zafrul Islam Lead Procurement Specialist SARPS Gaurav D. Joshi Environmental Specialist SASDI Syed Muhammad Latif Consultant SASDI Tapas Paul Senior Environmental Specialist SASDI Mohi Uz Zaman Quazi Consultant SASDA Co-TTL Reefat Sultana Infrastructure Specialist SASDT TTL Ismat Sultana Program Assistant SACBD Suraiya Zannath Sr Financial Management Specialist SARFM Fernanda Ruiz Nunez Economist SASDT Elena Y. Chesheva Operations Officer SASDT ICR TTL Debbie Wei Mullin Junior Professional Associate SASDT 22 Appendix B. List of Persons Met Name Designation Mohammad Nazrul Islam Transport Specialist, Asian Development Bank Akhtar Zaman Senior Social Development Consultant Partha Das Gupta Project Officer (Urban Transit Systems) Bangladesh Residence Mission Momin Mozibul Haque Shamaji Project Director, Bangladesh Trade and Transport Facilitation Services Project Mostafa Kamal Project Director, RTIP II Md. Abdul Quader Former Project Director, RTIP Dewan Abdus Sabur Senior Assistant Engineer, Road Safety and Transport Planning, RTIP II. LGED Javed Bari Senior Technical Consultant, RTIP II Mr. Rabiul Islam Senior Assistant Engineer, LGED, Sirajganj Mr. Mahfuzur Rahman Executive Engineer, LGED, Sirajganj Mr. Ismail Hossain Sub Assistant Engineer, Tarash Upazila, Sirajganj Md Abdur Razzak Chair, Market Committee Muhammad Abdurrazzak Sub Assistant Engineer, Tarash Upazila, Sirajganj Md Nazmul Haque Surveyor, Tarash Upazila, Sirajganj Babul Akhtar Upazila Engineer, LGED Md Ahsan Habib Sub Assistant Engineer, Raiganj, Sirajganj Ehsanul Hoque Sub Assistant Engineer, Raiganj, Sirajganj Md. Sajjad Hossain Assistant Engineer, Sirajganj Muhammad Samiul Assistant Engineer, RTIP, Sirajganj Moshiul Alam Sub Asssistant Engineer, Sirajganj Md. Shakhawat Hossain Field Engineer, TRIP, Sirajganj Farhad Ahmed Senior Transport Specialist TK Barua Consultant, World Bank Bakhtiar Shohag Consultant, World Bank Dr. Khorshed Alam Consultant, World Bank 23 Appendix C. Borrower Comments No comments were received from the Borrower.