ICRR 14180 Report Number : ICRR14180 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/29/2014 Country : Bangladesh Project ID : P071435 Appraisal Actual Project Name : Rural Transport US$M ): Project Costs (US$M): US$250 million Not available in Improvement Project US$ L/C Number : C3791 Loan /Credit (US$M): Loan/ US$M ): US$204 million US$204 million Sector Board : Transport US$M ): Cofinancing (US$M): 0 0 Cofinanciers : Board Approval Date : 06/19/2003 Closing Date : 06/30/2009 06/30/2012 Sector (s): Roads and highways (80%); Agro-industry marketing and trade (20%) Theme (s): Rural services and infrastructure (40% - P); Rural policies and institutions (40% - P); Rural markets (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : William B. Herbert Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: According to the Project Appraisal Document (PAD, page 2), and the Development Credit Agreement dated June 30, 2003 (DCA, page 13) the project's objectives were "to provide rural communities with improved access to social services and economic opportunities, and to enhance the capacity of the Borrower's agencies and institutions to better manage rural transport infrastructure ". Additional Financing of SDR 12.6 million (US$20.0 million equivalent) was approved by the Board on February 7, 2008. The Additional Financing was to help meet the Recipient's increased funding requirement caused by the impact of floods that hit the Recipient's territory in August 2007. The project objectives were not changed . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project contained eight components (while the ICR provides the appraisal costs by component in US dollars, it provides the actual costs only in Bangladesh Taka . It is not, therefore, possible to compare the appraisal estimates to the actual costs. 1. Improvement of about 1,100 km of Upazila roads (former feeder roads known by their acronym UZR ), (US$110.50 million at appraisal). This component was to improve rural access and connectivity to markets and production centers in the 21 project Districts by making key UZR passable all year around and by improving their riding quality and carrying capacity. 2. Improvement of about 500 km of Union roads (former rural roads) (U$23.50 million at appraisal). This component was to improve rural access and connectivity in the 21 project Districts by making key Union roads passable all year around and by improving their riding quality . 3. Periodic maintenance/rehabilitation of about 1,500 km Upazila roads. (US$39.0 million at appraisal). This component was to help reduce the periodic maintenance /rehabilitation backlog on the Local Government Engineering Department (LGED) managed portion of the road network by funding the overlaying, resealing and minor rehabilitation of 1,500 km of key UZRs in the 21 project districts. 4. Construction of about 15,000 m of culverts/bridges on Union Roads (US$30.50 at appraisal). This component was to make approximately 5,000 km of URs passable year-around in the 21 project districts, and to provide appropriate drainage structure on these roads . 5. Improvement/construction of about 150 rural markets and 45 river jetties. (US$17.50 million at appraisal). This component was to help lower the cost of goods and commodities in the 21 project districts by facilitating their trade . 6. Land acquisition, implementation of Resettlement Framework, Environmental Management Framework, Resettlement Action Plan, Environmental Management Plan, Indigenous People's Development Plan, utility relocation (US$14.00 million at appraisal). This component was designed to assess and mitigate the social and environmental impacts of implementing the civil works components of the project . Land acquisition and cash compensation was to be funded entirely by the Government of Bangladesh . 7. Design Supervision Monitoring consultant services, quality, financial and procurement audit services, and other consultant services. (US$144.00 million at appraisal). This component was designed to ensure the timely, quality, and cost effective construction, maintenance /rehabilitation of the civil works components under the project . 8. Technical Assistance, training, capacity building of Local Government Engineering Department (LGED) and Local Government Institutions (LGIs), equipment and pre-investment studies. (US$6.00 million at appraisal). This component was to finance a wide variety of activities and investments including enhanced training programs of LGED staff, LGED implementation of improved/new business processes and systems, rural government capacity building initiatives and rural road transport safety . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : Annex 1 of the ICR (page 21) shows actual costs of the project only in Taka . The Taka amount indicates a 26.5% decrease from the appraisal estimate . However, this contradicts earlier statements in the ICR which imply only a $4.5 million savings; less than 2% of the total cost. Although the absence of actual costs in US dollars was raised with the project team, it is still not possible to comment on actual costs of the project and compare these with appraisal estimates. Financing : At appraisal, it was expected IDA would finance 80.85% (US$206.2 million) with the Borrower financing the remaining 19.15% (US$48.8 million). At project closure about US$4.5 million of the total Credit, including the Additional Financing, remained undisbursed and was canceled . This amount included US$236,408 that was refunded to IDA as a result of ineligible expenses due to contracting issues . The balance of US$4.5 million resulted from (i) depreciation of the Taka against the dollar; (ii) the decision not to rebid cancelled contracts in 2010; and (iii) changes in the SDR/dollar rate. There was no other external financing for the project . Borrower Contribution : While the ICR states there was "strong commitment to the project from Government and there were no issues with counterpart funding", the ICR does not indicate the actual Borrower contribution (there is no financing table in Annex 1). Dates : The original closing date was June 30, 2009. At the time of approval of the Additional Financing, the closing date was extended to June 30, 2011 to allow completion of the originally planned civil works and to meet needs arising from the 2007 Flood. A further extension of one year to June 30, 2012, was subsequently granted to allow completion of flood rehabilitation contracts which experienced delays in implementation . 3. Relevance of Objectives & Design: a. Relevance of Objectives: High. High . The overall project development objectives are relevant to the July 2010 Country Assistance Strategy (CAS) covering the fiscal years 2011-2014, one of the goals of which is the maintenance and extension of transport networks, and within this, the rehabilitation of rural transport to be supported by a follow -on project to that under review. The objectives were also consistent with the December 12, 2000 CAS, which set out primary goals for rural development including increased opportunities and assets available to the poor; improved rural infrastructure services; and strengthened local institutions . The objectives are relevant the Government's Poverty Reduction Strategy which aims to reduce the incidence of rural poverty by improving the infrastructure of the rural economy . b. Relevance of Design: Substantial . The project design has an appropriate balance between the physical works for improving rural roads to enhance access to markets and production centers; rehabilitation and maintenance of roads; improvement of rural markets; and technical assistance and capacity building . It is reasonable to assume that improving rural roads would lead to a reduction in travel time and transport costs . Although the results framework does not specify the links between these intermediate outcomes and the project development objectives (improved access to social services and economic opportunities), the main text of the PAD discusses the expected impact of enhanced road infrastructure on agricultural and off -farm activities, household employment, access to markets, and to social service centers (clinics, schools, etc.). Design was also built on the experience of two previous road improvement projects in Bangladesh, which had demonstrated the impact of project interventions on access to social services and economic opportunities. With regard to the second development objective, it is reasonable to assume that the technical assistance and capacity -building activities provided under the project would enhance the ability of the Local Government Engineering Department and other local and national authorities better to manage rural transport infrastructure. 4. Achievement of Objectives (Efficacy): The project's objectives were to i ) provide rural communities with improved access to social services and economic opportunities, and ii) to enhance the capacity of the Borrower's agencies and institutions to better manage rural transport infrastructure. Improve Access to social services and economic opportunities . Substantial . Outputs: Land Acquisition : The total amount of land acquired was 427.91 acres which affected a total of 15,735 landowners and 14,304 households. About 92% of all affected landowners were compensated . The ICR does not provide the total amount disbursed for compensation . Completion of Civil Works : All planned civil works under the original components were completed . 732 km of roads and 1.518 meters of bridges damaged in the 2007 flood were rehabilitated. Increase in Maintenance Funding : The 5% annual real increase in maintenance funding was surpassed, from TK 2 billion to TK 6.25 billion. Rural Road Improvements/Maintenance :Targets for road improvement, rehabilitation and maintenance were achieved. This included 1630 km of UZ and UP roads; 15,956 meters of bridges and culverts and 2120 km of road rehabilitated and maintained. Construction of Markets and River Jetties . Targets not fully achieved were the construction of Center Markets and River Jetties (there was 80% achievement). Intermediate outcomes According to the ICR, travel speeds for motorized traffic increased from a baseline of 12 kilometers per hour to 33.5 kilometers per hour. It is unclear how these estimates were made . The project team subsequently explained that origin-destination surveys were carried out to compare weighted average speeds at the baseline with those at project closure . The final report of the third and final round of data collection (which took place in 2008-2009) for a Socio-Economic Monitoring and Evaluation (piloted under a previous project ) calculated reduction in travel times at 58% for motorized vehicles during the dry season and 65% during the monsoon and for non -motorized vehicles 53% during the dry season and 61% during the monsoon, surpassing the target of 25 percent (ICR, page 14). It is unclear how these estimates were made . The project team explained that the estimates also relied on origin-destination surveys. According to the ICR Data Sheet (ICR, page vii) “costs� fell from 2.25 Taka per kilometer to 1.64 Taka per kilometer between 2003 and 2012, presumably as a result of the project . However, there is no indication of what these costs consist of or how the reductions were estimated . There was a target to increase by 20% the percentage of rural roads in good condition . However, it was not possible to measure this due to an inconsistency between the PAD and what was monitored during implementation. “The PAD identified [a] road in ‘good’ condition as one with [an] IRI [International Roughness Index] below 4, however, the LGED used the definition of road condition as good with IRI< 6� (ICR, page 8). The ICR contends that the LGED definition is the appropriate one for the kinds of roads improved under the project and that the PAD definition sets the bar at too high a level . Be that as it may, “this discrepancy in interpretation was never brought up� (ICR, page 8). Final Outcomes Neither the PAD nor the ICR contain any indicators which would measure the extent to which the project improved access to social services or economic opportunities . Evidence is, however, available from the Socio-Economic Monitoring and Evaluation (SEME) referred to above. According to the ICR (page 29), the SEME was carried out in three phases – 2003-04, 2006, and 2008-09. The sample consisted of 22 UZRs (it is impossible to know the percentage of the total represented by this sample, since it is expressed in number of roads, whereas project outputs for UZRs are in terms of kilometers ), eight growth center markets (6.5% of the 123 constructed or rehabilitated ), and four river jetties (12.5% of the 32 constructed or rehabilitated ). The survey compared results in areas benefiting from the project with those in control areas not so benefiting . The following findings are reported (ICR, pages 29 and 30): (a) a 500% increase in the number of passengers using motorized vehicles in project areas compared with an increase of 40% in control areas (as a consequence of road improvements favoring the use of motorized vehicles ); (b) lower passenger fares on project than on control roads (the reduction ranging from 8% on market days to 38% on non-market days); (c) freight charges on project roads increased from 24% on non-market days to 182%, compared to “a much higher 1,300%� in control areas; (d) agricultural and non-agricultural production “appears to have been stimulated in project villages …as a direct consequence of traffic improvement; � (e) agricultural yield increases of 195% were observed in project villages; (f) the growth in the number of enterprises per road was higher in the project than in the control villages; (g) there were 265 new permanent shops in project growth center markets, representing an increase of 134%; it is stated that “in all the project areas overall turnover of shops increased dramatically in comparison to the control areas;� (h) in project villages, the number of health care service recipients increased by 32%, with a 35% growth in the number of female recipients; in control villages, the corresponding increases were 20% and 3%; (i) total school enrollment rose by 12.2% in project villages and decreased by 60% in control villages; (j) the number of new bank account holders increased by twice as much in project as in control villages; and (k) average monthly income and expenditure rose by 73.5% and 55.8% respectively in project areas and by 14.8% and 33.9% respectively in control areas . Although the the latest survey data were from 2008-2009, whereas the project closed on June 30, 2012, information provided to IEG by the project team indicates that the majority of civil works were completed by the time the survey took place (for example, 94% of rehabilitation works, 91% of bridges and culverts, and 64% of the UZR and UR road improvements). Most of the project activity after 2009 was directed to flood relief work. This facilitates the attribution of many of the results outlines above to the project . The ICR did not fully address some methodological issues raised by the survey, such as the the unknown proportion of UZRs represented by the sample, the number of control areas or villages researched (also unknown), and the lack of a description of interview methods, and of details on how estimates of income and expenditure were arrived at . However, the full survey report, subsequently provided to IEG by the project team, contained adequate explanations of these issues. According to the ICR, 50,275 person-years of additional employment were created by the project, more than double the target of 24,000. However, nearly all of this was linked directly to employment under the civil works contracts. It is not clear how many, if any, positions are permanent . Enhance capacity to better manage rural transport infrastructure . Modest Outputs: Implementation of the LGED Institutional Strengthening Action Plan :About 98% of the plan was implemented and almost 13,000 people received training in strategic development, organizational development, financial management, quality assurance, maintenance and asset management, rural transport safety, and environmental mitigation management. Implementation of LGED's assistance to Local Government Institutions : About 95% of the plan was implemented and over 20,000 people received training. Development of a Road Safety Strategy and action plan : Three road safety pilots were implemented, however the development of a Rural Road Safety strategy was deferred to a future project to allow for scaling up the initiative to 26 districts. A Rural Road Maintenance Policy for Bangladesh was developed (the project team subsequently stated that the policy was approved by the Cabinet in January, 2013). According to the project team, environmental strengthening facilitated by the project led to mainstreaming environmental management into the regular operations of the LGED . An Operational Risk Assessment was carried out for the LGED in 2007-2009. Outcomes: The proportion of LEGD’s annual budget spent on operating costs . fell from 13.7% in 2003 to 4.9% at closure, which surpassed the target of 11.7%. However, this can be partly attributed to the doubling of the maintenance allocation from Taka 3.2 billion at appraisal to Taka 6.25 billion in FY 2011-12. This increased the total size of the budget and hence reduced the percentage represented by LGED's operating costs . According to the ICR (page 15), LEGD was transformed into an agency with a “vision and strategy and a focus on continued modernization.� Nevertheless, a number of challenges still remained at closure . Establishing the Unified Financial Management System in LGED was not fully achieved . This resulted in delays in financial management reporting and timely audits . Likewise, the establishment of the Road Safety Cell within LGED was delayed. There was also limited success in building the Local Government Institutions (LGIs) capacity for infrastructure management and related revenue mobilization . The target of a 30% increase in LGI revenues could not be achieved (according to the ICR Data Sheet, page vii, the actual increase in revenues over the life of the project was zero) due to the weak capacity of the LGIs and inadequate commitment by Government to decentralize local infrastructure management responsibilities . 5. Efficiency: The PAD (pages 34-39) includes a cost benefit analysis for components with quantifiable benefits . These included three of the eight components : i) improvement of about 1,000 kms of roads; ii) first year periodic maintenance of about 1,500 roads; and iii) improvement and construction of about 150 rural markets. The three components represented about 72% of the project costs estimated at appraisal . The ex ante economic rate of return (ERR) was 20.5% for the road improvement; 52% return for the road maintenance and rehabilitation component; and 28.9% for the rural market improvement component . The ex-post economic analysis estimated an ERR for the road improvement component of 19.2% and for the rural markets component of 35.2%. These costs represented about 75% of the project cost at closure . For reasons which are unclear, the ERR for the maintenance component was not calculated at closure, but it would almost certainly have had a positive NPV, as the ICR points out . . The project was implemented over the course of nine years, three years longer than anticipated at appraisal . This was largely due to the need to address flood rehabilitation work (supported by Additional Financing ), rather than to operational or administrative inefficiencies, of which relatively few are recorded in the ICR or in supervision reports . Efficiency is rated Substantial ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re-estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of objectives is rated high and that of design substantial . The project over-achieved a number of output and intermediate outcome targets, especially in road improvement The evidence presented in the ICR and subsequently by the project team indicates that the efficacy of the first of the two objectives – improved access to social services and economic opportunities -- was substantial. However, that of the second objective -- enhanced capacity to manage rural infrastructure -- is rated modest. With efficiency rated substantial, overall outcome is assessed as moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Political risk: The Government remains committed to the project ’s objectives and concept . A number of initiatives started under the project are expected to continue and be further developed under the proposed second Rural Transport Improvement Project. Commitment is high at the district level. Institutional risk: Key areas of institutional risk include perceived weak commitment by the Central Government to decentralize management and development of rural infrastructure and to implement planned institutional strengthening measures for LGED, as well as interference by local politicians and businessmen with the leasing of local markets and river jetties. These institutional risks will require further commitment from the Government to continue to fund adequately LGED and its institutional strengthening program . Technical risk: . The chief concern here remains the adequacy of the road maintenance effort . Although the maintenance allocation increased significantly under the project, the annual amount actually required for routine and periodic maintenance does not appear to have been systematically determined . While the quality of completed works was considered acceptable, a major challenge is posed by the increased volume of traffic on rehabilitated roads, especially of overloaded trucks . Overloading is a national problem on the Bangladeshi road network . Legislation to address the issue is under preparation . LGED is planning to procure weigh bridges under the follow -up Second Rural Transport Project, which will also support further development of maintenance capacity . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The project built on lessons learned from previous IDA funded operations including two Rural Road and Markets Improvement and Maintenance projects . These included the need to anticipate delays in procurement; delays in handing over construction sites to contractors; lack of economies of scale; inadequate preparation and management of the social and environmental aspects of civil works; and weak involvement of users and local bodies. Given these lessons, the project preparation focused on early preparation of engineering, social, environmental, and participatory planning . There were also up-front procurement actions to ensure implementation readiness; substantial completion of site readiness before civil works contracts were approved; targeted capacity building initiatives to increase the role of Local Government Institutions; and strengthening of LGED monitoring and evaluation processes to improve management and public accountability . However, there were a number of shortcomings . M&E design was weak (see Section 10 below). Baseline indicators for key aspects of the project were not identified until after effectiveness and only during the first supervision mission. Failure to agree on the definition of "good condition" at the time of appraisal and throughout the life of the project was an important oversight . While it was appropriate to package a number of civil works contracts for International Competitive Bidding (ICB), given both the nature of rural roads (size and locations) and the difficult conditions in rural Bangladesh, it was overly optimistic to envision ICB as an effective tool to gain efficiencies for rural road construction and rehabilitation . Most principal risks – political, institutional and technical – were correctly identified and appropriate mitigation measures were proposed. While the project had eight components and at first glance could be considered overly “complex� for a rural infrastructure project, the reality is that four of the eight components were specific “targets� and not components. The targets specified the number of kilometers for improvement, construction, and maintenance of Upazila and Union roads. The targets could have been presented in the PAD as sub -headings under a single infrastructure component. The three other components (land acquisition and safeguards, project management, and technical assistance) were essential to the success of the project and relevant to the development objectives . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Formal supervision missions were carried out twice a year . While there were three Task Team Leaders (TTLs) over the life of the project, continuity benefited from having the co -Task Team Leader based in the Dhaka office. This provided on going supervision /implementation support that was needed throughout the project . Reporting on supervision activities appears to have been timely and recommendations were followed up with actions. For example, the team was proactive with respect to procurement issues and delays in financial /audit reporting. However, there were some significant shortcomings . It is not clear from the evidence provided whether safeguard policies were fully complied with for this Category “A� project (see Section 11 below). The indicator for the flood rehabilitation component, while listed in the Project Paper for the Additional Financing, was never formally added to the results framework and was not part of the monitoring of results in the Implementation Status Reports . The discrepancy between the PAD and LGED road roughness indicators was not resolved . Advantage was not taken of the opportunity provided by the Additional Financing to revise the outcome indicators better to measure the attainment of the project development objectives as written . Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Government commitment during preparation and throughout implementation was demonstrated by the provision of timely counterpart funding and an increase in the budgetary allocation for rural road maintenance . Government also took steps to provide LGED with greater autonomy and tools to strengthen its institutional capacity, although some doubts remain about the strength of its commitment to decentralized management of rural infrastructure. While there were procurement-related issues and delays, these were eventually resolved . An area where Government did not provide sufficient leadership was related to the development of the growth center markets and river jetties, important components for developing rural economic opportunities . Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: The implementing agency was LGED. A Project Management Unit was housed at LGED headquarters . The 21 district offices were responsible for implementation of project work in the respective districts including supervision, measurement and payment . LGED has considerable experience working with donor funded projects and is considered one of the best performing agencies in Bangladesh . LGED was able to manage a large volume of small contracts in a timely and transparent way, although there were a number of procurement irregularities related to mobilization payments for contracts that were subsequently canceled due to contractor performance issues. All “ineligible� disbursements related to these contracts were refunded to IDA . (See Section 11 below). There were also some delays in financial reporting which were addressed during supervision missions. LGED was able to utilize funds for capacity building, including support to its Quality Unit which resulted in “overall good quality of construction � (page 6 of ICR). An independent ex post review found that LGED performed well with respect to the compensation program for land acquisition, although there were issues regarding the performance of other government agencies . The extent to which environmental safeguards were complied with is unclear (see Section 11 below). Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Annex 1 of the PAD, "Project Design Summary" (Page 26), outlines the hierarchy of objectives and the related key performance indicators, including output indicators for each component and sub -component. However, there were no indicators to measure the project development objectives (“improved access to social services and economic opportunities�), and the so-called outcome/impact indicators measured only intermediate outcomes such as reduced costs and journey times . The definition of “good condition� was not clarified either in the PAD or in the Development Credit Agreement. In addition, while the PAD addresses “critical assumptions� it does not address, in any detail, how M&E was to be carried out or who was responsible and accountable for the quality of the work . The project team subsequently stated that responsibility for M&E lay with LGED, while indicators were to be monitored by the Project Management Unit. Independent consultants were retained for independent evaluations such as SEME and the assessment of the resettlement program . b. M&E Implementation: LGED assumed responsibility for monitoring the outputs for rural road construction, rehabilitation, and maintenance. LGED was supported by an internationally experienced Design, Supervision and Monitoring (DSM) Consultant. However, due to the Consultant ’s weak performance, much of the quality control of the civil works was carried out by LGED's Quality Unit. The socio-economic monitoring and evaluation system was used to collect data on the impact of the project on beneficiaries (see Section 4 above). A number of other independent evaluations were commissioned for specific activities in project areas, including safeguard issues concerning resettlement and the assessment of implementation of the Local Government Improvement Program . Neither the PAD nor the ICR provide information on who was responsible for conducting these evaluations . Indicators for the flood rehabilitation activities (supported by the Additional Financing ) were never added to the results agreement of the project and were not covered by the monitoring activities . c. M&E Utilization: It is not clear from the ICR if the data generated from the M&E system was used during implementation of the project. However, the fact that the lack of precision in the definition of “good condition� was known but no action was taken to resolve the issue raises questions on how other M&E data were used . One positive note is that the methodology developed for the Socioeconomic Monitoring and Evaluation system has been mainstreamed in LGED . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was classified as Category “A� for purposes of Environmental Assessment . According to the PAD (page 21), in addition to Environmental Assessment (OP 4.01), three other safeguards policies were triggered : Cultural Property (OPN 11.03), Indigenous Peoples (OP 4.10) and Involuntary Resettlement (OP 4.12). Environment: The ICR reports that an Environmental Management Framework was developed . Environmental issues were mostly related to civil works, including the impact on water quality, aquatic ecology, and management of soil and borrow pit areas. The management of the safeguard was the responsibility of the Project Environmental Unit located within LGED, with support from the environment specialists from the Design, Supervise, and Management (DSM) firm hired under the project. The ICR (pages 11-12) reports that an environmental screening checklist was carried out for each sub -project to identify environmental impacts. According to the ICR, an Environmental Assessment was done for sub -projects with expected moderate environmental impacts . An Environmental Management Plan (EMP) was prepared for each sub-project and was included as specific line -items in the Bill of Quantities of the bidding document . An Environmental Code of Practice was prepared, which was also used as a guideline for the contractors . The ICR reports that the billboard introduced by the project on a pilot basis with the EMP and related budget at the sub-project site created significant awareness at community level of environmental management issues . According to the ICR, while management systems were put in place, the DSM consulting firm did not manage these activities, so that this work became the direct responsibility of the LGED . The ICR (page 12) "found weaknesses in the documentation of EMP progress reports and monitoring results as well as the establishment of Environmental Management Information System (EMIS), which made it difficult for actual data assessment ." Despite this being a Category “A� project, there is no clear statement in the ICR that OP 4.01 was complied with. On the basis of the evidence presented, it is not possible to conclude that there was full compliance . Cultural Property The PAD indicates that “no designated cultural property will be impacted …..a small number of minor community and religious properties are close the civil works ….and all such properties will be carefully protected .� There is no reference to the cultural property safeguard in the ICR . Land Acquisition and Resettlement : Land Acquisition and resettlement was a specific component of the project . The ICR (p. 12-13) reports that the project was implemented in three phases with three separate Resettlement Action Plans (RAPs). Several issues arose: Average land holdings were very small, thereby increasing transaction costs . There was a large number of narrow strips to be acquired along the existing road alignments, and hence a large number of affected landowners with losses of small amounts of land . 173.24 ha were acquired, affecting 15,735 landowners, yielding an average loss per landholder of 0.011 ha. According to the ICR (page 13), “in many cases the landowners were reluctant to claim compensations that were too small in relation to the travel time and costs involved in going to the DC [District Commissioner] offices at the District Headquarters . In many other cases affected landowners lived abroad and did never turn up to claim compensation from DCs .� The compensation process was slow and unwieldly . Compensation was paid in two parts : the entitlement under Bangladeshi law, administered by the Deputy District Commissioners, and the “top-up� (to make up for shortfalls relative to market values and other stipulated entitlements ). In the case of the former, “the extremely time-consuming and cumbersome acquisition process caused persistent delays ...in payment� (ICR, page 13). LGED reportedly handled the top-ups more expeditiously and 100% of payments were made. “Lack of legal ownership records was the main reason for the vast majority of unpaid cases, followed by those where the land was acquired from co -sharers who could not show up together to make their claim � (ICR, page 13). There was weak capacity in managing social safeguards, particularly at District and Upazila levels . The law required officials who had no accountability to the authorities in charge of the project to become involved in land acquisition processes. An independent ex-post review of the resettlement program found “LGED generally adhering to the [Resettlement Framework], however, it raised questions about the practices in [compensation under law] determination by the acquisition officials and other participating [Government] agencies, and whether the compensation paid to the squatters was enough compared to their losses � (ICR, page 13). The ICR states (page 13) that “the achievement of about 92% of payments to project-affected people was one of the highest so far in Bangladesh .� However, it is unclear if this is 92% of the amount of compensation due, or 92% of the amount of claims made. Does it include those without legal entitlement and those unable or unwilling to sacrifice the time, money and trouble necessary to file a claim? The ICR contains no clear statement of compliance with OP 4.12. On the basis of the evidence presented, it is not possible to conclude that there was full compliance . Indigenous Peoples The ICR (page 12) reports that “an Indigenous Peoples Planning Framework consistent with the Bank ’s OP 4.10� was prepared. Reportedly, the project caused no adverse impacts on indigenous peoples who were dispersed in small settlements in some of the project districts . Nevertheless, LGED built two schools and provided water supply and sanitation facilities at two other locations in the northern region of the country . b. Fiduciary Compliance: During the early stages of implementation, financial management was weak . According to the ICR, this was attributable to the heavily decentralized financial structure of the project, which included mobilizing adequate accounting resources from many field offices and transmitting and consolidating the information at headquarters . However, a new Unified Financial Management System was developed for LGED . A subsequent review of the earlier financial issues indicates that most were resolved and that the new system was generating timely reports by the closing stages of the project . Nevertheless, throughout implementation, LGED had difficulty resolving material issues raised in the audit reports. These issues were related to obtaining timely financial data from sub -offices in rural parts of Bangladesh. The ICR reports (page 11) that “substantial progress� in resolving these issues was made during appraisal of the second Rural Transport Improvement project, and that “all audit objections on the financial statements [of the project under review] until 30 June 2011, that were material to IDA have been satisfactorily resolved.� Overall, the project did not have any significant mismanagement of funds, despite the relatively high financial risk attending a large rural infrastructure project . According to the ICR, procurement was carried out in accordance with Bank guidelines . Initially, there were a number of procurement issues including award of contracts beyond the initial period of bid validity . There were also some problems associated with two ICB contracts, which subsequently had to be cancelled due to nonperformance . Cancelling these contracts was not related to mismanagement or corruption but rather to the contractor ’s inability to mobilize in multiple construction sites . There were no reported cases of misprocurement . c. Unintended Impacts (positive or negative): d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately The Government could have provided Satisfactory stronger leadership in addressing issues related to development of new Growth Markets, and to expanding the impact of Local Government Institutions. There were issues concerning the management of safeguards compliance. Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: IEG draws the following lessons from the experience of preparing and implementing this project : To ensure value for money, it is important to ensure that a training program will have a favorable impact on institutional capacity, and to be able to measure that impact . The fact that a large number of people were trained under the project may be a necessary but insufficient condition for strengthened institutions . Better indicators need to be developed to measure the effectiveness of capacity building efforts . During project preparation it is important to clarify who is responsible and accountable for monitoring and evaluation. Merely providing indicators and targets is not sufficient if the benefits of M&E are to be realized . While it is important that the implementing agency is fully aware of safeguard policies, it is equally important that other organizations/stakeholders have the same awareness . For example, most of the delays in land acquisition were due, not to the LGED, but rather to a lack of knowledge of the safeguard compliance requirements by officials at the District level and by other Ministries not directly linked to the project . ICR preparation guidelines clearly state that component costs at appraisal and closure should be in US dollar equivalents. To monitor project costs in US dollars, it is important that any significant shifts in exchange rates are taken into account. Failure to do this results in a situation reflected in Annex 1 of the ICR where it is not possible to compare the actual expenditure to the appraisal cost estimates . Unless additional activities, such as the flood rehabilitation component of this project, are fully integrated into the monitoring and evaluation system, it will not be possible to supervise adequately the deployment of funds to support these activities . 14. Assessment Recommended? Yes No Why? An assessment of this project, together with its two predecessors, would enable verification of the socio-economic impact of the infrastructure improvements and an in -depth investigation of the handling of land acquisition, which is a major cause of delay in rural infrastructure projects . 15. Comments on Quality of ICR: The ICR provides a reasonably thorough and critical assessment of what went well, what did not go well, and what could have been done to improve the project . However, the assessment of achievement of objectives (Section 3.2) did not address the statement of objectives as written but limited itself to the discussion of “key outcome indicators� which, in fact, measured outputs and intermediate outcomes . There is no clear statement of compliance with Bank safeguards policies, a major shortcoming with regard to a Category “A� project. While the ICR provides actual project cost in Taka, it does not indicate the cost in US dollars . As a result, it is not possible to reconcile or compare the estimated cost at appraisal to the actual cost . The ICR also provides limited information on who was responsible for M&E and how effective they were in utilization of data . (The PAD is also very limited in its M&E description ). It would have been useful if the ICR had provided additional information on the flood rehabilitation component . There is little indication as to how well this component performed or how it was supervised . a.Quality of ICR Rating : Unsatisfactory