ICRR 12694 Report Number : ICRR12694 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/17/2007 PROJ ID : P057997 Appraisal Actual Project Name : Poverty Alleviation & US$M ): Project Costs (US$M): 22.2 22.1 Capacity Building Country : Gambia Loan /Credit (US$M): Loan/ US$M ): 19.0 20.3 Sector Board : UD US$M): Cofinancing (US$M ): Sector (s): General education sector (36%) Health (36%) General water sanitation and flood protection sector (16%) Sub-national government administration (7%) Central government administration (5%) Theme (s): Other urban development (25% - P) Access to urban services and housing (25% - P) Poverty strategy analysis and monitoring (24% - P) Municipal finance (13% - S) Municipal governance and institution building (13% - S) L/C Number : C3176 Board Approval Date : 03/16/1999 Partners involved : Closing Date : 12/31/2003 12/31/2006 Evaluator : Panel Reviewer : Group Manager : Group : Nilakshi M. De Silva Roy Gilbert Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: ( covering 69% original - (covering 69 % of disbursements ) To improve the living conditions of the urban population, and of the poor in particular by focusing on (a) the sustainable provision of adequate public infrastructure and services; and (b) the development of a lasting, enabling environment for municipal investment financing and for employment creation and income generation .Specific project objectives were to: (i) reduce the backlog of public infrastructure and assets, and improve their maintenance; (ii) alleviate poverty through creation of temporary jobs and improvements to the selection of small to medium -size investments; and (iii) strengthen the technical and managerial capacity of local authorities (with an emphasis on their financial situation), local private consultants and contractors and Gamworks (Gambian Agency for the Management of Public Works). ( covering 31% revised - (covering 31 % of disbursements ) - following July 5, 2005 Amendment to DCA To contribute to improved living conditions of the urban population, and the poor in particular through : (i) provision of public infrastructure and services based on demand and the fulfillment of clearly defined eligibility criteria; and (ii) implementation of a capacity building program aimed at improving the Local Government Authorities ’ (LGA) resource mobilization and management . b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 07/05/2005 c. Components (or Key Conditions in the case of DPLs, as appropriate): [unchanged despite project restructuring ] Part A : Investments (estimated cost US$16.0 million [original: US$13.0 million + supplemental: US$3.0 million]; actual cost US$16.25 million) to (i) finance subprojects to construct or rehabilitate small and medium scale infrastructure or public facilities identified by local authorities, NGOs and CBOs , in eight urban areas using labor intensive methods; and (ii) establish an area-wide solid waste collection and disposal system for Banjul City Council, Kanifing Municipal Council (KMC), and Brikama Area Council (BAC). Part B : Capacity building (estimated cost US$1.35 million [original: US$1.1 million + supplemental: US$0.25 million]; actual cost US$1.18 million). including (i) training, technical assistance and support to the central government as well as local authorities; (ii) training of local consultants and contractors who would receive specialized training through Gamworks to help them to better meet the needs of project beneficiaries; and (iii) technical assistance to Gamworks in the use of computerized Management Information Systems (MIS), support to improve its supervision of technical studies, services and works of local consultants and contractors, and management assistance to help the agency to adapt to meet the needs of its new range of clients . Part C : Studies (estimated cost US$1.2 million; actual cost US$1.32 million) including (i) physical infrastructure, public services, financial and management audits of LGAs; (ii) a study to develop a solid waste management system for the Greater Banjul Area; (iii) a detailed engineering study for the rehabilitation and improvement of the storm water drainage and sanitation system of the Banjul City Council; and (iv) a feasibility study to create a Municipal Development Fund. Part D : Project Administration and Monitoring (estimated cost US$1.57 million, actual cost US$2.60 million) to cover part of the operating costs of Gamworks, the Implementing Agency . Part E : PPF Refinancing (estimated cost US$1 million; actual cost US$0.78 million) to reimburse the Government for project preparation costs d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Original project cost estimate was US$ 18 million, of which IDA funded US$15 million. Because of cost overruns partly linked to the adverse macro economic environment since 2002, a supplementary credit of US$4.0 million was approved in July 2005. Final project costs are just under total (original plus supplemental) estimates. Counterpart funding of US$ 2.16 million was provided by beneficiaries /users, the Government, municipalities and NGOs (after the MTR, IDA finacing of local civil works was increased from 80% to 90%, effectively reducing the local government contribution. As a result, final estimate of counterpart funding required was less than the appraisal estimate of US$3.2 million). The project was scheduled to close on December 31, 2003 but due to project implementation difficulties which arose shortly after the MTR in November 2001, the project was extended thrice and finally closed on December 31, 2006, three years later than planned . 3. Relevance of Objectives & Design: original objective : improving access to services and poverty alleviation continue to be emphasized in both Bank strategy and Government policy . Original project objectives were closely linked to the FY 99-01CAS and are consistent with the priorities identified in the FY 03-05 CAS. The FY03-05 CAS is based on three out of the four main elements of the Government's PRSP, and focuses on improving public expenditure management, enhancing the equity and quality of social service delivery; and, promoting private sector led growth . revised objective : the main substantive change in the revised objectives is that the poverty alleviation objective has been dropped. However as the focus on service delivery and capacity building, particularly of LGAs, is retained the objectives remain substantially relevant. The (unchanged) project design is only modestly relevant to achieving the original objectives . Project components supporting the poverty alleviation objective are not always adequate and the causal chain between project provided temporary jobs and poverty alleviation is not clear . This objective was identified as being too ambitious during the MTR and subsequently dropped, and the revised objectives have been retrofitted to the objectives to suit the project design. 4. Achievement of Objectives (Efficacy): original : To improve the living conditions of the urban population, and of the poor in particular (Substantial) In all, 79 civil works subprojects in sectors such as water, transport, education and health were implemented in 8 LGAs in the country. More than 73.4% of these subprojects are reported to have been implemented in poor areas (though it is not clear how these areas were identified as poor ), as against a target of 50%. Areas that were inaccessible, particularly during the rainy season such as the Rev . J.C. Faye and Bakadaji Roads in the Serekunda Area were provided all-weather access roads, improving movement in the area and reducing health hazards . School sub-projects have contributed to improved access for marginalized groups (girls and the disabled), and improved the quality of education services . The upgrading of the Bakoteh dumpsite serving the densely populated Greater Banjul Area has improved sanitation conditions and urban environment in the entire area . According to a Beneficiary Survey, the social infrastructure, roads and drainage, revenue generation and water supply sub -projects provided under the Project have greatly increased access to basic services, improved the health status of the populations, provided income generation and employment creation opportunities, and improved living conditions . Overall, it is estimated that 378,000 persons (compared to target of 360,000) or 60% of the urban population in the country, have benefited from at least one subproject . (i) reduce the backlog of public infrastructure and assets, and improve their maintenance (Modest) Despite the completion of 79 civil works subprojects identified by LGAs, it is not clear whether the project has "reduced the backlog" as no baselines of the backlog were specified in the PAD nor reported in the ICR . In fact the ICR notes that a "significant backlog" still exists (p.15). The project has however helped to ensure that maintenance of public infrastructure and assets is improved; LGAs have established O&M Funds for maintaining assets funded by the Project and are implementing Revenue Improvement Action Plans (RIAPs) to increase their own sources of revenues to ensure adequate resources for O&M in the medium term . (ii) alleviate poverty through creation of temporary jobs and improvements to the selection of small to medium -size investments (Modest) While the project has provided 29,275 person months of unskilled labor and 9,808 person months of skilled labor through temporary employment in the construction sector, the impact upon poverty alleviation is not clear . Most of the subprojects provided infrastructure and public facilities in poor areas . (iii) strengthen the technical and managerial capacity of local authorities (with an emphasis on their financial situation), local private consultants and contractors and Gamworks (Substantial) The Project has contributed to strengthening LGAs capacity in prioritizing their investment needs through urban audits, operating and maintaining their infrastructure and services through the establishment of O&M accounts and monthly monitoring systems, and increasing their revenue by property revaluations . In the private contracting and consulting firms, a total of 367 staff were trained and 57 out of 79 contracts for civil works were awarded to local firms who carried out all civil works except asphalt road pavements and Bakoteh Dumpsite which specialized equipment not available in the Gambia. However, building local capacity of contractors and consultants for delivery of quality works and services remains a continuing challenge for the construction industry and the Government . The development of Gamworks suffered a setback with the management crisis in 2002-2003 when the Director General was removed by the Government and a nominee not acceptable to the Bank was appointed . However, with the recruitment of a new Director General selected on a competitive basis and supported by technical staff, Gamworks is now able to operate with a 6%-7% fees excluding audit costs, a reasonable fee based on the size of the portfolio implemented by the agency. revised : To contribute to improved living conditions of the urban population and the poor in particular (Substantial). (This objective is substantively the same as the orginal overall objective, and adds only the words "to contribute"). As noted under achievements against the original overall objective above, social infrastructure, roads and drainage, revenue generation and water supply sub -projects provided under the Project are reported to have greatly increased access to basic services, improved the health status of the populations, provided income generation and employment creation opportunities, and improved living conditions . More than 73.4% of the 79 subprojects implemented under the Project are reported to have been implemented in poor areas, as against a target of 50%. (i) provide public infrastructure and services based on demand and the fulfillment of clearly defined eligibility criteria (Substantial) Public infrastructure and services were provided based on demand as subprojects were selected after consultations with, and the participation of, LGAs, after ensuring that these projects were selected by the LGAs in consultation with the communities who were the ultimate beneficiaries . All subprojects are reported to have met the eligibility criteria, such as the absence of negative environmental impacts . (ii) implement a capacity building program aimed at improving the Local Government Authorities ’ (LGA) resource mobilization and management (Substantial) 250 LGA staff received training in computer applications, financial management and O&M, while technical assistance was provided to all LGAs on resource mobilization and improvement of revenue management . 89.8% of the trainees expressed satisfaction with the training and some have used it to train others in their departments . LGAs have established O&M Funds for maintaining assets funded by the project . The urban audits and medium term investment plans have provided LGAs with a rational basis for prioritizing their investments and to carry out systematic rehabilitation and O&M of their existing assets . Urban mapping has improved availability of land -related information for LGAs, enhancing their ability to better plan for the rapid urban growth and to maximize internal resources from land related taxes. Finally property revaluation has helped to increase own resource revenue of LGAs through improving property tax base and collection procedures . As a result, property tax revenue for all LGAs (except BAC) increased by 19.0% annually from 2001 to 2006 at current prices, and 9.0% at constant prices (compared to target of 6% increase per year in real terms). 5. Efficiency (not applicable to DPLs): Overall project efficiency is assessed to be modest because the project experienced substantial delays, cost overruns including in project administration, and also implemented fewer activities than planned - number of subprojects was reduced from 110 to 79, and only 4 out of 6 studies were completed while component cost estimates were exceeded. The ICR reports internal rates of return of 41% and 34% for Bakadaji and Rev. J.C. Faye road investments, respectively, which are based on estimates of traffic growth in similar operations (it is not clear from the ICR what percentage of project costs are accounted for by these two investments ). While other subprojects were reportedly screened through cost benefit and cost effectiveness analyses, most subprojects did not have easily quantifiable revenues, and benefits which could be easily measured in monetary units . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: (covering 69% of disbursements ) original objectives - (covering While objectives were substantially relevant, and the overall objective of improving the living conditions of the urban population was substantially achieved, there were significant shortcomings . The ICR does not present any evidence to indicate that the backlog of public infrastructure has been reduced or that project activities has lead to poverty alleviation. In the context of modest efficiency (section 5), outcome against the original objectives is rated as moderately unsatisfactory. ( covering 31% revised objectives - (covering 31 % of disbursements ) Against the revised objectives, project performance is significantly better and it has achieved its major relevant objectives with only minor shortcomings . Against the revised objectives outcome is therefore rated as satisfactory. Giving due weight to project performance before and after restructuring (69%:31%), the project outcome is rated moderately satisfactory overall. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The greatest risk to development outcome is posed by the potential failure on the part of LGAs to carry out timely and effective O&M on Project provided infrastructure and public facilities . On the whole, LGAs have made progress in creating O&M accounts and taking steps to ensure sustainable revenue streams to meet their O&M expenses . However, some LGAs have only established line items for O&M in their budgets, which are either under -funded or not funded at all, while some of LGA are yet to put in place comprehensive O&M plans . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry is rated unsatisfactory mainly because of weak project design in the absence of a clear statement of objectives, and weak M&E design . The ICR states that the project objectives were identified at MTR as being "unrealistic and not well defined", neccessitating a formal review of project objectives (p2). The project design also contained simplistic assumptions, for example about the impact of temporary jobs on poverty alleviation, without an adequate assessment of the possible impacts of such activities on the poor in the long term. Overall, a poverty alleviation project of this nature should be based on a more substantive assessment of the poverty situation in urban Gambia than was evident in the PAD . These weaknesses were by -and-large recognized during the MTR and appropriate remedial action taken . The Bank performed well to keep the development objectives in focus during several implementation crises to facilitate the achievement of agreed targets and completion of the project . at -Entry :Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The Government and implementing agency performed well to complete this project, to achieve the agreed targets and fulfill all financial and legal covenants . This is particularly noteworthy because a management crisis in Gamworks during 2002-2003 (which was precipitated by the actions of the Government in removing the Director General of Gamworks and appointing a nominee, rather than a competitively selected candidate, for the position ) almost brought project implementation to a standstill . Since 2004, the new Gamworks management team, supported by technical staff in three key departments (Technical, Finance and Capacity Building ) have significantly improved the quality of management through forward planning and intensive monitoring, and the Gov ernment has proactively supported Gamworks to enable the expeditious completion of project activities . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: The M&E design was weak. The Project Design Summary of the PAD is disconnected from the statement of objectives and appropriate indicators to measure project performance were not identified . Key baselines and targets, such as to measure achievement against the objective to reduce the backlog of public infrastructure, were neither specified at appraisal nor identified once the urban audits were completed . During implementation, most performance indicators identified in the PAD were revised or dropped and new, more appropriate, indicators were adopted. Two key outcome indicators, number of persons benefitting from at least one sub -project and the level of resource mobilization by LGAs, are well documented . Complementing a monitoring framework that was largely focused on outputs, the beneficiary survey (summarized in an annex to the ICR) adds substantial value. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project did not involve any involuntary resettlement or land acquisition, and it is in compliance with environmental safeguards. Environmental Impact Assessments (EIA) were carried out as part of the design of various civil works, particularly for the Bakoteh dumpsite in Banjul, and mitigation plans are being implemented (but questions remain as KMC is yet to procure necessary equipment and capability to implement mitigatory measures ). In addition, a standardized guide for carrying out EIA for small works was produced as part of project activities, reviewed by the National Environmental Agency and adopted by Gamworks . 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Giving due weight to project Satisfactory performance before and after restructuring (69%:31%), outcome is rated moderately satisfactory . While achievement against objectives after restructuring is satisfactory, there were significant shortcomings in the achievement of project objectives before restructuring. Risk to Development Negligible to Low Moderate While LGAs have taken steps to create Outcome : O&M funds and ensure sustainable revenue streams to meet their O&M expenses, in some LGAs these measures have not been effective . Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Some lessons identified in the ICR include : Building local capacity for delivery of quality works and services is key for small countries such as Gambia . However, it is also necessary to balance the need to strengthen local capacity with the need to provide quality works. Where possible the government should facilitate international contractors and consultants to help build and strengthen local capacity . Provision of relevant studies, training and technical assistance in a co -ordinated manner can provide significant results in improving local government capacity . 14. Assessment Recommended? Yes No Why? To provide an input into IEG's Special Study on Municipal Development . Also, to better understand the impact of project activities on the urban poor (including for example women, the elderly and disabled ), to identify which activities were particularly beneficial or otherwise, and to provide broader lessons for urban infrastructure and poverty alleviation projects in the country . 15. Comments on Quality of ICR: The major weakness of this ICR is the quality of evidence and analysis relating to the achievement of intended outcomes. The ICR focuses on project outputs, such as number of subprojects implemented, rather than the intended outcomes - in particular the impact of the project on poverty, living conditions in urban areas, infrastructure maintenance and institutional capacity . Given this shortcoming in the ICR, the assessment summarized in Annex 3: Beneficiary Survey adds substantial value in assessing project performance . On the other hand, the ICR also makes some evaluative statements without providing sufficient evidence or analysis, such as in relation to poverty alleviation (p8) "all targets were met or exceeded, confirming that project design had adequately addressed the issues during project preparation" . Greater care to draw out broader lessons from this project and better organization of the information presented to avoid repetition, would have greatly improved this ICR . Finally, some key factual information is missing, such as Board approval date for the revised project objectives, and discussion of reasons for project extensions and for approving a supplementary credit . a.Quality of ICR Rating : Unsatisfactory