ICRR 14242 Report Number : ICRR14242 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 10/24/2013 Country : Sierra Leone Project ID : P113757 Appraisal Actual Project Name : Sl-decentralized US$M ): Project Costs (US$M): 20.0 19.8 Service Delivery Program L/C Number : Loan/ US$M): Loan /Credit (US$M): 20.0 19.8 Sector Board : Social Protection Cofinancing (US$M): US$M ): 0 0 Cofinanciers : Board Approval Date : 10/15/2009 Closing Date : 12/31/2011 06/29/2012 Sector (s): Public administration- Water sanitation and flood protection (31%); Public administration- Health (27%); Public administration- Education (27%); Sub-national government administration (15%) Theme (s): Social safety nets (34% - P); Other human development (33% - P); Decentralization (33% - P) Prepared by : Reviewed by : ICR Review Group : Coordinator : Judith Hahn Gaubatz Judyth L. Twigg Christopher D. Gerrard IEGPS2 2. Project Objectives and Components: a. Objectives: According to the Financing Agreement (page 5) and the Project Appraisal Document (PAD, page 12), the project objectives were: To strengthen Government capacity to manage decentralized services; To improve the availability and predictability of Local Councils' funding; and To strengthen the inter -governmental fiscal transfer system . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: 1. Grants to Local Councils (Appraisal: US$16.0 million; Actual: US$15.2 million): This component aimed to supplement fiscal transfers that local government entities -- Local Councils (LCs) -- receive from the central government. Funds from this component were to be allocated and disbursed to LCs to deliver basic services in the sectors of education, health and sanitation, solid waste management, and water . The funds were to be disbursed from the central government to each LC by the 15th of the first month of each quarter, according to each LC's annual development plan. 2. Capacity Development and Technical Assistance (Appraisal: US$3.0 million; Actual: US$4.5 million (includes Component 3)): This component aimed to strengthen Local Councils' capacity to perform their core functions in basic service delivery, as well as the capacity of line ministries -- Ministries, Departments and Agencies (MDAs) -- to provide strategic guidance and oversight to LCs . Two channels of technical assistance were to be provided : centrally-provided support for all LCs to strengthen core functions (i.e. development planning and budgeting, monitoring); and demand-driven assistance for individual LCs or MDAs, according to their specific capacity needs . This component also included a pilot activity to develop a social accountability mechanism, whereby community groups -- Ward Development Committees (WDCs) -- provide regular monitoring and feedback on the LCs' performance in delivering services . 3. Program Management (Appraisal: US$1.0 million; Actual: US$4.5 million (includes Component 2)): This component was to support the operations of the project coordination unit in the Ministry of Finance and Economic Development, as well as technical assistance to the Local Governance Finance Department (which oversaw the fiscal transfers from central government to the LCs ) and the Decentralization Secretariat (which oversaw training and capacity building activities). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project cost The actual project cost was US$ 19.8 million, out of the estimated US$20.0 million. The slight variation was due to XDR exchange rate fluctuations . Financing The project was financed entirely from a Bank loan of US$ 20.0 million. The loan disbursed US$19.8 million. Borrower contribution There was no planned Borrower contribution . Dates June 2011: The project was restructured as follows : (i) the project closing date was extended from December 2011 to June 2012 to allow for completion of the Community Feedback pilot activities; (ii) loan proceeds were reallocated across disbursement categories; and (iii) the results framework was modified slightly to better align one project indicator and target with the triggers proposed for a follow -up Bank operation. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High. High Emerging from a decade of civil conflict, Sierra Leone has been in the process of restoring its economic and governance systems in order to sustain political stability and address low human development levels . In 2008, the country ranked in the bottom ten percent on "government effectiveness " according to the World Governance Indicators, and last on the UNDP Human Development Index . The government embarked on an ambitious decentralization reform process in an effort to strengthen sub -national democratic institutions and improve delivery of basic services, including the creation of a legal framework for the devolution of functions to Local Councils (LCs), of which there are 19 in the country. The Bank's Joint Country Assistance Strategy for FY 10-13 identifies one of its key pillars as ensuring sustainable human development through improved provision of social services (including improved governance, capacity, and transparency ). b. Relevance of Design: Substantial . The results chain provided a logical and plausible connection between the project's planned activities and its anticipated outcomes . Project design provided for both direct financing of social services and capacity building for central and local governments to improve governance and service delivery . The design focused on four "sectors" -- education, health and sanitation, solid waste management, and water -- in line with government priorities to focus on sectors with highest impact on human development indicators . Although the fiscal transfers were to take place within existing national systems and in a supportive policy environment, capacity building activities for LCs were needed early on to support effective development planning . 4. Achievement of Objectives (Efficacy): To strengthen Government capacity to manage decentralized services Achievement of this objective is rated Modest . The PAD (pages 6-7) indicates that Local Councils (LCs) would receive assistance particularly in the areas of development planning, linking budgets to desired results, monitoring service delivery, and promoting social accountability . Although there was evidence of increased capacity in development planning and monitoring, there was insufficient evidence of achievements in the remaining areas . In addition, sector line ministries were also to receive support in establishing technical /quality standards, supervising service delivery, and monitoring service delivery outcomes (PAD, page 55). Outcomes pertaining to improved sector line ministry capacity were not reported . Outputs Development of integrated development plans ("Agreements") and budgets in all 19 LCs. Establishment of M&E activities in all 19 LCs, including receiving, processing, and transmitting M&E data to key stakeholders. Training to LC, line ministry, and ward development committee staff on development planning, budgeting, procurement, and M&E. Provision of technical supervision by line ministries . The ICR (page 13) reports that 80% of project funding used by the line ministries went towards conducting field visits to LCs . Conducting of consultations on development Agreements by ward development committees . However, only 46% of committees received copies of development Agreements and annual updates . Development of Community Compacts as a pilot activity . This activity developed scorecards for health metrics in maternal and child health posts and community health posts . The Compacts represented mutual commitments between clinic staff and community representatives to improve health service delivery . Results from this social accountability activity are expected later in 2013. Conducting of one national Public Service Survey to assess impact on basic service delivery . The survey was modified to combine three related surveys into one consolidated instrument . Outcomes The 2011 Comprehensive Local Government Performance Assessment confirmed that all 19 LCs had prepared annual development plans with balanced budgets and approved procurement plans . 14 LCs were in compliance with their M&E plans and were preparing regular monitoring reports . The number of Local Councils meeting at least 75 percent of service-output targets in development Agreements was 11, falling short of the target of 15. To improve the availability and predictability of Local Councils' funding Achievement of this objective is rated Substantial due to increased provision and regularity of funding to Local Councils. Outputs Project funds provided grants to LCs according to Agreements between the LCs and the central Government . These grants, disbursed through the existing inter -governmental transfer system on a quarterly basis, funded the following : 84 projects to rehabilitate primary health clinics and hospitals . Procurement of drugs and medical supplies, vehicles, power supply equipment, and medical equipment . Public awareness and advocacy meetings on health topics . 101 projects to rehabilitate primary schools, junior secondary schools, and adult literacy centers . Procurement of school furniture and equipment, textbooks, vehicles, and examination verification equipment . 15 projects to upgrade collection and dumping facilities in 79 locations. Procurement of solid waste collection equipment . 20 projects to rehabilitate or construct water supply systems . Outcomes In 2010, the government transferred 96% of its planned budget allocations to the LCs . In 2011, the government transferred 86%. This compares to less than 75% transferred annually during the years 2005 to 2008. After an initial delay in disbursing the first quarter grants, transfers occurred with improved regularity and timeliness. The original target was disbursement within the first 15 days of each quarter; although this target was not met, disbursements were taking place within 3 or 4 months since the previous transfers . The ICR (page 15) reports feedback from LCs and MDAs that this entailed improved predictability of funding . The percentage of total domestic revenues transferred to Local Councils on an annual basis increased from 29.6% (baseline year not provided) to 38.0% in December 2012. As these percentages excluded the project funds, the increase reflects the additionality of project funds, rather than substitution for government funding to the LCs. The percentage of total government expenditures transferred to Local Councils on an annual basis decreased from 6% in 2010 to 5% in December 2012. However, the ICR (page 14) reports that the real value of government disbursements to the LCs more than doubled between FY 2009 and FY2011, from 28.2 billion Leones to 61.5 billion Leones. As the civil servant salaries of teachers and nurses increased substantially during the same time period, the proportion of the total government expenditures comprising fund transfers subsequently decreased . To strengthen the inter -governmental fiscal transfer system Achievement of this objective is rated Substantial due to evidence of streamlined funding and improved functioning of the transfer system. Outputs Adoption of the Second Generation Inter -governmental Fiscal Transfer System, with a focus on further reducing conditions on LC transfers to permit greater autonomy in defining local priorities . Preparation of annual development plans by all 19 LCs, with integration of planning and budgeting . Training to LCs and MDAs on development planning and budgeting . Outcomes The number of grants provided annually in the four sectors (education, health and sanitation, water, and solid waste management) was streamlined from 9 to 6, achieving the target of 6. In 2010, the government transferred 96% of its planned budget allocations to the LCs . In 2011, the government transferred 86%. This compares to less than 75% transferred annually during the years 2005 to 2008. After an initial delay in disbursing the first quarter grants, transfers occurred with improved regularity and timeliness. The original target was disbursement within the first 15 days of the each quarter; although this target was not met, disbursements were taking place within 3 or 4 months since the previous transfers . The ICR (page 15) reports feedback from LCs and MDAs that this entailed improved predictability of funding . 19 LCs had January to September budget execution rates of 90% in 2010 and 2011, although that dropped to 11 LCs in 2012. The 2011 Comprehensive Local Government Performance Assessment confirmed that all 19 LCs had prepared annual development plans with balanced budgets and approved procurement plans . 5. Efficiency: Efficiency is rated Modest due to insufficient information. Some aspects of program design likely contributed to efficiency: reduction in transaction costs at both central and local levels by streamlining the number of grants to LCs; the use of the existing inter -governmental fiscal transfer system to provide sector grants on a more predictable basis (these grants comprised US$16.0 million out of the total US$20.0 million loan). However, efficiency in the use of project resources for capacity building at all levels (central Government, MDAs, LC, WDCs) is unclear. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of the project's objectives is rated High, and relevance of the project's design is rated Substantial . Achievement of the objectives to improve the availability and predictability of Local Councils' funding and to strengthen the inter-governmental fiscal transfer system is rated Substantial . However, achievement of the objective to strengthen central government capacity to manage decentralized services is rated Modest due to shortcomings in outcomes. Efficiency is also rated Modest due to insufficient evidence . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Political commitment to decentralization of basic service delivery remains significant .. Funding for the sector grants to the LCs is being continued through a follow up Phase II Bank operation, which was approved in December 2011. The institutional arrangements and systems used by the project are likely to be sustained, through continued support of legal and policy frameworks. However, the supervisory and monitoring functions of the MDAs and central government will need further strengthening, and capacity at LC levels still needs to be improved . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The project built upon the institutional arrangements set up by the prior Institutional Reform and Capacity Building project (2004-2011), which assisted in the establishment of a local government system, decentralization of functions, and development of a funding mechanism to transfer grants from the central government to the LCs . The project also drew upon lessons from a joint Bank -Government review of the initial phase of decentralization (2003-2007), including the identified constraints of adequacy, timeliness, and predictability of resource transfers from the central government to the LCs . However, there were some inadequacies in the project's design . The disbursement condition of finalization of Agreements (LC annual plans) did not allow sufficient lead time in preparing the Agreements before the first disbursement was due . Although government transfers during the delay period minimized the impact of the delayed grant disbursements, the ICR reports (page 7) that there were inadequacies in setting accurate targets in the Agreements . Also, as noted in the ICR (page 6), the Agreements "would have benefited from clearer linkages among the targets established, the capacity building activities necessary to achieve these outputs, and the engagement of relevant MDAs in their monitoring and completion ." The risk assessment was candid and comprehensive, identifying seven risks as Significant or High . A number of these risks materialized but were effectively mitigated . The M&E framework had some shortcomings, including a lack of comprehensive indicators clearly linked to all of the development objectives . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: In conjunction with the Mid-Term Review in May 2011, an Independent Technical Audit Report was prepared, highlighting insufficient quality of some civil works, as well as procurement and filing problems . The Bank team responded by strengthening capacity building activities in these areas, although the 2012 In-depth Financial Management Review identified continuing weaknesses in file management . However, there were no major fiduciary problems reported. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The central government demonstrated strong commitment to the decentralization agenda, including through supportive institutional and legislative measures . The government also provided the planned fiscal transfers to the LCs to complement the project grants . Government Performance Rating Satisfactory b. Implementing Agency Performance: The Local Government Finance Department was effective in providing fiscal transfers from the central government to the LCs on a more predictable basis . The Decentralization Secretariat, which was tasked with training activities for LCs and MDAs, fell short in implementing some capacity building activities . The project management unit carried out its functions effectively, with no major fiduciary problems reported . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E framework had shortcomings, in that the choice of indicators was limited and did not clearly capture all intended outcomes. M&E arrangements built on existing systems to monitor implementation progress, while also drawing from the newly instituted Agreements to monitor LC outputs . The National Public Service Surveys and the Comprehensive Local Government Performance Assessment System were existing evaluative tools to help assess the project's impact on service delivery . A pilot activity was designed to introduce demand -side social accountability. b. M&E Implementation: A number of the M&E activities were carried out as planned . However, as noted in the ICR (page 6), the linkages between the Agreement plans and the targets were not clear, and the relevant MDAs were not sufficiently engaged in monitoring the Agreements. Data on all intermediate indicators were not tracked by the implementing agencies . c. M&E Utilization: The ICR (pages 8-9) cites examples of the use of M&E to inform project decisions . The LC performance assessments identified capacity weaknesses, which were addressed by providing additional training in financial management, development planning, and records management . The M&E information has also informed project design for the follow-up Phase II Bank operation. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was classified as a Category "B" project due to rehabilitation of basic infrastructure, triggering the safeguard policy on Environmental Assessment (OP/BP 4.01). An Environmental and Social Management Framework from the previous Institutional Reconstruction project was revised for the purposes of this project . The ICR does not report on whether overall safeguards compliance was satisfactory . Trainings were provided on solid waste management. During the Mid-Term Review, new construction was deemed as an ineligible expenditure due to the lack of a Resettlement Plan, and therefore these activities were not implemented (focusing instead on rehabilitation of existing structures ). b. Fiduciary Compliance: Financial management: Project audit reports were unqualified and submitted in a timely manner . An In-depth Financial Management Review conducted in June 2012 reported that FM systems at both the project coordinating unit and the LCs generally met the minimum Bank requirements . However, there remained instances of inadequate supporting documentation, and improper authorizations and approvals . There were also three ineligible expenditure claims amounting to US$26,000. Procurement: Procurement performance was overall satisfactory, building on capacity from the previous Bank operation. There was one instance of a significant procurement delay in finalizing contracts for the social accountability activities, which led to delays in implementation . c. Unintended Impacts (positive or negative): d. Other: 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lessons from the ICR, adapted by IEG : A phased, long-term approach to supporting the Government's decentralization reform agenda can be effective in low-capacity, fragile contexts . This project drew upon capacity built in a previous operation, and then reviewed outcomes and experience to inform the follow up operation . Key benchmarks were clearly identified to mark the Government's progress in implementing the reform agenda . Capacity building activities need to be properly sequenced to support effective project implementation . In the case of this project, although there was some element of "learning by doing" in allowing LCs to access sector grants quickly, capacity building support could have been front -loaded before allowing the fiscal transfers to take place. Adequate attention to and investment in M&E tools can produce useful data, even in low -capacity contexts. This project's National Public Service Survey and Comprehensive Local Government Performance Assessments were conducted within the existing national system, with some fine -tuning, and provided information that led to project adjustments . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is consistent with guidelines and overall satisfactory . The analysis of the results chain for the first objective ("to strengthen Government capacity to manage decentralized services ") could have drawn clearer linkages between the specific project outputs and the intended outcomes . Lessons were informative for other low -capacity, fragile contexts. The ICR does not report on whether safeguards compliance was satisfactory . a.Quality of ICR Rating : Satisfactory