ICRR 13294 Report Number : ICRR13294 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/16/2010 PROJ ID : P001807 Appraisal Actual Project Name : Decentralized US$M ): Project Costs (US$M): 46.29 48.68 Planning And Financing Project Country : Mozambique Loan/ US$M): Loan /Credit (US$M): 42.0 45.17 Sector Board : PS Cofinancing (US$M): US$M ): 4.29 3.51 Sector (s): Sub-national government administration (30%) Central government administration (30%) Roads and highways (20%) General water sanitation and flood protection sector (15%) Other social services (5%) Theme (s): Administrative and civil service reform (20% - P) Rural services and infrastructure (20% - P) Rural policies and institutions (20% - P) Participation and civic engagement (20% - P) Decentralization (20% - P) L/C Number : CH067 Board Approval Date : 11/20/2003 Partners involved : Closing Date : 06/30/2008 03/31/2009 Evaluator : Panel Reviewer : Group Manager : Group : Tim L. De Vaan Fareed M. A. Hassan Ismail Arslan IEGCR 2. Project Objectives and Components: a. Objectives: The overall objective of the Decentralized Planning and Financing Project (DPFP) was to improve the institutional capacity of District Administrations to plan and manage small infrastructure investments in response to community demand. No specific sub-objectives were formulated. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project had the following five components : US$ 8.76 million, Actual : US$9 1. Participatory District Planning (Appraisal cost : US$8 US$ 9.94 million ). This component was to provide support to district administrations to establish and operationalize District Planning Teams and to prepare strategic and annual plans . It would also develop capacity of civil society and rural communities to effectively participate in planning and ensure local administration accountability . Lastly, this component would support the development and implementation of a communications strategy . 2. Local Investment Grants (Appraisal cost : US$US$1919. 19 19. US$19 .24 million, Actual : US$ 19 .96 million ). This component was to provide grants for implementation of small rural infrastructure investments . Three grant types were envisioned. Type 1 was to finance investments in district administrative infrastructure, managed by the province . Type 2 grants would finance investments in small -scale public social infrastructure prioritized by communities, co-managed at the province and district level . Type 3 grants would provide small discretionary funds to District Administrations for direct contracting for micro -projects. 3. Capacity Building for Local Administration (Appraisal cost : US$8 US$ 7.84 million ). This US$ 8.52 million, Actual : US$7 component aimed to strengthen the training system to increase the capacity of local administration officials to undertake key local government functions more effectively (on the provincial and district level ). Activities were likely to include training in basic public administration, planning and budgeting, and selected technical subjects . US$ 1.51 million, Actual : US$2 4. Support for Decentralization Policy Reform (Appraisal cost : US$1 US$ 2.98 million ). This component was to finance (demand-driven) technical assistance to the Government of Mozambique (GOM) to improve decision-making through policy analysis and for the further development of the institutional and policy framework for decentralization. US$ 8.26 million, Actual : US$7 5. Project Coordination (Appraisal cost : US$8 US$ 7.96 million ). This component would consist of a small core team of project coordination staff at central and provincial level who would fulfill some World Bank-specific fiduciary functions and provide technical assistance to the government officials responsible for project implementation and GOM fiduciary function s. This component would also contribute to the development of a common decentralization Monitoring and Evaluation (M&E) framework, and project resources under this component could also be used for the preparation of a future national decentralization program . DPFP initially focused on four provinces in the central zone of the country : Zambezia, Sofala, Tete, and Manica--with a total of 49 districts. In the last year of the project, the province of Maputo (7 districts) was added, but no investment grants were made in this province . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: DPFP's initial closing date of June 30, 2008, was extended twice. The first extension of six months was to accommodate delays in the local investment grants . The second extension to March 31, 2009, was--according to the Implementation Completion Report (ICR)--to allow time to undertake a study of community participation in district planning, to finalize several studies, and to prepare inputs for the follow -on National Decentralization Planning and Finance Program. The total costs at appraisal were US$ 46.20 million (US$42 million from IDA and US$4.29 million from the government), while the actual costs were US$ 48.68 million (US$45.17 million from IDA and US$3.51 million from the government). 3. Relevance of Objectives & Design: The project’s objective was and remains relevant . Participatory district planning and finance was identified in the government’s Poverty Reduction Strategy Paper (PRSP) as key to improving rural governance and an important contribution to the higher-order objective of accelerating poverty reduction by increasing the provision of local services. DPFP was approved in response to government demand, reflected lessons learned from pilots in Mozambique and experiences in other (neighboring) countries, and its design was based on analytical work carried out in 2002. The project’s focus on participatory and institutional capacity building in four central provinces was appropriate given the early stages in the long -term program of decentralization that the country is still in. The project’s overall objective was phrased rather broadly and not specified by sub -objectives, but the results framework included generally sensible outcome and intermediate indicators . The Project Appraisal Document specified that the project’s main focus was on institutional impact rather than on developmental impact (in terms of poverty reduction and socio -economic development)—which meant that these were not captured in the M&E design. No quality assurance group (QAG) at entry assessment was conducted . 4. Achievement of Objectives (Efficacy): In order to measure progress towards achieving the overall objective of improving the institutional capacity of District Administrations to plan and manage small infrastructure investments in response to community demand, the project formulated six outcome indicators /key performance indicators (KPIs) without specifying specific (sub)objectives. Because of this, this section uses the KPIs (and related results) to guide the achievement of the overall objective. DPFP contributed to downward accountability through citizen participation in selecting project and overseeing their implementation, with some initial shortcomings . The first and second KPIs entailed the organization and training of District Consultative Councils to participate in District Development Plans and Annual Investment Plans, which e.g., selected small investment projects . All districts in the project were covered, which exceeded the target of 80 percent. Guidelines that the project supported ensured that consultative councils included at least 30 percent women, and that other disadvantaged groups were represented . In total, 713 local consultative councils, 56 district councils, 168 local councils in the administrative posts, and 48 local councils in the localities were established. Training in participatory planning under DPFP covered 11,300 consultative council members, 555 district technical specialists, and 28 provincial technical specialists . The project’s third KPI aimed at 60 percent of districts participating in the Local Investment Grants (LIGs) in which participants in planning processes express satisfaction with responsiveness and accountability of district administration, but could not be monitored . Instead, rather late in the project cycle, a 2009 study included a survey of people’s perception of the performance of district administrations . It revealed that 81 percent of respondents (not clear what the sample is) were aware of district investments, and 88 percent of these expressed satisfaction with the results . DPFP financed the training of provincial and district financial staff, these included 49 accountants and 51 works technicians, of whom 95 were still in their posts at the end of the project . In addition, staff at the department of public works and housing/education were trained to procure and supervise the LIGs . This training proved to be a crucial in meeting the fourth KPI : the percentage of districts that rated good or excellent in executing the LIGs under the annual performance evaluation established under DPFP . By the end of the project, the target of 50 percent was exceeded by 17 percentage points. DPFP also aimed at extending its methodology regarding participatory district strategic and budgeting planning to other districts that fell outside the direct scope of the project . By the end of the project, all 128 districts in Mozambique had adopted DPFP’s approach, which exceeded the fifth KPI ’s target of 60 percent. The Local Investment Grants (LIGs) under DPFP were meant to a learning-by-doing exercise by using government systems at the provincial and district level covered administrative infrastructure, small -scale socio-economic infrastructure projects, and small discretionary funds . Although there were some initial delays in LIGs (due to e.g., bidding issues), a total of about 800 small projects were considered successfully executed, while 52 were not considered executed successfully . The measure of success lies in financial accountability — reflecting DPFP’s focus on capacity building at the participatory and institutional level —rather than in a higher level development impact that could capture improved local service delivery or a reduction in poverty . The sixth KPI aimed at supporting transparent district planning and management by adopting five key administrative and fiscal instruments, which were met and exceeded the target of four . Examples are the publication of guidelines for district consultative councils, the adoption of a law that formalized the district as a budget unit, and the publication of rules and procedures for district finance . This KPI was allocated under the fourth component of DPFP that would provide demand -driven technical assistance to several key ministries . However, demand was lower than expected . In that sense, The KPI—although met—did not fully serve to measure possible success of this component . That said, this component did provide, for instance, critical assistance to the Tribunal Administrativo to conduct audits of the districts ’ financial accounts--which contributed to upward accountability. Achievement: substantial 5. Efficiency (not applicable to DPLs): The project appraisal document did not include specific ERR or FRR targets . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The outcome rating for DPFP is satisfactory . In the five selected provinces, the project met and often exceeded its outcome indicators--where monitorable. Training provided through DPFP resulted in increased capacity in planning, budgeting, financial management, and contracts management at provincial and district level administrations. Capacity building of the Tribunal Administrativo allowed it to carry out audits at the district level and institutionalized upward accountability . Downward accountability systems were set up through the establishment of consultative councils, preparation of district plans through participatory processes, and dissemination of information about the plans and their execution . The learning-by-doing approach using small LIGs in socio-economic infrastructure resulted in about 800 completed small projects--although the development impact of these in terms of enhanced service delivery at the local level was not monitored (no LIGs were used in the province of Maputo). Results are likely to be sustained as relevant laws, regulations, and guidelines were adopted by the government, key civil servants were mainstreamed into the district level, and training courses were institutionalized in national training institutions. Although no decentralization policy framework was implemented by the government, its pending National Decentralized Planning and Finance Program, and the associated Bank project that will follow, are expected to sustain results achieved under DPFP and further the decentralization agenda in Mozambique that is currently still in a relatively early stage . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome of the capacity built under DPFP is rated negligible to low . The government’s adaptation of relevant laws, regulations, and guidelines has mainstreamed decentralization into the country ’s development agenda. Also, district accountants and work technicians have been trained and hired into the civil service at the district level, and many training and project manuals are now part of public administration training . In addition, since 2007, the central government provides direct budget transfers to the districts . Although the government never formulated a decentralization policy framework, implementation is pending for the government’s National Decentralized Planning and Finance Program, which will scale up the decentralization approach to cover all provinces and districts in Mozambique . In response, the forthcoming World Bank National Decentralized Planning and Finance Project will continue the work under DPFP on a country -wide level. This risk to development outcome rating does not cover the sustainability of the small infrastructure investment through the LIGs as this was not monitored through the project ’s M&E system (see section 10). a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: DPFP’s design was consistent with the PRSP and the Bank ’s country strategy, and reflected lessons learned from pilot decentralization projects and from experiences in other countries embarking on decentralized service delivery. The project’s selection of four central provinces was based on consultation with the government and on a strategic division of labor with other development partners . The province of Maputo expressed interest in being included in DPFP, which took place about a year from the projects closing date (which meant that capacity building could be covered, but no LIGs were issued ). Supervision missions took place at least twice a year and included foreign and local consultants . A 2006 mid-term review was conducted jointly with development partners who financed decentralization projects in other provinces, which helped identify lessons learned across projects that could be reflected in the design of the nationwide program. An in-depth review of safeguard compliance in 2008 revealed some environmental and social safeguard issues (see section 11). at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The government has adopted an important set of initial institutional building blocks, such as : the adaptation of key legislation, regulations, and guidelines, as well as mainstreaming permanent posts in district administrations. However, a government decentralization policy framework is still pending . In a conversation with the projects TTLs, this was not considered a (major) obstacle to DPFP. Counterpart funds were not always timely or in the agreed amounts . In addition, the government did not (initially) ensure that implementing agencies applied the environmental and social safeguards (see section 11). Lastly, the ICR reports that, during the time of the project, the government took inconsistent positions with respect to the use of block grants to districts, which called into question its commitment to empower district administrations to make planning decisions. The country team reported strong leadership by the Ministry of Planning, with the exception of its coordinating role among other ministries . The project coordination team was well staffed, prepared quarterly progress reports on time, but its oversight of environmental and social safeguards was not adequate (see section 11). a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: Although the project’s overall objective is phrased rather broadly and not specified by any sub -objectives, the results framework included generally sensible outcome and intermediate indicators that aimed at quantified targets where possible, perhaps with the exception of component 4. The project’s main focus was on institutional impact rather than on downstream contributions to poverty reduction and socio -economic development. This meant that the development effectiveness of the small investments through the LIGs was not captured in the M&E design. To overcome delays in M&E implementation due to the lack of qualified staff, the project financed M&E specialists stationed at the provincial coordination units who produced e .g., the quarterly and annual progress reports. Staff reports steady progress in implementation during the project ’s cycle. The project’s fourth KPI, on beneficiaries perceptions, could not be monitored, and a study and survey were carried out late (early 2009), which prevents observing a trend over the project cycle . As part of component 5, DPFP aimed at contributing to the development of a common M&E framework that would standardize M&E across all provinces . The ICR does not provide information on the status of this goal . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): A 2008 World Bank review of safeguard compliance found that, because of a lack of adequate and continuous training, provincial and district field technicians did not correctly apply the project ’s environmental and social management framework prior to undertaking works . This resulted in the construction of two projects (a cattle dipping station and a market stall ) without adequate attention to environmental and social issues . In response, Bank supervision downgraded overall safeguard compliance to moderately satisfactory, which was also the rating in the last ISR. The Bank team advised the government to take action to mitigate potential environmental and social impact of the two projects, and requested that the government prepare a training plan . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately Higher rating is prevented due to e .g., Satisfactory issues with counterpart funds, lack of decentralization framework, among others (see section 9). Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: This Review agrees with the lessons presented in the ICR . Experience under DPFP illustrates that an M&E system with clear target values, coupled with intensive supervision, can help identify obstacles and allow for the incorporation of lessons learned during implementation . However, including an indicator on the application of the environmental and social safeguards, and institutionalizing training courses in government and Bank systems, could have avoided an incorrect application . This review would add that not including development impact indicators (on local service delivery or poverty reduction ) meant that the results of the LIGs could only be captured in accounting terms, and not in terms of development effectiveness . Lastly, DPFP illustrates the benefits of having in place a mechanism for development partner harmonization to help to learn from each other and facilitate the dialogue among partners and with the government . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The quality of the ICR is satisfactory . It provides a candid account of the activities undertaken under DPFP, and delivers a consistent discussion of the status of outcome and intermediate indicators . That said, it does not always fully capture the complexity of the project . An example is the ICR's discussion of the achievement of the project development objectives, which is somewhat simplistic . a.Quality of ICR Rating : Satisfactory