ICRR 13877 Report Number : ICRR13877 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/21/2012 Country : Russian Federation Project ID : P058587 Appraisal Actual Project Name : Regional Fiscal US$M ): Project Costs (US$M): 36.20 70.19 Technical Assistance Project (RFTAP) L/C Number : L4528 Loan/ US$M): Loan /Credit (US$M): 30.00 28.61 Sector Board : Public Sector Cofinancing (US$M): US$M ): Governance Cofinanciers : Board Approval Date : 12/23/1999 Closing Date : 12/31/2004 06/30/2011 Sector (s): Central government administration (50%); Sub-national government administration (42%); Law and justice (8%) Theme (s): Municipal finance (20% - P); Legal institutions for a market economy (20% - P); Law reform (20% - P); Tax policy and administration (20% - P); Debt management and fiscal sustainability (20% - P) Prepared by : Reviewed by : ICR Review Group : Coordinator : Clay Wescott Michael R. Lav Ismail Arslan IEGPS2 2. Project Objectives and Components: a. Objectives: "The principal development objectives ...are to build institutional capacity for advancing the reform of inter-governmental fiscal relations and improving fiscal performance at the sub -national level. To meet these objectives, the project will pursue four sub -goals: Improve the legal framework for sub -national public finance and inter-governmental fiscal relations; Establish a mechanism for improving compliance with federal laws and regulations in the area of fiscal management and promote fiscally responsible behavior at the sub -national level through appropriate economic incentives; Strengthen the Government ’s capacity to monitor sub-national performance and reform efforts; and Strengthen the institutional capacity of sub -national governments to carry out fiscal and structural reforms " (Memorandum and Recommendation of the President, p . 3). There is similar wording for the project objectives in the Loan Agreement . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: There are four components in addition to project management : 1. Strengthening Federal and Regional Fiscal Legislation (US$2.2 million at appraisal, US$17.92 million at completion): To improve the legal framework, including an inventory of the current assignment of revenue and expenditure responsibilities, design of formula -based equalization transfers for the regions, federal regulations for monitoring sub-national fiscal performance, and analysis of the consistency and compatibility of existing legislation. 2. Strengthening Federal Monitoring Capacity : Fiscal Monitoring Division and the PIU for the RFTAP (US$8.1 million at appraisal, US$11.69 million at completion) To strengthen capacity for fiscal and structural reforms, including technology for statistical analysis of regional data, organizing and disseminating information, monitoring of reforms and performance, analytical support and training . 3. Assistance to Sub -national Governments in Accounting and Budgeting (US$9.4 million at appraisal, US$28.66 million at completion): To support diagnostic reviews, financial planning, treasury and cash management, accounting and audit, expenditure restructuring, debt management, procurement, and related manuals, computer equipment and training . 4. Sectoral Public Expenditure Reviews (US$2.2 million at appraisal, US$3.1 million at completion): To support reviews of sectors which have been the largest recipients of subsidies, leading to recommendations on budget prioritization, pricing policies for cost recovery, and opportunities for private sector service provision . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The operation was approved on December 22, 1999, and became effective on August 30, 2000, as planned, although the reason for the delay between approval and effectiveness is not known . The original project cost was US$36.2 million, with US$6.2 million constituting the borrower's contribution . Based on amendments to legal agreements signed in 2004, the project was extended four times, closing on June 30, 2011. The Government decided that deeper public financial management improvements were needed at the federal level along with improved institutional capacity and compliance at the regional level . The extension was financed mainly by increasing the counterpart contribution to US$ 41.6 million. Initial component costs are from Table 3.1, Technical Annex, with "goods" split two thirds/one third between Components 2 and 3 as in ICR. Actual costs are from project financial statements, as reported in the ICR . The focus on deeper reforms led to large increases in cost for Components 1 and 3. There were smaller increases in Components 2 and 4. There were additional costs for project management, taxes, other recurrent costs, and IBRD front -end fee. 3. Relevance of Objectives & Design: a. Relevance of Objectives: The objectives were substantially relevant both at appraisal and completion . They built on the new Budget Code and Tax Code adopted by the Duma in 1998 and 1999 respectively, and the Concept of Reform of Inter-Government Fiscal Relations in the Russian Federation for 1999-2001, adopted with support from the Third Structural Adjustment Loan (1998 - SAL3). They were aligned with the Government's medium -term economic strategy in effect at the time of appraisal calling for eliminating unfunded mandates from the federal to sub-national governments, revising the formula for transfers to the regions, and rationalizing expenditure and tax assignments. These policy goals were given further emphasis after 2004, with a commitment for deeper public financial management reforms. The greater than anticipated contribution provided by the Government, and completion of related activities not planned for at approval, are further signs of the project's relevance to the Government., The Government's commitment to modernizing sub -national governance got a further boost in December, 2011 with the announcement of the intention to elect governors by regional voters . The objectives are consistent with the Bank's Country Partnership Strategy (2006), in effect at project completion with its goals of diversifying the economy for sustainable development and growth, improving public sector management and performance, and improving delivery of communal and social services . b. Relevance of Design: The design is substantially relevant . There is a clear statement of objectives, with a convincing causal chain linking Bank support from this and complementary operations (e.g. Community Social Infrastructure, Management Skills for Market Development, Enterprise Housing Divestiture, and SAL 3), and support from other development partners (e.g. UK Know How Fund, USAID, International Center for Accounting Reform ) with intermediate and final outcomes, with measurable indicators that were further enhanced during the project extension period. The design builds on extensive AAA, and lessons from previous operations both in Russia and in other transition economies . It addresses IEG's findings that governance problems and weak management have been the main reasons for the poor past performance of technical assistance loans . As pointed out in a 2004 QAG assessment, the design could have been improved with outcome and monitoring indicators focusing on outcomes (such as improved service delivery ) rather than just processes and outputs . IEG agrees with the ICR that better service delivery would be impossible without improved processes and outputs, but it would have been better to have some measures, however preliminary, of services actually improving in regions that had carried out fiscal reforms. 4. Achievement of Objectives (Efficacy): The achievement of efficacy of objectives is measured by considering the achievement of the four sub -goals: 1. Improve the legal framework for sub -national public finance and inter -governmental fiscal relations . According to the ICR, the relevant PDO was fully achieved . Three new laws and legal amendments on expenditure assignments were drafted, approved, and put into effect, and a fourth was drafted . Additional Performance and Monitoring Indicators (Technical Annex, Annex 1) were all achieved. The operation supported development of laws and regulations, along with new procedures, methodologies, manuals and training underlying a coherent framework for fiscal policy in Russia . Efficacy of this sub-goal is rated high. 2. Establish a mechanism for improving compliance with federal laws and regulations in the area of fiscal management and promote fiscally responsible behavior at the sub -national level through appropriate economic incentives . Bank support helped improve regional compliance with federal laws . Compliance on Budget Code caps on debt stock, budget deficit, debt service and revenue -current expenditure ratio improved, from 23 regions noncompliant in 2003, to only 2 in 2010. Regions are now annually graded on adherence to guidelines, with sanctions for failing to adopt recommendations . Multi-year budgeting and program based budgeting are being implemented, with the latter scheduled as part of the 2013 federal budget, and then rolled out to the regions. The PDO calling for designing formula-based equalization transfers for all regions was fully achieved. In addition, these transfers were instituted for all regions participating in the Regional and Municipal Fiscal Reform Fund (RFRF) that was supported by the project . Additional Performance and Monitoring Indicators (Technical Annex, Annex 1) were all achieved, including two new indicators added in 2008 on quarterly monitoring of overdue accounts, and annual debt monitoring . Efficacy of this sub-goal is rated high. 3. Strengthen the Government ’s capacity to monitor sub -national performance and reform efforts . Bank support improved capacity, procedures, and monitoring systems . Regional fiscal performance is now monitored monthly, with an annual report on each region published online . Ceilings on regional debt and guarantees are set annually, and performance agreements between the Finance Ministry and the region are reached in the event of breaches of ceilings . The PDO: "computerized systems, in particular financial reporting " was fully achieved, with a computerized information network for fiscal monitoring in place, and computerized systems introduced in selected regions for budget execution, financial reporting and medium term financial planning . These systems will be further enhanced with support from a parallel Bank operation : Treasury Development (p064508). However, there was no attempt to monitor any link between these achievements and improvement of service delivery. There are some indications of improved perception of public services; for example, the Life in Transition Survey found a significant increase between 2006 - 2010 in respondents saying they were satisfied with public services, although the survey doesn't distinguish between services provided by federal and sub-national authorities (Country Partnership Strategy, 2012-16, Annex 5, p. 3). The target: "Regional borrowing and debt servicing profiles improved " was achieved. The three regions participating in RFTAP complied with the Budget Code caps . While their debt stock to total revenue increased marginally over the period 2002-2011, these were initially at very low levels and the duration of debt liabilities improved . Efficacy of this sub-goal is rated high. 4. Strengthen the institutional capacity of sub -national governments to carry out fiscal and structural reforms . Bank support helped 24 out of 89 regions design basic public sector reforms, and provided more intensive support to 6 of these, including adoption of systems such as medium term financial planning, treasury and cash management systems, debt management, procurement, accounting and audit . Another 28 regions benefitted from the framework and standards developed with Bank support, and more for some aspects : e.g. In at least 70 regions revenue assignments and expenditure responsibilities are now made according to the relevant federal legislation; and debt stock in at least 90 percent of the regions is in compliance with the Budget Code caps A Best Practice Code on regional/municipal public financial management was developed, along with recommendations on expenditure management and restructuring at the sub -national level. Four out of five Performance and Monitoring Indicators (Technical Annex, Annex 1) were achieved, and one was was partially achieved: Improvement of regional borrowing and debt servicing profiles . Four new indicators added in 2008 were over achieved, including use of approved methodologies to allocate transfers, expenditure responsibilities made according to federal legislation, debt stock in compliance with Budget Code caps, and number of regions receiving federal grants from the RFRF not less than 25%. Efficacy of this sub-goal is rated high. 5. Efficiency: Non-evaluable. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The objectives and design were substantially relevant . A minor shortcoming was that there were no indicators, however preliminary, to measure services actually improving following the fiscal measures taken, despite a lengthy project extension and near doubling of project costs . There was high achievement of all sub -goals, and many targets were exceeded . While efficiency is rated "non-evaluable", it is important to note that the project implementation required more than 11 years, and the project closed some 6 and a half years behind schedule, which detracts from the benefits of the project . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: At appraisal, it had been feared that the consensus in favor of inter -governmental fiscal reform might be lost, and that the political will to implement the reforms would weaken as improved transparency imposed hard budgetary constraints on regional administrations . There is still a low risk of these changes materializing, particularly the latter as governors need to campaign for election, and high oil revenues might reduce overall fiscal discipline . On the other hand, there is a strong momentum to maintain and build on the new legal and operational framework, reinforced by incentives for performance and disincentives for poor performance built into the system. There are also strong disincentives for governors not to comply with fiscal rules, since violations can subject them to prosecution. The same fiscal rules apply regardless of movements in oil prices . Risks of setbacks in fiscal management are also mitigated by related operations such as Support for Program Budgeting and Expenditure Efficiency (p129493), and the Bank's plans for a follow up operation : Fiscal and Financial Development (p122998). a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: a. Quality at entry: The project provided a holistic approach to sub -national fiscal reform, covering legal frameworks, capacity strengthening at all levels, and improved monitoring . The approach built on AAA, and operations supported by the Bank and other development partners . The project extension reflected a deepening of reforms, not delays in the original reforms. at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: There was considerable overlap in supervision with the complementary Fiscal Federalism and Regional Fiscal Reform DPL (2002 - FFRFR), which provided innovative, competitive funding to selected regions undertaking fiscal reforms. The Bank also coordinated closely with other development partners working on fiscal federalism, including DfID, GTZ, OECD, and USAID . A 2004 QAG review found the project well designed and implemented. While there were delays in agreeing on expanded performance indicators following project extension, and a falling rate of loan disbursement after 2004 (Section 11.b), the extended operation with field based supervision from 2003 allowed the Bank's engagement with the Government on a new round of fiscal reforms during years of almost no borrowing from the Bank . The 14 year period of Bank engagement from project preparation to loan closure gave sufficient time for reforms to take hold . Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government met or exceeded targets, and built a good foundation for improvements in service delivery . There was high ownership at federal and regional levels, as evidenced by deepening of scope and counterpart funding after project extension . There could have been an attempt, in the later years, to coordinate with regional service delivery agencies to achieve improved performance linked to fiscal reforms . Government Performance Rating Satisfactory b. Implementing Agency Performance: The Ministry of Finance provided strong leadership, ownership, coordination with subnational bodies, and nearly flawless administration. The competitive grant procedure linked with FFRFR was innovative . The Ministry maintained its organizational commitment to fiscal reforms over a lengthy period, despite changes in key staff. Implementing Agency Performance Rating : Highly Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The objectives, four sub-goals, and respective indicators were well specified and were restricted to processes supported by the operation . The operation provided considerable support to the Fiscal Monitoring Division, Ministry of Finance to obtain and evaluate quality data . b. M&E Implementation: Implementation was extensive and robust . As discussed in Section 3.b, the monitoring framework would have been improved had there been some attempt to measure indications of service delivery improvement that would be expected to follow from these process improvements . While perhaps not feasible to do this at appraisal, the extension of the project and its counterpart contribution should have made this possible . The project also failed to conduct a final stakeholder survey or final seminar, though there are plans to discuss the project results at a workshop for regional CEOs in spring 2012, and to conduct surveys in the future to provide important feedback benefits. Furthermore, the introduction of program budgeting starting in 2013 will provide a good opportunity for designing a robust evaluation framework to measure the impact of better fiscal management on service quality . c. M&E Utilization: As discussed in Section 4, there was monthly monitoring of regional fiscal performance, annual reports on each region published online, and performance agreements reached to address shortcomings in regional performance. Monitoring also fed into selection of regions for competitive grants supported by FFRFR, and was the foundation for the deepening of fiscal reforms after the project extension in 2004. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The operation supported improvements in fiscal management that do not have direct effects concerrning safeguard issues, and the operation was not required to be classified in a safeguard screening category . The project was classified in Category C for the purposes of environmental assessment . b. Fiduciary Compliance: Financial management and procurement processes were satisfactory . There are no reported issues arising with quarterly financial monitoring, annual audit reports, and procurement, except for requests just prior to loan closure for retroactive clearances for contract amendments or extensions . Counterparts gave their full cooperation, as they appreciated the greater weighting given to quality by the Bank's procurement rules vis a vis the Government's own rules. The rate of loan disbursement fell after 2004, reflecting the increasing counterpart contribution. c. Unintended Impacts (positive or negative): A positive unintended impact was the increased scope and achievement following the project extension, and increased counterpart contribution . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: All lessons cited in the ICR are supported . A crucial lesson is the importance of a mutually reinforcing package of properly sequenced analytical work, technical assistance, policy lending, and parallel assistance from other donors. As part of such a mutually reinforcing package, technical assistance can be vital to build up technical capacities so that subnational units can benefit from the financial incentives available in quick disbursing operations. By focusing on technical rules and frameworks, such support packages can provide a measure of stability and predictability to help maintain technical standards despite the uncertainties of politics . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is comprehensive in reporting on a complex and lengthy operation, and is clearly written . The lessons were instructive and thoughtful . a.Quality of ICR Rating : Satisfactory