ICRR 11816 Report Number : ICRR11816 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 07/09/2004 PROJ ID : P046496 Appraisal Actual Project Name : Soc Prot Impl Project Costs 35.95 29.74 US$M ) (US$M) Country : Russian Federation Loan/ US$M ) 28.6 Loan /Credit (US$M) 26.3 Sector (s): Board: SP - Compulsory Cofinancing pension and US$M ) (US$M) unemployment insurance (65%), Central government administration (19%), Sub-national government administration (16%) L/C Number : L4234; LP322 Board Approval 98 FY ) (FY) Partners involved : Closing Date 12/31/2001 12/31/2003 Prepared by : Reviewed by : Group Manager : Group : Michael R. Lav Alice C. Galenson Kyle Peters OEDCR 2. Project Objectives and Components a. Objectives This Social Protection Implementation Loan (SPIL) was designed to support implementation of the Social Protection Adjustment Loan (SPAL) to gradually develop an effective system of income support and poverty relief for a market economy. A mid-term review which focussed on correcting lagging implementation took the opportunity to build on a reform program implemented by the government by adding a new objective, namely, explicitly support development and implementation of the Government's social protection reform agenda . b. Components This ICR Review is based on the project according to the original components because the project was restructured as a result of poor performance (the PSR prior to project restructuring rated PDO "unsatisfactory" with important delays in implementation). The ICR guidelines specifically state that a project should be assessed on the basis of the revised components "only if implementation was proceeding well but unforeseeable external factors largely outside the borrower's control required a change in project objectives (s)". The original project comprised technical assistance and training in (amounts for each component are actual /latest estimate from ICR Annex 2.): (1) Pension Reform (US$4.34 million): (i) assist the Pension Fund in managing individual accounts and establishing a governing regulatory framework for accounts; (ii) audit the pension fund; (iii) implement an action plan for improving collection of pension contributions; (iv) prepare a detailed concept paper and legislation on systemic pension reform for a three -tiered system; (v) design and implement a public relations campaign; (vi) implement systemic pension reform; (vii) organize actuarial activities; (2) Social Assistance (US$3.33 million): (i) design and implement pilot programs to means test benefits; (ii) disseminate results of the pilots and conduct a national evaluation; (iii) review social benefits, subsidies, compensation and payments; and (iv) review local delivery systems for social services; (3) Social Protection (US$3.14 million) to: (i) put in place a monitoring system of all regional and national social transfers; (ii) analyze date and monitor poverty and nutrition levels; (iii) conduct a national survey of household welfare and program participation, and (iv) review government and non-government programs directed at child protection, the handicapped and the homeless; (4) Child Allowances (US$10.56 million): (i) design and implement programs to transfer payment of child allowances from enterprises to local social assistance offices; (ii) national dissemination of results from pilot programs; and (5) Unemployment Assistance(US$2.08 million) : (i) design and implement a system to monitor unemployment and changes in the type and duration of unemployment, and (ii) draft amendments to the Employment Law; (iii) audit the Employment Fund; (iv) assist the Government to draft a new Labor Code; (v) review programs to support the long -term unemployed, and (vi) evaluate experience with public works programs, and (6) Project Implementation Unit (US$5.29 million). In the revised project, several new components were added in Social Assistance : (1) development of standards in occupational classification; (2) financial analysis of the Social Insurance Fund (SIF) and the Federal Fund for Health; (3) improvement of the operation of the State Medical and Social Examination Service (SMSES) used for management of disability benefits, and (4) assistance to the SIF in the introduction of mandatory social insurance for occupational injury and disease . The revised project dropped two Unemployment Assistance components (evaluation of experience with public works programs, and review of programs to support the long -term unemployed) as these were transferred from national to regional programs. c. Comments on Project Cost, Financing and Dates The project was appraised in July, 1997, approved by the Board on October 7, 1997, made effective on July 1, 1998, and closed on December 31, 2003, two years behind schedule, with a cancellation of US$ 2.35 million. 3. Achievement of Relevant Objectives: All components of the revised project were completed, except as noted below in Section 5. In particular: 1. Pensions: A three-tiered pension system was implemented with individual worker accounts . A new methodology was introduced to calculate state pensions, although further refinements will be needed . While the Government did not select a single actuarial agency as envisaged, it did use several agencies for pension reform modeling activities and for actuarial research and evaluation . Pension delivery costs were reduced through expanded use of Post Offices in the delivery of pensions and a reduction in Post Office delivery charges from 2 percent to 1.5 percent as a result of a review conducted uner the project . 2. Social Assistance: Means tested methodologies were piloted as foreseen, and two Federal Laws on State Social Assistance and on qualification for the poverty benefits were adopted . Regional officials were trained in targeting benefits, and software and other relevant materials widely distributed . 3. Social Protection: Household surveys (NOBUS) on poverty and welfare were carried out . Additional surveys enabled recommendations to be formulated for subsistence food baskets for different regions . 4. Child allowance payments were transferred from enterprises to local social assistance offices based on successful SPIL pilots . Child allowances were targeted based on family income . The federal budget became responsible rather than regional budgets for child welfare thus strengthening the safety net . State managers of Child Welfare programs were trained . 5. Unemployment Assistance: The Employment Fund was integrated into the Federal Budget . Four new Federal Labor Laws were enacted making more stringent the criteria to register as unemployed and participate in unemployment programs. Background drafts of Labor Codes were prepared, and the Duma approved a new Labor Code based on this work. A new occupational classification was drafted and a new methodology to develop occupational standards was designed and tested. 4. Significant Outcomes/Impacts: Pension reform was substantially enhanced by this project, and pension arrears were eliminated (although the lack of acceptance of the financial review methodology means that further reforms /pension management will be more difficult). NOBUS was the first poverty survey made available to those outside the statistical agency, and so was the first which could be examined for objectivity and accuracy . It was found to be of exceptional quality by the Bank . Recommended levels of food support and methodological guidelines for subsistence minimum food baskets for different regions were developed with a high degree of confidence . The improved child allowance payment system enables savings of about 20 percent as multiple payments were eliminated and those not needing such assistance were excluded. With the Federal budget now responsible for Child Welfare payments, arrears were reduced . (Previously, the budgets of the poorest regions with the least resources had the highest incidence of Child Welfare payments resulting in backlogs .) The new labor code should improve the functioning of the labor market . 5. Significant Shortcomings (including non-compliance with safeguard policies): Improvements in targeting social assistance have not been implemented as broadly as envisaged, as decisions on benefits were left to the discretion of regional administrations . The SMSES component lagged badly, was not implemented as foreseen, and most of its funding was cancelled (accounting for most of the US$2.3 million which was cancelled from the loan.) Poor Borrower performance, including the leadership of the PIU, during the first years of the project led to lags in implementation and delayed benefits (which might have been mitigated by effective Bank supervision). 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: 1. This was an extremely complex project which made supervision difficult . The Bank should be wary of projects where detailed technical assistance programs are spread out over so many sectors, resulting in hundreds of contracts to be supervised . 2. If the Bank uses a PIU for project implementation, it needs to ensure that its staff is of high quality (PIU management had to be changed in mid -stream) and that it does not act as a barrier between staff and government ministries. 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: While ICR quality is rated satisfactory because it effectively captures the main outlines of the project, there are important deficiencies. The ICR should have rated the project based on its original concept, as specified in the ICR guidelines. The ICR notes the loan cancellation of US$ 608,815 on account of misprocurement, but does not mention that the total loan cancellation came to US$ 2.3 million, mostly on account of foregone activities in the SMSES component due to serious lags and the decision not to continue extend the project by more than two years after the original closing date. Borrower's comments are unusually extensive and are very helpful in understanding the project . Finally, the ICR presents inconsistent cost data in the text and annex tables .