ICRR 11760 Report Number : ICRR11760 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 05/18/2004 PROJ ID : P035761 Appraisal Actual Project Name : Community Social Project Costs 288.15 176.56 Infrastructure Project US$M ) (US$M) Country : Russian Federation Loan /Credit (US$M) Loan/ US$M ) 200 140.24 Sector (s): Board: ED - Health (36%), Cofinancing na na General education sector US$M ) (US$M) (32%), General water sanitation and flood protection sec (22%), Sub-national government administration (7%), Central government administration (3%) L/C Number : L4009; LP275 Board Approval 96 FY ) (FY) Partners involved : None Closing Date 12/31/2002 09/30/2003 Prepared by : Reviewed by : Group Manager : Group : Nalini B. Kumar Roy Gilbert Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The principal goals of the project were to (i) Reduce the deterioration of social infrastructure by financing the rehabilitation and limited replacement of high priority facilities for providing health care, education, water supply and sanitation; (ii) Improve the efficiency of managing public resources by ensuring that the selection of subprojects is based on sound criteria; raising facility design standards and improving the incentives to observe them; improving the quality and sustainability of construction by improved site supervision, and by encouraging beneficiaries to take a more active role as clients during the design and the construction process; introducing competitive procurement practices; introducing improved methods of expenditure programming and budgetary management by public authorities; strengthening environmental management practices through a program of studies and training linked to the health, water supply and sanitation components; (iii) Promote private sector development through encouraging the break -up and commercialization of existing design institutes into independent engineering and architectural firms; providing training in procurement processes for engineers, architects, local construction firms, and public bid evaluation committees; and providing an environment that can facilitate the emergence of private providers of medical care . (iv) Contribute to the Government's policy of decentralization by promoting broad participation, including non-governmental participation, in project decision -making. (v) Monitor the effectiveness of new practices, and disseminate the results, to promote their use in similar activities elsewhere in the Russian Federation . It was expected that the project design would be replicated across the country . As designed, the project was ambitious and complex . It had to be downsized during implementation . Objective (iii) to promote private development was dropped when the loan agreement was amended in September 1999. Other aspects of the project were also modified, for example the SAR had envisioned substantial institutional development of health systems in both regions and also expected loan repayment for the health component by health providers . These expectations were overly optimistic and were dropped . Further, an institutional study for the water sector reform options in the two regions was not acceptable to authorities in the two regions and was replaced by a TA on energy efficiency and conservation in the water sector . b. Components The project had five components (with appraisal estimates and actual costs in parenthesis ). (a) Education (app. US$ 64.1m., act. US$ 38.9 m.); The component rehabilitated 29 schools in the two oblasts. In both oblasts the project assisted the Department of Education in various aspects of facility management, including identification, preparation and implementation of education subprojects . (b) Health (app. US$ 72.1m., act. US$54.1 m.); The component financed the rehabilitation and upgrading of selected city and rayon hospitals and polyclinics . Two sites, one in each oblast, were chosen as pilots prior to full project implementation. (c) Water and Sanitation (app. US$44.5 m., act. US $ 34.8 m.); The component was targeted at rehabilitation, refurbishment and renewal of the existing infrastructure, including rehabilitation of water mains to reduce leakage, refurbishment of chlorination installations to provide better security in drinking water disinfection and renewal of electrical and mechanical pumping . (d) Budget and Expenditure Management (app.US $3.3 m., act.US$ 3.6 m.); The component was to provide support for identification of sources of local revenue, budget formulation and expenditure management . It was transferred to another PIU (see section 3 below). (e) Monitoring and Management (app.US$16 m., act.US$ 11 m.) The component focused on lessons for project replication and assistance in preparation of future projects . c. Comments on Project Cost, Financing and Dates The project was approved in April 1996 and became effective in January 1997 but actual start up was only in January 1998. US$45.5 million of the loan was cancelled in 1999 at the request of the Borrower including a part that was not specified in detail during preparation and was intended to be identified during implementation . At the request of the Ministry of Finance, the Budget and Expenditure Management component was transferred to another project implementation unit (PIU) to become an independent World Bank project . In July 2001, another US$6.5 million was cancelled from the Novosibirsk part of the Loan program due to inadequate local counterpart funds . The Loan closing date was extended from the original date of 12/31/2002 to 09/30/2003 to complete the investment program as construction works could be completed only during the warm season in both regions and also because of some delays in procurement of medical equipment for the Novosibirsk oblast . 3. Achievement of Relevant Objectives: The achievement on the five objectives was as follows : (i) Achieved . The project was successful in rehabilitating social infrastructure in the two pilot regions --though the number of facilities to be rehabilitated was reduced after the restructuring . (The number of schools to be rehabilitated was reduced from 36 to 29, hospitals from 11 to 9 and water and sanitation sub -projects from 61 to 53.) Improvement in facilities like oblast hospitals that served the entire region and water utilities benefited a high proportion of the population. (ii) Partially achieved . The Budget and Expenditure Management component was transferred to another PIU and it is not clear from the ICR whether it left behind any improved methods of expenditure programming and budgetary management. In addition, schools and hospitals did not have clear incentives for achieving cost saving and hence project supported energy and cost saving programs were of little use . However, the project created a transparent mechanism for government tenders for public works and established sound criteria for the selection of socially important investment projects at the regional level . It also helped raise facility design standards and improved the incentives to apply them in the future . (iii) Objective dropped through amendment to the Loan Agreement . (iv) Partially Achieved . The project included beneficiary participation at all stages of sub -project selection, design and implementation. In both oblasts, local municipalities, administrators of health, education and communal services were fully involved in selection of project sites, in discussion and approval of the design, contract award and in supervision of the contract implementation . However the ICR reports that some components were not discussed in sufficient detail with Russian counterparts with consequent delays and difficulty in achieving objectives . (v) Partially Achieved . Monitoring reports were produced twice a year and presented to the Bank and to all involved parties in the Government. Monitoring included surveys of beneficiaries, public discussions of the project outcomes and participation in sector development programs . The project results were well publicized among authorities in other regions but the project experience could not be replicated as the project approach became outdated after regional finance reform took place in 1998. 4. Significant Outcomes/Impacts: The project helped raise the local priority of social investment programs . Before 1997 most social investment programs received the lowest priority . The project also resulted in the attraction of substantial additional funds into social infrastructure in both regions . The project provided oblast and municipal authorities with tools for broader participation including participation of NGOs and groups protecting public interests through a set of hearings and public discussion of the project priorities and outcome. A significant number of schools and hospitals were rehabilitated . Organizational reforms in the hospital sector were promoted through the development and implementation of a new organizational concept for a rayon (district) hospital. Significant number of workshops and seminars provided training to a large number of professionals in various aspects of sector reform and project implementation . A transparent mechanism for government tenders for public works was created . 5. Significant Shortcomings (including non-compliance with safeguard policies): Over ambitious project design out of tune with reality on the ground : little strengthening of the capacity of regional and municipal administrations; Several institutional reforms visualized under the project were dependent upon changes in the Federal legislation and policies in health, education and water and sanitation sectors . As a result several specific sectoral goals were not met, such as health reform and tariff regulation for the municipal water sector . Economic validity and priority of some sub -projects is questionable. Many sub-projects were not part of the investment plans of the local utilities . Misprocurement of a major contract despite improvements in procurement procedures . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory Achievement of most project objectives was partial at best.Weak Bank and Borrower performance prevented the project from attaining a fully satisfactory outcome. The option of rating a project moderately satisfactory is not available under the ICR four point rating scale . Institutional Dev .: Substantial Substantial Sustainability : Likely Non-evaluable Several critical policy reforms could not be implemented. These have bearing on the sustainability of efforts to reduce deterioration of social infrastructure . The Budget and Expenditure management component was transferred to another unit and it is not clear how much success was achieved in introducing improved methods of expenditure programming and budgetary management by public authorities within activities undertaken under the project. Bank Performance : Satisfactory Unsatisfactory Quality at entry was highly unsatisfactory . The Bank should have used its vast international experience to make an assessment of what could and could not be realistically implemented. The borrower was not sufficiently involved in project preparation. The project was largely prepared by international consultants who overestimated the project's capacity to influence the social infrastructure management reform at the federal level. Bank action in resolving the issue of counterpart funds was also inadequate. There was also lack of continuity in Bank task management . Borrower Perf .: Satisfactory Unsatisfactory Project implementation was negatively affected by several reorganizations of the federal implementing agency, lack of stability in project management at the regional level, inadequate provision of counterpart funds in Novosibirsk and misprocurement. Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: Based on the ICR the key lessons are : (i) A multi sectoral project can be complex and challenging to implement . Designing of such projects requires a thorough assessment of the institutional capacity in the country . Where capacity is limited, there may be a strong advantage in limiting an investment that aims at infrastructure development and significant institutional change to just one sector. (ii) The demonstration effect of Bank projects on the government, particularly in terms of quality of construction and rehabilitation of infrastructure can be immense . The improved architecture, design and materials brought by the Bank project helped develop an understanding among local officials of the importance of quality and the linkage between quality, durability and consequent cost savings . (iii) The importance of making the design of regional projects consistent with the federal government reform agenda cannot be underestimated. In the current project, several critical institutional reforms could not take place as they were dependent upon changes in the Federal legislation and policies in health, education and water and sanitation sectors that did not take place . (iv) Instability of the project implementing and executing agencies can undermine project outcomes . In the current operation the federal implementing agency went through several reorganizations and the administration in the Novosibirsk oblast also changed twice . 8. Assessment Recommended? Yes No Why? For several reasons: (i) To verify the outcome, sustainability and institutional development impact; (ii) The operation was the first multi-sectoral project and the first World Bank project with investments in any type of social infrastructure in Russia managed and implemented at the regional level, and its experience would provide important lessons for the implementation of other similar operations in the country; (iii) To draw lessons of experience for other transition economies which are facing similar problems of deterioration of social infrastructure . 9. Comments on Quality of ICR: The ICR is satisfactory but for four shortcomings : (i) The Budget and Expenditure Management component was transferred to another PIU and the ICR does not clarify whether the project was able to bring about any improvement in methods of expenditure programming and budgetary management which were critical to improving the efficiency of managing public resources; (ii) There are some contradictions in the report --for example, para 4.1 notes that the project financed the rehabilitation of high priority facilities . Whereas, under para 3.5 the report notes "many sub-projects were not part of the investment plans of the local utilities, and thus their priority was considered questionable." (iii) It does not explain why the objective to promote private sector development was dropped . This is particularly puzzling since the report provides substantial evidence on how the project supported private sector development. (iv) Target figures of performance indicators are reported as being exactly what was achieved . The ICR should have explained why and how there was a perfect match . The exact match otherwise raises credibility concerns (ICR pages 10 and 29).