ICRR 13437 Report Number : ICRR13437 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/11/2012 PROJ ID : P064238 Appraisal Actual Project Name : Northern US$M ): Project Costs (US$M): 95.24 47.86 Restructuring Project Country : Russian Federation Loan/ US$M ): Loan /Credit (US$M): 80.00 34.69 Sector Board : SP US$M): Cofinancing (US$M ): Sector (s): Other social services (88%) Central government administration (8%) Sub-national government administration (4%) Theme (s): Improving labor markets (33% - P) Municipal governance and institution building (33% - P) Other economic management (17% - S) Social analysis and monitoring (17% - S) L/C Number : L4611 Board Approval Date : 06/07/2001 Partners involved : Closing Date : 09/30/2005 09/30/2009 Evaluator : Panel Reviewer : Group Manager : Group : Robert Mark Lacey Soniya Carvalho IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: According to the Project Appraisal Document (PAD, page 2), “the project development objective is to test a set of actions that will allow participating municipalities in the Russian North to realize the benefits of economic restructuring and facilitate the implementation of sustainable municipal policies; and thereby to assist the Government in the development of a national strategy for restructuring of the economy of the northern areas .� The statement of objectives according to the Loan Agreement is virtually identical : “the objectives of the Project are to: test measures that will allow Participating Localities to realize the economic benefits of restructuring and facilitate the implementation of sustainable municipal policies; and thereby to assist the Borrower in the development of a national strategy for restructuring of the economy of the northern areas .� This Review is based on the statement of objectives in the PAD . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project had five components : 1. Migration assistance (US$76.23 million at appraisal, US$30.72 million at closure). This component was to finance out-migration assistance to specified population groups in the participating regions . The allowances were to help defray the costs of transportation, including that of household effects, and of alternative housing elsewhere in the Russian Federation. The component was also to support the development of an information system about housing, education, health care and employment possibilities in different regions of the country, as well as a public awareness campaign about the assistance available . 2. Local restructuring support (US$5.7 million at appraisal, US$7.31 million at closure).This was to assist the participating municipalities to reduce their dilapidated and unused housing stock, and to support the implementation of reforms in housing and utility services management . The component included: (i) technical assistance to municipalities to ensure compliance with the Environmental Management Plan when demolishing abandoned housing and other facilities; (ii) technical assistance for institutional reforms in housing management; (iii) training for the staff of participating municipalities and regions in municipal management; and (iv) technical assistance in other critical areas identified during implementation . 3. Monitoring and evaluation (US$1.2 million at appraisal, US$0.36 million at closure). This component was mto support the elaboration and implementation of an M&E system that was to address real time management needs as well as maximizing learning from the project -supported initiatives. The system was to include: (i) exit surveys of migrants about the services provided by the program; (ii) tracer studies of migrants following their departure; (iii) annual studies of the economic and fiscal impact of the local restructuring plans in the municipalities; and (iv) a review of the cost of housing and the program ’s effects on the housing markets in the migrants ’ destination regions. 4. Federal policy (US$3.34 million at appraisal, US$3.28 million at closure). This was to support the introduction of new procedures aimed at achieving the gradual deregulation of the Northern economy . It was to include: (i) advisory services to the Federal government and it agencies, the legislature, and regional authorities on legal, economic, fiscal, and social aspects of restructuring; (ii) funding for research on issues such as the interaction of the Northern labor market with the rest of the economy; (iii) dissemination and consensus building activities based on project experience. The component would also support the development of a national information system on housing and other relevant regional services . 5. Project management (US$7.90 million at appraisal, US$5.39 at closure). This financed the operating costs of the Central and Local Project Implementation Units . There were no changes to the components . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project cost . Total project cost at closure was US$ 47.86 million, just under 50% of the appraisal estimate of US$95.24 million ( both including an IBRD front end fee of US$ 800,000). The reasons for this are as follows : According to the ICR (page 11), implementation can be divided into two distinct phases . In the first, from effectiveness in August 2002, to October 2007, the project was basically on track, with the financing scheme working well, the Project Implementation Units working effectively, the technical assistance proving helpful, and the project providing a comprehensive approach to voluntary outward migration from the North . During this period, three waves of outward migration were completed . However, this description of the first phase does not altogether coincide with that given in the Borrower ’s ICR (ICR, Annex 7) which refers to slow progress and slow disbursement during the first three waves of outward migration, and the consequent need to adjust incentives and procedures in the assistance program . In the second phase, from November 2007 onwards, the ICR and the Borrower's ICR agree that implementation was characterized by Government inaction, delays, and the failure to initiate the proposed fourth wave of outward migration. In March 2007, the Bank had proposed a level one restructuring which would (i) extend the closing date by 30 months to September 30, 2009 to enable a fourth wave of outward migration to be implemented; (ii) expand the project’s scope beyond the original participating localities; (iii) increase the subsidy paid to outward migrants to maintain the subsidy’s real value and the attractiveness of the migration assistance scheme; (iv) introduce uniform mandatory provisions for transportation costs from local budgets; (v) increase government cofinancing for component 4 (Federal Policy); and (vi) transfer responsibility for the financing of the local Project Implementation Units to the participating localities . The Government did not formally agree to the proposed restructuring until October 22, 2008. Since the implementation of the fourth outward migration program would have required at least 18 months, it was decided that a further extension, to March 2011, would be requested. By September 2009, according to the ICR (page 12), only three (according to the Borrower’s ICR, two) out of six agreed actions needed to initiate the fourth wave of outward migration and the extension of the closing date had been completed. The Government did not send the request for extension, and in December 2009, asked the Bank to cancel the undisbursed balance of the loan, amounting to US$ 45.31 million. The Borrower’s ICR (ICR, Annex 7, page 58), reports that “the generic problem causing implementation delays was related to long internal and inter -agency agreement of [i.e. disagreement about] key issues and Project documents, which has led to a 2-year suspension and finally to the closure of the Pilot Project .� However, the project team subsequently explained to IEG that there were three main factors leading to the problems which culminated in the cancelation of the project . First, the strong personal and institutional leadership from which the project benefited in the preparation and early implementation phases diminished when the Vice Minister of Economic Development and Trade left for another position . Shortly thereafter (in August, 2008) there was a shift in responsibility for project oversight from the Ministry of Economic Development and Trade to the Ministry of Regional Development . The latter was not consulted concerning the revision of project documents until a very late stage . Second, the favorable transformation in the Russian Federation's macroeconomic situation following the rise in energy and other commodity prices in the latter half of the last decade alleviated the fiscal pressure to reduce the burden of northern subsidies and other benefits . Third, the incentives for individuals to migrate from the North were de facto reduced, both by the fact that the fiscally strengthened Federal and local governments were now actually able to deliver on their obligations -- for example by paying pensions and subsidies fully and on time -- and by the economic fall-out in the North of higher prices for the commodities produced there, in particular, gold, nickel, copper and coal . Financing . The project was entirely financed by the Bank and the Borrower . As noted above, $45.31 million of the US$80 million loan (57%) remained undisbursed at closure, and was cancelled . Borrower contribution . The Borrower’s contribution was US$13.17 million, US$0.7 million (5%) less than the US$13.87 million anticipated at appraisal. The ICR (page 27), notes that the pilot municipalities contributed more counterpart funds than envisaged, though no figures are provided . Dates . The project suffered serious delays in becoming effective . Effectiveness, which was expected on October 11 2001, within 90 days of Board approval, did not in fact take place until August 15, 2002, because of the time taken for internal and inter-agency agreement over the documents that were to be prepared as effectiveness conditions . The original closing date of September 30, 2005, was twice extended during implementation by an accumulated total of four years, thereby doubling the originally envisaged four year implementation period . The first extension, from September 30 2005, until December 31, 2006, was granted to compensate for the one year delay in effectiveness, and to allow enough time for the implementation of the third wave of outward migration . In March 2007, a second extension of 30 months until September 30, 2009, was granted, as noted above, in order to carry out a level one restructuring and allow sufficient time for the implementation of a fourth wave of outward migration . 3. Relevance of Objectives & Design: Relevance of Objectives : Substantial . The latest Bank strategy document for the Russian Federation available at project closure was the July 2009 Progress Report. This concludes that the four pillars of the 2006-2009 Country Partnership Strategy were still relevant for the Fiscal Years 2010 and 2011. The project’s objectives were relevant to two subsets of one of these pillars (“Sustaining Rapid Growth�) -- “Correct regional imbalances� and “Maintain macroeconomic stability.� Migration policy also features under the human capital development pillar of the current 2012-2016 Country Partnership Strategy, although the 2009 Progress Report had recognized (page 25) that the migration process out of the underdeveloped northern territories had stalled, and demand did not develop as expected for regional growth strategies. The objectives were also relevant to the Government ’s Program of Priority Measures encapsulated in its 2001 Letter of Development Policy for the Russian North . Actions under this program included : (i) new income tax legislation eliminating mandated salary supplements and augmenting benefits for private sector workers in the North; (ii) legislation removing the funds for Northern privileges; (iii) planned cuts in mandatory benefits that would further reduce the federal subsidy; and (iv) elimination of mandatory benefits for non -public sectors by early 2002. Together these actions were to place the North on a more equal footing with other Russian regions, and reduce its attractiveness for in-migration. Relevance of Design : Modest . The PAD contains no Results Framework that links in a logical and coherent manner the activities funded by the project to the outputs and outcomes related to the attainment of the development objectives . Most of the activities financed by the project were geared to encouraging outward migration . Those directed towards enhancing strategic and policy decision making appeared insufficient to achieve the goals of implementing sustainable municipal policies or developing a national strategy for restructuring the economy of the northern areas. Although the extent of the demand for outward migration support and services was unknown, and the project was meant to test approaches, design lacked flexibility in two important respects . First, the definition of eligible categories of beneficiary was too narrow . Many of those deemed eligible demonstrated low mobility because of health factors or because they lacked the resources to cofinance housing in a new location . Some were benefiting from other government resettlement programs, thereby diminishing their interest in what the project had to offer. Second, the Loan Agreement and the Federal budget specified a fixed amount of financing for each participating locality without the option to reallocate funds among municipalities on the basis of actual demand . This latter issue had to be addressed during implementation through amendments to the Loan Agreement Whereas the project development objective was stated in terms of developing a national migration strategy through testing of different measures, the outcome target focused on reducing local budget expenditures through outward migration, while the outputs of the project were designed to facilitate and promote such migration. The proposed implementation period of four years was unrealistic . Design failed to learn from the experience of substantial delays in effectiveness and implementation of previous projects in the Russian Federation . 4. Achievement of Objectives (Efficacy): Test a set of actions that will allow participating municipalities in the Russian North to realize the benefits of economic restructuring and facilitate the implementation of sustainable municipal policies . Modest . Outputs A migration assistance program was introduced . It included eligibility criteria, a housing certificate, a specific formula for determining the amount of housing subsidy, and logistical assistance to socially vulnerable persons . The formula for determining the size of the housing subsidy was changed in a manner conducive to budgetary savings. Whereas, under previously existing legislation, the calculation was based on the average square meter cost of housing in the region selected by the migrant, under the project the subsidy was based on the Russia-average price of standard housing . This removed an incentive to select the most expensive recipient regions. Municipalities tested and implemented procedures of selection and registration of participants and calculation of their benefits. Information and public awareness campaigns were developed at the municipal level including advertising and distribution of written material concerning housing, employment and public services in recipient areas . A comprehensive information package was also provided to registered migrants . Transportation was provided for participants and their household effects . Municipalities received technical assistance in inter -budget relations, reform of housing and communal services and demolition of housing and other facilities . Training in municipal management was provided at regional and municipal levels . 12 settlements were closed. A total of 642 buildings (404,689 sq meters) of consolidated housing and social infrastructure, including 60 km of heating networks and 29.6 km of water supply networks, were decommissioned. Intermediate Outcomes 4,527 households (10,028 persons) migrated outwards from the North, significantly below the projections contained in the PAD (see table). The proposed fourth wave of migration did not take place . Pilot territories Projected out -migration (1) Actual out -migration (2) (2) as % of (1) Households Persons Households Persons Households Persons Susuman 2,114 6,002 1,428 3,338 68% 56% Norilsk 4,599 15,105 623 1,152 14% 8% Vorkuta 3,688 6,422 2,476 5,538 67% 86% Total 10, 10,401 27, 27,529 4,527 10, 10,028 44% 44% 36% 36% Source: ICR, page 12. Cumulative budgetary savings resulting from the outward migration and accompanying measures had reached an estimated US$20.7 million equivalent by September 2009 (that is, over a seven year period ), compared to the appraisal estimate of US$25 million annual budgetary savings (ICR Data Sheet, page iii). At appraisal, savings equivalent to about 7.5% of annual municipal budgetary expenditures were expected . Actual annual savings were equivalent to about 0.9% of annual municipal budgetary expenditures . Outcomes The set of actions that the project supported and tested consisted of a comprehensive approach to assist voluntary out-migration from the North, which supplemented existing migration programs . The numbers of outward migrants and accumulated budgetary savings were considerably below the appraisal projections . No evidence is presented in the ICR to show that these lower levels of savings would suffice to attain the objectives of facilitating the implementation of sustainable municipal policies or of enabling municipalities to garner the benefits of economic restructuring . The Borrower's decision not to proceed with the fourth wave of outward migration and to close the project with more than half the loan amount undisbursed, may be interpreted as the logical outcome of a pilot exercise that did not produce the hoped-for results, although the results could have been better monitored for lesson learning (see Section 10 below). To assist the Government in the development of a national strategy for restructuring of the economy of the northern areas . Modest . Outputs Support was provided for the implementation of Federal Law 125, the legal framework for the provision of migration certificates and targeted support of socially vulnerable groups . Project-financed analytical work supported preparation of various Federal Government resolutions . The work focused on issues such as zoning, assessment of the potential effectiveness of restructuring the economy of the Far North, and analysis of the factors determining the volume of outward migration and resettlement . Support for applied research on issues such as the interaction of the Northern labor market with that of the rest of the economy led to the production of more than 60 analytical notes and reports . Two information systems (on the housing market and on migration ) were developed under the Project . They included information about the real estate market, regional labor markets, healthcare, education and training opportunities, climate in recipient regions, etc . Outcomes Although the outputs described above were probably helpful to policy makers, there is no evidence presented in the ICR that the project assisted in the development of a national strategy for restructuring the economy of the northern areas. 5. Efficiency (not applicable to DPLs): The PAD and the ICR both performed a cost benefit analysis of the project, though the PAD calculated a net present value (NPV) only (of US$239 million), while the ICR estimated both an NPV (of US$9.8 million) and an economic rate of return (28%). There were three clusters of quantified benefits at appraisal : (i) fiscal savings from reduced public expenditures resulting from outward migration; (ii) non-fiscal or quasi-fiscal savings resulting from lower levels of support for residents in the northern areas (these were mostly difficult to identify and assumed or imputed ); and (iii) increases in the economic well-being of migrants (for example, for some social groups, the reduction in migrants' incomes due to the elimination of benefits for dwelling in the North were more than compensated for by the lower cost of living in the recipient regions). At closure, only direct benefits in the form of identifiable fiscal savings and increases in migrant well-being were included. Costs were those of the project and were equal to the disbursements of the loan plus the counterpart contributions . The very considerable reduction in the NPV from appraisal to closure is due mainly to : (i) less than half the anticipated number of migrants; (ii) lower per capita budgetary savings; (iii) a pay-back period of seven years at closure compared to just over three years at appraisal; (iv) much lower benefits arising from the well -being of migrants (US$5.2 million per year at appraisal and US$790,000 altogether at closure); and (v) the inclusion in the benefit calculation at appraisal, but not in the ICR, of assumed or imputed benefits such as reductions in overdue payables (US$6.3 million per year), reductions in quasi-budgetary expenditures (US$5.4 million per year), and lower expenditures from extra-budgetary funds (US$359,000 per year). Per capita migrant annual benefits were calculated as US$3,187 at appraisal and US$703 at closure. In view of the lower than anticipated benefits (both in absolute and per migrant terms ), the cancelation of over half of the loan amount, and the fact that implementation took double the time estimated at appraisal (due principally to administrative inefficiencies), efficiency is assessed as modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate Yes 28% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Project objectives were substantially relevant, although outward migration from the North has subsequently stalled and does not feature in the 2012-2016 Country Partnership Strategy. Design relevance was modest - there was an incongruity between the expected outputs and the full statement of objectives . Regarding efficacy, the project did test a series of actions designed to encourage outward -migration, but the results were below expectations . Evidence is presented on the budgetary savings resulting from outward migration, but there is no evidence to demonstrate whether these savings would be sufficient to facilitate the sustainability of municipal policies . There is, moreover, no evidence that the pilot exercise has assisted the Government in developing a national strategy for economic restructuring of the North. Both parts of the project development objectives are, therefore, rated modest . Efficiency is rated modest due to a significant fall in the NPV and per capita benefits between appraisal and closure, and to administrative inefficiencies resulting in a doubling of the originally estimated implementation period . These constitute significant shortcomings, and overall outcome is assessed as moderately unsatisfactory . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome is considered to be high: high The Federal Authorities canceled the Loan with more than half the proceeds undisbursed, and with the planned fourth wave of outward migration uninitiated . The 2009 CAS Progress Report states that the demand for regional restructuring has diminished and such restructuring of the Northern Region, accompanied by further planned outward migration, does not feature in the 2012-2016 Country Partnership Strategy. The “champion� of the program within the Russian Federal Government was the Ministry of Economic Development and Trade. When that Ministry experienced high staff turnover, and responsibility for the project and the program it supported was transferred in 2008 to the Ministry of Regional Development, the Government ’ s interest in the program began to wane . The data bases on housing and related services -- which the ICR (page 22) describes as being of high quality -- have not been utilized by the Ministry of Regional Development (responsible since August, 2008, for the outward migration and restructuring program ), nor by any other Federal or regional institution . The four points above are indicative of a significantly diminished interest on the part of the Authorities of the Russian Federation in the program . They are, therefore, highly unlikely to allocate resources to ensure its continuation or the preservation of the outcomes of the pilot exercise . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: Quality at Entry On the positive side: In order to build the project's information base and improve understanding of the status of potential migrants and those who had already departed from the North, a considerable body of analytical work, promoted by the Bank, had been completed. This included two socio-economic studies followed by a socio -economic survey. The focus on testing an approach and the choice of a limited number (three) localities for the pilot exercise were appropriate. Design incorporated important shifts in the policy of encouraging outward migration, including a housing certificate scheme that allowed individuals to choose the location of their future residence rather than having it chosen for them, introduction of greater transparency in selection criteria and allocation of benefits, and elimination of inappropriate incentives (such as those encouraging migrants to settle in the most expensive regions). However: As noted in Section 3b above, there was an incongruity between the project development objectives and the instruments chosen to achieve them . The outcome indicators were not suitable for measuring progress towards attainment of the objectives . Several indicators lacked targets . These and other weaknesses in the design of the M&E system (see Section 10 below) were particularly significant for a project that was meant to be a learning exercise. The implementation period of four years was too short, especially in the light of experience of other projects in the Russian Federation, many of them less innovative than this one . In particular, the extent and intensity of internal procedures in the Federation required to reach the necessary degree of consensus concerning changes in legal and policy documents governing project implementation were underestimated . There was over-reliance on the Ministry of Economic Development and Trade as the internal "champion" of the program supported by the project . With hindsight it is clear that the effort to build consensus among other government stakeholders was insufficient . After the Ministry no longer held responsibility for the project, official interest waned markedly. Although, as noted, the decision to focus on only three locations for the pilot exercise was appropriate, the choice of locations could have been made more carefully . The Bank took insufficient account of the factors in Norilsk (from where more than half the projected number of migrants were expected to originate ) which would reduce the incentives for outward migration . These included relatively high living standards and quality of social services. The beneficiary eligibility criteria and the inability to reallocate funds in accordance with changing demand patterns led to unnecessary inflexibility and the need for amendments to the Loan Agreement . No provision was made at the preparation stage for a mid -term review. Supervision There was good continuity (the same Task Team Leader remained with the project from preparation through implementation). Team composition was appropriate and included a social scientist and urban specialist in the early stages of implementation . Financial management issues received systematic and timely attention . Considerable efforts were made to promote close collaboration and smoother coordination within and between the Federal and local authorities involved . Equally intense were the Bank's attempts to to expedite implementation after it began to slow down in late 2007 and 2008. As well as the constant exertions of the project team, the ICR (pages 24-25) lists eleven specific actions including special missions, letters and deputy ministerial level meetings with the Country Director . It appears as if the Bank made all possible efforts to try and get the project back on track . However, no mid-term review was undertaken. Such a review would have provided an opportunity to assess the prospects for continuation and broadening of the pilot exercise, and to restructure either the results framework or the objectives to make them more congruous with each other . at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government During preparation and in the early stages of implementation, government commitment to the project's goals, in particular on the part of the responsible Ministry of Economic Development and Trade, appeared strong . As noted, this diminished significantly in the later stages . Even in the early phase, however, there were delays, with over a year elapsing between Board approval and effectiveness . An Inter-Ministerial Working Group provided good guidance early on to the different Federal and local agencies involved. However, after the transfer of responsibilities for project oversight to the Ministry of Regional Development, changes in the Group's composition were necessary and it met only once (in April 2008). The substantial delays in reaching consensus within Government on the procedures and documents necessary for the continuation of the project, expansion of its territorial scope (within the northern areas) and further extensions of the closing date, led to curtailment of the work and cancelation with more than half the Loan proceeds undisbursed . Implementing Agencies The implementing agencies at the Federal level were the Ministry of Economic Development and Trade until the transfer of project oversight in August 2008, and the Ministry of Regional Development after that . At the local level they were the participating municipalities . There was a Central Project Implementation Unit (PIU) housed in the Ministry and three local PIUs housed in each of the participating municipalities . During preparation and the early stages of implementation, the Ministry of Economic Development and Trade demonstrated strong leadership with the Deputy Minister participating actively in mission discussions and field visits. However, following changes of personnel at the Ministry, and then the transfer of project oversight responsibilities, implementation was characterized by poor coordination between agencies, inability to reach consensus on key project documents, and lengthy processing delays . For example, an agreement authorizing the Foundation for Enterprise Restructuring (an agency dependent on the Ministry of Finance ) to implement the project on behalf of the Ministry of Regional Development was not signed until February 2009, seven months after the transfer to the latter of oversight responsibility . Against this difficult background, the Central PIU performed as well as could have been expected, generating weekly monitoring reports and periodic project management reports . The Unit’s financial management and procurement activities were carried out efficiently (ICR, page 27). The participating municipalities demonstrated strong commitment to the local restructuring, economic and social goals of the project. This was demonstrated by their willingness to contribute a higher counterpart funding than originally envisaged . The ICR (page 27) reports that the Local PIUs were “particularly effective and pro-active in supporting beneficiaries at all stages of the application process for out -migration assistance, including counseling and providing information to participants, and tracking their progress through each stage of the out -migration process.� a. Government Performance :Moderately Unsatisfactory b. Implementing Agency Performance :Moderately Unsatisfactory c. Overall Borrower Performance :Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: Design There were a number of weaknesses in M&E design . Whereas the project’s objectives sought as an outcome the development of a national migration strategy through testing of different measures, the outcome indicator focused on reducing expenditures of the local budgets resulting from out -migration and output indicators focused on the actual number of beneficiaries. The M&E system also failed to include targets for most indicators . There was no indicator capable of capturing what the ICR (page 13) calls “the underlying project objective of reducing human suffering under extremely harsh climate conditions .� Operation of the M&E system was to be the responsibility of the Central and Local PIUs. Implementation Although no attempt was made during implementation to improve the monitoring framework, a number of M&E activities were developed to monitor various aspects of project implementation . M&E was supported by a comprehensive electronic data base, designed for the project, which provided timely information on implementation progress, the economic and fiscal impact of project activities, social aspects of migration, and housing purchases by migrants in the recipient regions. The monitoring of social dimensions was carried out through a series of three qualitative social assessments, all related to the first two waves of migration . Information gathered from these exercises was used in order to enhance implementation of the next wave -- for example, to broaden eligible categories of migrant, adjust the incentives package, and respond to shortcomings in the information campaigns . Three quantitative social assessments were also carried out . The first (November, 2003) measured beneficiary satisfaction with the program. While participants found the data base useful in selecting specific destinations, dissatisfaction was expressed at the downgrading of living standards through moving to less comfortable apartments, and (especially by pensioners ) at the "dramatic reduction or complete loss of of customary social protection offered in the North" (ICR, page 65). The second assessment (September, 2004) was to identify reasons for the unexpectedly high levels of refusal to participate in the pilot project . The main reasons were (i) the lack of mobility among the designated project beneficiaries (see Section 3b above); (ii) the level of assistance offered; (iii) the novelty of the housing certificate arrangement; and (iv) the fear of outward migration to new and unknown places of residence . The third assessment (October, 2005) was to identify the causes of return migration (this covered all return migrants and not just those who had participated in the project; the ICR offers no statistics on return migration ): (i) the wish to work for additional years in the North in order eventually to enjoy a higher living standard elsewhere; (ii) the wish to help relatives (especially children and grandchildren ) who had remained in the North; (iii) difficulties in adapting to new locations; and (iv) the wish to earn a "Northern Bonus" as an addition to pension benefits . No further surveys were carried out after 2005 as there was no fourth migration wave . Utilization The ICR contains no information on utilization of data gathered beyond that mentioned above . There is no evidence that the data from the project-supported pilots have been used to develop strategies for outward migration from the North or for regional economic restructuring there . As noted in Section 7 above, the data bases developed have not been used in the work of relevant Federal Ministries . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards According to the PAD (page 30), the only safeguards policy to be triggered was Environmental Assessment (OP 4.01). The project was classified as Category "B" for purposes of environmental assessment . The ICR (page 16) reports that an Environmental Management Plan (EMP) was developed for the project to cover the consolidation and demolition of dilapidated housing and social facilities . The project did not deal with any industrial sites . Project-financed technical assistance supported the implementation of the EMP, including the disposal of hazardous materials and rehabilitation of sites following demolition . The ICR reports (page 16) that "supervision reports from project consultants confirm that the EMP was followed in all localities, " and on the basis of these reports rates safeguard implementation as satisfactory . Fiduciary According to the ICR (page 16), financial management was rated satisfactory or highly satisfactory during the life of the project. Reviews of financial management arrangements in the Central PIU demonstrated that the financial management system, including accounting, internal controls, and reporting, was effective . Financial records and reports were submitted regularly. The ICR (page 16) reports that all external audits were prepared on time and with unqualified opinions. Prior to the closing date, the Central PIU signed a contract for the final audit which was underway at the time the ICR was completed . The project team confirmed that the final audit was also unqualified . The ICR (page 16) reports that the Central PIU managed procurement effectively . No cases of misprocurement were reported. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Significant High There is evidence of a significantly Outcome : diminished interest on the part of the Authorities of the Russian Federation in the program. They are, therefore, highly unlikely to allocate resources to ensure its continuation or the preservation of the outcomes of the pilot exercise. Bank Performance : Moderately Moderately There were significant shortcomings in Satisfactory Unsatisfactory Quality at Entry, including an incongruity between the project development objectives and the instruments chosen to achieve them; over-reliance on one internal "champion" of the program supported by the project; an over-ambitious implementation period; and insufficient flexibility in design. Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The following lessons are selected from those presented in the ICR with some adaptation of language : When an operation supports the formulation and implementation of socially sensitive policies, then efforts should be made during preparation to extend and deepen consensus for such policies as much as possible among all concerned stakeholders, especially within government . Relying on a single "champion," as in this case, leaves the project vulnerable to changes in staff and in institutional oversight responsibility . Project design should ensure coherence between development objectives and the activities and inputs financed by the operation. Where demand is unknown, flexibility in design is important to enable the project to react smoothly to unforeseen changes in the volume and pattern of demand . Monitoring and evaluation should not be left to project implementation units or other structures outside the mainstream, permanent government institutions . Despite the reportedly high quality of the data bases developed under this project, relevant ministries felt no ownership of them, and did not use them . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is thorough and clearly written . It presents a comprehensive review of the project's activities, and is candid about the design weaknesses and about the shortcomings in Bank and Borrower performance . More analysis of the link between outward migration (a means to an end) and the expected development outcomes of the project would have been helpful. a.Quality of ICR Rating : Satisfactory