ICRR 13405 Report Number : ICRR13405 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/19/2011 PROJ ID : P094167 Appraisal Actual Project Name : Privatization Social US$M ): Project Costs (US$M): EUR450.0 EUR370.6 Support Project 2 Country : Turkey Loan/ US$M ): Loan /Credit (US$M): 465.4 442.2 Sector Board : SP US$M): Cofinancing (US$M ): Sector (s): Compulsory pension and unemployment insurance (94%) Central government administration (3%) General industry and trade sector (1%) Other social services (1%) General education sector (1%) Theme (s): Improving labor markets (33% - P) Small and medium enterprise support (17% - S) Education for the knowledge economy (17% - S) Social risk mitigation (17% - S) Social analysis and monitoring (16% - S) L/C Number : L4791 Board Approval Date : 06/14/2005 Partners involved : Closing Date : 06/30/2009 10/31/2009 Evaluator : Panel Reviewer : Group Manager : Group : Victoria Monchuk John R. Eriksson IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: The development objective of the Second Privatization Social Support Project (PSSP 2) was "to support the Government's privatization program through mitigating the negative social and economic impact of the privatization of State Owned Enterprises (SOEs)" (PAD p. 6, p. 27, Loan Agreement p. 14). In helping to meet the market demands of EU accession, the Turkish government was undertaking a privatization program to enhance the efficiency and competitiveness of the Turkish economy . The privatization program would involve selling or liquidating several SOEs which would result in labor redundancies . The project aimed at helping make privatization more smooth and effective and mitigating the negative social and economic impacts on the retrenched workers through providing mainly severance payments and labor market redeployment activities . The project followed the Bank-supported PSSP 1. PSSP 1 financed job loss compensation payments for more than 18,000 workers displaced from SOEs and worked with the Privatization Administration (main implementing agency) to design a broad framework for mitigating the negative effects of privatization . PSSP 2 maintained the same framework whereby the Privatization Administration worked with firms prior to privatization to assist them in getting ready for the point of sale including facilitating lay offs and providing workers with services and compensation . The PSSP 2's key performance indicators involved a ) improving the productivity of the industrial sector, b ) mitigating the effect on poverty of the affected workers, c ) maintaining job placement rates equal or better than similar programs in Turkey and the region, d) evaluating the impact of the privatization programs and labor redeployment services and identify policy alternatives. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project had three components : Job Loss Compensation (JLC) (appraised estimate EUR420.1M, actual estimate EUR353.4M): The objective of this component was to ameliorate the temporary negative social and economic impact on workers displaced during privatization of SOEs. JLC would also be provided to workers who would be unemployed due to secondary employment effects and retirees who were eligible for severance according to the Turkish Employment Law . This component financed severance and related payments to displaced workers . The loan agreement allowed to use a maximum of EUR210M to retroactively reimburse 81 percent of job loss compensation payments to workers displaced from privatizations that took place after January 1, 2003 (revised to 95 percent in February, 2008 with the aim of further incentivizing privatization ) . Labor Redeployment Services (LRS) (appraised estimate EUR27.2M, actual estimate EUR14.9M): The objective of this component was to provide labor redeployment services to workers who have been displaced by the privatization of SOEs to assist them to rapidly re -enter the labor market. This component financed a variety of LRS including job counseling and placement services, retraining, temporary community employment, small business assistance services, and small business incubators . Management, Monitoring and Evaluation (revised shortly after effectiveness ) (appraised estimate EUR0.9M, actual estimate EUR0.474M): The objective of this component was to monitor the impact of LRS and manage PSSP 2 effectively. The component was to finance : a) surveys to evaluate the effectiveness of the LRS in mitigating the social costs of labor redundancies resulting from employment and privatization on selected communities, and b ) undertake in-depth socio-economic analyses of specific communities where privatization has taken place . Following a Government Treasury management decision, the monitoring and evaluation part of this component was dropped by an amendment to the loan agreement . Part of the reason for the revisions was the sensitive nature of the previous impact assessment but also a response to organizational priorities . The revised component which was renamed to "Project Management" only included provision of technical assistance and goods for project coordination and implementation. The project's fourth key performance indicator became partially irrelevant following the loan amendment. No changes were made to the original monitoring and evaluation framework . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The loan was approved on June 14, 2005 and became effective seven months later on January 31, 2006. On June 29, 2009 a partial loan extension was granted until October 31, 2009 to finance expenditures on consultants and training for completing activities related to project closing . Funds were provided to the government of Turkey through reimbursements after the firms had been privatized and workers displaced . Because several changes had to be made to the project after effectiveness it was informally restructured five times (without requiring board approval or changes to the development objective ).Total project costs were appraised at EUR 450.0M including the costs for the three project components and a EUR 1.8M front-end fee. At project closing, total costs were EUR 370.6M or 82.3 percent of the appraised estimate . EUR15.0M from the loan was cancelled from the JLC component at the government's request and an additional EUR 15.3M was cancelled as part of the Bank's loan closing procedures . At closing EUR329.7M or 91.6 percent of the EUR360.0M IBRD loan was disbursed. Total borrower contributions were EUR40.9M. 3. Relevance of Objectives & Design: Relevance of objectives: Substantial . The objective of the project was relevant with regards to Turkey's large reform and privatization efforts which have been ongoing since the crisis in 2001 and are mentioned as continued priorities in the government's Ninth Development Plan (2007-13) and the Country Partnership Strategy (2008-11). Privatization of SOEs speaks directly to one of the three pillars in the Development Plan : Improved Competitiveness and Employment. However, the second part of the development objective (to mitigate the economic and social impacts of privatization of SOEs) is not specifically mentioned as a specific area for Bank engagement in Turkey . Relevance of design: Modest . The project design was largely based on the PSSP 1 design and experiences but not all lessons were factored into PSSP 2's design. The two project components were aligned with the objective and were based on knowledge that both financial compensation as well as active labor market training and counseling are needed to help displaced workers regain employment without suffering socially and economically . Although not a specific component, the project's support to the Privatization Administration's Labor Adjustment Group (supported under PSSP 1), actively helped firms go through labor adjustment prior to privatization . However, the project's design had significant shortcomings . These were a) poor consideration of the age, work incentives, and financial needs of the majority of laid-off workers (who were close to the retirement age ), and b) no targeting mechanism in place to ensure that the LRS benefitted those workers who were laid off due to SOE privatizations . 4. Achievement of Objectives (Efficacy): The achievement of the development objective : to support the Government's privatization program through mitigating the negative social and economic impact of the privatization of State Owned Enterprises was Substantial . Outputs : All 13,614 workers laid off from the 20 privatized SOEs received JLC payments (13,535 received severance and special job loss compensation payments; 70 received social assistance payments for civil servants ). This number was less than half of the 29,000 targeted workers. This discrepancy is a result of a significant double counting of beneficiaries at appraisal and not a reflection of poor coverage . Eighty percent (80%) of JLC beneficiaries had reached maximum retirement age at the time of privatization or were close to retirement and chose to retire when they qualified for JLC. (According to the Turkish Privatization Law retirees are entitled to severance payments .) Hence, severance payments mainly helped fund the transition of workers to retirement and may also have created an incentive for early retirement and positively influenced the privatization efforts (ICR p.18). The workers who had reached maximum retirement age and those who chose to retire early benefitted from both their regular retirement benefit and from the JLC. The JLC severance payments represented an additional income to the regular pension . The average JLC severance payment was TL 47,741 which, if spread out evenly, could keep a family without any other income above the poverty line for between 42 (family of 6) and 62 (family of 3) months. Of the remaining displaced workers who did not retire (20% of beneficiaries) almost all joined the Temporary Employment Program (TEP) which functioned as a bridge to retirement rather than a temporary arrangement in transition to new employment (TEP is a low-wage, low-skill public employment program given to eligible participants on a rolling 11-month basis). Hence, both for retiring and non -retiring workers JLC payments served as an income top -up for a period of time. Virtually no laid-off workers from the privatized SOEs benefitted from LRS . In sum, 9,420 people participated in LRS programs (8,546 completed them, 4,896 were employed); 5 Small Business Incubators (SBI) were established generating 1,086 jobs; and 1,536 people participated in Small Business Assistance programs which generated 210 start-up enterprises employing 332 people. These numbers were broadly in line with the target of 11,000. However, the people who participated in the LRS were not the targeted workers of SOEs that were being privatized (primary or secondary lay-offs). Rather they compromised the generally unemployed . The planned targeting of LRS services was not applied. The main reasons for not reaching the targeted workers was that the definition of secondary lay -offs was too broad and that LRS rolled out nationally and not only in the 36 targeted provinces where privatizations would occur. Other reasons for the very low participation include a ) most of the laid-off workers were close to retirement age and chose to take early retirement instead of re -entering the labor market, b) laid-off workers were not aware of LRS training or did not consider it valuable, and c ) TEP activities attracted many of the younger laid -off workers who may have otherwise enrolled in LRS activities . Outcomes : There is some evidence that JLC payments reduced the negative impact resulting from SOE privatization . Because the impact studies were cancelled there was no rigorous evaluation of the social and economic impacts of the JLC on the displaced workers and their families . Information provided to IEG by the Region (based on surveys with workers in two large SOEs) indicate that regular pensions were generally below the national poverty line for the majority of affected workers and that both younger and older workers experienced a worsening of their economic situation upon privatization. The younger terminated workers and those who chose to retire early claimed more negative impacts (ICR p. 19, the ICR does not specify what these impacts were ). For instance, 73 percent of survey respondents said that their income level decreased compared to their working salary and 24 percent declared a dramatic drop in their income which they said was not good enough to support their living . But without JLC payments, which topped up pensions, the survey shows that the majority of households affected by the lay -offs would have had incomes below the poverty line, suggesting that JLC payments helped provide some sort of security for affected workers and their families. However, workers who had reached the maximum retirement age were less affected by the privatization process and were able to sustain themselves through their regular pension according to the ICR . There is no information on pension replacement rates, the amount of severance payment required by law, or the ratio of the JLC severance payments to individual salaries so it is not known how generous the compensation was . There is no evidence to show whether the LRS benefitted laid off workers or mitigated the negative impact of SOE privatization . But the LRS labor redeployment training did have some positive impact on re -employment and the SBI program, providing some boost to small business survival rates . There are no data available on whether the outcome of the LRS helped the participating workers and their families increase their social and economic welfare and avoid poverty. Nevertheless, the project may have contributed to creating some employment and helping other unemployed workers enter the labor market . Official placement rates for those who completed labor redeployment training was 57 percent (exceeding or matching targets, ICR p . 19-20) although the result of the focus groups show that only 31 percent of participants found that the skills acquired during the training helped them find a job and 39.5 percent said that they have found a job following the training . Moreover, the SBI model was strengthened with the support of the project, nine out of ten SBIs created under PSSP 1 and 2 are still operating today. Firm survival rate after two years of the SBI program was 65 percent which compares well with US small businesses . Consistent with the overall PDO, the project likely helped facilitate the privatization process and created efficiency gains in privatized SOEs because it assisted in moving redundant workers into retirement . Because it is not possible to establish a counterfactual it cannot be demonstrated whether the JLC and LRS provided by the project made the privatization process more effective . However, two project features may have contributed to making the privatization process more efficient than in the absence of the project . First, the arrangements set up under PSSP 1 - to use the Labor Adjustment Group to provide in -company pre-layoff labor redeployment assessment and planning sessions for workers who would be displaced helped the social transition out of employment for the affected workers . Second, guaranteeing through World Bank financial support that the government would be able to fund the privatization costs and reimburse the Privatization Administration for its services (LRS and JLC payments) to workers and the affected firms made privatization more feasible economically and politically . It is therefore possible, albeit not fully known, that the project assisted in creating efficiency gains for the privatized SOEs . Data collected by the ICR team from a sample of two large privatized SOEs indicate that efficiency gains were made as a result of labor shedding and large scale investment programs during the privatization process . Tupras, Turkey's largest industrial enterprise, decreased the number of employees by 15 percent since the 2005 privatization while increasing expor ts and sustaining production. Petkim, Turkey's main petrochemicals producer, halved its operating expenses on a year to year basis due to restructuring and gradually cutting back its work force since privatization begun in 2001. 5. Efficiency (not applicable to DPLs): The ICR does not present any data on rates of return or net present value of the project . At appraisal, the efficiency of the project was justified based on analysis that indicated that JLC payments to the affected workers would be cheaper than paying salaries over a four year period for each eligible SOE . At closing, the analysis of the Petkim and Tupras enterprises indicated that post -privatization performance was improved and that JLC payments produced significant savings (ICR p. 26, the ICR does not state the amount of savings ). But because replacement rates are not known and the age structure and life expectancy of workers were not calculated it is not clear how much savings were generated. Most workers chose to retire and exit the labor market on their pensions and topped up their income with JLC payments. Also, because the firms that benefitted from the project would have undergone privatization even in the absence of the project (clarified by TTL in interview) it is not certain to what extent those savings were generated by the project. Moreover, LRS did not benefit displaced workers and did not help actively move displaced workers back into the labor market . Finally, the fiscal justification for the Bank loan, which was rather strong at appraisal given the poor economic situation of the government, had significantly diminished by project closing . Therefore, the efficiency of the project was Modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project had a relevant objective but suffered design shortcomings and had modest efficiency . However, the project substantially achieved its objective of supporting the government's privatization agenda through mitigating the negative social and economic impacts of privatization . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome is rated Negligible to Low . SOE privatization continues as a strong priority for the government and picked up considerably in 2006-07. Privatization revenues in those two years (exceeding US$16 billion) were almost twice as high as combined privatization revenues in the previous 18 years (CAS, 2008, p. 2). Furthermore, the government's financial situation has significantly improved since 2001 which bodes well for fiscal room for further privatization and severance payments . JLC continues to be provided to laid off workers and retirees . LRS activities also continue to be supported partly by the World Bank . Pre-lay-off labor redeployment assessments and planning sessions for workers, created under PSSP 1 and continued under PSSP 2, are no longer in effect. The ICR provides no indication of whether JLC activities will be modified in the future and whether there will be an assessment for learning if JLC payments have an impact on mitigating negative impacts of the privatization program . a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: Quality at entry : As noted, the project design was largely based on the PSSP 1 design and experiences. The overall project design with two components providing financial compensation and active labor market services were aligned with the objective. In addition, a third component which would monitor and evaluate the impacts of the project's activities on the beneficiary population was present with an adequate results framework . Compared to the first project phase the PSSP 2 strengthened the accountability and service quality provided through the small business assistance and SBI modules . But because of sensitivities surrounding the findings on the social and economic impacts on workers in PSSP 1, some of the lessons from the earlier phase were not made public and factored into PSSP 2. The Bank did not ensure that there was enough demand for JLC and LRS by displaced workers in order to temporarily help them from suffering negative impacts of the lay -offs and replace them into the labor market. Project preparations should have better analyzed the needs and incentives of the workers who were close to retirement age. Also, even though the PSSP 1 ICR stated that LRS needed to be flexible to adjust to demand it did not do so. Another reflection of inadequate quality at entry was that the project had to be restructured five times (informally without requiring board approval ) including amendments to the loan agreement and operations manual. Finally, the layered implementation arrangements with the Privatization Administration as the main coordinating body between the Bank and other implementation agencies actually limited coordination and information sharing. The Privatization Administration housed the Project Coordination Unit (PCU) but the LRS component was managed by another group within the Privatization Administration - the Labor Adjustment Group. Although identified as a risk during preparation, project design did not include any mitigating measures for the lack of institutional coordination . Quality at entry was Moderately Unsatisfactory . Quality of supervision : During the project the Bank team - largely based in the field - maintained a sustained dialogue with the government. The fiduciary aspects of supervision were especially strong . But supervision lacked in several aspects . First, when the government decided to drop the impact studies the Bank team did not accordingly adjust the results framework . Other sources of information should have been sought at an early stage to verify the projects results and the Mid -Term Review (MTR) should have given greater importance to monitoring results. Instead, no formal MTR was undertaken . Secondly, although the revisions made to the operations manual to allow TEP workers to receive JLC payments were justified, adjustments to the arrangement with the TEP program should have been made so that the displaced workers would be reintegrated into the labor market as planned. Thirdly, ISR monitoring of project implementation during the first half of the project was less than satisfactory. However, with the arrival of new Bank staff to the team, implementation monitoring and results orientation picked up substantially towards the last year of the project especially in collecting data for measuring the achievement of objectives and outcomes . Quality of supervision was Moderately Unsatisfactory . Overall Bank performance was Moderately Unsatisfactory . at-Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government performance : Throughout both phases of the PSSP government commitment to privatization and to the project objectives have been, and continues to be, strong . The Treasury's decision to no longer provide technical support to the M&E part of the project and to drop the impact studies proved detrimental to documenting project outcomes. This decision came about partly due to the sensitive nature of the impact study from the first project phase which the government chose not to make public . The limited disclosure of the phase 1 study also negatively affected the extent to which important lessons from phase 1 could be factored into phase 2, especially in terms of workers' demand for LRS. Moreover, the fact that in practice displaced workers enroll in the TEP program as a permanent employment rather than as a temporary arrangement made it difficult to reintroduce workers into the labor market as planned and desired . Government performance was Moderately Satisfactory . Implementing agency performance : Commitment from the Privatization Administration as well as from the other involved implementing agencies was also strong . However, the layered implementation arrangements channelling information through the coordinating unit in the Privatization Administration and the Labor Redeployment Group made communication slow and incoherent at times . Implementing agency performance was Moderately Satisfactory . Overall borrower performance was Moderately Satisfactory . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: The overall results framework was relatively good including a freestanding M&E component undertaking impact studies which would measure the social and economic impacts of privatization on the displaced workers . The weakness in M&E design includes somewhat outputs focused indicators (people served, replacement rates immediately following training, and number of impact studies undertaken ) without baselines. Project M&E could not be implemented as planned due to the amendment to the loan agreement dropping most if the impact and evaluation studies . A formal mid-term review was never undertaken . Project outputs (not outcomes) were however closely monitored and accounted for . Despite dropping the planned evaluation activities the project's results framework was not revised and continued to include indicators on mitigation of poverty among the affected workers which proved to be difficult to measure at project closing . As a result of the implementation revisions and the lack of mid -term review the M&E utilization was low. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): There were no safeguard issues and the fiduciary management was satisfactory . It was initially intended that the LRS component only benefits primary and secondary lay -offs due to SOE privatizations. But because the definition of secondary lay-offs was too broad these came to include all of the unemployed population in the country . While the lack of targeting meant that most of the laid off workers did not benefit form LRS there were, however, a number of other unemployed that could take advantage of the LRS activities and be reintegrated into the job market . 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Moderately Moderately Bank performance is downgraded to Satisfactory Unsatisfactory Moderately Unsatisfactory because of significant shortcoming in quality at entry especially in terms of inadequate needs assessments of the laid -off workers. Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Providing and institutional structure and working with firms and workers prior to lay -offs can improve the readiness of firms at the point of sale and ease any social, economic and political consequences of privatization . The project used the PSSP I structure which supported the Privatization Administration to work with firms that were on the list to be privatized and get them ready for privatization and the point of sale . The Labor Adjustment Group in the Privatization Administration worked with workers and provided them guidance on retiring or getting labor market counseling and training so as to help guarantee that firms would smoothly shed their labor force by the time of sale. In a project which is trying to affect people's behaviors (such as finding new employment) is essential to base project design on assessments of needs and incentives of the intended beneficiaries . Had project design more candidly factored in the sensitive but important findings from the impact studies of the first phase as well as assessed the incentives and needs of the affected SOE workers the project may have chosen to alter design details of the JLC and LRS to achieve the objectives more effectively and efficiently . When M&E plans are significantly altered during project implementation appropriate adjustments need to be made to the results framework early on to ensure that project performance can be rigorously assessed . Not until project closing did the ICR team try to compensate for this shortcoming and collect information about project outcomes, albeit only to a limited extent. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clear and comprehensive . Because of the poor M&E the ICR did not have good data to work with . The ICR should be commended on its efforts to try to collect ex -post data for assessing achievement . The ICR has some shortcomings, especially it draws too broad and positive conclusions based on relatively weak and mainly qualitative data. For example, the data presented in the ICR suggest that there is no indication of the welfare of affected workers either before or after the project . Yet the ICR claims that PSSP 2 prevented workers from falling into poverty . a.Quality of ICR Rating : Satisfactory