ICRR 12538 Report Number : ICRR12538 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 10/05/2006 PROJ ID :P058308 Appraisal Actual Project Name :China Pension Reform Project Costs US$ 6.3 US$ 6.3 Project US$M ) (US$M) Country :China Loan/ US$M ) US$ 5.0 Mil. from IDA US$ 5.0 Mil. from IDA Loan /Credit (US$M) and US$ 1.3 Mil. from and US$ 1.3 Mil. from the Peoples Republic the PRC. of China (PRC) Sector (s):Board: ): SP - Compulsory US$M ) Cofinancing (US$M) N/A N/A pension and unemployment insurance (96%), Non-compulsory pensions insurance and contractual savings (4%) L/C Number :C3241 FY ) Board Approval (FY) 99 Partners involved : Closing Date 12/31/2002 12/31/2005 Evaluator : Panel Reviewer : Division Manager : Division : Marcelo J. Bueno Chad Leechor Lily L. Chu IEGCR 2. Project Objectives and Components a. Objectives The objective of the China Pension Reform Project (Project) was to assist the Borrower in reforming its pension system through the formulation of a new funding strategy and implementation of the pension systems in a pilot province (Heilongjiang) and a pilot city (Qingdao), in order to obtain experience and develop a template for establishing a national unified pension system . b. Components (or Key Conditions in the case of Adjustment Loans ): The Project consisted of four components which remained unchanged throughout the Project's life cycle : Component 1, Policy Development (US$ 0.7 million ). This component was aimed at supporting policy studies at the central, provincial, and municipal levels, including : (i) projected cash-flows analysis for four (4) types of pension system benefits (Basic, Transitional, Mandatory Individual Accounts, and Voluntary Supplementary Accounts ), and (ii) sensitivity analysis and test modifications to effective contribution rates, coverage rates and labor force growth, compliance rates, rates of real wage growth, rates of real returns accrued to pension accounts, adjustments to retirement ages, individual account accumulation to more closely appropriate provincial and municipal life expectancies, and modifications on the basis of pension benefits indexation . These analyses were intended to provide policy makers with a sense of direction and clarification of policy necessary to achieve viability and sustainability in pensions reforms and development . Component 2, Information Systems Development (US$ 4.2 million ). This component was to: (i) support the development of updated and revised information, including related administrative systems in the Ministry of Labor and Social Security (MOLSS), and in the Heilongjiang and Qingdao labor and social bureaus, and (ii) create greater capacity, increase processing efficiency, strengthen controls and oversight, and establish record -keeping capabilities. Component 3, Training (US$ 0.9 million ). This component was to support the: (i) identification of key personnel skills requirements in the MOLSS and in the Heilongjiang and Qingdao labor and social security bureaus, and (ii) development and implementation of training programs to improve the skills of their personnel in such areas as pension management and oversight, pension reform policy, financial projections and modeling, and pension asset management. Support under this component was to be provided only if the personnel were part of the staff development plans for pension reforms at the MOLSS, Heilongjiang, and Qingdao labor and social security bureaus . Component 4, Project Administration (US$ 0.5 million ). Component 4 was to provide: (i) technical assistance in project administration at the level of the MOLSS Project Implementation Unit (PIU), and the Project Management Offices (PMO) in Heilongjiang and Qingdao, and (ii) financing support for (a) personnel costs associated with PIU and PMO operations, (b) costs associated with counterpart meetings, (c) printing, communications, postage and translation costs, (d) in-country travel for coordination between the PIU and the PMOs, (e) office overhead and operating costs, and (f) costs of special hired assistants . c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The Project was completed within the original Project Cost at Appraisal of US$ 6.3 million. Where there were savings in the Heilongjiang and Qingdao programs (see Components 2, 3, and 4 above), Bank approval was sought to re-allocate the funds toward technical assistance to the MOLSS to develop a pension system actuarial model . This was in reference to the development of voluntary supplementary schemes in Heilongjiang and Qingdao . During Project implementation, the central government issued regulations for a national supplementary plan "Enterprise Annuities" to be developed, thus Project support for a voluntary scheme at the provincial and municipal levels was abandoned so as not to duplicate efforts . Savings from funds originally allocated to Heilongjiang were then re-allocated to the MOLSS to develop a short to medium -term actuarial forecasting model known as the Projection System of the China Pension Funds . 3. Relevance of Objectives & Design : The Project objectives reflected the Bank's CAS objective of social protection by : (i) assisting the government in developing financially sustainable parameters for pension policies that lead to improved pension systems, and (ii) supporting the government in overcoming barriers to establishing a national unified, multi -pillar pension system, while strengthening pension regulatory and oversight institutions at the provincial and city /municipal levels. Given the state of pension reforms in the PRC and the government's goal of strengthening them, the Project objectives were quite responsive to the Borrower's priorities . By choosing and focusing on Heilongjiang (provincial focus) for its market composition and existing pension system coverage, and Qingdao (municipal focus) for its large SOE presence, relatively high dependency rates, and rapid growth in uncovered pension in joint -ventures and private sector enterprises, the central government aimed at learning from the experience before implementing pension reforms in the rest of the country. In addition, strong and direct government support of the Project at the central (MOLSS), provincial (Heilongjiang), and municipal (Qingdao) levels was evident by its explicit role in project design, provision for administrative and financial support at each level, and in coordinating the efficient execution of Project tasks . 4. Achievement of Objectives (Efficacy) : Most of its objectives were already achieved while additional results are expected. The project's overall institutional development impact includes the new skills of appropriate staff in policy-making, operations, and management of pension insurance systems . Training broadened the understanding of issues relevant to the PRC's pension system and exposed participants to new ideas for pension reform that could be adopted in individual jurisdictions . The design and implementation of the information system for the pension network in Heilongjiang and Qingdao did, and will continue to, improve the data quality, effectively reducing data processing time, and providing a more accurate representation to policy -makers and administrators of participation and compliance rates, as well as the pension system's financial status and performance . Component 1, Policy Development was to forecast the financial viability and sustainability of the provincial and municipal pilots' pension system and provide the inputs based on the forecasts to central policy -makers on key pension parameters to be applied more broadly . Project funds were used to support a short to medium -term forecasting model developed by the MOLSS's Projection System of China Pension Funds (PSCPF) to incorporate a wider range of pension parameters . With modifications, the PSCPF model has the flexibility to run forecasts based on various sets of parameters, a useful function given the disparity in pension systems across the country . Training for the Pension Reform Options Simulation Toolkit (PROST) was also carried out at the central government level as well as in Heilongjiang and Qingdao. In addition to the outputs above, the project also brought about the following outcomes in Heilongjiang and Qingdao : Higher coverage due to increased participation of private employees; Harmonization of contribution rates across cities and types of enterprises; Improved compliance (contribution made) due to new procedures and monitoring . Component 2, Information System Development was for system design, hardware procurement, software development for Heilongjiang and Qingdao, site preparation for servers and other hardware, and operating procedure development and personnel training to use the new system . As a result, information collection, management, information dissemination, and expenditure tracking in the pilot province and municipality were automated . By digitizing the information and providing access via wide -area networks, the system in place provided for more informed policy making and clearer understanding to policy -makers of the state of the pension system and the parameters that need to be changed or modified . Component 3, Training was to support pension-related personnel skill requirements in the MOLSS, and in the Heilongjiang and Qingdao labor and social security bureaus . In the MOLSS, pension training seminars, IT overseas training and study workshops were carried out . For Heilongjiang and Qingdao, pension policy training workshops, and domestic and overseas pension training programs and seminars were also carried out for appropriate personnel . Component 4, Project Administration was to provide assistance in Project administration at the MOLSS PIU, and PMOs in Heilongjiang and Qingdao. The allocation of sufficient and appropriate staffing to these units allowed them to effectively carry out their organizational and financial management responsibilities . This component was accomplished within its budget ceiling with substantial savings realized from the procurement process . Due to the Severe Acute Respiratory Syndrome (SARS) which delayed travel and visa restrictions vis -a-vis study tours and overseas training, the Project completion date was twice delayed . Project completion was also delayed by the re-allocation of funds from savings in the Heilongjiang and Qingdao programs to the MOLSS for improvements in the MOLSS' actuarial model (see 2c above). 5. Efficiency : The project appraisal document (PAD) and the ICR state that the Net Present Value (NPV) or Economic Rate of Return (ERR) is not quantifiable. While all components were executed in a satisfactory manner, the direct benefits, values, and returns are deemed not quantifiable . This assessment, however, is incorrect and may deprive the borrower of an opportunity to use this pilot project as a basis for re -evaluating the economic benefits of pension reforms before implementing them nationwide . See related comments in section 11 (on quality of the ICR) below. 6. M&E Design, Implementation, & Utilization: Overall monitoring, evaluation, and utilization during Project implementation was satisfactory . The preparation of the project appraisal document (PAD) was comprehensive in the background and rationale for the objectives of the Project, including the clear performance indicators for project implementation . More importantly, the Project was prepared jointly with the participation and expressed interest of the MOLSS, Heilongjiang provincial and Qingdao municipal representatives resulting in substantial ownership and commitment on the part of government to the design, implementation, and monitoring of the Project . Over the 5 year span, the project leader was changed twice, yet there was consistency in the make -up of the project team from preparation to implementation, with no apparent negative impact on the supervision and monitoring of the project . Supervision missions were in accordance with the Bank's requirements, and reports following the missions provided useful assessments of the Project's status including appropriate feedback and suggestions for the resolution of issues and "next steps". The PRC's implementation performance was also rated as Satisfactory . Staff members were assigned to the Project at the central level and in Heilongjiang and Qingdao, and all the participants implemented their respective tasks in earnest . They were mindful in following the guidelines set by the Bank particularly in the area of procurement through competitive bidding and took pride in the results of the Project as well as what they learned in the process . 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): Since the Project was closely in -line and consistent with the PRC's pension development objectives and strategies, it is difficult to differentiate whether the final achievements, which could materialize long after the Project closes, resulted from the Project, the government or both . On the other hand, there is no doubt that the Project contributed to the many positive impacts in pension reforms and related policy initiatives in the PRC . The development in human resource capacity and improvements in skills via training under the Project, while difficult to measure, will, nevertheless, have a lasting impact of the future development of the pension insurance system in the PRC . 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Highly Likely Highly Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Three lessons can be learned from the objectives, design, and implementation of the Project : (i) the importance of aligning project objectives with those of the Borrower to bring a shared sense of purpose in moving the project forward, (ii) pilot trial basis prior to design and implementation at the national level brings focus and clarity to attaining project objectives, and (iii) clarity regarding the responsibilities and repayment obligations at the onset of the project launch would make project management and monitoring more effective and efficient . 10. Assessment Recommended? Yes No 11. Comments on Quality of ICR: The quality of the ICR is unsatisfactory . On the positive side, it provides a clear summary of : (i) project objectives and their alignment with the Bank's CAS objectives and government priorities, (ii) the actual and expected achievements of the components, and (iii) the institutional development impact on the prospect of the pension system. However, a major shortcoming of the report is the failure to do an adequate analysis of the economic rate of return, cost/benefit or other analysis of efficiency . The pilot project provided an opportunity to collect and quantify the economic benefits of pension reforms, especially the reduction in intermediation costs and the improvements in compliance envisaged for the project . With the aid of the actuarial forecasting model, the project team or ICR team, which included modeling experts and actuaries in addition to financial analysts, could have computed likely economic rates of return, which would serve as valuable inputs for eval uating the next steps of pension reforms . Indeed, the PAD for the project had originally envisioned such an analysis "... the short, medium and long-term fiscal impact will be quantified for the pilots through the financial sustainability modeling exercise supported in project preparation as well as executed under the project itself . Utilizing the output from this exercise as well as the data generated from similar exercises with other project support, improved estimates of the potential fiscal costs of transition can be developed." (PAD, pg. 8) Given that this is a pilot project, such an analysis would be very helpful in assessing the desireability of expanding this program to other areas . In addition, the ICR could be significantly strengthened by : (i) taking advantage of the dialogue and cooperation with other partners to incorporating the linkages, benefits and impact on the Project, if any, of the Asian Development Bank project aimed at supporting the formulation and implementation of the administrative, legal, regulatory, a and supervisory frameworks for the new pension system, as well as the initiatives and bilateral support provided by the governments of the Netherlands and Australia to the Chongqing municipality and to the MOLSS, respectively, and (ii) providing the value added of IDA's experience in pension reforms, specifically, its country - specific knowledge based on earlier analytical work undertaken on China's pension system in 1996. In addition, although the ICR notes that there has been active participation by counterparts in the project, and the " MOLSS as the borrowing agency concurred with this ICR and agreed that the project provided valuable lessons in many aspects which were subsequently applied in other activities, " there is no explanation of what those lessons were and how they were subsequently applied . On balance, taking the strengths and weaknesses into account, the quality is unsatisfactory .