ICRR 12716 Report Number : ICRR12716 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/13/2007 PROJ ID : P069939 Appraisal Actual Project Name : Public Administration US$M ): Project Costs (US$M): US$8.97m US$9.84m Reform Project Country : Albania Loan /Credit (US$M): Loan/ US$M ): US$8.5m US$8.82m Sector Board : PS Cofinancing (US$M ): US$M): US$0.47 US$1.02m Sector (s): Central government administration (100%) Theme (s): Administrative and civil service reform (25% - P) Public expenditure financial management and procurement (25% - P) Conflict prevention and post-conflict reconstruction (24% - P) Economic statistics modeling and forecasting (13% - S) Law reform (13% - S) L/C Number : C3328 Board Approval Date : 03/21/2000 Partners involved : Closing Date : 12/31/2004 12/31/2006 Evaluator : Panel Reviewer : Group Manager : Group : Steven Benjamin Webb Gita Gopal Jaime Jaramillo IEGCR 2. Project Objectives and Components: a. Objectives: "The overall objective of the proposed Public Administration Reform Project is to provide required resources for technical assistance, training and incremental operating costs that are needed to implement the Government's Institutional and Public Administration Reform agenda effectively ." (PAD p. 3) The meaning of effective implementation of the institutional and public administration agenda is spelled out in a table of indicators of institutional performance, which the ICR does a good job of summarizing--see 2B. The initial indicators were never formally dropped, but in 2001 a new set was informally adopted, which was more focused and linked to the activities in the project, as elaborated in section 2.c, below. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Initially there were three groups of objectives with multiple indicators, plus a component for project management: Management: Policy and Public Expenditure Management : (i) aggregate fiscal discipline including inflation rate, revenue predictability, and fiscal aggregates; (ii) strategic prioritization including policy volatility, delays in auditing, and deviation from functional appropriations; (iii) operational efficiency including representative deviation by spending units at sector level, and transparent, competitive procurement. (Original cost estimate was US$5.74m, of which Bank financing was to be US$5.59. Actual financing for this component was US$6.63m.) Management: Public Sector Human Resource Management : (i) fiscally sound pay and employment practices including the number of civil and public servants in comparison with international practices, and fiscal weight of public employment in comparison with international practice; (ii) competitive and non-arbitrary remuneration including civil service pay (vertical and horizontal compression), and comparisons with the private sector; and iii) human resource management based on rules, performance and fairness including rule credibility, and limited incidence of political appointees in the civil service . (Original cost estimate was US$2.32m, of which Bank financing was to be US$2.24. Actual financing for this component was US$1.66m.) Coordination: Policy Formulation and Coordination : Policy credibility measures based on periodical surveys. (Original cost estimate was US$0.40m, of which Bank financing was to be US$0.31. Actual financing for this component was US$0.34m.) Project Management -- (Original cost estimate was US$0.51m, of which Bank financing was to be US$0.36. Actual financing for this component was US$0.28m.) In 2002, the objectives were informally rephrased for monitoring purposes : --Improve key dimensions of public sector institutional arrangements and practices in Albania, including : Budget formulation practices that more effectively support rational budget allocation decisions Budget execution practices that better support effective management by budget institutions Continued affordable human resource management Employment conditions within the civil service that make it easier to attract, retain and motivate qualified civil servants More meritocratic and depoliticized management of the civil service Policy formulation deliberations that are more contestable . --In the latter part of project implementation, the following PDO indicators were used in the Implementation Status Report (ISR). Integrated Planning System approved by CoM and launched Treasury modernization system fully operational in MoF, all District Treasury Offices and pilot-tested in line ministries and local governments Civil Service Law (CSL) effective and enforced. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project cost was modest. Part of the TA activities were picked up by DFID directly, and several development partners including UNDP, DFID, SIDA and EC provided resources in a trust fund for training MoF staff. This allowed reallocation of funds toward the Treasury part of component 1. The project was extended three times for a total of two years, partly due to a major change of government in the middle of the project cycle. Initially the project's duration was intentionally set relatively short, with the intent "both to keep pressure on the Government to ensure minimal slippage during project implementation, as well as to permit preparation of a follow -on operation." (PAD p. 3). As the ICR notes, the preparation of the project was rushed to deliver it along with an adjustment loan, which was reasonable, but then more time should have been given to the project itself . 3. Relevance of Objectives & Design: The objectives of the project were all desirable and consistent with the CAS, but there was wide variation in the degree to which the measures in the project seemed to have potential to affect the target variables. Some of the indicators of institutional performance were substantially under the influence of factors in the project (timeliness of audits, minimized deviations of actual from budgeted spending, transparent procurement practices, restraint of public service hiring, civil service pay reform for vertical decompression and horizontal compression, decrease in the share of political appointed civil servants ) but others were not, at least in the medium term (inflation rate, revenue predictability, external current account deficit of the economy [private as well as public sector ], private sector salaries relative to those of the civil service ). The initial objectives were too diffuse and did not prioritize according to client needs . In the hurried preparation of the project, some of the participatory interaction with stakeholders was apparently planned for the implementation phase, which can be appropriate for a TA project . Some of this occurred within the government, although coordination between the two implementing agencies remained a problem . There was little or no consultation with stakeholders outside the government, which may have contributed to the questions about the commitment of a new government (previous opposition party) to sustain the reforms, particularly in the personnel management area. An informal revision of objectives and design at midterm led to more focus, as some sub -components received greater emphasis, while others were dropped or greatly reduced . 4. Achievement of Objectives (Efficacy): Policy and Public Expenditure Management -- Overall, moderately unsatisfactory (same as ICR). Achievement was satisfactory on the macroeconomic targets, although it is not clear how much the project itself contributed to the outcome. On public financial management, budgeting became more organized and the implementation of revenue and expenditure more in line with the approved budget . On procurement, progress was moderately unsatisfactory . The treasury system component went through several design changes and delays, and only became operational in the first half of 2007, after the project closed, so it is too early to judge its success . Restructuring the project put the majority of resources into the treasury sub-component, so its delays and uncertain outcome carry a heavy weight in the overall project rating. The internal audit function in the Ministry of Finance improved . Public Sector Human Resource Management -- Overall, moderately satisfactory (same as ICR). The new civil service law (CSL) was passed in 1999, just before project approval, so its implementation is the core of the component. Initially there was good progress, in reducing to nil the share of employees recruited initially as consultants, increasing by 30 percent (to 89 percent) the share of positions subjected to performance appraisal There was a large scale retrenchment in 2005, not envisioned in the project, after a change in government; this raised concerns about the sustainability of compliance with the new CSL. There were a number of complaints about politically motivated dismissals, although the new appeals process to the civil service commission seemed to operate in that the cases were resolved quickly and mostly in favor of the appellants. Still, there remains a lot of political contention about the process, which the opposition (former governing party) criticizes, as well as the prospect that the present government will change the law in ways that could lead to re-politicization. Policy formulation -- This component successfully supported some rationalization of policy making agencies, and donors have committed large TA funding to carry this further . Project management by the PIU was satisfactory, with timely and regular reporting and some surveys of citizen perceptions. 5. Efficiency (not applicable to DPLs): NA for this type of project. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: There were some positive achievements supported by the project, but there are important short comings. In the public expenditure management area there were important improvements to budget formulation . A lot of the project funding went to a new treasury system, which has good potential, but it was not completed until after the project closed (and is still running in parallel with the old system ) and its outcome therefore is not known. For procurement, the legal framework was improved, but implementation has been poor, as noted in the CPAR 2006. The new civil service law has been implemented, with some steps to professionalize and depoliticize personnel management. On the other hand, the current government (not the originator of the project) downsized personnel in a way that the opposition says was politically motivated and is taking some other actions not in compliance with the law. It also intends to revise the law and perhaps make it explicitly more political. The human resource management system has not been properly implemented. The small policy formulation and coordination component had a satisfactory outcome, particularly in reducing the number of overlapping and conflicting strategies . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The civil service law and the recently installed treasury system offer the possibility of important contributions to development outcomes. As the ICR points out, however, the government has plans to amend the civil service law, which could jeopardize the achievements . The treasury system is too new to assess its outcome and still needs strong political and technical support to achieve its potential . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: The project was not well prepared, being rushed, but subsequent restructuring has allowed it to support some things that have at least the potential for satisfactory outcomes, although these are not yet certain. The initial project design was too complex and assigned resources to activities like anticorruption and management capacity that were underspecified in the PAD and later were dropped from the project. Ownership by the government and the wider political environment (including the opposition, which became the new government) was not adequately assessed. The project design provided too little time and financial resources to areas like treasury and personnel management systems that offered important opportunities for value added. Midterm restructuring moved more resources to treasury systems with some positive results . Although extending the project by two years allowed time for the initial implementation of these systems, the project closed too soon to do the technical fine-tuning that is usually needed to assure the technical capacity as well as political commitment to sustain the new systems. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: Borrower performance was mixed. The borrower did implement some parts of the program successfully, but serious capacity constraints and limits to commitment hindered the full realization of the ambitious project goals. The government's commitment to reform, particularly in the personnel area, has come into question since the change of government in 2005. The two implementing agencies -- Ministry of Finance and Department of Public Administration -- did not cooperate in the way envisioned in the original project design. The project implementation unit performed well in monitoring, reporting, and management of project finances. a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: The project was designed with indicators for regular monitoring, and the government and the Bank followed through and elaborated on these during project implementation . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Fiduciary management of the project was solid . There were no environmental or resettlement issues . 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development High High Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Exemplary NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The design of a public administration project should avoid just providing a pot of money that could be used for consultant advisory services in diverse areas . Rather, it should think precisely about which institution-building activities can be done on a significant scale and then give time and funding to bring those to fruition. Projects should not over-promise and should allow adequate time to achieve even modest goals . Changes of government are an important risk to any project of this type . To mitigate the potentially adverse impact of such events, preparation could include consultation with major stakeholders from across the political spectrum, to take some account of their concerns and to increase the likelihood that they will understand the advantages of the project and support it if they come to power . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: ICR is clear and explains both the strengths and weakness of the project and its implementation . a.Quality of ICR Rating : Exemplary