VIETNAM Evaluation of Public Financial Management Reform Report No. 106635 JUNE 29, 2016 © 2016 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. The World Bank does not guarantee the accuracy of the data included in this work. The Any queries on rights and licenses, including boundaries, colors, denominations, and other subsidiary rights, should be addressed to information shown on any map in this work do World Bank Publications, The World Bank not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No. 106635 PROJECT PERFORMANCE ASSESSMENT REPORT VIETNAM PUBLIC FINANCIAL MANAGEMENT REFORM PROJECT (P075399, P120613, CR.3767, CR.4863, AND TF-50988) AND MULTI-DONOR TRUST FUND FOR PUBLIC FINANCIAL MANAGEMENT MODERNIZATION (P110525, TF-94240) June 29, 2016 IEG Human Development and Economic Management Department Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Vietnamese Dong (D) 2004 US$1.00 D15,744 2005 US$1.00 D15,851 2006 US$1.00 D15,966 2007 US$1.00 D16,085 2008 US$1.00 D16,237 2009 US$1.00 D17,778 2010 US$1.00 D19,497 2011 US$1.00 D21,034 2012 US$1.00 D20,840 2013 US$1.00 D21,095 2014 US$1.00 D21,097 Abbreviations and Acronyms CAS country assistance strategy COA chart of accounts CPS country partnership strategy CTA chief technical advisor DMFAS Debt Monitoring and Financial Analysis System DFID U.K. Department for International Development EMCC Economic Management Competitiveness Credit FDS Financial Development Strategy FMIS financial management information system GDP gross domestic product GFS Government Finance Statistics ICB international competitive bidding ICR Implementation Completion and Results Report IDA International Development Association IMF International Monetary Fund IP implementation plan IV&V independent validation and verification ISR Implementation Status and Results Report M&E monitoring and evaluation MDTF multi-donor trust fund MTR mid-term review MOF Ministry of Finance MPI Ministry of Planning and Investment MTFF Medium-Term Fiscal Framework MTEF Medium-Term Expenditure Framework PAD project appraisal document PDO project development objective PEFA Public Expenditure and Financial Accountability PFM public financial management PFMRP Public Financial Management Reform Project PMU Project Management Unit PRSC Poverty Reduction Support Credit SDR special drawing rights SEDP Socio-Economic Development Plan SOE state-owned enterprise SSD single strategy document TA technical assistance TABMIS Treasury and Budget Management Information System TTL task team leader TSA treasury single account UNCTAD United Nations Conference on Trade and Development VDR Vietnam Development Report Fiscal Year Government: January 1 – December 31 Director-General, Evaluation : Ms. Caroline Heider Director, Human Development and Economic Management : Mr. Nick York Manager, Economic Management and Country Programs : Mr. Mark Sundberg Task Manager : Mr. Moritz Piatti v Contents Principal Ratings ................................................................................................................ ix Key Staff Responsible........................................................................................................ ix Principal Ratings ................................................................................................................ ix Key Staff Responsible........................................................................................................ ix Preface................................................................................................................................ xi Summary .......................................................................................................................... xiii 1. Background and Context................................................................................................. 1 Scope ............................................................................................................................... 1 Implementation Background ........................................................................................... 1 2. Objectives, Design, and Relevance ................................................................................. 8 Objectives ....................................................................................................................... 8 PFMRP........................................................................................................................ 8 MTDF II ...................................................................................................................... 8 Related Operations ...................................................................................................... 9 Design ........................................................................................................................... 10 PFMRP...................................................................................................................... 10 MDTF II .................................................................................................................... 12 Implementation ............................................................................................................. 14 Principle Changes and Dates .................................................................................... 14 General ...................................................................................................................... 15 TABMIS Implementation Experience ...................................................................... 16 Fiduciary Issues and Safeguards ............................................................................... 19 Relevance of Objectives ............................................................................................... 20 Relevance of Design ..................................................................................................... 22 PFMPR...................................................................................................................... 22 MDTF II .................................................................................................................... 22 3. Achievement of the Objectives ..................................................................................... 23 PFMRP.......................................................................................................................... 23 Objective 1: Strengthen the Borrower’s Capacity to Plan Its Budget: Substantial... 25 This report was prepared by Iradj Alikhani, under the guidance of Moritz Piatti, task manager, who assessed the project in March 2016. Ali Hashim authored appendix C. The report was peer reviewed by Salvatore Schiavo-Campo and panel reviewed by Clay Wescott. Viktoriya Yevsyeyeva and Yezena Yimer provided administrative support. vi Objective 2: Strengthen the Borrower’s Capacity to Execute Its Budget: Substantial ................................................................................................................................... 27 Objective 3: Strengthen the Borrower’s Capacity to Report on Its Budget: Substantial ................................................................................................................. 27 Objective 4: Strengthen the Borrower’s Capacity to Improve the Transparency of the Budgetary Systems and Processes: Substantial ........................................................ 27 Objective 5: Strengthen the Borrower’s Capacity to Improve the Accountability of the Budgetary Systems and Processes: Substantial .................................................. 28 MDTF II ........................................................................................................................ 28 Policy Actions Undertaken since Project Closing .................................................... 29 Cross-Cutting Theme ................................................................................................ 29 Contribution of Project Results to Objectives .......................................................... 29 4. Efficiency ...................................................................................................................... 33 PMFRP.......................................................................................................................... 33 MDTF II ........................................................................................................................ 34 5. Outcome Ratings ........................................................................................................... 35 6. Risk to Development Outcome ..................................................................................... 36 Cross-Cutting Risks .................................................................................................. 36 PMFRP.......................................................................................................................... 37 MDTF II ........................................................................................................................ 38 7. World Bank Performance ............................................................................................. 39 Quality at Entry ............................................................................................................. 39 PFMRP...................................................................................................................... 39 MDTF II .................................................................................................................... 41 Quality of Supervision .................................................................................................. 43 PFMPR...................................................................................................................... 43 MDTF II .................................................................................................................... 47 8. Borrower Performance .................................................................................................. 49 Government................................................................................................................... 49 Implementation Agencies ............................................................................................. 50 PFMRP...................................................................................................................... 50 MDTF II .................................................................................................................... 50 9. Monitoring and Evaluation ........................................................................................... 51 What Constitutes a Strong M&E System for PFM Operations?................................... 51 PFMRP.......................................................................................................................... 53 MDTF II ........................................................................................................................ 55 vii 10. Lessons ........................................................................................................................ 57 General .......................................................................................................................... 57 FMIS (TABMIS) .......................................................................................................... 58 References ......................................................................................................................... 62 Appendix A. Basic Data Sheet.......................................................................................... 64 Appendix B. DPO PFM Related Prior Actions ................................................................ 69 Appendix C. TABMIS ...................................................................................................... 72 Appendix D. Updated Results Matrix of MDTF II........................................................... 93 Appendix E. Review of PFMRP ISRs ............................................................................ 109 Appendix F. List of Persons Met .................................................................................... 114 Appendix G. Borrower Comments ................................................................................. 116 Boxes Box 7.1. Feedback from a Sectoral Team Regarding Its Degree of Involvement in a Subproject ......................................................................................................................... 48 Box 9.1. Social Media Monitoring of Government Finances through Facebook ............. 55 Tables Table 1.1. Key Provisions of the 2002 Budget Law ........................................................... 1 Table 1.2. Key Economic Indicators 2004–2014................................................................ 4 Table 1.3. Fiscal Situation 2004–2014 ............................................................................... 6 Table 1.4 World Bank Operations Supporting PFM Reforms in Vietnam (2001–2015) ... 7 Table 2.1. MDTF II Subobjectives ..................................................................................... 8 Table 2.2. Project Cost Planned and Actual by Component and Financing Source ......... 12 Table 2.3. Project Cost Planned and Actual by Activity and Financing Source .............. 14 Table 2.4. Key Dates and Modifications .......................................................................... 14 Table 2.5 PEFA Performance Measurement 2013 ........................................................... 18 Table 3.1. Mapping of Legal and Regulatory Achievement to MDTF II PDOs .............. 30 Figures Figure 4.1. Key FMIS Indicators for Selected Countries ................................................. 34 ix Principal Ratings Project 1: Vietnam—Public Financial Management Reform Project ICRa ICR Reviewa PPAR Outcome Moderately Moderately Satisfactory Satisfactory Satisfactory Risk to Development Negligible Negligible Moderate Outcome Bank Performance Moderately Moderately Satisfactory Satisfactory Unsatisfactory Borrower Performance Satisfactory Moderately Satisfactory Satisfactory a. The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate Independent Evaluation Group (IEG) product that seeks to verify independently the findings of the ICR. Key Staff Responsible Task Manager or Division Chief or Project Leader Sector Director Country Director Appraisal Edward Mountfield Barbara Nurnberg Klaus Rohland Completion Quyen Hoang Vu Sandeep Mahajan Victoria Kwakwa Principal Ratings Project 2: Vietnam—Multi-Donor Trust Fund to Support Public Financial Modernization ICRa ICR Reviewa PPAR Outcome Moderately Moderately Highly Satisfactory Unsatisfactory Satisfactory Risk to Development Moderate Moderate Negligible Outcome Bank Performance Moderately Moderately Satisfactory Satisfactory Unsatisfactory a. The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate IEG product that seeks to verify independently the findings of the ICR. Key Staff Responsible Task Manager or Division Chief or Project Leader Sector Director Country Director Appraisal Quyen Hoang Vu Barbara Nurnberg Ajay Chhibber Completion Minh Van Nguyen Sandeep Mahajan Victoria Kwakwa x IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: ieg.worldbankgroup.org. Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, and Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. xi Preface This Project Performance Assessment Report (PPAR) reviews the performance of the World Bank’s Public Financial Management Reform Project (PFMRP) and the Multi- Donor Trust Fund to Support Public Financial Management Modernization – Phase 2 (henceforth MDTF II) in support of the public financial management (PFM) reform program in Vietnam. The former was approved by the Board on May 22, 2003 in the amount of SDR39.9 million (equivalent to US$54.3 million), consisting of an International Development Association (IDA) credit of US$44.3 million and cofinancing of US$10 million by the U.K. Department for International Development (DFID), with a closing date of February 28, 2009. The project received an additional financing of SDR9.2 million (US$14 million equivalent) approved on January 6, 2011. Following extensions, it closed on October 31, 2013. MDTF II was approved on April 1, 2009, and was financed through a grant of US$5.8 million provided by the Australian Agency for International Development, Canadian International Development Agency, Danish Development Agency, the European Commission, the government of the Netherlands, and Switzerland State Secretariat for Economic Affairs. Its original closing date was June 6, 2011, which was extended twice to June 30, 2012. The PFMRP’s project development objectives (PDOs) were to: “to strengthen the Borrower’s capacity to (i) plan, (ii) execute and (iii) report on its budget and to improve the (iv) transparency and (v) accountability of the budgetary systems and processes.” MDTF PDOs, as evaluated here, are “to support Vietnam to strengthen its capacities for: (a) increased (i) coordination and (ii) transparency in public finance management; (b) improved revenue mobilization; and (c) (i) effective and (ii) efficient public expenditure management.” The lending instrument for the first operation was a specific investment loan and for the second a trust fund. This report presents findings based on a review of the project appraisal documents, the Implementation Completion and Results Reports, Implementation Completion and Results Review, aide-memoires, International Monetary Fund and World Bank reports, and other relevant materials, including a number of publically available studies by various donors. An Independent Evaluation Group (IEG) mission visited Hanoi during March 6–18, 2016, to interview government officials, the staff of the World Bank, and other development partners and stakeholders (see appendix F for a list of persons interviewed). The assessment aims to review whether and how the operations achieved their intended objectives. The report provides additional evidence and analysis of relevant and comparative data for a more complete picture of the outcomes and the factors that influenced them. By covering the period between 2013 and 2016, it offers an opportunity for broader lessons and a longer time perspective as well as reflection on the sustainability of policy reforms and long-term factors that facilitated outcomes. This assessment is part of a larger body of public sector and governance evaluations in Africa and East Asia that will feed into a synthesis report to draw lessons on cross- country experience. The reports provide a dedicated appendix on the design, xii implementation, and utilization of financial management information systems (FMIS), which have been at the core of the reform programs. Following standard IEG procedures, a copy of the draft report was sent to the relevant government officials and agencies for their review and feedback. Their comments were taken into account and Borrowers comment are included in appendix G. xiii Summary This Project Performance Assessment Report (PPAR) reviews the performance of the World Bank’s Public Financial Management Reform Project (PFMRP) and the Multi- Donor Trust Fund to Support Public Financial Management Modernization – Phase 2 (henceforth MDTF II) in support of the public financial management (PFM) reform program in Vietnam. The former was approved by the Board on May 22, 2003, in the amount of SDR39.9 million (equivalent to US$54.3 million), consisting of an IDA credit of US$44.3 million and cofinancing of US$10 million by DFID, with a closing date of February 28, 2009. The project received an additional financing of SDR9.2 million (US$14 million equivalent) approved on January 6, 2011. Following extensions, it closed on October 31, 2013. MDTF II was approved on April 1, 2009, and was financed through a grant of US$5.8 million provided by the Australian Agency for International Development, Canadian International Development Agency, Danish Development Agency, European Commission, the government of the Netherlands, and Switzerland State Secretariat for Economic Affairs. Its original closing date was June 6, 2011, which was extended twice to June 30, 2012. The PFMRP’s project development objectives (PDOs) were to “to strengthen the Borrower’s capacity to (i) plan, (ii) execute and (iii) report on its budget and to improve the (iv) transparency and (v) accountability of the budgetary systems and processes.” MDTF PDOs, as evaluated here, are “to support Vietnam to strengthen its capacities for: (a) increased (i) coordination and (ii) transparency in public finance management; (b) improved revenue mobilization; and (c) (i) effective and (ii) efficient public expenditure management.” The lending instrument for the first operation was a specific investment loan and for the second a trust fund. Both projects experienced slow initial starts and were delayed due to different factors. The PFMPR implementation was initiated at a time Vietnam’s shared prosperity strategy was quite effective and the country was moving quickly toward becoming middle income by 2010. However, the pace of reforms slowed down during the 2000s, as evidenced by the late implementation of the treasury single account and adoption of the chart of accounts. The delays also were due to technical reasons, notably the implementation of an information technology system, the procurement of which proved more complicated and time-consuming than expected. Implementation delays were compounded during the financial crisis and the period between 2008 and 2012, when the government was preoccupied with a significant tightening of the macroeconomic and fiscal framework and PFM reforms were not considered priority solutions to these problems. Nevertheless, various systems—notably the Treasury and Budget Management Information System (TABMIS), the treasury system which consumed the bulk of financing—were completed between 2012 and 2013. An additional financing approved in 2011 helped in this respect. Initial implementation of MDTF II was also slowed by the financial crisis. Another contributing factor was the need to prepare implementation plans (IPs) based on the demand-driven approach embedded in project design. Because of the semi-experimental nature of the approach and broad initial criteria, the first IP plan took time to be finalized. The process was smoother for the second and last IP that, learning from earlier xiv experience, empowered the various departments within the Ministry of Finance (MOF) in the design and implementation of activities. Despite progress in terms of delivery of output, project outcomes appeared at risk in 2013 when the Public Expenditure Financial Accountability (PEFA) report was published. The various indicators were compiled based on 2011–2012 data, and most rated C or below on a scale of A–D. The analysis identified policy actions needed to improve various areas, notably adopting a new budget law and an effective TABMIS, generalizing medium-term expenditure frameworks, and completing various reforms that had been initiated but not completed. The situation had evolved favorably during the last two years of implementation of both projects, but many actions were still pending as they closed— notably in the case of MDTF II. The Vietnamese experience is different from that of other countries. Capacity building not only proved to be effective and sustainable, but also top-down and bottom-up ownership policy actions not yet completed at closing were followed through by the government from 2014 to 2016. This review estimates that at least 22 important PFM reforms are directly attributable to the MDTF II. Similarly, with regard to coverage, functionality, and reliability, TABMIS appears to be one of the best performing systems implemented under similar World Bank operations. As a result, this assessment’s appreciation of both operations differs from their completion reports and IEG validation exercises. Both operations benefitted from strong fundamentals, notably high strategic relevance. Similarly, their results framework had shortcomings, one as the result of the use of log frame, the other from the demand-driven nature of subprojects funded under MDTF II, which was partly addressed through restructuring of results once subprojects were known. Otherwise, both operations’ objectives were sufficiently ambitious, while realistic. The chain of logic between components and their objectives was clear and convincing, notably in the case of PFMRP where the sequencing of subcomponents where meticulously thought through. Finally, the lending instruments were appropriate. Notwithstanding some weakness, relevance of design for both operations is considered substantial. PFMPR achieved its various objectives, principally on the successful implementation of TABMIS as well as development of the debt database and piloting of the medium-term expenditures framework in four ministries and three regions. As a result, the approach is being mainstreamed through the new budget law. Considering progress toward achieving objectives, all four are rated substantial. The mapping of various finalized policy actions against the five objectives of the MDTF II leads to the conclusion that one has been fully met with a rating of substantial, while the other four have exceeded expectations and are rated high. Delays in implementation are a potential source of inefficiency, however, the analysis indicates that implementation periods were reasonable, the TABMIS was cost-effective, and, most importantly, policy action under MDTF II conceptually yielded high rates of return. For these reasons, PFMRP efficiency is considered substantial and MDTF II’s is xv high. Based on the ratings for relevance, achievement of objectives, and efficiency, PFMRP outcome is rated satisfactory and MDTF’s is highly satisfactory. Both operations are expected to be sustainable. The recurrent costs of TABMIS are fully covered by government, the new budget law strengthens PFM systems, and the policy actions have been implemented and seem largely irreversible. Based on consideration of these and other factors, risks to sustainability of achievements under PFMRP and MDTF II are considered respectively moderate and negligible. PFMRP’s strengths included a well thought through design that is reflected in highly satisfactory quality at entry. Uneven supervision is reflected in moderately satisfactory supervision performance. Bank performance is thus rated satisfactory. MDTF II benefited from sound quality at entry, given the iterative demand-driven nature of the project, and effective supervision and project restructuring. Both dimensions are rated satisfactory, as is overall rating for Bank performance. The government gave increasing priority to PFM reforms. The failure to better integrate the development and recurrent budget represents the most significant issue. Its performance under both operations is rated satisfactory. Implementation agency performance is broadly defined to include various departments within the MOF. It was satisfactory in the case of PFMPR and highly satisfactory in the case of MDTF II largely because of proactive follow-up of yet to be completed actions after the project had closed. In both cases, borrower performance is rated satisfactory. The project offers a number of important lessons. General  An evolving project-level monitoring and evaluation (M&E) may be a good substitute for a formal one that is preset at appraisal. This can be the case when the iterative, demand-driven nature of the activities does not allow clear measurable objectives to be set from the outset. Also, approaches such as use of social media may be informative and cost effective. Finally, standalone studies and assessment provide important complements to results-based M&E.  Outcomes for PFM operations are difficult to measure at closing. The Vietnamese cases show how ongoing policy actions take time to mature as do systems. In this instance, project outcomes became fully apparent in early 2016. In cases elsewhere, the initial results may be undermined by a number of factors including poor governance. Even though periodic PEFA assessments are not suitable as M&E for operations because of lags in indicators, they can provide useful information on PFM program progress over time.  PFM reforms need decades to complete. Vietnam is well into its second decade of reform. Both the government strategy and actions needed to reach the higher level objective of good governance show the need for this effort to be sustained into the 2020s. In this context, continued effective provision of financial and, more importantly, knowledge by development partners, will be needed. Strong donor harmonization and coordination have been enabling factors so far. Coordination has become more important now that some partners are no longer xvi participating in the follow-up to the MDTF under preparation, providing separate support instead.  The funding of operations may not be commensurate with their impact. Significant results were attained under MDTF II even though the grant amount was small. The supervision resources devoted to the project were large compared to disbursements because they underpinned an important policy and institutional dialogue. The World Bank correctly continued to support project implementation, even at a time were difficulties were experienced. On the government side, capacity built was sustained and, together with substantial ownership, resulted in actions being continued beyond the timeframe of the project.  Technical assistance operations that are initially focused on a single ministry may have a greater chance of success. Clarity in ownership and leadership of the activities by the MOF, which is the leading institution in PFM, was essential and avoided overcomplicated coordination mechanisms in the case of MDTF II. However, future PFM reforms will inevitably have to cover multiple entities and would need to ensure activities are well-sequenced and coordinated. FMIS (TABMIS)  Strong and sustained government and MOF commitment is essential. Since these projects can take 7–10 years or longer for completion, it is necessary to maintain this commitment over a long period which can extend beyond an election cycle. This commitment can be achieved better if projects are framed as public expenditure management systems reform, rather than just accounting systems reform. In Vietnam, government support was particularly important to address project-staffing issues, resolve business process issues, and tackle interorganization issues.  Comprehensive FMIS implementation as in Vietnam takes about 10 years to complete. These projects are characterized by slow disbursement rates over the initial years, which are spent in systems design, testing, and piloting. The disbursement profile needs to be adjusted accordingly.  Continuity in the project team contributes to success. There was stability on both the government and Bank side, and key members of the team that designed the project remaining engaged in its implementation for the first few years ensured an effective transfer of know-how to field-based and other staff.  Systems sequencing is key. TABMIS implementation in Vietnam illustrates how project implementation for similar systems needs to be phased to achieve significant outcomes, such as good budgetary control and cash management early in the project. It is first necessary to implement modules to cater to core budget execution processes and processing of payments and receipts transactions, across government, before going on to other noncore elements, such as fixed assets and inventory management and human resource or fleet management.  Systems deployment strategies should make sure that a large part of the budget is covered from the outset. A major part of the budgetary resources should be subjected to ex-ante budget and cash control so that meaningful fiscal control and cash management are possible at an early stage in the deployment. A deployment across treasury offices through which these transactions are routed, may be sufficient for this purpose. xvii  Spending unit access is a challenge in the use of treasury centric systems. The lack of spending unit access inhibits ownership at the line ministry and end-user level. This often leads to line ministries developing independent financial systems for their own management use, as in the case of Vietnam. Avoiding this outcome requires that once the basic treasury system has been implemented and is operating in stable mode, the large spending units be empowered and made responsible for the bulk of transactions.  Availability of specialized information technology (IT) expertise within the Bank project team is critically important to assure the quality of design and supervision of the projects that support development of complex IT systems. The Bank team needs to have experienced specialists on board or available to advise the client on important technical issues during the design and implementation phases. These specialists also need to be familiar with procurement practices and procedures for IT related procurements and the World Bank rules under which they will need to be applied.  Staff in the implementing agencies should recognize the inevitability of change. Resistance often comes from staff responsible for operating the legacy systems as well as those who are used to doing their regular work in a given way and are reluctant to change. MOF management has a major role to play.  The design of the main consultancy package has proved to be a significant bottleneck for TABMIS implementation. The consultancy package should be designed to ensure that appropriate consulting help is available throughout the various stages of the project.  Not all needs may be foreseen during the project's preparation and original design. The number of system users of TABMIS increased threefold between design and full rollout and had to be accommodated. Nick York Director Human Development and Economic Management Independent Evaluation Group 1 1. Background and Context Scope 1.1 This Project Performance Assessment Report (PPAR) reviews the performance of the World Bank’s Public Financial Management Reform Project (PFMRP) and the Multi-Donor Trust Fund to Support Public Financial Management Modernization—Phase 2 (henceforth MDTF II) in support of the PFM reform program in Vietnam. Information pertaining to project objectives, funding, key dates, and more are provided in chapter 2. Implementation Background 1.2 The political economy of reforms in Vietnam, including governance and PFM, is discussed in an assessment by the Independent Evaluation Group (IEG) of various budget support operations implemented between 2006 and 2012 (IEG 2015). This PPAR draws on its summary of experience with PFM reforms. However, it should be recognized that the socialist accounting system in Vietnam only prepared a record of financial transactions for statistical and revenue purposes. Accountants were given the same status as bookkeepers, and regulation and law determined accountability, not efficiency and effectiveness. A major change in mind-set was needed to move from top-down central planning and budgeting to a results-oriented, more transparent, and accountable approach, in parallel with decentralization of responsibilities to subnational governments. 1.3 Changes began with the adoption of a state budget law in 1996, leading to reforms such as: (i) implementing a consistent format for Government Finance Statistics (GFS) on fiscal information; (ii) adopting of a uniform budget manual for use starting with the 1998 budget; (iii) prudential regulations on capital adequacy, related-party lending, foreign exchange operations, bank inspection, collaterals, and deposit insurance; (iv) issuing accounting and audit standards; and (v) budget publication starting with the 1998 state budget (Wescott, Nguyen, and Vu 2009). The new budget law in 2002 laid down further building blocks of a modern budgetary system. The project appraisal document (PAD) for PFMRP presented an assessment of the 2002 law (see table 1.1). Table 1.1. Key Provisions of the 2002 Budget Law Area Improvements Managing  The new law further clarifies the roles and responsibilities of central and subnational decentralization government.  Provincial people's councils are given more explicit powers and duties to prioritize resources, including determining allocations to different sectors and transfers to lower tiers.  They also are given more explicit powers and duties to mobilize resources, with some taxes that were assigned to central government now shared.  Local spending units will gradually be given greater budget flexibility.  Local planning will be promoted through the agreement of transfers from the center to provinces for stable periods of three years, in return for which provinces must produce medium-term expenditure plans. Strengthening  Builds on recent efforts to promote fiscal transparency and accountability, allowing for more transparency detailed publication of final accounts for 2000 and the budget plan for 2002 and providing an enforcement mechanism to make communes post their budgets outside offices. 2 and  Henceforth, budgets will be published as soon as they are approved, along with all accountability procedures and regulations.  The National Assembly is given powers to decide detailed appropriations for ministries and transfers to provinces, whereas previously they only approved allocations at the sector level. Promoting  The division of responsibilities between the Ministry of Finance (MOF), the State Treasury administrative Department (within the MOF), the Ministry of Planning and Investment (MPI), and the State rationalization Bank of Vietnam are further clarified. and  The MOF will be the lead agency responsible for budget formulation and debt management. streamlining The State Treasury Department will be the lead agency responsible for budget execution and for government financial management information.  A major effort is under way to simplify and strengthen financial management and reporting requirements. Duplicative and low-value added stages in the budget formulation and allotment processes are being eliminated or consolidated. Area Issues Fiscal  Potential declines in oil revenue. sustainability  Growing mismatch between increasing capital expenditure and declining nonwage recurrent expenditure. Prioritization is carried out separately for capital spending (by MPI) and for recurrent spending (by the Ministry of Finance), with significant imbalances between the two.  Pressure for further civil service pay increases.  Uncertainties associated with extra-budgetary accounts, nonperforming loans to state-owned enterprises, and other contingent liabilities.  Processes for prioritizing expenditures remain ineffective.  The absence of a credible multi-year fiscal framework means that expenditure planning is conducted without reference to medium-term resource constraints.  Operational effectiveness is limited. Vietnam lacks even a basic mechanism for monitoring the actual outcomes of public spending and for feeding this information back into future resource allocation decisions.  Weaknesses in transparency and accountability continue to exist, perpetuating corruption and waste. Cross-cutting  The lack of a fully consolidated budget makes it difficult to monitor total revenues and PFM MIS expenditures as well as the true fiscal position.  Extra-budgetary funds, on-lent official development assistance, and much of commune-level spending are not consolidated into the budget.  The lack of common accounting structures results in numbers that are inconsistent and hard to compare.  The lack of integrated, electronic data recording and reporting results in laborious manual consolidation and manipulation of data from multiple satellite databases and in financial reporting which is neither timely nor accurate.  These deficiencies contribute to the poor flow of budgetary information between government ministries, provinces, donors, and the public. 1.4 This was followed by the enactment of new budgetary oversight regulations that mandated disclosure of budgetary data at all levels of government and increased transparency in public investments projects. Establishment of an Independent State Audit in 2005 was important in promoting accountability of public agencies. As a first step in improving the budget’s mid-term planning framework, the government began piloting a Medium-Term Expenditure Framework (MTEF) in four sectors and for all provinces (with the support of PFMRP) with the intention of mainstreaming the approach in due course. This is now mandated under the latest budget law approved in 2015. Improvements included 3 consolidation of capital and recurrent budgets into a single budget and introduction of budget norms of expenditure planning. 1.5 Despite these developments in Vietnam’s PFM system, by the mid-2000s much remained to be done for PFM systems to meet international standards. The key challenges to improving budgetary efficiency included using internal standards of accounting in the public sector, introducing internal audit mechanisms in public agencies, changing the practices of off-budget borrowing for public investment projects, limiting carry-over of large a portion of annual spending without subsequent reporting, reducing inefficiency in public investment program, and addressing shortcomings in public financial management information systems (World Bank 2005). Some of these areas became the focus of operations. 1.6 Between 2007 and 2012, the PFM system was in a process of change, although progress was slower than expected given the ambitious timing built into the government’s reform program. A modern treasury management system (Treasury and Budget Management Information System, TABMIS) and enhanced external audits were introduced. Nevertheless the 2013 Public Expenditure and Financial Accountability (PEFA) report diagnosed PFM issues (based on 2011 and 2012 information) and revealed significant remaining shortcomings (Government of Vietnam 2013). The PEFA findings are discussed in chapter 2. A major finding of the present review is there was an acceleration of policy actions and capacity improvements between 2012 and early 2016 (not captured by the PEFA) that are largely attributable to or associated with support provided by both operations. To avoid duplication, this is detailed in the efficacy section of the report.1 1.7 However, it is important to note an important institutional issue has not yet been fully addressed. The responsibility for recurrent expenditures rests with the Ministry of Finance (MOF), while that for public investment lies with the Ministry of Planning and Investment (MPI)—a legacy from central planning. The problem of dual budgeting in Vietnam, as elsewhere, consists of lack of integration of capital and recurrent expenditure planning—not solely whether the capital and recurrent budgets are prepared by the same ministry or different ministries. If separate ministries of finance and planning are merged without any other action, coordination between investment and current expenditure processes may remain weak. In contrast, even with two separate ministries, if coordination between the two decision-making processes is close at every major step, the capital and current budgets end up consistent with each other and with government policies, and the dual budgeting problem does not arise. The issue of dual budgeting is therefore institutional, not organizational, and rooted in the political economy of the country. 1.8 As noted in the 2003 project appraisal document of PFMRP, “the government is increasingly aware that it needs to move beyond ‘dual budgeting’ to integrated planning of development policy, recurrent and capital expenditure within a single medium-term framework, consistent with a single, forward-looking fiscal framework” (World Bank 2003a). Even though these issues have been alleviated through various measures, institutionally budget formulation and implementation remain segmented. This issue is taken up again under sustainability. The concept note for the 2016 PER2 notes a series of other issues, notably related to public administration reform. 4 Macroeconomic, Fiscal, and Public Expenditure Performance3 1.9 Macroeconomic stability of PFMRP was implicitly assumed in 2003, whereby high gross domestic product (GDP) growth would be sustained and fiscal deficits would remain manageable at around 2.5 percent of GDP (World Bank 2003a). Internal and external factors changed this situation toward the end of the decade, possibly contributing to slow-down of reforms observed under the Project Implementation Review (PIR) and the last two operations under the Poverty Reduction Support Credit (PRSC) series. In summary, Vietnam’s economy went through a turbulent period between 2007 and 2013. After a prolonged period of growth at an average of about 7 percent per year, the Vietnamese economy began overheating during 2007. GDP growth in 2007 rose to 8.5 percent, and by early 2008, inflation surpassed 20 percent (table 1.2). 1.10 The government recognized the presence of a financial bubble and adopted a package of policy measures in March 2008 to prevent further overheating and to curb inflation. The stabilization program included a reduction of recurrent expenditures by 10 percent and a discontinuation of several nonessential public investment projects. Credit growth was kept to 25 percent during 2008. The authorities switched from stabilization to stimulus in the second half of 2008 as the financial crisis spread from the United States to the rest of the world and affected Vietnam’s exports. This resulted in a substantial increase in the budget deficit in 2009 to 7.2 percent of GDP, compared to 2.2 percent in 2008 (see table 1.2). Table 1.2. Key Economic Indicators 2004–2014 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Output, Employment, and Prices GDP (% change year- 8.4 8.2 8.5 6.3 5.3 6.8 6 5 5.4 6.0 to-year Industrial production 17.2 17.0 16.7 13 7.6 9.3 6.8 4.8 5.9 7.6 index (% change) Unemployment (% 5.3 4.8 4.6 4.7 4.6 4.3 3.6 4.2 3.6 3.4 urban areas) Consumer price index 8.8 6.7 12.6 23.1 6.7 9.2 18.7 9.1 6.6 4.1 (%, period average) Fiscal Balance (as % of GDP) Overall balance, — — -3.6 -2.2 -7.2 -3.0 -1.7 -7.5 -7.4 -6.2 including off-budget items Foreign Trade, BOP, and Debt Exports of goods (US$, 32 40 49 63 57 72 97 115 132 150 billion, FOB) Imports of goods (US$, 35 43 59 76 65 85 107 114 132 148 billion, FOB) Current account balance -1.1 -0.3 -9.8 -11.9 -6.5 -3.8 0.2 6.0 5.6 4.8 (as % of GDP) 5 FDI (net inflow, US$, 1.9 2.3 6.6 9.3 6.9 7.1 7.1 7.2 8.1 8.7 billion) Total PPG debt (as % of 45.6 42.9 46.9 48.4 46.7 48.5 54.5 59.6 GDP) Foreign exchange 3 3 3 3.8 1.9 1.4 1.4 2.3 2.6 2.8 reserves (months of imports of G&S) Financial Markets Domestic credit (% 31.7 25.4 53.9 25.4 39.6 32.4 14.3 7.0 8.8 12.7 change) Short-term interest rate 7.8 7.9 7.8 8.1 10.7 11.6 14.9 9.0 6.5 5.5 (3-month deposit) Exchange rate (1,000 16.5 17.5 17.8 19.5 20.8 20.8 21.1 D/US$) Stock market—VN 308 752 972 316 495 484 351 414 505 546 index (July 2000 = 100) Memo: nominal GDP 79.8 90.8 92.4 103.6 122.8 141.5 171 186 (US$, billion) Source: World Bank estimates. Note: — = not available; BOP = balance of payment; FDI = foreign direct investment; FOB = free on board; G&S = goods and services; GDP = gross domestic product; PPG = public and publicly guaranteed; VN = Vietnam Ho Chi Minh Stock Index. 1.11 Vietnam experienced a period of macroeconomic destabilization in 2009–2010 from a major fiscal expansion and subsidized credit growth. Although it regained macrostability in 2011, weaknesses remained. The procyclical fiscal position of recent years and resultant buildup of public debt increased macroeconomic risks. The economy was vulnerable to internal and external shocks. Vietnam’s current public debt level was higher than that of its regional comparators. The International Monetary Fund (IMF) estimates that a debt to GDP ratio of no more than 40 percent to 45 percent should be the medium-term benchmark for Vietnam (IMF 2014). The latest estimate of debt to GDP (2015) suggests that it has risen above 60 percent.4 Substantial improvements in fiscal management and institutional transformation in the state-owned enterprise (SOE) and banking sectors were still required to reduce the risks to more manageable levels. The situation may improve somewhat once the impact of recent reforms—notably those associated with MDTF II—are reflected in the key indicators. Also, inflation has abated and is estimated at around 0 percent (2015–2016). 1.12 Public expenditures were affected by macroeconomic conditions. The overview of public expenditures is based on available data.5 As noted in table 1.3, government expenditures as a percentage of GDP have declined slightly since the mid-2000s. The fiscal envelope increased substantially in 2009 as a result of the government’s response to the financial crisis. However, total expenditures fell from 33 percent of GDP in 2009 to an estimated 27.6 percent in 2013, mainly on the basis of a 4 percentage point decline in capital expenditures. The government has maintained its focus on social spending. A positive factor is that the coverage of expenditures by the budget has improved. Off-budget expenditures are recorded and include items such as social security payments that are being brought under the budget under new law. 6 1.13 The decrease in capital expenditure can be initially considered as a positive development given the inefficiencies in Vietnam’s large public investment program, on the condition that the reduction in the envelope has been made through the elimination of projects with the lowest rates of return. This has not been established.6 However, the latest investment figure for public capital investment hovers around 5 percent of GDP, which may be too low to sustain high growth, unless it leverages considerable public-private partnerships (PPP) in infrastructure and other sectors in activities with high rates of return. Table 1.3. Fiscal Situation 2004–2014 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Revenue 27.8 28.4 29.7 29.4 29.3 25.8 27.3 26.0 22.7 23.1 21.9 Current revenue 25.2 26.1 27.1 26.1 26.4 23.1 24.3 23.6 20.9 22.8 21.8 Tax 21.8 22.8 24.3 23.5 24.4 20.6 22.4 22.3 19.0 19.5 18.2 Grants 0.4 0.5 0.8 0.5 0.6 0.4 0.5 0.4 0.3 0.3 0.2 Expenditure 27.7 29.9 30.1 33.0 31.5 33.0 30.3 27.7 30.2 30.5 28.2 Recurrent 16.9 17.9 18.5 20.3 19.7 18.1 18.7 17.9 19.9 21.1 20.3 Administration 2.2 2.2 1.9 2.8 2.6 2.2 2.6 2.6 2.7 2.7 2.5 Economic 1.4 1.4 1.5 1.4 1.4 1.5 1.7 1.6 1.8 2.08 1.9 Social 7.7 7.3 8.0 8.0 7.7 8.2 8.4 8.0 8.3 9.9 10.9 Interest payments 1.0 0.8 0.8 1.1 1.1 1.1 1.2 1.1 1.2 1.3 1.7 Other recurrent 4.6 6.1 6.3 6.9 6.9 5.0 4.8 4.6 5.8 5.2 3.4 Capital 10.7 12.0 11.6 12.7 11.8 14.9 11.6 9.7 10.3 7.6 5.3 —On budget NA NA NA NA 8.0 10.0 8.5 7.5 8.3 5.9 2.8 —Off budget NA NA NA NA 3.8 4.9 3.1 2.2 2.0 1.7 2.5 Budget balance 0.1 (1.4) (0.4) (3.6) (2.2) (7.2) (3.0) (1.7) (7.5) (7.4) (6.2) Primary balance 1.1 (0.6) 0.4 (2.5) (1.1) (6.1) (1.8) (0.6) (6.3) (6.1) (4.5) Public savings 10.8 10.6 11.2 9.1 9.6 7.7 8.6 8.0 2.8 2.5 1.6 Source: World Bank estimates. Note: NA = not available. Bank and Development Partners’ Support to PFM 2003–2015 1.14 The operations funded by development partners supported policy actions and capacity building to improve public financial management in Vietnam. The assessment of operations and their analytical underpinning should take into account this factor and that they were mutually reinforcing. Specifically, budget support (PRSC and PIR) would include policy actions that had been identified through analytical work, often prepared with the help of technical assistance (TA), and implemented and followed through with the support from traditional projects (notably PFMRP and MDTF). PFMRP in turn allocated much of its funding to support the implementation of a treasury system that provided the information technology (IT) backbone for sound, accountable, and transparent financial management. 7 Finally, MDTF helped units within the Ministry of Finance define small subprojects that would help prepare and implement various policy actions, build capacity within MOF, and install a new mind-set that built ownership and understanding of the goals from the ground up. This change may help explain why even in the absence of World Bank and development partner operations, many initiatives that had been started prior to 2014 were followed through afterward. This complementarity also explains various references to the early PRSC evaluation and why the two operations covered by this report are evaluated together and with full consideration of the context described above. 1.15 More specifically, some key areas, such as planning process, audits, debt management, and public investment, became the focus of the second PRSC series (2007– 2012), with complementary capacity building and TA provided under PFMRP and MDTF I and II. A parallel development policy operation (DPO) series (PIR) was launched in 2009 as an additional boost for public investment reforms that had been largely stagnant between 2007 and 2009. These operations were part of the series of complementary programs and operations overseen by the World Bank, implemented by the government, and cofunded by the World Bank and donors. Finally, the Economic Management Competitiveness Credit (EMCC) replaced the PRSC series. Its operations focus on fewer sectors and aim to tackle, with apparent success in the PFM area, “big ticket items” such as the public investment and 2015 budget laws that had eluded previous attempts. Timelines from approval to closing are summarized below in table 1.4. The EMCC third and final operation in the series was approved in May 2016. 1.16 This reinforcing combination of operations is conceptually strong and contributes to the desired development outcomes—when properly coordinated. The PRSC provided potential impetus for accelerating and focusing on policy actions, thus framing the program. The PFMRP delivered the large building blocks for setting-up a sound PFM system, including funding the financial management information system (FMIS), called TABMIS in Vietnam, and providing capacity building. The MDTF availed focused grant resources to the MOF for diagnostic studies and TA to ensure important details and needed institutional and policy reforms at the MOF level did not fall through the gap, providing building blocks for the PFMRP. Table 1.4 World Bank Operations Supporting PFM Reforms in Vietnam (2001–2015) 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Operation PRSC BS BS BS BS BS BS BS BS BS BS BS BS PIR BS BS BS EMCC BS BS BS PFMRP TA TA TA TA TA TA TA TA TA TA TA MDTF I SIL SIL SIL SIL SIL MDTF II SIL SIL SIL SIL SIL AAA (MDTF III) SIL Note: AAA = analytical and advisory assistance; BS = budget support; TA = technical assistance; SIL = specific investment loan; EMCC = Economic Management Competitiveness Credit; MDTF = multi-donor trust fund; PFMRP = Public Financial Management Reform Project; PIR = Project Implementation Review; PRSC = Poverty Reduction Support Credit. 8 2. Objectives, Design, and Relevance Objectives PFMRP 2.1 The project development objectives (PDOs) from the Development Credit Agreement (DCA) are unchanged by additional financing: “to strengthen the Borrower’s capacity to (i) plan, (ii) execute and (iii) report on its budget and to improve the (iv) transparency and (v) accountability of the budgetary systems and processes”. The statements in the DCA and PAD are largely identical, even though the transparency objective is not mentioned in the PAD (appendix A) as a project objective, but rather a higher level one. This review encompasses all five objectives. PDOs, while numerous, are well defined and reflect the appropriate level of ambition (strengthen and improve), which as noted in the monitoring and evaluation (M&E) section is contradicted by some indicators that refer to “best practice.” MTDF II 2.2 Original (as per grant agreement, section 2.01): “The objectives of the Project are to support the Recipient to develop capacities to strengthen its public finance management system through: (a) developing robust and efficient systems for (i) raising and (ii) deploying resources; (b) improving the (i) quality and (ii) effectiveness of public expenditures in achieving the Recipient’s social and economic policies; (c) supporting the introduction of market orientation in public finance management; and (d) promoting (i) transparency and (ii) accountability at all levels in the management of public finances”. 2.3 Restructured (as per amended grant agreement, section 2.01): “The objective of the Project is to support Vietnam to strengthen its capacities for: (a) increased (i) coordination and (ii) transparency in public finance management; (b) improved revenue mobilization; and (c) (i) effective and (ii) efficient public expenditure management”. 2.4 Both the MDTF II PAD and the grant agreement refer to objectives (plural) which suggest that “through” was intended to describe subobjectives and not component or activities as is usually the case. This notion is further reinforced at project restructuring (five subobjectives). Table 2.1 reorders and matches the subobjectives. 2.5 As per PPAR guidelines, intentions that appear evident in the original project documents, and given the clarified PDOs in the restructured project, IEG management has approved undertaking the evaluation based on the subobjectives. In practice, this leads to three unchanged subobjectives (i, iv, and vi), one dropped from the original (deploying resources covered by iii and iv), two modified (iii and vii), and one dropped at restructuring (v). Table 2.1. MDTF II Subobjectives Original Restructured Comment Develop capacities to strengthen its Strengthen capacities for: — public finance management system through: 9 Raising resources Improved revenue Largely identical mobilization Deploying resources — In practice subsumed within quality and effectiveness of public expenditures. Quality of public expenditures Efficient public Quality and efficiency not quite expenditure management the same but substantial overlap. Effectiveness of public Effective public Identical expenditures expenditure management Market orientation in public finance — This was essentially about price management control. Dropped at restructuring. Transparency at all levels in the Transparency in public Largely identical management of public finances finance management Accountability at all levels in the Increased coordination in More modest after restructuring management of public finances public finance management Note: — = not applicable. RELATED OPERATIONS 2.6 The objectives for PFMRP and MDTF II were linked to that of other noted operations and have a bearing on strategic relevance. They are:  MTDF I: Provide technical assistance to support efforts to modernize the management of public finances in six areas. The implementation completion memorandum concludes that the project attained its objective.  PRSC First Series: Improving public expenditure management. According to IEG, this objective was met.  PRSC Second Series: The relevant objective was “building institutions that can support the country’s development strategy through improving governance.” The IEG assessment of modest progress related to PFM7 was based on the following: “In the policy area of planning no substantial progress was achieved in public investment management and integration of modern planning in provinces. In PFM, in spite of some achievements in external audit and introduction of a modern treasury management system (a major contribution of PFMRP), the overall progress in PFM in 2007–2012 was below expectations. Specifically, the issues of off-budget expenditures, conformity of the budget classification with international standards, and lack of a major progress in internal audit remain major shortcomings.”  PIR: Support modern governance. Support to the stimulus package was an additional objective. According to the IEG assessment, this project failed to achieve its main objectives. A new public investment law was not adopted , macroeconomic stability was not maintained, and the stimulus package was ineffective.  EMCC: Contribute to enhanced competitiveness to boost growth and poverty reduction. The third and final operation in the series has not yet been approved. It is noteworthy that some tax reform under EMCC 2 has been prepared with partial contribution from MDTF II. 10 2.7 The objectives of the DPOs were set at a higher, more ambitious level than those for the PFMRP and MTDFs. This has a bearing on how achievements of each are being or were assessed. The full list of PFM-related measures is provided in appendix B, together with tax reforms, which appear under the growth pillar of the PRSC but benefited from MDTF support. Design PFMRP 2.8 The components were defined based on a clear diagnostic of the PAD. Their internal sequencing, among subcomponents, was well thought through and considered the best among PFM operations assessed by IEG during FY16. This reflected strong conceptual and operational thinking and attention to detail with clear guidance on how to implement the operation and to anticipate and deal with issues, which was not relegated to an operational manual. . The components8 are: Component 1: Strengthening Treasury and Budget Management 2.9 Component 1 involved primarily the implementation of TABMIS. It built on substantial previous government investment in information and communication technology. An international consulting firm designed the system based on international experience and global best practice. Functional and technical specifications were closely aligned with the World Bank and IMF Treasury Reference Model which was considered good practice at the time and still remains relevant today (Hashim and Allan 2001). Subcomponents were:  Subcomponent 1 (a): Procurement assistance and Independent Bid Evaluation.  Subcomponent 1 (b): Independent Validation and Verification (IV&V). Assistance to not only select its turnkey solution partner and negotiate a contract with them, but also thereafter to verify and validate delivery against the milestones stipulated.  NB. At the government's discretion, the two areas of support above might have been contracted directly to the international firm responsible for the TABMIS design (competitively selected originally) in order to ensure continuity with earlier work. While this option was not exercised, it reflected a forward-looking approach that tried to reduce procurement constraints.  Subcomponent 1 (c): Training and change management. A "training of trainers" approach would be used to deliver training to an estimated 14,000 trainees in the State Treasury Department, Ministry of Finance, Ministry of Planning and Investment, and provincial and district finance departments and line agencies.  Subcomponent 1 (d): Installation and configuration of TABMIS in the treasury head office, MOF, MPI, and one pilot province.  Subcomponent 1 (e): Roll out to all treasury offices in provinces and districts.  Subcomponent 1 (f): Roll out to finance departments and planning departments at provincial and district levels. At the completion of this subcomponent, the legacy systems will be de-commissioned. 11  Subcomponent 1 (g): Pilot implementation in selected large spending units.  Subcomponent 1 (h): Government TABMIS implementation team.  Subcomponent 1 (i): Supplementary technical assistance with financial management rationalization and reform. A major input will be to support the establishment of a treasury single account (TSA). Similarly, the new system will support a gradual move to accruals accounting.  NB. Recurrent operating costs. The originally estimated amount was US$3.2 million annually.9 Component 2: Strengthening State Budget and Investment Planning  Subcomponent 2 (a): Assistance to MOF and MPI with preparation of a Medium- Term Fiscal Framework (MTFF) and oversight of MTEF pilots. Funding provided for a peripatetic rather than resident chief technical advisor (CTA) who will support subcomponents 2 (b) and (c).  Subcomponent 2 (b): Assistance with preparation of MTEFs in four pilot sectors and four pilot provinces. The sectors are (i) education and training, (ii) health, (iii) transport, and (iv) agriculture.  Subcomponent 2 (c): Workshops to support establishment of medium-term budget planning.  Subcomponent 2 (d): Design and implementation of a budget preparation system. Component 3: Strengthening Management of Public Debt and Monitoring of SOE Fiscal Risks  Subcomponent 3 (a): TA and capacity building to support strengthening of recording and management of domestic debt.  Subcomponent 3 (b): Procurement of systems for recording, consolidation, and management of public debt.  Subcomponent 3 (c): Monitoring of SOE fiscal risks. This subcomponent will not address issues of SOE reform itself but will focus on the aggregate monitoring of the basic SOE liabilities that may impact Vietnam’s fiscal situation. Component 4: Project Management Support 2.10 The project was originally supported by an International Development Association (IDA) credit of SDR39.9 million (US$54.3 million equivalent), including US$10 million cofinanced by the U.K. Department for International Development (DFID). An additional financing of SDR9.2 million (US$14 equivalent) was approved in 2011. It should be noted that other reasons for the difference between total planned actual costs include fluctuations in the U.S. dollar and SDR exchange rate as well as lack of information on actual recurrent operating costs, funded by the government. Furthermore, while the original IDA credit was fully disbursed, US$4.7 million of the additional financing was cancelled after closing. An overview of planned and actual expenditures by component is provided in table 2.2. 12 Table 2.2. Project Cost Planned and Actual by Component and Financing Source Cost Percentage Component (US$, million) of Planned Actual Planned Strengthening treasury and budget management 46.97 77.94 165.9 Strengthening state budget and investment planning 3.80 3.13 82.4 Strengthening management of public debt and monitoring of SOE 2.62 2.43 92.7 fiscal risks Project management support 2.70 4.22 156.3 Other (recurrent operating costs and contingencies) 15.36 0.00 "NA" Total 71.45 87.72 122.8 By Financing Source Borrower 7.14 8.10 113.4 DFID 9.99 10.46 104.7 IDA 54.33 69.15 127.3 Total 71.46 87.71 122.7 Note: DFID = U.K. Department for International Development; IDA = International Development Association; SOE = state- owned enterprise. N.B. The data in the "planned" column only reflects the original budget of the project, and is not updated with the additional financing, exchange rate gain/loss and increasing counterpart funding of the Government. Update figures as reported by Government are provided in appendix G. MDTF II 2.11 Specifying details and the funding level for each component was not possible at preparation. The grant was designed to be flexible and available to respond to the needs of activities as they arise (demand-driven). Prioritization and strategic focus will be identified by MOF in the annual implementation plans (IPs) based on the proposals submitted by beneficiary departments and agencies. Furthermore, MOF intended to keep the single strategy document (SSD) as an open document in the sense that it will allow for including supplemental reform initiatives to be identified through future diagnostic studies. The project originally consisted of the following parts (components):  Part I: Strengthening the recipient’s capacity in the implementation of selected reforms and managing technical assistance programs.  Part II: Conducting analytical and diagnostic work in the seven key areas as specified in the 2005 SSD (at time of appraisal 2006–2010 SSD) which included: o Budget management: strengthening institutional framework, building capacity, developing TABMIS as well as developing a medium- and long-term public expenditure framework to meet the requirements of public financial management. 13 o Budget revenue management (taxation and customs): strengthening the tax administration system and customs modernization to meet the requirements of economic growth and intensive international economic integration. o Debt management: strengthening the system for recording domestic and external debt, and developing debt management and risk management systems. o Financial market supervision and bond market development: establishment of a financial service supervision agency and a road map for bond issuance and development of a bond market. o Corporate finance management: strengthening financial management capacity in the government’s business activities to speed up the equitization process and increase the efficient use of capital by state enterprises. o Public assets management: strengthening the legal framework, strengthening the capacity of the officials in charge of managing public assets, developing a better system of monitoring public assets, and enhancing the examination and inspection over the management and utilization of public assets. o Price control: developing and improving the legal system for price control and developing a professional price valuation system under the socialism-oriented market economy in Vietnam. 2.12 The project-restructuring document does not formally present revised components or subcomponents, even though this happened in practice. The Implementation Completion and Results Report (ICR) notes that that components may be linked to the IPs, which as reflected in the revised results framework of the restructuring paper (appendix D) are the following, and comingle activities previously listed under Part I and Part II:  Increased coordination and transparency across the national PFM system through a clear reform strategy, greater disclosure, improved oversight capacity, more secure IT systems, and compliance with PFM legislation.  Improved revenue mobilization through equitable and efficient tax policies and efficient customs administration.  Strengthened expenditure management through higher accounting standards and improved capacity in financial reporting, cash management, and state asset management. 2.13 Allocations to components and subcomponents were not adjusted in any of the project documents, following the two main restructurings.10 2.14 The project was originally supported by a trust fund of US$5.12 million, which was subsequently increased to US$7.2 million through additional financing provided by existing and new donors.11 The following donors funded MDTF II: Australia, Canada, Denmark, the European Commission, the Netherlands, and Switzerland. Canada, the European Commission, the Netherlands, and Switzerland had undisbursed commitments at closing, which were reimbursed. Neither the PAD and the restructuring document, nor the ICR provide information on planned and actual cost by components. Instead, the ICR groups the expenditures around 10 themes or subcomponents based on the IPs, Because of the lack of other data, this approach is also adopted herein and presented in table 2.3. 14 Table 2.3. Project Cost Planned and Actual by Activity and Financing Source Cost Percentage Activity (US$, millions) of Planned Actual Planned State budget expenditure management reform 0.85 0.87 102.4 State budget revenue management reform 1.53 1.11 72.5 Government debt management reform 0.15 0.11 73.3 Financial market 0.17 0.15 88.2 Public asset management reform 0.39 0.58 148.7 Price management reform 0.03 0.03 100.0 Corporate finance management 0.4 0.38 95.0 Support for PFM sector 1.23 1.27 103.3 Support for implementation of components 0.77 0.61 79.2 Project operational support 0.26 0.28 107.7 Total 5.78 5.39 93.3 By Financing Source Borrower 0.37 0.28 75.7 Six development partners 5.41 5.1 94.3 Total 5.78 5.38 93.1 Implementation PRINCIPLE CHANGES AND DATES 2.15 The key information for each project is summarized in table 2.4 below. Table 2.4. Key Dates and Modifications Area PFMRP MDTF 2 Key dates: Approval 05/22/2003 04/01/2009 Effectiveness 09/04/2003 09/07/2009 MTR (planned) 12/11/2006 (11/30/2006) 08/15/2011 (07/31/2011) Closing Original 02/28/2009 06/30/2011 First extension 02/28/2011 06/30/2012 Second extension 02/28/2013 12/31/2013 (actual closing) Third extension 10/31/2013 (actual closing) Closing DFID grant 06/30/2011 Other changes or restructuring: First 02/03/2011. Changes: 15 (i) Adjusting the scope of 01/06/2011. Following several subcomponents. changes: (ii) Allocation of additional (i) Additional financing: The financing, reallocation of funds, Swiss State Secretariat for new disbursement category Economic Affairs joined the (consultants’ services, training trust fund. and workshops), and changes to (ii) Revision: Reallocation. percentage of financing. Second 02/26/2013. Changes: (i) Revising project activities 25/06/2012. Change in: and their associated output (i) PDO. indicators. (ii) Project financing. Due to (ii) Reallocating proceeds. slow progress, donors were unable to disburse all committed amounts. The Netherlands exited at the mid- Third 09/19/2013. Changes: term review. (i) Remove selected activities which are no longer appropriate technically or feasible within timeframe. (ii) Revise a number of sector and output indicators. Source: Project documents. Note: The DFID grant was extended a third time but the relevant document does not appear to have been filed. NA = not available. GENERAL 2.16 The implementation context has been discussed in chapter 1 and the key changes to the projects are listed above. It should be noted that both operations suffered from delays, due in part to the fact that the initial steps involving either launch of complicated procurement or preparation of a demand-driven sound implementation plan took longer than expected, but not unreasonably so. Additional implementation factors pertaining to each operation are discussed below. 2.17 In the case of PFMRP, various factors, most listed in the ICR and verified by the IEG mission through interviews, affected implementation. The TABMIS experience was reviewed as a case study and is summarized further below. With the benefit of hindsight, the mid-term review (MTR) in 2006 took place was unable to identify all the issues affecting the project as some were not apparent due to slow and limited implementation. The additional financing provided an opportunity to assess and accelerate progress, which appears to have been taken. 2.18 The interest and participation of various ministries in the MTEF pilot were uneven. Issues included lack of ownership and this task not being perceived as a core responsibility, key expected reforms needed for MTEF rollout were delayed, and the budget preparation subcomponent was dropped in order to simplify the project and focus on TABMIS (events proved this decision to be the correct). The main issue hindering the implementation of the Debt Monitoring and Financial Analysis System (DMFAS) was time taken for World Bank to approve deviations from standard contracts. 16 2.19 The implementation of PFMRP was based on two implementation plans (IP 1 and 2) agreed to in March 2010 and June 2011. Their preparation process proved quite lengthy. The MTR that took place in 2011 triggered additional financing by donors (as anticipated in the PAD) as well as project restructuring a year later. 2.20 Factors that affected implementation included the need to enhance MOF’s capacity to manage a complicated TA program with many beneficiaries and activities.12 The transition to the new Financial Development Strategy (FDS) and ensuring the alignment of subprojects to it was another source of delay, as was insufficiently clear criteria for screening subprojects. TABMIS IMPLEMENTATION EXPERIENCE 2.21 The TABMIS experience is given special prominence here as it accounted for the majority of the funding under PFMRP and is a major contributor to improvements in PFM. In the early 2000s, the government of Vietnam’s Financial Management Information Systems were fragmented and had overlapping and conflicting functionality. Little attention was paid to critical flows of information between system components and beyond the system. The result was lack of integrity in overall fiscal data, transparency and control. The problems could be traced to weaknesses at two levels: (i) institutional weaknesses, notably lack of a common accounting standard and of consolidated budget; and (ii) technological weaknesses. 2.22 The acquisition of TABMIS was based on:  A turn-key contract covering: (a) The procurement of a new TABMIS; (b) installation and configuration of TABMIS in Treasury head office, MOF, MPI and one pilot province; (c) roll out to all Treasury offices in provinces and districts and to Finance Departments and planning departments at provincial and district levels; and (d) Pilot implementation in selected large spending units.  Technical assistance for: (a) procurement assistance and Independent Bid Evaluation; (b) IV&V; and (c) training and change management.  Supplementary technical assistance with financial management rationalization and reform.  Implementation complemented by: (a) a government TABMIS implementation team; and (b) payment by government of recurrent operating costs. 2.23 In 2005, IBM Singapore was awarded the contract for implementing TABMIS after a Bank “no objection letter” was issued on in June 23 2005. The contract covered all implementation activities envisaged after two-stage international competitive bidding (ICB) based on tender documents and technical specifications prepared by a consultant. The contractor proposed Oracle EBS as the application software to be used as a basis for TABMIS implementation. 2.24 The design parameters for the systems ensured that (i) the scope of the implementation would be restricted to the modules to cater to Core Budget Execution Processes, and processing of payments and receipts transactions; (ii) a Treasury Centric System would first be implemented and then decentralized to spending units as necessary; (iii) initially budget preparation will be undertaken outside the system or by another system, with in year budget transactions entered in the TABMIS system; (iv) Budget Department, 17 Treasury, and Line Ministries would use the same system to process their transactions and would share databases; (v) transactions should be captured in real time as they occur; and (vi) the TABMIS databases should be treated as the primary source for financial reporting within government; there should be no second set of books. 2.25 Design parameters and sequencing conformed to the Bank’s experience in implementing such projects. Systems procurement was carried out through a competitive two stage ICB process based on well-prepared detailed technical specifications that described the scope of the system in terms of the functional processes that would be covered and the agencies where the systems would be installed. 2.26 The two main contracting assignments that were involved in project implementation were a diagnostic and design consultancy and a contract for actual system implementation. The design of the main consultancy package proved to be a significant bottleneck for systems implementation in Vietnam. It was split into various contracts with the same consultant. However, the consultancy for the supervision/implementation phase was called an IV&V consultancy. The consultants misread the purpose of the hiring and saw it as an audit type of exercise where they were to critique the initial design and the implementation work, instead of actually helping the government to ensure that the contractor implements the system in accordance with the design that had been specified and approved earlier. The government finally terminated the contract and was left with limited support in the contract management and for supervising implementation. This resulted in a situation that the government had to rely exclusively on the implementing contractor advice during this phase. 2.27 TABMIS in Vietnam came into operation in phases over the period 2010–2011. It is now operating in stable mode. It is a mission critical system and is the backbone of the government of Vietnam’s financial operations. The system serves as a common platform to all treasury offices and financial agencies at all four levels of the government for budget allocation and control, payment and receipt processing, expenditure and revenue accounting and reporting. 2013 PEFA 2.28 The PEFA issued in 2013 was funded by the MDTF II (Government of Vietnam 2013). Its measurement of various performance indicators is shown in table 2.5. 2.29 The PEFA is based on 2011–2012 data. It does not capture the project’s main contributions, which were back-loaded toward 2012–2013 and only fully visible since 2015. Nevertheless, the PEFA is an important exercise as captures some of the implementation challenges faced by the two projects. The fact that this highly critical report was made public is an important step toward fiscal transparency. Furthermore, if repeated in 2019 as apparently planned, it will provide a useful baseline to assess achievements under the two operations and confirm the findings of this assessment concerning efficacy and outcome. 2.30 Only 12 out of 31 indicators were rated B or better (only one was rated A). This result was below that of most comparators and was an indication of the challenges ahead. The report nevertheless notes reasons to expect progress in several areas. For example, TABMIS, which was not fully operational at the time, is highlighted as a positive factor13 (e.g., in limiting future contingent liabilities). The planned review of budget law is also seen as an important step forward, now completed with support provided under MDTF II. The PEFA 18 assessment concludes by listing other areas where further policy actions are needed, most of which were covered by one or both projects. They include the medium-term fiscal and expenditure framework; tax management; public debt management; public asset management; public procurement; and auditing. Table 2.5 PEFA Performance Measurement 2013 PEFA Dimension Score A. Credibility of the Budget Score PI-1 Aggregate expenditure out-turn compared to original approved budget C PI-2 Composition of expenditure out-turn to original approved budget D+ PI-3 Aggregate revenue out-turn compared to original approved budget D PI-4 Stock and monitoring of expenditure payment arrears NR B. Comprehensiveness and Transparency Score PI-5 Classification of the budget D PI-6 Comprehensiveness of information included in budget documentation B PI-7 Extent of unreported government operations C+ PI-8 Transparency of Inter-Governmental Fiscal Relations B+ PI-9 Oversight of aggregate fiscal risk from other public sector entities C+ PI-10 Public Access to key fiscal information B C. Budget Cycle C (i) Policy-Based Budgeting Score PI-11 Orderliness and participation in the annual budget process B PI-12 Multi-year perspective in fiscal planning, expenditure policy and budgeting C C (ii) Predictability and Control in Budget Execution Score PI-13 Transparency of taxpayer obligations and liabilities C+ PI-14 Effectiveness of measures for taxpayer registration and tax assessment C+ PI-15 Effectiveness in collection of tax payments C+ PI-16 Predictability in the availability of funds for commitment of expenditures B+ PI-17 Recording and management of cash balances, debt and guarantees B PI-18 Effectiveness of payroll controls B PI-19 Competition, and appealing mechanism in procurement C+ PI-20 Effectiveness of internal controls for non-salary expenditure D+ PI-21 Effectiveness of internal audit D+ C (iii) Accounting, Recording and Reporting Score PI-22 Timeliness and regularity of accounts reconciliation B+ 19 PI-23 Availability of information on resources received by service delivery units A PI-24 Quality and timeliness of in-year budget reports D+ PI-25 Quality and timeliness of annual financial statements D+ C (iv) External Scrutiny and Audit Score PI-26 Scope, nature and follow-up of external audit C+ PI-27 Legislative scrutiny of the annual budget law B+ PI-28 Legislative scrutiny of external audit reports B+ D. Donor Indicators Score D-1 Predictability of Direct Budget Support D+ D-2 Donor information for budgeting and reporting on project/program aid B D-3 Proportion of aid managed by national procedures C Source: PEFA 2013 FIDUCIARY ISSUES AND SAFEGUARDS 2.31 Given the similarity in both projects’ experience, the fiduciary and safeguards issues are treated together. Neither project triggered any social issues or safeguards. MDTF II risk was rated as modest with respect to both procurement and financial management, on the assumption that the Project Management Unit (PMU) would be adequately staffed. Procurement risks were identified in the PFMRP PAD and mitigated through actions such as hiring a specialist and training (World Bank 2010). While project financial management risks were considered low, issues at country level were considered more significant. Risk mitigation included appropriate staffing of the PMU. The minor difference in risk perception of the two project is explained by the latter’s greater complexity, including a challenging TABMIS procurement, and the fact that national systems improved over time and became less risky. 2.32 An unusual but quite appropriate provision under PFMRP at time of appraisal is that the Bank agreed to the sole-source extension of a key contract (also discussed under quality at entry). Even though the contract was eventually retendered and awarded to the same firm, this feature is noteworthy. 2.33 Neither the Implementation Status and Results Reports (ISRs) nor the ICRs describe any issues with fiduciary compliance, and this was confirmed during the field visit. The only issue noted was in the MTDF II ICR—difficulty in finding qualified local experts, even though interviews with the beneficiary department suggest this was not seen as a binding constraint. The PFMRP ICR states that initial problems early on did not persist. Both projects submitted all audits on time, and these were unqualified. 2.34 The review of fiduciary arrangements raises two issues concerning their efficiency. First, even though the MDTF II PAD recognized that full-time fiduciary staff would not be needed, it required such staffing at the PMU. This review could not establish why this capacity could not be shared with the PFMRP PMU (both at MOF), even if two PMUs were needed to manage the substance of each operation. Second, the low procurement thresholds 20 (for example in the case of consultants US$100,000 for firms and US$50,000 for individuals) may have been typical in the early 2000s. However, it is unclear why these thresholds were not increased over time and why the same levels were kept for the MDTF II project. All terms of reference for consultants are subject to prior review by the World Bank, which is a good way to add value and bring knowledge to bear on individual activities. However, low thresholds would have taken away supervision resources from substantial implementation support, with questionable benefits. 2.35 Finally, the difficulties under PFMRP in contracting with the United Nations Conference on Trade and Development (UNCTAD) are discussed under Bank performance. What is worth repeating is that the process for obtaining the procurement waver was time consuming and inefficient. Relevance of Objectives 2.36 Both operations supported the same government and Bank objectives, complemented each other, and were implemented during overlapping periods. MDTF II did not overlap with the first years of PFMRP, but both operations closed during the last quarter of 2013. Notwithstanding the differences in their PDO statements (in the case of MDTF II both pre- and post-restructuring and needlessly complicated), they shared the same strategic relevance. 2.37 PFM reforms were given prominence in both overall development strategies of Vietnam as well as in specific ones. In terms of overall government strategies:  The 2002–2005 Poverty Reduction and Growth Strategy supported three themes, one of which concerned the adoption of a modern public administration, legal, and governance system.  The Socio-Economic Development Plan (SEDP) 2006–2010 set out four broad objectives, including improve governance. As further developed in the 2007 Vietnam Development Report (VDR), PFM reform was a substantial part of this theme. 2.38 The operations were also supported by specific strategies adopted by government, including:  Public financial management was central to the government's Administration Reform Master Program for the period 2001–2010, endorsed by the prime minister in September 2001.  The Public Administration Reform Master Plan for the period 2001 to 2010 stated that treasury management (the basis for TABMIS) will remain a central government function, and this was clearly stated in the new state budget law.  In 2004, the prime minister approved Decision 211 to guide the country's PFM development, focusing on the broad strategic areas of resource mobilization, budget management (preparation and execution), debt management, and public administration reform.  To operationalize these umbrella plans, the Minister of Finance approved the 2007 SSD, which outlined seven key reform areas. The seven subcomponents of the MDTF II project were directly mapped to these reform areas. It should be noted that some 21 development partners felt that the SSD lacked sufficient sequencing, which contributed to the MDTF II’s slow start.  A new 2010–2015 FDS was adopted to replace the SSD. This strategy paved the way for bringing to fruition policy and institutional actions started under both projects, especially MDTF II. 2.39 From the standpoint of development partners, PFM reforms appeared to be a very important element of the overall reform agenda in Vietnam, which accounted to a substantial amount of their financial programming, especially once DPOs described in chapter 1 are taken into account. This importance is further reflected by the fact that throughout the period under review, the bulk of this support was fully harmonized and provided through the cofinancing of World Bank managed operations as well as harmonized dialogue, mainly through the PRSC and coordinated policy dialogue under EMCC. 2.40 All relevant Bank strategies were fully aligned with government strategies and priorities. PFM reforms were this given high prominence in all country assistance strategies (CASs) since 2002, including the current country partnership strategy (CPS) approved as both projects closed. More specifically:  The FY03–06 CAS third goal of promoting good governance was supported by improving public financial management, information, and transparency. Indirect but important contributions were made to achieving other governance subgoals, including public administration reform. The CAS identified four risks, one of which was failure to address governance issues. The PFMPR and PRSC operations launched at this time aimed to minimize this risk from the PFM standpoint.  In line with the SEDP, the FY07–11 CPS maintains improving governance as a pillar. It considers continued PFM reforms as essential to improving governance and to strengthening key government systems.  The FY12–16 CPS focuses on improving competitiveness. More specifically, it supports measures to strengthen CPS Outcome 1.1: Improved Economic Management and Business Environment. This includes reforms to strengthen Vietnam’s macroeconomic policy framework and PFM. The CPS further notes that implementation risks would be partially mitigated by capacity building for PFM.  The two operations under review are explicitly mentioned as lending Bank instruments in all three strategies. 2.41 High prominence has been given during the past decade and half by government, development partners, and the Bank to governance reforms, of which PFM constitutes a significant part of financial support. The objectives of both operations, which complemented others, notably large DPOs, were fully aligned with this priority and covered goals that were attainable and realistic, and yet sufficiently ambitious. These objectives might have benefitted from being restated. In the case of MTDF II, there were too many, and they could have been clustered. As for PFMRP, as was done for the PFM operation in the Russian Federation, the objective might have been specified in the form of putting in place functioning information systems, including a TABMIS with adequate coverage of expenditures. Despite this minor shortcoming, strategic relevance for both projects is rated high. 22 Relevance of Design PFMPR 2.42 The PFMRP was appropriately focused on three clear priority areas, with other important areas related to PFM, public expenditure management, and external debt management, which were supported through other operations or advisory services provided by the Bank and other donor agencies (World Bank 2003a, 17–18). The project’s components and subcomponents were well related to the objectives and were sequenced well, one subcomponent often reinforcing the other. Their design addressed the key institutional and technological challenges, in light of the available lessons from the international experience. 2.43 The choice of lending instrument (a single, technical assistance project) was also reasonable. The alternative of breaking down the financing into two or more tranches (as in the case of adaptable program lending, which had been the chosen instrument in some cases such as Russia) had been considered during the preparatory phase. However, in the case of Vietnam, the choice was made of procuring TABMIS under a large, turnkey contract covering all hardware, software, and implementation services. This approach addressed concerns over weak contract management and technical expertise available within the government to oversee separate contracts for these components, and reduced uncertain over availability of future financing. Under this approach, breaking down the financing into two or more phases would have been impractical and inefficient. In practice, when additional resources were needed to fill a financing gap, they were provided through IDA’s additional financing. 2.44 The results framework for this project was limited to the results matrix in appendix D of the PAD, which reflected the standard logframe approach in use at the World Bank at the time of the project’s appraisal. One of the general weaknesses of this approach, which is no longer used by the World Bank, was the lack of explicit discussion about how project outputs would translate into the postulated intermediate and final outcomes and these into the project’s specific objectives. Moreover, two of the key associated outcomes proposed by the PAD for this project, that is, “better planning” and “fiscal sustainability” were not specific, but were clearly related to debt management and MTEF, which were project activities. The logframe also included higher level objectives (promoting good governance and strengthening PFM information and transparency). The latter could have also been set as a project objective, and both were closely related to PDOs. The higher level indicators were also relevant. 2.45 Overall, the logical chain from outputs to intermediate outcomes and to specific objectives or their key associated outcomes that was implicit in the project’s results matrix was broadly plausible. There was a similar close relationship between subcomponents, components, and PDOs. The relevance of design is therefore rated substantial. MDTF II 2.46 The MDTF II design has to be seen in the context of the overall PFM program it was supporting. As mentioned earlier, the main policy actions were supported by DPOs and to some extent PFMRP, which also funded major investment in technology infrastructure. 23 However, the MOF needed resources to undertake strategic studies and capacity building on PFM-related activities. The small flexible, demand-driven TA project was thus the right instrument that complemented the others. 2.47 The MDTF was intended to support the implementation of the SSD. At approval, it consisted of two parts (components), one designed to build capacity and the other to conduct analytical and diagnostic work. The second component specified seven thematic themes (subcomponents) fully aligned with the SSD (implicitly the first component would also cover the same themes). Even though the link between the components and the higher level objective seemed quite clear, an unnecessary complication was introduced by a complicated PDO statement, each element of which nevertheless could be associated with one of the components. Project restructuring streamlined both PDOs (which remain somewhat elaborate) and components were simplified. Subcomponents were altered and redefined as components, and capacity building and studies were folded into each. Furthermore, the components were based on IP 1 and 2, which can be thought of as inputs to the intermediate outcomes in the chain of logic. 2.48 The original design struggled with how to establish indicators to measure project outcome when activities were not known. Higher level ones from hypothetical PEFAs were considered, but ultimately the PAD included tentative results linked to the SSD themes. These were still set at too high level. The PAD also noted that these indicators would be revised once the IPs would be known. Following project restructuring, the project redefined results that were based on the expected outcomes of IPs. 2.49 The chain of logic had weaknesses that were not fully addressed at restructuring. Nevertheless, the choice of instrument was appropriate, and there were credible relationships between PDOs, components and activities, and results, given the demand-driven nature of the operation. The relevance of design is therefore rated substantial. 3. Achievement of the Objectives PFMRP 3.1 PFMRP’s PDOs were to: “to strengthen the Borrower’s capacity to (i) plan, (ii) execute and (iii) report on its budget, and to improve the (iv) transparency and (v) accountability of the budgetary systems and processes.” It should be noted that this assessment considers that the objectives of “strengthening” and “improving” various aspects of PFM clearly reflected the understanding that PFM reforms are part of a long-term process that would for instance be measured through PEFA indicators being rated “A” or “B.” The operation’s ambition was thus to make satisfactory advances toward that goal, and efficacy is judged here on that basis as well as on the basis of results outlined in the PAD or from other sources but attributable to the project. Centrality of TABMIS as Contributor to Objectives 3.2 The second to fifth PDOs all rely on the functionalities of the TABMIS and how it operates in practice. To avoid repetition, a consolidated discussion of the TABMIS is presented, with additional results reported for each later. An assessment of TABMIS functionality was carried out separately and is detailed in appendix C.14 The main 24 conclusions, which were validated by the IEG during and after its mission, are summarized below. 3.3 The following features determine the effectiveness of the TABMIS as a budget management tool:  Enabling environment. This includes the status of the TSA and an evaluation of the degree of consolidation of government cash balances in a TSA.  FMIS coverage. This includes an evaluation of the coverage of government financial transactions.  Core system functionality. This includes functionality related to budget management; commitment management; payments management and associated controls; payroll related payments; debt service payments; fiscal transfers and subsidies; evaluation of the comprehensiveness and timeliness of tax and nontax receipts posted in FMIS; the nature of the banking interface; quality of fiscal reporting; basis of accounting; and use of advanced budgeting features.  Noncore functionality. This includes use of other systems modules and interfaces with other systems, for example, budget preparation; MTEF capability; establishment control and its integration with payroll payments; debt management; fixed assets; and auditing.  Some technical aspects. Nature of information systems support for budget execution and treasury processes; systems architecture; and use of a data warehouse and analytical tools. 3.4 The main findings are the following:  Around 85–90 percent of the total government resources are banked in the TSA or TSA-linked arrangements. This level of coverage is considered high.  It is estimated that around 85–90 percent of the total government resources are processed through the TABMIS. The main exception is that transactions against donor funds (other than grants) are not processed through the TABMIS. This level of coverage is also considered high.  TABMIS has the core functionality required for effective budget execution and incorporates key controls that are applied in ex-ante mode. Considering its coverage, transactions related to 85–90 percent of the government financial resources are subjected to controls prior to payment.  TABMIS focuses on core budget management functionality and does not include ancillary functions. While these may need to be developed in due course, the current state of affairs reflects acceptable sequencing. 3.5 Based on the above considerations, this report concludes that that TABMIS is a well- performing tool for effective budget management for the government of Vietnam. Its scope covers 85–90 percent of the financial resources available to the government. Its geographic coverage extends across the country and across all four levels of government. The system functionality implemented includes effective ex-ante budget control for government financial resources and commitment control above specified thresholds. Systems functionality covers core budget execution processes. Some 10,000 treasury and finance officials use it for the 25 government’s day-to-day financial operations. The successful TABMIS implementation is taken into account in the ratings of the various objectives it affects (2–5). Cross-Cutting Theme 3.6 All five objectives are defined in terms of capacity building. Some of this occurs through the various systems being put in place. However, the training provided to government staff, notably MOF, is also quite important. There is no systematic assessment of whether the training provided resulted in improved functionality, even though all officials interviewed, many of whom had directly or indirectly participated in the project, stated the various activities had been very useful. The second issue, also taken up under sustainability below, was whether trained civil servants had remained within government in a position where they could use the training. In the absence of a tracer study, the IEG mission asked about this in every meeting. The consensus response was that almost no one had left MOF positions, except a minority who had retired. Bank staff also confirmed this. In view of this, one can consider capacity-building efforts to have been successful. OBJECTIVE 1: STRENGTHEN THE BORROWER’S CAPACITY TO PLAN ITS BUDGET: SUBSTANTIAL 3.7 This objective was largely aligned with the second project component, to which most relevant results are attributable. Output 3.8 The following principle indicators were included in the logframe (indicators such as workshops are not reported here as were lower level and, as indicated in the additional financing documents, the targets were surpassed):  Budget planning module of TABMIS tested and implemented. o This was part of TABMIS’s ancillary functions that were dropped during the design and implementation phase to focus on core functions. Instead, the MOF operates a separate Excel-based budget preparation system. Budget figures from this system are loaded by the MOF and budget controllers into TABMIS at the start of the year. The IEG mission reviewed the system’s functionality and concluded it was adequate. The indicator is considered as having been met.  A five-year MTFF was prepared as part of the 2005 budget cycle, and rolled over, updated and published as part of each subsequent budget cycle.  MTEFs were successfully piloted in two sectors and two provinces as part of the 2005 budget cycle, and rolled over and updated in the first two sectors and provinces. In addition, they were piloted in a further two sectors and provinces as part of 2006 budget cycle, and rolled over, updated, and published, in all four sectors and all four provinces in subsequent years.  These two intermediate indicators were subsequently revised in the additional financing. As follows, three-year MTFFs and MTEFs were piloted as part of the 26 budget cycle in four sectors and four provinces; and rolled over, updated, and published as part of each subsequent budget cycle. o The project successfully piloted the application of MTFF and MTEF in four line ministries (education and training, health, transport and agriculture) and three provinces (Binh Duong, Vinh Long, and Hanoi City) for three budget year cycles. The scope of the MTFF was a bit narrower than in the original indicator15, there were three provinces instead of four, and the exercise was delayed. However, the revised indicator was achieved. Nevertheless, keeping in mind that the objective of the exercise was to demonstrate the feasibility of pilots, for future generalization, the indicator is considered as having been met. Outcome 3.9 The PAD indicator was “better planning of the State Budget and the Public Investment Program to achieve the growth and poverty reduction goals set out in the SEDP.” Specific measurements were not proposed, and this review draws from its mission and other sources. The relevant PEFA indicator is PI-12, rated “C.” The PEFA notes that the lack of overall budget constraint was a key shortcoming. However, a new budget law was approved in 2015. According to the World Bank staff’s assessment, it is of good quality and addresses most of the shortcomings of the previous law. The revised State Budget Law promulgates the preparation of the five-year financial plan and the 3-year financial-budgetary plan. Guiding decree on MTEF is being developed and will be passed during 2016. There is also a separate medium-term investment plan, which was issued in 2015. Most of sectoral strategies are for 10 years and sectoral plans are for 5 years. These were issued, following the latest social- economic development strategy and plan. 3.10 It mandates the use of MTFF and MTEF in budget planning. This law was a prior action under EMCC and benefitted from technical assistance from MDTF II. Therefore, the main question asked here is whether there is also attribution to PFMPR. The IEG mission was satisfied that, without the successful piloting of these frameworks, the budget law is unlikely to have supported their generalization. Attribution therefore exists. Furthermore, these pilots have improved collaboration between MOF and MPI, which in the face of continued segmentation of responsibilities between the recurrent and investment budget, will be essential. 3.11 Based on analysis, the objective may be considered as having been attained. Other Objectives 3.12 Before discussing the four remaining objectives, it should be noted that their analysis was hampered by lack of useful outcome indicators. Project documents fail to propose a clear indicator for transparency, while an outcome indicator covers this and other PDOs without proposing a quantifiable measure other than it should be “in compliance with best practice” which reflects a level of ambition more aligned with a longer term goal and higher level objectives. This review therefore largely relies on its own analysis and interviews from the field visit and various reliable public sources. 27 OBJECTIVE 2: STRENGTHEN THE BORROWER’S CAPACITY TO EXECUTE ITS BUDGET: SUBSTANTIAL 3.13 The ICR argues “the TABMIS business processes aligned with the World Bank Treasury Reference Model together with a unified chart of accounts aligned with GFS and TABMIS itself helped to establish an orderly and transparent budget execution process with strict control throughout four levels of government.” IEG’s own assessment is in agreement with this, also considering the importance of expenditure control mechanisms and of the reliability and timeliness of the information that is entered into TABMIS. 3.14 Furthermore, according to PEFA, “the majority of revenues and expenditures of the State are captured in the budget. However, there are some elements of revenue and expenditure that are treated as off-budget, even though they are still budgeted, reported, disclosed, and audited annually.” The relevant PEFA indicator PI-7 is rated “C+.” This indicator is expected to improve. A large portion of off-budget expenditure originates from the social security and health insurance funds, which accounted for 5 percent of total budget. According to the revised state budget law, to enhance fiscal transparency, expenditure from these extra budgetary funds will have to be disclosed in the budget. OBJECTIVE 3: STRENGTHEN THE BORROWER’S CAPACITY TO REPORT ON ITS BUDGET: SUBSTANTIAL 3.15 There is strong evidence from PEFA and a review of TABMIS functionality that the government can report on its budget in real time. Similarly, the development of DMFAS enables the regular monitoring and reporting of domestic and external debt. DMFAS key staff have been trained and remain in place (thus achieving intermediate outcome in revised indicators). Debt sustainability is analyzed periodically in collaboration with the IMF and World Bank and published soon thereafter. 3.16 The government has the capacity to report internally on its budget, which implies the objective has been met. This review considers its actual publishing and disclosure to the public to be covered under the next objective. OBJECTIVE 4: STRENGTHEN THE BORROWER’S CAPACITY TO IMPROVE THE TRANSPARENCY OF THE BUDGETARY SYSTEMS AND PROCESSES: SUBSTANTIAL 3.17 The Fiscal Transparency Report published by the U.S State Department (2015) classifies Vietnam among the countries that have met its minimum standards for fiscal transparency. In contrast, its 2012 report classified the country among those that did not. The World Bank’s fiscal transparency review of Vietnam also comes to a similar conclusion but notes areas for improvement (Allan and Rab 2014).  Increased public availability of fiscal information in Vietnam in recent years is matched by strong interest and demand among stakeholders. The Ministry of Finance is rated highly in terms of its progress in communicating fiscal information.  The budget transparency survey also indicates much scope to improve official budget documents.  Although the media is the key channel for communicating fiscal information to the markets, they themselves have difficulty in interpreting primary budget reports. Civil 28 society organizations play an important role in promoting budget literacy and transparency particularly at the subnational level.  But general access to and awareness of budget issues at the subnational level remains poor, in part because disclosing only financial information may not provide meaningful information to ordinary citizens. 3.18 The assessments reflect an improved situation, consistent with project objectives being met. Additional information pertaining to budget transparency is also provided in box 9.1 below. Furthermore, PI-10, the relevant PEFA indicator, was “B” reflecting relative strong performance in this area. However, the issue of attribution needs to be examined. The review also identified TABMIS, TSA, and the chart of accounts (COA) as drivers of, or contributors to, these improvements. There is also expectation than the new approved budget law will improve matters further.16 The analysis is consistent with progress toward the objective with credible attribution to the results to support under the project, notably TABMIS. OBJECTIVE 5: STRENGTHEN THE BORROWER’S CAPACITY TO IMPROVE THE ACCOUNTABILITY OF THE BUDGETARY SYSTEMS AND PROCESSES: SUBSTANTIAL 3.19 The strengthening of PFM systems (TABMIS, COA, and DMFAS) has resulted in improvements in the accountability of budgetary systems and processes. An important issue in assessing PDOs is whether the information in turn feeds into the accountability institutions (State Audit of Vietnam and National Assembly), which even though they were not beneficiaries under the project are part of the accountability systems, broadly defined. According to the PEFA, this flow of information and subsequent follow-up happens satisfactorily: “The State Audit Agency carries out a range of different types of audit — financial, compliance and performance—and works to international standards. But its resources only enable it to cover fully each year about 60 percent of central government units and 50 percent of provinces. Reports are submitted to the National Assembly without delay, and there are effective arrangements for follow-up” (Government of Vietnam 2013, vii). 3.20 The conclusion of this review is that all objectives have been met. This has been primarily through project achievements during its last two years of implementation, especially once TABMIS became operational, as well as sustainability of capacity building efforts during the project life. The project also appears to have contributed to the higher level objective of improved governance, as reflected in the key performance indicators. However, available information also suggests most such indicators have been partially met, and continued policy and institutional actions will still be required. MDTF II 3.21 MDTF PDOs as evaluated here are “to support Vietnam to strengthen its capacities for: (a) increased (i) coordination and (ii) transparency in public finance management; (b) improved revenue mobilization; and (c) (i) effective and (ii) efficient public expenditure management.” 29 POLICY ACTIONS UNDERTAKEN SINCE PROJECT CLOSING 3.22 There is an important difference between the outcomes attributable to the project at closing and at present. The main factor is that the project was fully or partially responsible for preparing an impressive list of policy actions that were ready in draft but not yet approved by decision makers, adopted formally, or implemented. Experience in other countries has shown that such policies may remain in draft form for a long time before some are finalized. Indeed Vietnam’s track record in this area had been mixed in recent years, as evidenced by the number of triggers under DPO series that were dropped or modified in a way that made them less impactful. However, the IEG mission noted that not only had this risk not materialized, but also an impressive number of policy actions had been brought forward by MOF and adopted by the appropriate decision-making entity (National Assembly, prime minister’s office, or others). As noted elsewhere, this favorable outcome (favorably comparable to achievements under large DPOs) reflects the high degree of proactive commitment by MOF staff and its decision makers and the effectiveness of capacity building under the project (as well as in some cases additional leverage provided by the last operation in the EMCC series). It also underpins a case whereby the project outperformed its goals. CROSS-CUTTING THEME 3.23 All five objectives are defined in terms of capacity building. This occurred in a number of ways under the project. The project was apparently quite successful in attracting highly qualified international consultants who were supported by strong local consultants. These teams worked closely with MOF staff from various departments and imparted their tacit knowledge to them, for example, with respect to the development of bond markets. This type of capacity building occurred for instance while drafting laws and regulations, through the development of specific manuals, formal training, and study tours. In case of training and study tours, the conclusion made under PFMRP, whereby MOF staff tended to remain in the ministry, is also valid here. As noted under M&E, in a minority of cases, the beneficiaries of training programs were tested on their acquired knowledge, and the best performers were eligible for various recognitions and rewards. In this case also, officials interviewed stated the various activities had been very useful and that almost no one had left MOF positions. In view of this, one can consider this aspect of capacity building efforts to have been successful. CONTRIBUTION OF PROJECT RESULTS TO OBJECTIVES 3.24 Many of the results achieved by the project affect one or more objectives. To avoid repetition, the mapping between the key results and objectives is provided in table 3.1. Additional details are provided under the analysis of various objectives. 30 Table 3.1. Mapping of Legal and Regulatory Achievement to MDTF II PDOs Transparency Coordination Effectiveness Efficiency Revenue Result PEFA report was used extensively to inform new government policy including the X X X X X recent issuance of the revised state budget law in 2015, which among other things, (i) builds a robust fiscal policy framework with the medium-term fiscal framework that aligns revenue and expenditure with the development objective; (ii) increases fiscal transparency by disclosing the executive budget proposal to the public when it is submitted to the National Assembly; and (iii) aligns recurrent budget classification with capital budget classification. PEFA report served as the baseline result framework to monitor the PFM X analytical and advisory assistance TA provided to the government during 2015– 2020. The medium-term action plan 2015–2017 (for PFM strategy) was issued by the X X X X X MOF Decision 704/QD-BTC dated 17 April 2015, which comprised 8 pillars and 45 activities to be implemented during 2015–2017; 36 of 42 activities were completed in 2015. One activity was adjusted and one more activity was added to the action plan. Some results are highlighted below. Pillar on revenue mobilization: the government has already made progress on X X several tax policy and tax administration including increasing tax revenue from domestic market and collection of tax arrears. Pillar on the tax and custom administrative procedure reform has been progressing X X very well. The government recently issued resolution 19 to shorten substantially the tax compliance time. Pillar on public debt management was being consciously managed by MOF. X X Although public debt has been under the debt threshold, debt servicing and interest payment are rising, leading to the need to restructure the debt portfolio. Government issued Decision Number 689/QD-TTg, dated May 4, 2013, X X approving a medium-term debt management program for the period 2013–2015. The Ministry of Finance has prepared an updated medium-term debt management X X X X program for the period 2016–2020, with strengthened risk analysis based on additional scenarios and updated macroeconomic assumptions. The government has issued Decree Number 87/2015/ND-CP, dated October 6, X X 2015 which replaced Decree Number 61/2013/ND-CP, dated June 25, 2013, adopting the audited reports on the public dissemination of the key financial performance of all state economic groups. Based on the security review results, IBM (contractor) has made corrective X measures to improve the security of TABMIS. MOF issued a decision 3317/QD-BTC dated 24/12/2014 on regulations on X X financial information security for public financial information systems. The regulation has been approved by the minister of MOF through his Decision X X X X 2247/QĐ-BTC dated 06/9/2013, which serves as the legal basis for carrying out evaluating the legislation implementation situation in PFM across the country. The government adopted accounting law in 2015, which mandates establishment X X X X of the internal audit function in government agencies. The accounting and audit 31 Transparency Coordination Effectiveness Efficiency Revenue Result policy department of MOF is tasked with drafting the internal audit regulation pursuant to the accounting law. The internal audit function will be managed by the supervision department at the line ministries. The revised law on import-export tax was issued in April 2016 that promulgates X X X the new contents and revised tax rates in accordance with the recently signed trade agreements. Law on fees and charges was issued in November 2015 that promulgates the list X X X of fees and charges to be collected as well as the authorities responsible for setting the price of user fees and charges. Revised law on special consumption was issued in November 2014 that applies X X revised tax rates toward selected special consumption goods. The government adopted the accounting law (2015) to develop the consolidated X X X X the government financial statement in which public accounting standards will be applied. The accounting law of 2015 requires MOF to prescribe accounting standards X X X X based on international standards. The most recent circulars issued by MOF are Circular 200/2014/TT-BTC applicable from January 2015, Circular 210/2009/TT- BTC, and Circular 202/2014/TT-BTC. They provide guidance on the Vietnamese Corporate Accounting System and try to align VAS more closely with international accounting standards. The State General Accounting (SGA) function was established pursuant to X X X Decision 1188/QD-BTC in 2014. The SGA will be responsible for preparing the consolidated government financial report in accordance with public accounting standard. The Ministry of Finance issued Circular 111/2015/TT-BTC regulating the X government bond issuance in the domestic market. It also provides procedure for bond buyback. Decree 24/2016/ND-CP on cash management policy was issued on April 5, 2016. X X X The state treasury has implemented TSA arrangements, consolidating cash balances from more than 700 accounts. Cash balance consolidation has helped optimize the use of cash balance, minimized borrowing requirements, and minimized the risk of temporary cash shortages. The Ministry of Finance has drafted the revised state asset management law X X X following the Constitution 2013. The output of MDTF provided helpful background and analyses to the revision of the state asset management law. The draft law was approved by the prime minister in May 2016 and will be submitted to the National Assembly in late 2016. It is expected to be passed by the assembly by the first half of 2017. The prime minister has issued Decision 08/2016 dated February 26, 2016 to X X X institutionalize the centralized procurement mechanism adopting the framework contract concept that the MOF learned from U.K. experience through the study visit to the United Kingdom under the MDTF-2. 32 Transparency Coordination Effectiveness Efficiency Revenue Result Tax Law on Environmental Protection (2010); Decree 67/2011/NĐ -CP providing X X X X X guidelines to implement the Tax Law on Environmental Protection; circular guiding the implementation of Decree 67/2011/DN-CP; Decree 69 to supersede the Decree 67/2011; Circular 159 to revise the circular guiding the implementation of Decree 67/2011. Effective since January 1, 2012. Tax Law on Utilization of Non-agricultural Land (2010). Effective since January X X X X X 1, 2012. Law on Amendment and Supplementation of a Number of Articles of the Law on X X X Corporate Income Tax (09/6/2013). Effective since January 1, 2014. Law on Amendment and Supplementation of a Number of Articles of the Law on X X X Personal Income Tax (23/1/2012). Effective since July 1, 2013. Law on Amendment and Supplementation of a Number of Articles of the Law on X X X Value Added Tax (19/6/2013). Effective since January 1, 2014. 3.25 In summary, the project played an important role in preparing and adopting 22 important laws, decrees, and decisions that improve the PFM system in a significant way. The Trans-Pacific Partnership also provided a strong incentive for this. The new budget law, which is considered key by both government and development partners, should also be included on this list as there is a credible logical link between the PEFA results and the revisions of the law. Achievement of the Objectives Objective 1: Strengthen Capacities for Increased Coordination in Public Finance Management: Substantial 3.26 A number of actions contributed to improved coordination capacity; however, there is no specific evidence of the objective being surpassed significantly. Objective 2: Strengthen Capacities for Increased Transparency in Public Finance Management: High 3.27 There are strong, nonincremental results listed in table 3.1 that are associated with substantial improvements in transparency. This, for instance, includes the publication of audits of state economic groups, which for a long time have been seen as “black boxes.” Objective 3: Strengthen Capacities for Improved Revenue Mobilization: High 3.28 Part of the policy actions completed were aimed at allowing Vietnam to meet its commitment under the Association of South-East Asian Nations. The others strengthened the revenue base or tax administration at various levels of government 33 Objective 4: Strengthen Capacities for Effective Public Expenditure Management: High 3.29 Almost all the key policy action had a bearing on effectiveness of public expenditure management. A specific example concerns average maturity of bonds, which has been doubled in two years thanks to the measures prepared under the project and subsequently put in place. Objective 5: Strengthen Capacities for Efficient Public Expenditure Management: High 3.30 This PDO was also affected by most of the actions discussed above. For example, the procurement action is expected to increase efficiency, while also contributing to higher level PFM objectives, such as improved governance. 3.31 In summary, policy actions taken were deep and largely irreversible, while capacity building has been very significant. The project exceeded its goal of merely strengthening capacity in various areas. 4. Efficiency PMFRP 4.1 The main potential driver of inefficiency relates to delays in project implementation. Some may in theory be justified by the cost overrun and the need to provide additional financing, but without procurement delays, this could have been done sooner. Also, the delays in the development of the TSA and COA resulted in delayed benefits from these measures. 4.2 On the other hand, efficiency is also about whether project resources were used in a way that suggests high rate of return. (Economic or financial rates of return are not calculated for this project.) Given that the bulk of project resources (at least 80 percent) were used on TABMIS, the clear driver of efficiency relates to the rollout of this system. According to the Vietnam Treasury Department, the cost breakdown was: (i) Oracle licenses, US$14.5 million; implementation services, US$40.7 million; hardware and system software, US$11.6 million; design and IV&V consultancies, US$2.1 million for a total of US$68.9 million. The cost of the Telecom network, which is shared with other government applications, was borne by the MOF and is not included. Training costs are also not included. Therefore, according to these figures, the total cost of TABMIS is around US$70–75 million. This is in line with implementation user costs for other large budget execution systems projects (e.g., Indonesia and Pakistan). The time taken for implementation (10 years) is also comparable. 34 Figure 4.1. Key FMIS Indicators for Selected Countries 100 100 76 80 59 60 54 51 Score 46 40 40 34 34 25 22 25 23 15 14 16 20 1010 87 10 88 10 11 959 3 574 3 54 2 0 Max score Zambia Cambodia Vietnam Malawi Ghana TSA coverage Tech Features Non Core Func IFMIS Coverage Core func System strength Source: IEG 4.3 A related issue concerns the system’s functionality. Chart 4.1 above provides comparison with four countries with PFM projects assessed by IEG during FY16, as well as a composite score reflecting best practice (not necessarily from the same country). The chart indicates that not only Vietnam’s FMIS outperforms the other four comparators, but also all but one (noncore functions) dimensions of the score are above 75 percent of best practice, reflecting a high level of functionality and efficiency. 4.4 Therefore, even though minor inefficiencies in terms of delays were present, they were within the expected norms for this type of operation. There is also strong evidence that the large majority of project proceeds were used efficiently. Considering these various factors, efficiency is rated substantial. MDTF II 4.5 The operation was affected by a number of inefficiencies: (i) initially delayed implementation and extension of closing date by 30 months; (ii) completion of two IPs instead of the expected four (even if it should be noted that undertaking the fourth would have required topping-up the fund); and (iii) cancelation of part of the grant, and return of unused funds to donors (amounting to the additional financing). However, there were technical reasons behind these events. The original timetable was unrealistic, especially as developing a pipeline of subprojects and putting implementation capacity in place should have been expected to take at least a year. Furthermore, the additional financing in 2011 resulted in the need to extent the project (by a year, which should have been seen as insufficient). Finally, based on IEG interviews, it appears that the timetable was driven in significant part by some donors’ internal procedures regarding disbursements.17 4.6 As also noted in the ICR, elements of this operation contributed to efficiency. Specifically, the polling of donor resources decreased transaction costs for the government. The concentration on a single ministry helped in this respect by avoiding potential “turf battles.” 4.7 The PMU and Bank team could have accelerated commitments and disbursements, and thus avoided cancelation of funds by funding additional subprojects. However, they 35 appropriately chose to preserve subproject quality over low reward or insufficiently prepared activities. 4.8 Ultimately, the quality of the subprojects is reflected in the results they achieved or to which they contributed. This was a real strength of the operation considering that one may argue that this small operation contributed to more policy actions and institutional change than the last two PRSC operations (World Bank 2015). This assessment disagrees with the ICR conclusion that the project was inefficient because “transaction costs incurred by the Bank were rather high. The Bank task team, including senior management, spent a higher than usual amount of time and resources on project implementation support for a project of the size of the MDTF-2.” Bank inputs were well utilized and contributed to the important outcomes. The main reason behind this different perspective is that at the time the ICR was prepared the full outcomes of the project had not yet been attained. 4.9 The project experienced inefficiencies that were important at the micro-level but were dwarfed by the overall achievements of the project that are conceptually consistent with a high economic rate of return, which cannot readily be measured. This assessment gives prominence to the latter and as a result, rates efficiency high. 5. Outcome Ratings 5.1 This assessment has benefitted from the fact that results attributable to the two operations are now visible that were not measurable when the ICRs and IEG reviews were prepared one to two years ago. Policy actions implemented in 2014–2015 that originated from the projects were especially important in this respect, notably in the case of MDTF II. These factors together with validation from the fieldwork undertaken by IEG have resulted in the appreciation of both outcome and efficiency being somewhat different from earlier reviews. 5.2 The high strategic relevance for both operations is not only related to strong links between objectives and government and Bank priorities, but also to how the operations complemented each other and others (e.g., DPOs). Similarly, the relevance of design for both operations was substantial. The results framework was a weakness, in one case because of the logframe utilized and insufficient restructuring and in the other case because of the demand-driven nature of the operation. 5.3 All five objectives of the PFMRP project are rated as substantial. The drivers of this rating include the realism of objectives that are set as intermediate steps within a longer term process, the effectiveness of the TABMIS notably in terms of coverage and supporting arrangements (e.g., COA and TSA), capacity building having been sustained, and positive contributions to higher level objectives. 5.4 Despite delays of almost five years in implementation, PFMRP efficiency is considered substantial. The length of implementation of 10 years for this type of project is within the normal range, and project resources were used efficiently for TABMIS. Based on the preceding findings, the outcome rating for PFMRP is satisfactory. 5.5 MDTF II met its various objectives—some being rated high, others substantial. The main driver of these ratings was both the sustainability of capacity building, and the fact that important goals that were not yet reached at project closing were achieved in the two years 36 that followed. Project resources were used in a highly efficient manner to attain these objectives. The project suffered from delays, but the overall implementation period of four to five years is typical for this type of demand-driven technical assistance. Based on the previous assessments, MDTF II outcome is rated highly satisfactory. 6. Risk to Development Outcome CROSS-CUTTING RISKS 6.1 Both operations share a common risk of the PFM reform being given lesser priority and achievements to date being undermined. Given the track record during the two and half years since the operations closed, this risk appears to have been mitigated, except for the following important aspects:  The macroeconomic situation is relatively sound, even though there are vulnerabilities, and the budget is quite tight. This may result in some parts of government resisting a deepening of PFM reforms, which according to the 2013 PEFA18 remains a priority, and potentially arguing for reversal of some actions. Considering the level of commitment shown during the past three years by government, information collected by the mission on plans to continue the reforms and measure progress around 2019 through another PEFA, and taking into account further planned and ongoing donor supported (even if more segmented than before) this risk appears quite manageable and not significant.  The segmentation of the budget between recurrent under MOF and capital under MIP remains in place. The coordination between the two has improved thanks to measures supported by the projects and with the progressive introduction of multi-year budgeting through MTEF, but not sufficiently. The declining share of public investment to GDP—which has been cut in half since the mid-2000s—and the new public investment law aimed at increasing the efficiency of investments also reduces the scale of the problem. Current public investment, hovering at about 5 percent of GDP, needs to rise to accommodate sustained growth. But, attention needs to be paid to the recurrent implications of investments, which in turn require continued improvements in the institutional coordination. Furthermore, some of the investments (notably in infrastructure) are expected to be based on public-private partnerships for which an enabling law was recently passed. These, too, may put added pressures on the budget, especially if the projects are not well designed and subject to competitive tender, and if contingent liabilities are not well managed.19 6.2 The two projects also share a common experience that contributes to sustaining development outcomes at the “bottom” level:  Many MOF staff and other civil servants were trained under both projects. In the absence of a tracer study, it is difficult to verify whether those beneficiaries have moved to other positions inside or outside government and, if still employed by government, have used the skills acquired. Mission interviews revealed that turnover of civil servants with MOF was quite low—retirement being the most commonly cited reason for leaving government. Almost all of the officials interviewed 37 confirmed either they had benefitted from the capacity building directly or their department included people that had, even if they now often held higher positions.  Similarly, counterparts confirmed that the contributions of external consultants, notably procedure manuals, were still being used in operations and to train new recruits. They also explained how study tours had contributed to a mind-shift within the MOF. In parallel, the different information systems developed under the projects was maintained and used.  Even though the above observations are not based on a rigorous quantitative assessment, the sheer number of identical responses from stakeholders representing all activities funded by the operations and the specificity of examples cited indicate that capacity building efforts have been sustained and are likely to continue, at least in the medium term.  Finally, both government and donors realize that the PFM reform still contains a rich agenda that needs to be completed. Continued support by donors, even if more segmented than before, should contribute effectively to that goal. PMFRP 6.3 The PAD had rated the risks being faced by the operation (prior to mitigation) as “high” and had identified the following issues, which have been updated to reflect present circumstances:20  Lack of ongoing support for cross-departmental working, particularly between MOF, State Bank of Vietnam, and MPI. This risk has not been eliminated.21  Lack of ongoing support to ensure "buy in" and ownership at line ministry, province, and spending unit level. This risk remains present but appears modest.  Lack of partnership working among donors and between donors and government. The segmentation of donor support will not affect past achievements and should be manageable going forward thanks to existing coordination mechanisms.  Additional risks from the present standpoint are the following: o Insufficient budget allocation for operations and maintenance of the TABMIS. Average annual costs for telecommunications, service fees, and hardware upgrades are estimated in the range of US$7 million to US$10 million. The IEG mission concluded that this has been adequately budgeted so far and was likely to remain so.22 o Proliferation of information technology systems undermining the TABMIS and resulting in inefficient parallel systems. Even though there are instances of parallel systems being developed, the IEG mission found they are largely compatible with, and appear complementary to, one another. o The development of systems is only effective if used effectively. In the case of debt management, while recording of external and internal debt has improved, this has not prevented an increase in public debt to the point where sustainability may prove more challenging, according to the IMF. 6.4 The issue of system integration calls for the following clarification on how risks have been mitigated: 38  Excel-based budget preparation systems (not automated): Budget estimates are loaded into TABMIS and all in-year changes are entered directly into TABMIS.  DMFAS: An interface between DMFAS and TABMIS has been developed and is operational.  Tax and nontax receipts: Tax and customs receipts are deposited in Treasury Single Accounts (TSAs) of the State Treasury. Budget revenue information is exchanged among the State Treasury, Tax and Customs Departments and commercial banks through centralized Tax Collection systems/applications (TCSs), the Tax Collection e-Portal, Customs e-Portal. The State Treasury receives and controls budget revenue information and runs the automated interface between TCS and TABMIS GL. Non- tax revenues are recorded in the TABMIS GL on the basis of information received from TCS, Finance Departments or Spending units  TABMIS is connected with banking systems via the VST payment applications for both bilateral settlement and inter-bank clearance IBPS and then via enterprise service bus connects to banks under the two mechanisms of webservices and MQ.  There is no central human resource or payroll system: Payments for payroll are routed through TABMIS based on payment requests entered by spending units in the TABMIS. 6.5 The analysis does not identify any substantial risks, especially as TABMIS maintenance seems likely and system integration works reasonably well. The macroeconomic and some institutional risks remain present, but the government has a track record for managing them. There is therefore moderate risk that project achievements will not be sustained. MDTF II 6.6 The PAD rated overall risks as modest and identified key areas where mitigation would be needed. Aside from macroeconomic risks, the following issues remain relevant:  Impetus for PFM reform would lose momentum or political support. PFM reform is a complex task and requires considerable time and effort over many years. This risk remains relevant even if some of the specific needed reforms (including the new budget and the public investment laws) have been adopted and are being implemented. This risk continues to appear quite manageable at present, and the likelihood of reversal of implemented policy actions appears low.  Institutional capacity in MOF will not be strengthened sufficiently to manage PFM reforms over the next few years. Departments in MOF would drive PFM reform activities in an uncoordinated manner, likely reducing the impact of reforms. Not only was this risk avoided, the MOF appears well positioned, at least at the technical level, to champion further reforms.  Essential PFM reform initiative activities outside MOF’s mandate will not keep pace, particularly procurement reform and external audit and legislative oversight strengthening. Also, the slow pace of implementation may undermine PFM reforms, and complementary governance reforms23 need to be accelerated. These have more to do with the sustainability of the reform program than of achievements under the project. However, as reflected in the 2013 PEFA, there are challenges ahead. 39 6.7 The project restructuring paper, which was issued at a time implementation was facing delays, includes a less sanguine appreciation of risks, which are raised to “substantial.” The risks identified were broadly correct and properly mitigated, including: (i) Continued delays. Once the decision was made not to attempt to prepare IP 3 and extend the closing date, this was no longer an issue. (ii) Lack of synergy within MOF and too many activities. The new FDS provided a reference framework that not only guided the activities, but also appears to have been a factor for sustaining them beyond the project to attain the ultimate objectives. Furthermore, the fact that all activities were within MOF limited this risk. (iii) Spreading the project too thin and not achieving substantive results. This risk was still present at closing. However, the main stakeholders recognized they were embarked on a process that needed to be sustained through additional actions, such as the adoption and implementation of laws and regulations. These actions were therefore continued after the project closed. 6.8 In the absence of other risks not identified by this assessment, the analysis suggests there is negligible risk to achievements under MDTF II being reversed. However, it should be noted that the risk being faced by the PFM reform program appears higher.24 7. World Bank Performance 7.1 Bank performance was satisfactory or better for both projects. In the case of PFMRP, the high quality at entry contributed to attainment of outcome, as did proactive implementation support of the MDTF II. Quality at Entry PFMRP 7.2 Overall, PAD quality, a reliable reflection of quality at entry, was very high at the time it was designed. Paying greater attention to public fiduciary systems, and doing so systematically as a complement to public expenditure work, was still a recent evolution in the development community and the Bank.25 Some the elements of the operation’s design are no longer utilized (logframe) or are not considered good practice (e.g., less than eight-year implementation period for PFM operations). However, other aspects of design may still provide a blueprint for a first generation PFM operation. 7.3 Component design: The components were defined based on a clear diagnostic of the PAD. Their internal sequencing, among subcomponents, was well considered and best among four PFM operations assessed by IEG during FY16, reflecting strong conceptual and operational thinking. There was considerable attention to detail, with clear guidance on how to implement the operation and to anticipate and deal with issues that was not relegated to an operational manual. For instance, the document included no objection by the Bank to use of the sole-source method to retain service providers, recruited previously, with unique knowledge and comparative advantage. 7.4 The operation was selective in areas it supported: budget and debt management. The importance of assessing SOE risks was also integrated into the design—a risk that remains 40 present and potentially significant to date. Asset management was part of the TABMIS module. However, revenue management was excluded, in part because the necessary diagnostic was not done. This appears to have been a sound choice because it avoided greater complexity and in light of the failed experience under the 2007 Tax Administration Modernization Project cancelled in 2015. Some aspects of tax reform were devolved to MDTF I (and then to MDTF II). 7.5 The TABMIS design is assessed in some detail in appendix C. The system consisted of modular packages that could be expanded and was suitable for the situation in Vietnam. Also, a study fund included a follow-up project if needed, which constitutes good practice. The additional financing obviated the need for a follow-up operation, while also redressing the optimistic implementation timetable. 7.6 Broader context: The link to other operations quite clear. Even though the PFMRP PAD could not anticipate most post-2003 related operations (second PRSC series and MDTF II), it envisaged a complementary relationship between the project and the first PRSC series and MDTF I. Another important aspect of the approach was that it served a donor harmonization mechanism through cofunding of all operations—except for the PIR and EMCC—with the government in charge of implementation. Finally, the operation was underpinned by solid analytical work. 7.7 Operation and institutional design parameters: The operation benefited from sound arrangements.  The steering committee concept was appropriate. Institutional arrangements avoided overcomplicated institutional structures (such as in the Cambodia PFM operation).  Sound PMU design consisting of four teams.  The CTA approach (based on a long-term, part-time retainer contract with an international expert) for the MTEF, which was a good way to reinforce implementation capacity without undermining country ownership.26  MTR conducted two years after effectiveness. 7.8 However, some narrow shortcomings affected disbursements.  The project would be implemented during 5.5 years and complete disbursements in 6 years (including a grace period). This implementation period has proved too short for operations supporting FMIS, but perhaps this was not well established in 2003.  The fairly linear disbursement profile seems inconsistent with the large investment in TABMIS.  Procurement: Thresholds were set too low. This seems to be a Bank-wide issue and is discussed in the fiduciary section of this report. 7.9 Alternatives considered, analytical underpinning, and integration of lessons learned: The design found the right balance between defining the scope of the project too broadly and too narrowly. A very narrow (IT only) operation was rejected in favor of a more holistic approach. An approach including revenue management was also rejected so as not to increase complexity. Subnational governments, which managed 50 percent of expenditures, were included as opposed to a sole focus on central government—resulting in a longer implementation period. These choices were validated by implementation experience. 41 7.10 The project took account of experience in Vietnam and PFM projects elsewhere effectively. It is also took stock of existing and planned operations to minimize overlap with them (World Bank 2003a). The design was based on solid analytical work, including analysis of the budget law, a joint IMF-World Bank study, various PFM related assessments, and the 2003 VDR. The risk of slow implementation was noted too, but a mitigation measure (start implementation using a grant from the Japan Policy and Human Resources Development Fund) proved insufficient. 7.11 M&E: Project M&E arrangements reflected knowledge and practice at the time of design, but proved insufficient in practice. 7.12 Overall, the project at entry was based on a thorough appraisal and benefitted from attention to conceptual and implementation details rarely encountered in other similar operations designed at the time and later. There were weaknesses associated with M&E and optimistic assumptions over time needed for implementation, driven in part by unwritten Bank guidelines. Despite these issues (which were or should have been addressed during implementation), the overall strength of all other aspects of the operation’s design warrants a highly satisfactory rating for quality at entry. MDTF II 7.13 The project design was finalized during 2008 and 2010, with three different PADs prepared each year. The reason for this unusual feature was the project was being funded from a multi-donor trust fund. Its scope and content had to be adjusted to take into account various factors, including additional funding from donors. It also resulted in a few awkward passages where past events such as effectiveness are discussed ex-post. The adjustments were relatively simple to process as project approval was at the level of the Regional Vice- Presidency and Board approval was not needed. Even though three PADs inform this review, all references in this text pertain to the last (2010) version. 7.14 Another aspect of the project was that it was demand driven and activities were not pre-determined. Appraisal was thus based on a pipeline of potential subprojects. These were bundled together and eventually formed the basis of two implementation plans (IP 1 and 2). A question worth asking at the outset is whether this concept could be justified a priori, given the risk of spreading resources too thinly. This assessment concludes that it could for a number of reasons other than the various aspects of quality at entry discussed below: (i) additionality and complementarity with the PRSC series and PFMPR; and (ii) lessons learned elsewhere, even if much earlier – namely the Uganda TA III,27 which was subject to an IEG (then OED) performance audit in 1995. Even though the project was designed two decades before this one, they both share the characteristic of being based on the principle of flexibility (within an overall PFM framework) and evolving objectives (i.e., indicators). 7.15 Component design: The original project had seven components (in practice thematic areas and not traditional components) that were seemingly unrelated and relatively small in terms of average financing (less than US$1 million). However, an important factor was that all the activities were taking place within the Ministry of Finance, which helped ensure overall coherence. Furthermore, the first six components were closely related to ongoing PFM reforms. The justification of including the last, price control, was and remains less apparent especially as Vietnam was increasingly becoming a market economy with a 42 competitive internal market (World Bank 2010). The project restructuring reduced the number of components to three, each covering a number of themes. The most notable change was that price control was no longer prominent and appears as a minor activity (US$30,000) under the project costs presented in the ICR (annex 1). 7.16 Broader context: The operation complemented other PFM-related reforms and operations and benefitted from multi-donor support. It filled existing gaps by building capacity in PFM within the MOF. The design recognized that better integration between MOF and MPI would be desirable, but correctly did not over-extend itself by covering two ministries. 7.17 Operation and institutional design parameters: The operation was supported through appropriate arrangements:  The project was primarily aimed at supporting the MOF, which was appropriate at the time.28  The IP approach was suitable, even though the expectations that such plans would be developed annually was over-optimistic because of the time needed to complete each. This caused disenchantment for some donors who decided to provide standalone subsequent support.  The PMU design called for support by a peripatetic CTA on a long-term contractual basis. This helped ensure that tacit knowledge for managing the program was present. This aspect of the design may be considered best practice as it empowers national entities while ensuring implementation agencies have access to know-how.  The original implementation period was too short because of the delayed start.  Given the nature of the operation and constant exchanges on subprojects, the planned annual supervision seems appropriate.  The criteria for selecting subprojects were not well defined. This issue was resolved prior to approval of IP 2.  Procurement: Thresholds were set too low, especially in light of ownership shown by stakeholders and willingness to recruit a short-term international adviser.  The institutional arrangements were somewhat complex from the standpoint of the number of beneficiaries. However, excessive complexity was avoided through oversight responsibility being given to the PMU and the steering committee.  An MTR resulted in project restructuring that clarified and streamlined project activities and improved various aspects of implementation and results-orientation. A notable change was that beneficiary departments were empowered to manage their own activities, which may have contributed to the activities initiated being sustained and successfully completed after project closing. 7.18 Alternatives considered, analytical underpinning, and integration of lessons learned: The project refers to a limited number of PFM-related studies undertaken during the second half of the 2000s. Given that provision for specific studies (notably PEFA) was made during project implementation, the knowledge base was dynamic and provided as needed. The lessons integrated into the design are drawn mainly from those learned under MDTF I 43 (World Bank 2010). These refer mainly to the strategy and have limited project specific relevance. Nevertheless, lessons considered also correctly led to periodic implementation plans to underpin the operation as well as improved donor harmonization. This area could have been enhanced through greater consideration of lessons learned elsewhere from demand-driven public sector capacity building schemes—even though it is not evident that such considerations would have resulted in much different design. The PAD does not offer any alternative designs. But, it correctly justifies the project in terms of the Paris Declaration and its subsequent agenda. 7.19 M&E: Project M&E was appropriately flexible at the outset and improved over time. 7.20 Quality at entry was generally strong with minor and manageable weaknesses, notably in the form of an optimistic implementation timetable. It is rated satisfactory. Quality of Supervision 7.21 Quality of supervision and reporting of PFMRP was uneven and strong throughout for MDTF II. For both operations, the high quality written feedback from the Country Management Unit is noteworthy. In contrast, the sector’s inputs overall were minimal. PFMPR 7.22 Project implementation took place over 10 years during which time Bank supervision was uneven. There are four distinguishable periods or milestones.  During project launch and the first two years of implementation, considerable efforts were devoted to initiating activities, albeit using optimistic timetables not only concerning complicated IT procurements, but also more “straightforward” actions such as COA and TSA where the need to build political consensus was underestimated.  During the second half of the 2000s, implementation delays were substantial and evident as reflected by the original disbursement lag, which by the end of the decade had accumulated to four years. However, this was neither acknowledged or recognized by the team (other than in country director [CD] feedback) nor dealt with effectively.  Contributing to the problem, the MTR undertaken in December 2006 did not go far enough.  During the last three years of project implementation, policy and implementation issues were progressively resolved, thanks to both the efforts of the task team and advice provided by additional experts. These efforts paved the way to achieve various outcomes. 7.23 The ISRs provide insights into the quality of Bank supervision, even though formal interaction with the PMU and other stakeholders was more frequent than ISRs, which were not prepared after all missions. These documents were reviewed as part of the present assessment, which resulted in a number of observations supervision activities are reflected in reporting of project implementation progress through periodic status reports (ISRs).29 The first four project status reports were comprehensive and detailed. Reporting quality in ISRs declined and became unsatisfactory, as basic required information (notably pending actions) 44 was no longer provided until the last few ISRs. Various portfolio managers providing feedback on ISR on behalf of the CD repeatedly noted the various aspects of this problem. The task team never effectively incorporated the requested improvements. Other aspects of supervision reflected in status reports are detailed in appendix E and may be summarized as follows: Quantitative  The reporting interval was on average every 8 months, with significant variations during the implementation period—low of 2 months and high of 15 months. Reporting every 6 months is consider good practice, while exceeding 12 months disregards Bank guidance.  Supervision mission frequency was every 8 months, which is an over-estimate because no ISRs were associated with two supervision missions. However, there is no record of the project being formally supervised more than once after May 2011, which is insufficient even if by then the project was finally broadly on target. The interval between supervision missions from 2005 to 2008 was about 10 months. The mounting disbursement lags should have triggered a greater frequency—especially given the bulk of the team was field based. On the other hand, initial supervision was intense.  Interval between mission and ISR was on average 4 months, which implies that management does not always receive up-to-date information—good practice is typically considered two months or less. However, the aggregate figure is affected by four late filings by 8 months or more, without which the average interval would be less than 2 months.  CD comment is entered regularly from the fifth ISR onward (expect in two cases with no submission by team). The model in Vietnam is for the CD to delegate to the portfolio manager (there were four successive ones), who provided generally substantive and clear guidance. The only shortcoming relates to the fact that requested modifications and improvements were not introduced by the team, and more proactivity to ensure this outcome might have been desired.  Sector manager comments were either not entered or of quite limited scope and guidance. The lack of clear guidance to the team, including on the need to deal with CD comments, constitutes a weakness. It contributed to the persistence of fundamental issues, such as overhauling indicators, not being addressed. Another outcome was that realism of ratings were neither raised nor addressed, and this may have contributed to inefficiencies in terms of length of implementation of components other than TABMIS. Qualitative  Team composition included sectoral expertise only once at the outset. This may have affected the implementation of the MTEF. Initial continuity in the task team constitutes best practice. There was substantial delegation of periodic dialogue to the field-based team and participation by the original task team leader (TTL) in supervision during the first two years of implementation. Similarly, during the last year of implementation, supervision teams included highly qualified and well- 45 respected experts who, according to all stakeholders, were instrumental in ensuring key objectives were attained.  Basic information is missing from all ISRs, including a full list of covenants and their status. Similarly, apart from its shortcomings, the logframe included 17 useful intermediate indicators that were never included or reported.  ISRs generally provide a good presentation of status and issues. But, information on what follow-up is needed, by whom, and by when is lacking—the implementation roadmap not evident.  Between 2005 and 2011, the ratings, notably for implementation and M&E (throughout implementation), fail to reflect realities and slow progress.30 Numerous instances of unrealistic expectations were reflected in ISRs, notably with respect of the adoption of COA and TSA, which were finally realized about six years behind the original schedule (as discussed in the efficiency section).31  There seems to have been limited linkages established with the two PRSC series, which is mentioned in only one ISR.  The hesitancy of the Bank to extend the operation once the need became evident and formally recorded in ISRs may have been counterproductive as it increased uncertainty and undermined planning of activities. On the other hand, the task team assumed the necessary extension would be approved in due course. When it was done, it was for a period long enough to complete key activities. 7.24 As noted above, management performance in terms of strategic and process-related advice differed widely when the country and sector were compared. The former was specific and to the point, but largely ignored. The latter was frequently not provided, at least not in writing or pro-forma. There was generally no mechanism for ensuring that the task team followed board management comments. The lack of formal in-depth restructuring of indicators and strengthening of M&E, which was largely controlled by the task team, is a reflection on this shortcoming. 7.25 The MTR took place as scheduled in December 2016. It consisted of a detailed review of progress achieved at the time, identification of issues, and development of a detailed work program to ensure implementation would get back on track and be accelerated. The analysis of mission outputs, made part of preparing the present report, reveals a comprehensive review of progress during the initial implementation period and a detailed roadmap for future years that identified policy and institutional actions as well as intermediate benchmarks to monitor implementation progress. The mission aide-memoire thus provided a comprehensive update of the project implementation manual. Furthermore, the review effectively addressed accumulating delays by revising the implementation timetable and planning for a subsequent extension of the closing date by about two years— the disbursement profile was extended accordingly. It also avoided added complexity by agreeing with counterparts that support to tax administration and policy would be covered by the future MDTF II.32 7.26 The main shortcoming of the MTR was it did not identify or address the main shortcoming in the original project design related to its chain of logic and M&E. Specifically, the results framework for the project, based on a logframe,33 was weak and should have been overhauled. Only intermediate outcomes were changed two years before eventual closing. Furthermore, consideration should have been given to including impact assessments into the 46 design. More minor issues, which do not significantly undermine the overall quality of the MTR, were the following:  No consideration was apparently given to raise the low prior review thresholds, even though procurement was deemed satisfactory. These thresholds, notably for consultants, were set at lowest levels applicable Bank-wide at the time.  There is no explicit discussion of legal covenants. A question that might have been asked concerns whether implementation delays and a possible need to restructure the project in light of experience would justify a second MTR to be undertaken later, say in 2008 or when circumstances justified it. 7.27 In practice, there was not any immediate project extension (apparently due to management push back in favor of a “wait and see’ approach) and restructuring of indicators. Dropping one or both second and third components could have been considered then to decrease project complexity and focus on TABMIS. However, no compelling evidence suggests that persisting with these other activities ultimately undermined outcomes or efficiency. 7.28 TABMIS accounted for the bulk of expenditures under the project. Together with systemic changes (notably COA and TSA) that accompanied its implementation, it was the primary contributor to the project outcome. The implementation support provided by the Bank warrants scrutiny. To succeed, the Bank team needs experienced specialists on board or available to advise the client on important technical issues during the design and implementation phases. These specialists need to be familiar with procurement practices and procedures for IT-related procurements and the World Bank rules. As verified by the mission, specialized IT expertise within the Bank project team was made available to the MOF to support the design and implementation of the TABMIS, notably during the critical system roll-out period in the early 2010s. Also, as noted in the ICR, “during early critical stage of TABMIS contract implementation, the Bank TTL participated in monthly meetings of TABMIS Project Monitoring Committee to closely monitor contract execution and provide necessary facilitations.” 7.29 An area of weakness concerns the misunderstanding of the role of the IV&V consultants. The consultants misread the purpose of the hiring and saw it as an audit type of exercise to critique the initial design and the implementation work, instead of helping the government to ensure the contractor implements the system in accordance with the design that had been specified and approved earlier. This caused major difficulties, and this consultancy did not achieve the purpose for which it was intended. The government terminated the contract. As a result, the government was left with limited support in contract management and supervision of implementation. This led to a situation where the government had to rely exclusively on the advice of the implementing contractor (IBM) during this phase. The terms of agreement for the consultants should have clearly specified their role. 7.30 Had the Bank been clearer on this point or intervened when issues arose to clarify the consultants scope of work, the problem might have been avoided. The delays in implementing the Domestic Debt Recording System should also be noted. A decision was made at the time of appraisal to fund its expansion by expanding the existing UNCTAD system that had already been installed in two units at MOF and the State Bank of Vietnam 47 (World Bank 2003a). While this is not clearly stated in project supervision and related documents, there were implementation delays. More complications arose when in the mid- 2000s the Bank introduced an anticorruption clause in its procurement manual, which was not acceptable to the UN system because of its own governance system. A waiver of this clause had to be processed, and took a long time as it had to be approved by a managing director. This delay reflects on Bank performance. The whole process raises two issues.34  The Development Credit Agreement refers to the 1999 amendment of the World Bank procurement manual and does not state that further updates would be automatically applicable to the operation. Perhaps such a close was introduced as part of some portfolio restructuring, but it was not done at the level of the project. While the modification of Bank rule was based on solid governance concerns, this evaluation cannot confirm that its application to this procurement was legally necessary.  There are other instances of involvement by UN agencies in World Bank operations, either as a service provider or implementation agent (e.g., in South Sudan). Presumably, the same issue had to be addressed in other such circumstances and should have either been dealt with more broadly or by recognizing precedents, if any (especially as the waiver was ultimately given). 7.31 The review of project implementation, the analysis of ISRs, and experience with TABMIS reflect a mixed and variable supervision performance, which ranges from excellent in some aspects to substandard reporting. There are documented instances where the Bank team’s advice was vital, for instance in obtaining cost savings for Oracle licenses. On the other hand, weaknesses in M&E were not addressed, and the present assessment is handicapped by lack of data. Nevertheless, thanks to the persistence of the task team to address the Bank’s internal procurement requirements that may not have been well adapted to information systems, the main public fiduciary and governance systems were developed and are fully operational despite various challenges encountered and substantial delays, notably in the TABMIS. This ultimately reflects partly the efforts of the Bank team, which also ensured the cofinancier (DFID) was kept abreast of development and participated in key supervision activities. Everything considered, Bank performance, including inputs from sector management, might have been less uneven, and a moderately satisfactory rating best reflects its varying quality of supervision. MDTF II 7.32 The supervision of the MDTF was quite challenging because of the very nature of the project and its demand-driven design and complexity. The support was principally given to the Ministry of Finance, which has capable staff. This simplified implementation support somewhat. On the other hand, there were a dozen or so departments to deal with, and each subproject required advice based on specific knowledge. Technical advisers provided some of this knowledge. However, the Bank team, broadly defined, needed to complement it and undertook quality assurance on inputs and at times output—even though the exchanges were at times incomplete (see box 7.1). Compared to the amounts financed, transaction costs for the Bank team were relatively high. Nevertheless, the Bank team, while struggling at times on how to deal with issues such as results measurement and experimenting with different approaches, exhibited a high degree of competence. 48 Box 7.1. Feedback from a Sectoral Team Regarding Its Degree of Involvement in a Subproject Was the sector consulted at any stage of formulation and execution of subprojects? Yes. What type of feedback/support was provided? The team led the discussions with another team participating at time and the TTL. After two to three rounds of discussion, there was an agreement to support this activity. The team was engaged in initial supervision stage but not later on. Is there a view on the quality, relevance, and implementation status of the work done? It was difficult to obtain good information on the project status. The MDTF arrangement did not require the client to share all of the output. The other Bank teams involved may have done more of the follow-up. Source: IEG mission interview. 7.33 Supervision was conducted regularly from project approval to 2011, but not recorded in ISRs. They were not mandatory until then for projects supported by trust funds. Five ISRs were prepared by the task team (one per year on average), and their review provides a window into supervision quality. There were long gaps between the first three ISRs (16 and 17 months). The gap between the second and third ISR was in many ways compensated by project restructuring that took place halfway through the period, which provided detailed information on the state of project implementation. The first three ISRs were informative, quite complete, and clearly described status and issues—both conceptual and related to implementation. For instance, the struggle to come-up with relevant indicators for demand- driven activities is clearly explained. Similarly, the legal covenants were monitored, and their status is clearly listed—this was not done for PFMRP. A systematic shortcoming was found on entry and updating of pending actions, which was also present in PFMRP ISRs. Management feedback was initially concise, except in the last two ISRs where, in contrast to detailed CD feedback, the sector’s comments were not provided in one case and were minimal in the other. 7.34 A strength of the reports was the candor with which performance was assessed and reflected in the rating. For instance, the second ISR downgraded a number of ratings, including objectives, in view of difficulties being experienced at the time.35 Similarly, at closing, the various aspects of project performance were conservatively rated “moderately satisfactory” even though a higher ratings might have been argued based on the assumption that ongoing activities would yield important outcomes—as was eventually the case. 7.35 The dialogue with counterparts appears to have been conducted in a constructive manner. The task team showed flexibility and accommodated emerging priorities, for example, support to the Financial Sector Development Strategy 2011–2020 (ISR 2). The team also identified bottlenecks and proposed solutions that were adopted by counterparts, for example, by improving procurement capacity and simplifying the PMU’s complex internal procedures. These ultimately contributed to project success. 7.36 Finally, the MTR was conducted in a timely manner. It paved the way for addressing some of the issues but not all. Restructuring of indicators and modifications to the results matrix were insufficient. As noted in the ICR and confirmed by the mission through multiple 49 exchanges with the PMU and various MOF departments, “the MTR findings and agreements led to the critical decisions that included the program restructuring, strengthening mechanism for policy dialogue between DPs, the Bank and MOF, simplification of the MDTF II results matrix, and measures to improve project management (particularly the decision by the MOF to delegate the authority to implement project activities by responsible departments).” Furthermore, “following the MTR conclusions, the Bank enhanced quality of implementation support and supervision with more frequent and effective communication and engagement with the PMU and all responsible departments to provide technical support on terms of references or project proposals. The Bank team worked closely with the PMU to help screen and select projects to be financed under MDTF II with more direct links to the PFM priorities” (World Bank 2014b, 19).36 7.37 The ultimate success of the project is largely attributable to high quality supervision and ancillary, well-focused dialogue led by the Bank with the various stakeholders. Implementation support was more frequent and intensive than typical, especially for such a small TA project. The locally based task team members appear to have gone beyond expected norms to ensure a successful outcome. The demand-driven project required the Bank and counterparts to experiment iteratively with various types of support and fine-tune the approach based on lessons learned under the initial subprojects. In view of the eventual achievements under this project thanks to proactive implementation support, and notwithstanding issues noted under the efficiency section that were related partly to donor procedures and factors outside the Bank’s control, quality of supervision, including post- closing work on activities not yet completed, and is rated highly satisfactory. 8. Borrower Performance Government 8.1 Government involvement in both projects was similar, especially as both contributed to the same overall objective. For the purpose of this assessment, it is useful to decompose the government into two separate components. The first was the MOF, which oversaw both projects. The second consisted of the rest of central government, including the office of the prime minister, MPI, and line ministries, which were responsible for higher level policy reforms and budget management. 8.2 PFMRP started slowly, which may be partly attributable to MOF as well as the rest of government not initially giving PFM sufficient high priority (e.g., adopting a chart of accounts). However, the pace of implementation accelerated in the 2010s and continued after both projects had closed. Both ICRs as well stakeholders interviewed by IEG highlighted the key role played by MOF and its high level decision makers as a champion for enhanced PFM systems and of the required policy actions. Many reforms supported by MDTF II were completed in 2014 and 2015 well after the closing of both projects. 8.3 The performance of other parts of government was generally strong, even if uneven at times. Macroeconomic management weakened for a few years around the financial crisis, and the government response may have delayed policy actions (as seen under the PRSC). Nevertheless, improvements in macroeconomic stability in the last few years is associated with greater emphasis on PFM reforms and achievement of key milestones, such as an 50 operational TABMIS and the new budget law. The main failure of government as a whole has been an inability to address the key issue of budget segmentation between MOF and MPI—the latter remaining responsible for the investment budget and the former for the recurrent with insufficient interaction between the two. The need to reposition MPI37 and to consolidate oversight over the budget remain relevant. 8.4 According to project documents and IEG mission interviews, the steering committees functioned well. 8.5 The conclusion of this assessment is that despite fluctuations in government performance, which may have affected PFMRP more than MDTF II, overall government performance for both operations during their preparation and implementation was satisfactory. Implementation Agencies PFMRP 8.6 The project was implemented through component technical units, and the PMU was organized into specialized component implementation teams. Largely, this assessment concludes that implementation on the government side was sound. Implementation delays encountered during project implementation were to largely attributable to the Bank’s slow procurement processes. 8.7 The main issue for the PMU was related to the nonperformance of the IV&V consultants. Following the change in team members, performance was no longer deemed satisfactory. While the government continued to use some individual and firm consultants in the support of system functionality, the government also relied on the advice of the TABMIS provider, which lacked independence. According to information received, while suboptimal, this arrangement proved workable and did not appear to have resulted in a cost overrun. 8.8 Another issue concerns the effectiveness of the CTA. This area is not highlighted in project implementation documents reviewed by IEG (e.g., no mention in the ICR). Interviews with both government and Bank staff proved similarly inconclusive as some counterparts have left, and recollection was imperfect. Overall, this adviser appears to have been helpful to the PMU. 8.9 Except for minor problems at the start of the project, the PMU fulfilled effectively its procurement and financial management responsibilities. Based on this information implementation agency performance is rated satisfactory. 8.10 Overall borrower performance is satisfactory reflecting both government and implementation agency ratings. MDTF II 8.11 Implementation agencies for this project include both the PMU, supported by the CTA, and the various departments within MOF that were responsible for implementing the subprojects—increasingly so after the MTR. As noted in the ICR, “after delegating the implementation of project-specific activities to functional departments, the PMU stepped up its management role as a facilitator, advisor, and supervisor.” Furthermore, the PMU fulfilled 51 its fiduciary functions well and ensured that procurement and financial management was undertaken effectively. 8.12 A main risk potential associated with project design was the various departments within MOF would see the project as a “right” and source of funding for various activities, including for low priority training and other types of capacity building without significant ultimate contributions to PFM reforms. This could have resulted in financial resources not being used in the absence of a sound subproject, or not being used effectively. The fact this risk did not materialize demonstrates the careful selection of subprojects by the PMU, based on the advice of the CTA and the Bank, and ownership of the activities by the various units within MOF and their proactive approach in support of PFM report. The latter is evidenced by the fact that these departments continued to make good use of the training provided under the project. For instance, in some cases, the IEG mission was informed that training manuals developed by experts were still being used to train new staff. Furthermore, various studies, draft regulations, and laws were finalized, presented to decision makers, and adopted. 8.13 Another important contribution to sound implementation was the PMU and MOF units were able to recruit highly qualified international experts and utilize them effectively, according to information gathered during the IEG mission. This is in contrast with projects elsewhere where a bias against recruiting international experts may result in under provision of know-how. 8.14 The ICR rating for implementation agency performance reflects the initial slow start in project implementation. However, this assessment takes into account how the various MOF departments have sustained and deepened the activities initiated under the project and have brought to completion or contributed to an impressive number of policy actions since the project closed. For this reason, the implementation agency performance is rated highly satisfactory. 8.15 The overall Borrower performance is rated satisfactory, reflecting strong performance at all levels, especially implementing agencies. 9. Monitoring and Evaluation What Constitutes a Strong M&E System for PFM Operations? 9.1 The 2005 IEG (then OED) evaluation Capacity Building in Africa made four main general recommendations also applicable elsewhere, including: “Sector and thematic leadership should develop sector-specific guidance on diagnosing public sector capacity needs and ways of monitoring and evaluating interventions” (OED 2005). This reflects the difficulties encountered around the time (2002–2003) PFMRP was being designed in setting- up appropriate M&E systems for public sector projects, of which PFM operations are a large subset. The MTDF II designed at the end of last decade in part addressed the diagnosis of the needs by supporting a demand-driven TA scheme, which still faced M&E challenges as described below. 9.2 The 2005 report goes on to note “since it is not possible to directly attribute development impacts to capacity building efforts, the evaluation looks to intermediate 52 outcomes “to assess the effect of efforts to enhance capacity” (OED 2005). This was an approach also largely followed herein. 9.3 Except for a few operations,38 the assessment of Bank-funded projects in support of PFM approved during the 2000s has now been completed.39 They have generally revealed systematic weaknesses in M&E. This is largely due to excessively ambitious and complicated PDOs and misaligned indicators. Furthermore, the assessment herein of operations in Vietnam, as well as that of others undertaken in parallel with this study,40 reveals that an M&E system needs to track results associated with three types of project activities. They are: (i) establishment of a functional information system; (ii) capacity building through TA, training, and study tours; and (iii) TA for preparing legal and regulatory actions and their implementation. A comprehensive M&E system should thus be based on the features detailed below. 9.4 Indicators should be closely related to activities funded by the project and further results attributable to them, at project’s end. Typical issues frequently documented include indicators that are not attributable, readily quantifiable, or are set “too high” or “too low” (i.e., PDO indicators that are project outputs or related to higher level objectives). In designing the M&E system, it is important to keep in mind the indicators that appear as intermediate outcomes from the standpoint of the PFM program may be legitimate PDO outcomes from the standpoint of the project. For example, in the case of a FMIS, the percentage of transactions being processed through the system is a good proxy for transparency and accountability, as is the share of public procurement not subject to special procedures (i.e., sole-sourced or negotiated contracts). On the other hand, PEFA indicators need to be used with caution. These indicators and their change over time may provide good indication of progress in PFM. However, as evidenced in this assessment and elsewhere, they are imperfect indicators for many operations because of the required timing and measurement lags. An indicator calculated at project’s end is based on data that may be two years old and therefore would not capture improvements during the last one to two years. Similarly, there is a need for at least two observations for the period just preceding project approval and at closing. 9.5 Periodic suitably designed standalone assessments offer complementary information to that provided by indicators. Impact or independent assessments are rarely included within the design of PFM operations, even though they too provide invaluable information on efficacy. There seem to be two reasons for this: (i) such assessments may be seen as costly in terms of financial and human resources and a source of distraction by the implementation team without commensurate benefits; and (ii) they are not considered at the design stage and not retrofitted into the project M&E during implementation. The issue of cost and benefit of good quality M&E goes beyond the scope of this report, which restricts itself to noting that some complementary assessment appears warranted in all cases. The principal approaches seen in various operations are:  Periodic independent evaluations. This approach was an integral part of M&E in the Cambodia operation. It was undertaken about every two years and aimed to identify bottlenecks and suggest actions to address them. To maximize their usefulness, these evaluations should be not only at the program level, but also at the project level, and their recommendations proactively taken on board by government and the PMU as well as the Bank. They also may provide key inputs to the mid-term review and ICR. 53 Another type of study may be useful for operations aimed at supporting improved service delivery and decentralization. Expenditure tracking studies (typically undertaken in the social sectors and agriculture) may provide answers to questions such as “is the flow of fund to service delivery centers better,” which in turn reflects improved PFM.  Specific assessments. An important set of question to be asked at the end of capacity building activities concerns include: (i) was the training, study tour or other capacity building activity (e.g., manuals, draft policy papers) useful, and what evidence is there of this; and (ii) are trained civil servants still working in government, and if so, in a capacity where the acquired skills are still utilized? The questions may be largely answered through a tracer study, such as the one undertaken under the Malawi PFM project. The MDTF II project provided an interesting experience related to the first question involving post-training tests being administered and the top performers receiving an award and the opportunity for advanced training.  Qualitative assessments. This is a commonly used approach, based for instance, on perception studies, which asks questions such as: (i) to civil servants, is your work environment better thanks to the operation; and (ii) to users of government services, have public service improved and in which way?  Use of PEFA. PEFA indicators provide a useful way to measure progress in PFM reform. They can be especially useful for assessments undertaken after the operation has been completed for over a year, especially when they provide both a baseline and an end-of-project metric, as was the case for Cambodia. However, their use is more limited as M&E of PFM operations. Timing is typically unsuitable—even if undertaken at the end of project. The data on which the PEFA indicators are based related to the previous two years and would not capture project contributions, as in the case of both Vietnam operations. Furthermore, there are certain limitations to PEFA assessments, and their results need to be interpreted with some caution.41 9.6 Modern information technology and social media may act as key complements to project level M&E. Modern technology may offer the possibility of cost-effective gathering of information on PFM that is not yet used in World Bank operations and would not have been possible prior to the digital age and modern telecommunication. The Vietnam assessment identifies an example of effective use of social media by both the government and independent commentators to disseminate information pertaining to the budget. PFMRP 9.7 Design. At time of project approval, the results framework was based on a logframe, which presented the chain of logic from inputs to output to outcomes to higher level objectives (World Bank 2003a). The only provisions in project design with a bearing on M&E were periodic supervision and the MTR, the CTA position and institutional responsibility to monitor indicators and report delegated to the PMU by the Steering Committee (World Bank 2003a, 16). There was no provision in the original project design to undertake any type of qualitative or quantitative assessments along the lines described above, and none was undertaken during implementation. 54 9.8 The original results framework was a weak part of project design. Largely, the indicators were related to activities and objectives, many with good attribution. Seven clearly defined higher level (CAS) indicators might have also served as PDO indicators, even if attribution would have been partial. This reflected the fact that that the higher level subgoal was closely associated (and in some way better stated) than the PDO. Three PDO indicators suffered from various degrees of deficiency, including imperfect mapping with objectives (there were more objectives than indicators) and, most important, the absence of baselines and targets:  PDO Indicator 1: Accuracy, timeliness, relevance, transparency, and compliance with international best practices in budget execution and reporting at each level of government. The PAD does not explain how the indicators would be measured. In any event, the goal of conforming to international best practice would be hard to measure (e.g., would it mean an “A” rating for most PEFA indicators) and more ambitious than the PDO statement whereby various aspects of PFM would be strengthened.  PDO Indicator 2: Better planning of the state budget and the Public Investment Program to achieve the growth and poverty reduction goals set out in the original government strategy (subsequently SEDP). Conceptually, this indicator was associated with project activities, notably the MTEF, but its target was unspecified. For instance, taking into account recurrent implications of public investments would have been important.  Greater fiscal sustainability through improved and more integrated recording of external and domestic public and publicly guaranteed debt, improved capacity to monitor SOE liabilities, and improved ability to assess associated fiscal risks. This indicator was also related to activities under the project. However, as events demonstrated during the late 2000s to early 2010s, it was also affected by external factors. 9.9 The 17 output indicators were related closely to components. However, they included targets for achievement dates that were unrealistic (mostly 2004 to 2006). Most, if not all, were readily measurable without reinterpretation. 9.10 The 2011 amendment to the credit agreement changed the results framework. It left the higher level and PDO indicators, but modified intermediate outcomes. The target dates were thus modified and a few new indicators were added. However, these changes came late in the implementation process and did not fully address the need to simplify the PDO and the results framework. 9.11 Aside from indicators, there were no ad-hoc studies to assess project impact. For example, a tracer study of beneficiaries of training would have been helpful (this also applied to MDTF II). It should be noted that tracking studies in one or more sectors may not yield significant information given that the relevant PEFA indicator PI 23 is rated “A.” 9.12 Implementation. The implementation of M&E was partial, as intermediate outcomes were not reported in the ISRs, even though they provided useful goalposts.42 However, according to the ICR, this information was regularly compiled and monitored by the MOF. The shortcomings in the results framework were pointed out to the task team by the management unit, but not addressed, notably at mid-term review or when processing the 55 additional financing. A PEFA was prepared in 2013, with the support of the MDTF II project. This was a useful exercise as it identified PFM issues at the time (2011–2012) but was unable to capture project activities such as the TABMIS coming on line. As a result, it does not allow for the measurement of project outcomes at closing. 9.13 Utilization. According to the ICR, the M&E data were effectively used to steer project implementation. As a result, certain activities were reoriented or dropped. Nevertheless, some questions remain over how well M&E information was used—other than lack of monitoring of intermediate outcomes in ISRs. For instance, setting up a TSA by 2004 was an initial project objective that kept slipping. However, this did not appear to have resulted in greater urgency being given to implementing this measure. Furthermore, given the difficulties encountered, the profile of this policy action could have been raised by including it within the PRSC agenda. This was eventually done in 2014 through the EMCC2 (see appendix B). Since then, EMCC-3 was instrumental in facilitating the resolution of disagreement between the MOF and the CB on the collateral requirements on Treasury’s participation in the interbank payment system (IBPS) to enable full roll-out of TSA to all branches of CB. 9.14 Other. Aside from the formal M&E process, it appears that social media is being used widely and effectively as a means to communicate and provide feedback on government activities in general and more specifically PFM. Box 9.1 summarizes the relevant information. Box 9.1. Social Media Monitoring of Government Finances through Facebook Government Facebook page: https://www.facebook.com/thongtinchinhphu/ Budget transparency: https://www.facebook.com/todocabi/ Views of taxpayers: https://www.facebook.com/groups/Gocnhinnguoidongthue/ Budget allocation and planning processes: https://www.facebook.com/quyentiepcanthongtin/?fref=ts The government’s Facebook page is followed by more than 60,000 people; the one on budget transparency by some 43,000 people. 9.15 In view of shortcomings in all dimensions, M&E for the project is rated Modest, even though increased use of social media may have been a partial substitute for monitoring overall progress in the area of PFM outside the project (and MDTF II) purview. MDTF II 9.16 Design. The PAD candidly recognized the difficulties with setting up an appropriate M&E system for this demand-driven operation that would fund activities specified in successive implementation plans. This requires a bottom-up M&E that bases expected outcomes on results from yet unknown subprojects. A deterministic M&E is impossible to define under such circumstances. One approach would be to formulate results in terms of outputs (e.g., so many studies and training activities), which does not capture impact. The approach put forward in the PAD is more ambitious, unrealistically so, even though it explicitly recognizes the measurement and attribution difficulties by defining an indicative 56 framework: “given the flexible nature of the project, the results framework at this stage could only provide indicative outcomes and indicators, which was built on the overall objectives of PFM reform in 2006–2010 in related strategies, the Single Strategy Document (SSD)” (World Bank 2010). It was also expected that “a pilot and baseline Public Expenditure and Financial Accountability (PEFA) assessment will be conducted in 2010 and a follow-on assessment would be conducted at the end of the project to allow measuring progress.” In practice, an end-of project PEFA would only measure the results one to two years prior to closing. 9.17 The outcome indicators included in the PAD results matrix were thus aligned with the overall objectives of the PFM program. Attribution of changes at that level to small subprojects averaging a few hundred thousand dollars was problematic. The matrix also included 47 intermediate results—some of which were potentially legitimate PDO outcomes while others were simple outputs. In any event, these results were too numerous. Furthermore, many of the indicators lacked baselines while others were not readily measurable. The original design was thus quite weak. 9.18 The task team recognized the shortcomings in the original results matrix, which was overhauled following the MTR by aligning it more closely with project activities—most of which were known by then. This pragmatic approach was appropriate even if the modifications were insufficient. The revised results matrix reduced the number of indicators. Three new PDO indicators were included. All suffered from lack of clarity or measurement problems and lack of baselines. For instance, attribution of improved revenue mobilization to the project would be weak given the dominance of external factors such as the structure of growth. The original intermediate outcomes were replaced by 15 results, which were largely both attributable and measurable. These results were aligned with the reduced number of components. 9.19 Arrangement for results monitoring included timing of data collection, source of data, and strengthening of the MOF’s capacity to collect and analyze data (World Bank 2010). The need for analytical work to complement M&E, with specific reference made to two PEFAs. 9.20 Implementation. The implementation of M&E was initially uneven but improved over time. On the Bank side, annual standard reporting of results, to the extent data were available and indicators lent themselves to being reported, started with the first ISR issued in 2010. The formulation of the two implementation plans helped determine the series of activities to be funded, each with their own timetables and goals, which facilitated the restructuring of indicators based on the aggregated revealed demand from approved subprojects. This also allowed for monitoring of output and process-oriented results at the subproject level. In addition, a PEFA study was completed in 2013, which provided stakeholders with important information on the state of PFM in Vietnam. However, the PEFA data pertained to early 2010s and provided neither a baseline for the project nor a good indication of the situation at its closing. 9.21 The most important aspect of M&E implementation was not necessarily at the broad project level, but rather what was being done at the subproject level. Aside from the ultimate goal that was set for most of these (e.g., law prepared, strategy formulated and adopted, or some aspect of PFM becoming more efficient), there were experiments with novel approaches. A noteworthy case based mission interviews concerns the training provided 57 under a subproject. A typical question associated with training is whether there is good retention. To promote this goal and test the efficacy of training, the relevant managers decided to test participants on their knowledge, which was apparently high overall. They took this approach a step further by formally recognizing a handful of those who performed best in the test and giving them the opportunity to take more advanced training (ultimately increasing the likelihood of their promotion). 9.22 Utilization. The formal M&E was used to monitor revised input,43 output, and outcome indicators. Use of M&E data improved with better quality of reporting. However, ultimately the strength of the M&E was not its formal set-up in project documents but how subproject information was used to steer it toward specific outcomes. This information was subsequently used to bring various activities to conclusion, and its availability was most useful in the preceding assessment of efficacy. 9.23 Despite weaknesses in how project results were measured at the aggregate level, project stakeholders set up a subproject-level M&E system that was both effective and useful. Therefore, project M&E, while unconventional, is rated substantial. 10. Lessons General 10.1 An evolving project level M&E may be a good substitute for a formal one that is pre-set at appraisal. This can be the case when the iterative, demand-driven nature of the activities does not allow clear measurable objectives to be set from the outset. Also approaches, such as use of social media, may be informative and cost-effective. Finally, standalone studies and assessment provide important complements to results-based M&E. 10.2 Outcomes for PFM operations are difficult to measure at closing. The Vietnamese cases show how ongoing policy actions take time to mature as do systems. In this instance, project outcomes became fully apparent in early 2016. In cases elsewhere, a number of factors including poor governance may undermine the initial results. Even though periodic PEFA assessments are not suitable as M&E for operations because of lags in indicators, they can provide useful information on PFM program progress over time. 10.3 PFM reforms need decades to complete. Vietnam is well into its second decade of reform. Both the government strategy and actions needed to reach the higher level objective of good governance show the need for this effort to be sustained into the 2020s. In this context, continued effective provision of financial and, more importantly, knowledge by development partners, will be needed. Strong donor harmonization and coordination have been enabling factors so far. Coordination has become more important now that some partners are no longer participating in the follow-up to the MDTF under preparation, providing separate support instead. 10.4 The funding of operations may not be commensurate with their impact. Significant results were attained under MDTF II even though the grant amount was small. The supervision resources devoted to the project were large compared to disbursements because they underpinned an important policy and institutional dialogue. The World Bank correctly continued to support project implementation, even at a time were difficulties were 58 experienced. On the government side, capacity built was sustained and, together with substantial ownership, resulted in actions being continued beyond the timeframe of the project. 10.5 TA operations that are initially focused on a single ministry may have a greater chance of success. Clarity in ownership and leadership of the activities by the MOF, which is the leading institution in PFM, was essential and avoided overcomplicated coordination mechanisms in the case of MDTF II. However, future PFM reforms will inevitably have to cover multiple entities and would need to ensure activities are well sequenced and coordinated. FMIS (TABMIS) 10.6 Strong and sustained government and MOF commitment is essential. Since these projects can take 7–10 years or longer for completion, it is necessary to maintain this commitment over a long period, which can extend beyond an election cycle. This commitment can be achieved better if projects are framed as public expenditure management systems reform, rather than just accounting systems reform. In Vietnam, government support was particularly important to address project-staffing issues, resolve business processes issues, and tackle interorganization issues. 10.7 Comprehensive FMIS implementation as in Vietnam takes about 10 years to complete. These projects are characterized by slow disbursement rates over the initial years, which are spent in systems design, testing and piloting. The disbursement profile needs to be adjusted accordingly. 10.8 Continuity in the project team contributes to success. There was stability on both government and Bank side, and key members of the team that designed the project remaining engaged in its implementation for the first few years ensured an effective transfer of know- how to field-based and other Staff. 10.9 Systems sequencing is key. The TABMIS implementation in Vietnam illustrates how project implementation for similar systems needs to be phased44 to achieve significant outcomes such as good budgetary control and cash management early in the Project. It is necessary to first implement modules to cater to core budget execution processes and processing of payments and receipts transactions, across government, before going on to other noncore elements, such as fixed assets/inventory management, human resource or fleet management. 10.10 Systems deployment strategies should make sure that a large part of the budget is covered from the outset. A major part of the budgetary resources should be subjected to ex-ante budget and cash control so that meaningful fiscal control and cash management is possible at an early stage in the deployment. A deployment across treasury offices through which these transactions are routed, may be sufficient for this purpose. 10.11 Spending unit access is a challenge in the use of treasury centric systems. The lack of spending unit access inhibits ownership at the line ministry and end-user level. This often leads to line ministries developing independent financial systems for their own management use, as in the case of Vietnam. Avoiding this outcome requires that once the 59 basic treasury system has been implemented and is operating in stable mode, the large spending units be empowered and made responsible for the bulk of transactions. 10.12 Availability of specialized IT expertise within the Bank project team is critically important to assure the quality of design and supervision of the projects that support development of complex IT systems. The Bank team needs to have experienced specialists on board or available to advise the client on important technical issues during the design and implementation phases. These specialists also need to be familiar with procurement practices and procedures for IT related procurements and the World Bank rules under which they will need to be applied. 10.13 Staff in the implementing agencies should recognize the inevitability of change. Resistance often comes from staff responsible for operating the legacy systems as well as those who are used to doing their regular work in a given way and are reluctant to change. MOF management has a major role to play. 10.14 The design of the main consultancy package has proven to be a significant bottleneck for TABMIS implementation. The consultancy package should be designed to ensure that appropriate consulting help is available throughout the various stages of the project. 10.15 Not all needs may be foreseen during the project's preparation and original design. The number of system users of TABMIS increased threefold between design and full rollout and had to be accommodated. 1 The conclusion of the IEG assessment of progress in the PFM area and its attribution to budget support operations was not sanguine: “During implementation of PRSC there were some important achievements in PFM, including establishment of a modern treasury management system (TABMIS), strengthening of external audit functions, and a PEFA exercise conducted in 2012/13. However, achievements in internal auditing and reporting of expenditures were less noteworthy. PFM in Vietnam continues to suffer from long-standing problems such as nonconformity of financial reporting with international standards, lack of multi-year fiscal projections, carry-over of expenditures to following years (these expenditures also lack of economic and functional classification), and lack of reporting at the commitment stage that limits the efficiency of TABMIS. The CPIA subrating for the quality of budget and financial management was lower in 2012 than in 2006.” The present report denotes substantial progress in most if not all areas because of recent policy actions. 2 This internal note only uses end-2014 data and is being updated to reflect more recent information. 3 See World Bank (2010, 18–22) and IEG (2015) for a more detailed analysis of the 2007 –2013 experience. 4 This assessment does not make a judgment on the appropriate level of debt other than to note that sustainability is not only a function of debt to GDP, but also interest rate on the debt, and whether or not incremental borrowing is for activities with high rates of rates of return. 5 Budget data were made available to IEG by the Country Management Unit. 6 The 2015 IEG evaluation of budget support operations was unable to obtain information on this point. 7 The objective includes other areas such as public administration reform and anti-corruption. 8 See World Bank (2003a) for details. 9 As noted in the sustainability section, actual requirements are more than double and are adequately provisioned by government. 10 Allocation is only by disbursement category. 60 11 These resources were first allocated to a parent trust fund and subsequently to the project. The parent multi- donor trust fund arrangements are described in World Bank (2010). 12 A question asked later in this report is whether greater collaboration with the PFMRP PMU would not have been helpful. 13 One that would improve the scores, for instance PI8, PI9, PI18, PI20, PI22, PI24, and areas such as comprehensiveness and transparency, predictability and control, and fiscal performance. 14 Figures C1–C3 show how the system relates to various core PFM functions. 15 The piloting of the Medium-term Fiscal Framework and the Medium-term Expenditure Framework from 2005 - 2007 was launched in 04 sub-national governments including Hanoi city, Ha Tay, Binh Duong and Vinh Long. In 2008, Ha Tay province was merged into Hanoi city, then the pilot was in 3 provinces. 16 It is beyond the scope of this review to assess this new law. But according to World Bank staff interviewed, it covers most of the issues that were discussed with government upstream to its preparation. As also noted earlier, the project contributed to certain aspects of the law. 17 Some donors such as EU have stringent rules concerning the need to disburse funds within a specified period. However, their definition of disbursement varies somewhat from the World Bank’s. Specifically a contract once signed is considered as disbursed and is then subject to a grace period of about two years, during which service may be provided. 18 It should be noted that the PEFA fails to capture the impact of the TABMIS and recent reforms. Nevertheless, some of the “C” and “D” ratings would not have been affected substantially and will need to be tackled through additional actions. 19 This issue falls outside the scope of this assessment. 20 “Self-liquidating” risks associated with project implementation are not cited. 21 The risk is being reduced through the support of the Ministry of Finance (the State Treasury) for the use of TABMIS by sector ministries, including user training; central help desk to support via email; hotlines; regular direct phone calls to understand obstacles; and to provide on-site supports (for difficult issues) or off-site support (for simple issues). 22 See appendix C for details. 23 According to transparency international (transparency.org https://www.transparency.org/country/#VNM) based on its perception index, corruption in Vietnam remains a significant problem. In 2015, the coun try’s ranking was 112 out of 168. This ranking reflects limited improvement over time. In 2002, Vietnam was ranked 85 out of 102, and in 2008 its raking was 121 out of 180. 24 Not rated in this evaluation as it is a country level risk that would need to be assessed in the Country Partnership Strategy. 25 For instance, development policy [adjustment] operations circa 2000 began to provide reference to the adequacy of public financial management and procurement systems. This judgement as well as more traditional assessments of the macroeconomic framework and allocative consideration of public expenditure provided the cross-cutting basis for budget support. Subsequently, PFM reform became a typical pillar of PRSCs and some other general DPOs. 26 “A peripatetic rather than resident CTA should be appointed to assist in implementation. The Director of the State Budget Department in preparing the MTFF and in coordinating the preparation of MTEFs in pilot sectors. This Chief Technical Adviser will spend some three months per year in Hanoi for three years, making approximately four visits per year to review progress at critical junctures and agree next steps. The CTA's primary responsibility will be to support the development of the MTFF ” (World Bank 2003a). 27 Cr. 1951 UG. Even though the World Bank’s external site states that the assessment may be disclosed, it was not readily available. The internal document was consulted for the purpose of this report. The assessment concludes that the [demand-driven] project outcome was highly satisfactory, in large part because of the flexibility to finance studies, activities, and other inputs as the need arose. It was strength of the design that considerable flexibility was allowed to the project managers to modify project objectives and instruments 61 during the course of the projects.” A key lesson worth noting here was that “well -chosen and well-managed long-term expatriate advisers had in many instances been effective in fostering capacity building.” 28 However, future actions should focus on budget preparation and execution capacity in the line ministries. 29 The report was called Project Status Report until 2004 and ISR thereafter. Aside from the name change, the content and coverage of these reports changes twice during the implementation period of the PFMPR. The first change took place when the ISR format was adopted. The ISR presentation was further changed in 2010, notably through the introduction of a detailed risk assessment framework. 30 Downgrading of ISR ratings was reportedly discouraged by reform minded senior civil servants who feared that this would undermine their credibility needed to fight internally the reforms. 31 The MOF team sought to develop a COA to cater to the needs of SUs which in Vietnam are operating with modified accrual accounting. Due to complexities of the budget system in Vietnam (being unitary, covering all four levels of the government), the MOF decided to drop this idea as they wanted to have proper cash accounting in TABMIS first. That was why the COA issue had been raised in a number of ISRs as the pending issue. To the credit of the State Treasury, when the COA for TABMIS was agreed, they implemented the upgrade of their legacy system to operate with the TABMIS COA , which helped with data conversion during TABMIS roll-out. 32 Support subsequently provided to this area is discussed elsewhere under the MDTF II. 33 At time of the MTR, the Bank was no longer designing projects and related M&E based on logframes. M&E is discussed more extensively in the relevant section of this report. 34 The present assessment raises these issues so that if necessary they may be studied in a broader context if still unresolved. However, looking more broadly into how the Vietnam portfolio was managed and Bank-UN relationships clearly falls outside this review. 35 Arguably, implementation should have also been downgraded to moderately unsatisfactory. 36 Much of the analysis in this paragraph is from the Implementation Completion and Results Report (World Bank 2014b), validated by the consistent stakeholder feedback received by the mission. 37 For instance, as was done decades earlier in Korea through the Korean Development Institute, which contributed to the “Korean Miracle.” 38 For example, the Russia—Treasury Development Project. 39 More recent projects are either ongoing or have not yet been evaluated. 40 Cambodia, Malawi, and Zambia (completed) and Ghana (ongoing). 41 Various sources, including the IMF, note limitations notably due to the fact that PEFA does not provide the basis for prioritizing reforms. Additional analysis may be found in a recent report from the Overseas Development Institute (Hadley and Miller 2016). 42 Following change in the report format, indicators were introduced in ISR #5. It only shows the three PDO indicators. The entry for intermediate outcomes reads “n/a – old format PAD” suggesting that there were not any. However, the logframe include a number of useful intermediate outcomes. 43 The ICR states that this monitoring allowed dropping subprojects that could not be completed before closing, while the PMU was able to identify savings that could be redeployed elsewhere (World Bank 2014b). 44 This strategy was also employed in Kazakhstan, Pakistan, and Russia where a focused effort on Core budget execution processes from the start has yielded good results with a minimum of wasted time and effort. 62 References Allan, Bill, and Habib Nasser Rab. 2014. Vietnam Fiscal Transparency Review: Analysis and Stakeholder Feedback on State Budget Information in the Public Domain . Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/2014/05/19554712/vietnam-fiscal- transparency-review-analysis-stakeholder-feedback-state-budget-information-public-domain. Government of Vietnam. 2013. Vietnam Public Expenditure and Financial Accountability (PEFA): Public Financial Management Performance Assessment July 2013. http://www.pefa.org/en/assessment/files/1205/rpt/9497 Hashim, A., and Bill Allan, 2001. Treasury Reference Model. Washington, DC: World Bank. http://www1.worldbank.org/publicsector/pe/trm.pdf. Hadley, Seird, and Mark Miller. 2016. PEFA: What Is It Good for? London: Overseas Development Institute. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=7&ved=0ahUKEwi9pKT rn6XNAhVHKx4KHaKJDPkQFghFMAY&url=https%3A%2F%2Fwww.odi.org%2Fsites%2Fodi .org.uk%2Ffiles%2Fresource- documents%2F10484.pdf&usg=AFQjCNH37GrC_q7NGvpTlMMaoEIRBON3nA IEG (Independent Evaluation Group). 2005. Capacity Building in Africa. Washington, DC: World Bank. ———. 2015. Project Performance Assessment Report: Vietnam. Report No. 96203. Washington, DC: IEG. IMF (International Monetary Fund). 2014. Vietnam: 2014 Article IV Consultation-Staff Report. Washington, DC: IMF. Irwin, Timothy, and Tanya Mokdad. 2010. Managing Contingent Liabilities in Public-Private Partnerships. Washington, DC: World Bank. OED (Operations Evaluation Department). 2005. Capacity Building in Africa: An OED Evaluation of World Bank Support. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/7468. Wescott, Clay G., Nguyễn Hữu Hiếu, and Vũ Quỳnh Hương. 2009. Public Financial Management: How to Deliver Better Value for Money in Vietnam’s Public Administration System? Hanoi: United Nations Development Programme. World Bank. 2003a. PFMRP, Project Appraisal Document. Washington DC: World Bank. ———. 2003b. Country Assistance Strategy. Washington, DC: World Bank. ———. 2005. Cross Sectoral Issues. Vol.1of Vietnam Managing Public Expenditure for Poverty Reduction and Growth. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/2005/04/5813703/vietnam-managing-public- expenditure-poverty-reduction-growth-public-expenditure-review-integrated-fiduciary- assessment-vol-1-2-cross-sectoral-issues. ———. 2006. World Bank. 2006. Vietnam—Aiming High: Vietnam Development Report 2007. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/2006/12/7273035/vietnam-aiming-high-vietnam- development-report-2007 ———. 2007. Country Partnership Strategy. Washington, DC: World Bank. ———. 2010. MDTF II Project Appraisal Document. Washington, DC: World Bank. ———. 2012. Country Partnership Strategy. Washington, DC: World Bank ———. 2014a. PFMPR Implementation Completion Report. Washington, DC: World Bank. ———. 2014b. MDTF II Implementation Completion Report. Washington, DC: World Bank 63 ———. 2014c. Vietnam Fiscal Transparency Review. Washington, DC: World Bank. ———. 2015. Vietnam—Sixth-Tenth Poverty Reduction Supports Credit Project; Second Phase of the 135 Programs Supports Credit Project; and One-Two Public Investment Reform Project. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/2015/08/24829471/vietnam- sixth-tenth-poverty-reduction-supports-credit-project-second-phase-135-programs-supports-credit- project-one-two-public-investment-reform-project. U.S. Department of State. 2015. Fiscal Transparency Report. Washington, DC. http://www.state.gov/e/eb/ifd/oma/fiscaltransparency/243692.htm. 64 APPENDIX A Appendix A. Basic Data Sheet VIETNAM: MULTI-DONOR TRUST FUND FOR PUBLIC FINANCIAL MANAGEMENT MODERNIZATION. (TF-94240) Key Project Data (amounts in US$ million) Actual or Actual as % of Appraisal current appraisal estimate estimate estimate Total project costs 5.78 5.39 93% Loan amount 7.19 5.10 71% Cofinancing - - - Cancellation - 2.09 - Cumulative Estimated and Actual Disbursements FY10 FY11 FY12 FY13 FY14 Actual (US$M) 0.50 0.93 2.48 3.38 5.10 Date of final disbursement: May 1, 2014 Project Dates Original Actual Initiating memorandum 01/22/2008 02/29/2008 Negotiations 05/05/2009 05/05/2009 Board approval 09/15/2009 04/01/2009 Signing 06/02/2009 06/02/2009 Effectiveness 09/07/2009 09/07/2009 Closing date 06/30/2011 12/31/2013 APPENDIX A 65 Task Team Members Responsibility/ Name Title Unit Specialty LENDING/GRANT PREPARATION Minh Van Nguyen Senior Economist (TTL until EASPV TTL Oct. 2008) Quyen Hoang Vu Economist (TTL from Oct. 2008) EASPV TTL Hisham Abdo Kahin Senior Country Lawyer Counsel Cung Van Pham Financial Management Specialist EASFM Financial Mgt. Nguyen Chien Thang Senior Procurement Specialist EASRP-HIS Procurement SUPERVISION/ICR Quyen Hoang Vu Economist (TTL from Oct. 2008 to EASPV Task Team Leader Sept. 2011) Yasuhiko Matsuda Senior Public Sector Specialist EASPW Public Sector Mgt. Quynh Xuan Thi Phan Financial Officer GEFOB Financial Mgt. Tuyet Thi Phung Program Assistant EACVF Task Assistant Nguyen Chien Thang Senior Procurement Specialist EASRP-HIS Procurement Senior Economist (TTL from Sept. Task Team Leader Habih Rab EASPV 2011 until July 1, 2013) Senior Economist (TTL from July 1, Task Team Leader Minh Van Nguyen EASPV 2013 until ICR) Phuong Anh Nguyen Research Analyst EASPV Operation Support Hanh Thi Huu Nguyen Financial Management Specialist EASFM Financial Mgt. Nguyen Hoang Nguyen Procurement Specialist EASR2 Procurement Khanh Linh Thi Le Program Assistant EACVF Task Assistant Deepak Mishra Lead Economist EASPV LE/Manager Sandeep Mahajan Lead Economist EASPV LE/Manager Vu Thi Anh Linh Program Assistant EACVF Task Assistant Hisham Abdo Kahin Senior Country Lawyer Counsel Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs) Supervision/ICR FY10 11.93 36.60 FY11 24.73 75.00 FY12 35.27 245.52 FY13 3.17 30.77 FY14 35.13 113.46 Total: 111.23 501.35 66 APPENDIX A VIETNAM: PUBLIC FINANCE MANAGEMENT REFORM PROJECT (IDA-EY670 IDA-48630 TF-50988) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 71.45 87.72 123 Loan amount 71.45 82.72 123 Cofinancing - - - Cancellation - Cumulative Estimated and Actual Disbursements FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 FY13 FY14 Appraisal 1.04 0.02 3.78 4.71 6.10 5.71 10.17 16.29 23.07 3.80 0.00 estimate (US$M) Actual 1.04 0.02 3.78 4.71 6.10 5.71 10.17 5.51 11.32 14.81 5.99 (US$M) Actual as % of 100 100 100 100 100 100 100 49 34 0.00 appraisal Date of final disbursement: May 1, 2014 Project Dates Original Actual Initiating memorandum 04/22/2002 02/04/2002 Negotiations 01/05/2004 03/04/2003 Board approval 05/27/2004 05/22/2003 Signing 06/06/2003 06/06/2003 Effectiveness 09/04/2003 09/04/2003 Closing date 02/28/2009 10/31/2013 Task Team Members NAME TITLE UNIT APPENDIX A 67 Ed Mountfield Manager (TTL until June 2005) OPSPQ Gloria M. Elmore Temporary EASPR Robert J. Gilfoyle Sr Financial Management Specialist EASFM Hung Viet Le Sr Financial Management Specialist EASOS Minh Van Nguyen Senior Economist (TTL June 2005- EASPV August 2009 and between Aug. 2011- Dec.2011 ) Hoang Xuan Nguyen Procurement Specialist EASR2 Cung Van Pham Sr Financial Management Specialist EASFM Quynh Xuan Thi Phan Financial Officer GEFOB Tuyet Thi Phung Program Assistant EACVF Ronald Points Consultant EASPC Ramesh Sivapathasundram Lead Information Officer TWICT Nguyen Chien Thang Senior Procurement Specialist EASRP-HIS Jennifer K. Thomson Chief Financial Management Off OPSOR Quyen Hoang Vu Economist (TTL since August 2009 ) EASPV L. S. Christine Wong Shui Wan Operations Officer EASPW Cem Dener Senior Public Sector Specialist PRMPS Anna L. Wielgorska Senior Procurement Specialist EASR1 Bruce Pollock ICT System Consultant EASPV 68 APPENDIX A Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY02 38.00 261.55 FY03 38.00 135.28 FY04 14.00 57.05 FY05 5.00 28.91 FY06 4.00 22.87 FY07 2.00 15.69 FY08 0.00 0.00 Total: 101.00 521.35 Supervision/ICR FY04 21.00 57.13 FY05 40.00 111.27 FY06 14.00 36.28 FY07 16.00 37.43 FY08 27.00 57.50 FY09 10.00 0.00 FY10 17.20 87.80 FY11 20.40 76.30 FY12 16.42 63.70 FY13 22.00 65.90 FY14 10.00 67.50 Total: 214.00 660.80 69 Appendix B. DPO PFM Related Prior Actions Planning Process 2007 PRSC Allocate state capital expenditure transparently, using criteria like population, poverty and ethnicity – NB: 6 Also related to public investment and social protection 2008 PRSC Establishment of clear criteria for the selection of public investment projects and of mechanisms for their 7 financing and monitoring. NB: Also related to public investment Policy measures not included as prior action 2007 PRSC  Adopt regulations for master and regional plans specifying issues, process, agency responsibility, and -12 6-10 require disclosure  Establish list of indicators, data sources and reporting mechanism for monitoring of SEDP  Establish a legal framework for strengthening urban planning and urban infrastructure management PFM 2007 PRSC 6 Disclose results of audits conducted by the State Audit of Vietnam (SAV) and its annual audit plan 2008 PRSC-7 Issuance of regulations regarding content and timing in the disclosure of reports by State Audit of Vietnam, including audit reports of individual entities. 2009 PRSC 8 Formulate a public debt management law, consolidating the management of domestic and external debt. 2010 PRSC 9 The Recipient has formulated a regulation for development and implementation of internal audit of public sector agencies and defining organizational responsibilities. 2010 PIR 1 The Borrower, through MOF, has issued a Decision to align time of recording ODA funds in government accounts with the timing of actual receipts and expenditures. 2011 PIR 2 The Recipient, through MPI, has issued a Circular, including technical guidelines, to implement the use of electronic procurement (e-procurement) in pilot provinces and agencies. 2011 PIR 2 The Recipient, through MOF, has issued a Circular to mandate the sharing of independent audits of financial statements of ODA-funded projects with MOF and SAV. 2011 PIR 2 The Recipient, through MOF, has issued a Circular to streamline payment procedures and clarify and simplify payment-supporting documents for ODA-funded projects. 2012 PRSC 10 The Recipient, through MOF, has started an annual exercise of publishing the synthesis report based on financial statements of state-owned economic groups and general corporations. NB: Also related to state reform. 2013 EMCC 1 The Recipient, through Prime Minister, has issued Decision Number 958/QD-TTg dated July 27, 2012 to strengthen the institutional framework for debt management and establish prudential debt thresholds for medium-term fiscal sustainability. 2013 EMCC 1 The Recipient has issued Law Number 21/2012/QH13 dated November 20, 2012, amending Law Number 78/2006/QH11 dated November 29, 2006 on Tax Administration, to streamline procedures, introduce advance pricing arrangements, increase risk-based management, and improve transparency. 2013 EMCC 1 The Recipient’s Government has submitted to the National Assembly Report Number 283/BC -CP dated October 19, 2012, on the development investment status of 2012 and medium-term investment plan for the period 2013-2015 to set medium-term capital expenditure priorities in the state budget and including off-budget bond financing for 2013. 2014 EMCC 2 Government through Prime Minister, has issued Decision Number 689/QD-TTg, dated May 4, 2013, approving a medium-term debt management program for the period 2013-2015 2014 EMCC 2 Government through National Assembly, has adopted the Amended Law on Procurement No. 43/2013/QH13to strengthen transparency and competition in public procurement. 2014 EMCC2 The Recipient, through State Treasury, has implemented the treasury single account procedures in the Joint Stock Commercial Bank for Investment and Development of Vietnam, and the Vietnam Joint Stock Commercial Bank for Industry and Trade, and provided reports on the implementation. Policy measures not included as prior action 2007- PRSC 6-  Publish report on budget execution for current year and budget plan for forthcoming year, on an annual 12 10 basis  Issue regulations for periodic disclosure of external public debt and its composition APPENDIX B 70  Implement regulation to guide the establishment and issuance of benchmark government bonds. NB: Also related to Financial Sector  Consolidate administration procedures for all taxes into a single law, modernizing assessment and enforcement methods. NB: Also related to PSD  Revise the investment laws to improve efficiency in state capital expenditures and strengthen the management of public investment projects  Conduct public financial management reforms including commitment accounting, vendor management and new chart of accounts  Adopt procedures for annual audit plan ensuring sufficient frequency and adequate provincial coverage  Strengthen the legal framework for taxing the use of non-agricultural land and the capital gains to increase efficiency and equity, and dissuade speculation. NB: Also related to PSD Legal Development 2007 PRSC 6 Establish separate legal, judicial, economic, and budget committees to strengthen NA’s supervisory role Policy measures not included as prior action 2007- PRSC 6-  Adopt policies to encourage the participation of non-state establishments in the delivery of public 12 10 services  Enhance the responsibility of enterprises vis-à-vis consumers and strengthen the handling of consumer complaints  Develop effective legal framework on consumer protection Public Administration Reform 2010 PRSC 9 The Recipient has issued a regulation and developed pilot schemes to strengthen competition, merit orientation, and transparency in the recruitment, appointment, and promotion of civil servants. 2012 PRSC 10 The Recipient, through the Prime Minister has adopted a Public Administration Reform Master Plan for 2011-2020; and through MOHA, has started piloting a results-oriented monitoring and evaluation system. Policy measures not included as prior action 2007- PRSC 6-  Extend One-Stop-Shop (OSS) to all ministries and agencies and introduce inter-linked OSS to further 12 10 simplify administrative procedures  Adopt principles of competition, merit-orientation and transparency in the recruitment, appointment, promotion and dismissal of civil servants  Dissociate civil service pay structure from minimum wage setting and widening the range of minimum wages  Formulate common standards for IT applications and e-government interfaces at central and provincial levels  Develop indicators for monitoring and evaluating the implementation of PAR and provision of public administrative services Public Investment 2010 PIR 1 The Borrower has issued a Decree to establish a consistent monitoring and evaluation framework for public investment projects, including standardized monitoring tools. 2011 PIR 2 The Recipient, through MPI, has issued Circulars to establish standard templates for project monitoring reports and to set up the required legal criteria for individuals and institutions to carry out investment evaluation. 2014 EMCC 2 GOV through Prime Minister, has issued Directive Number 14/CT-TTg, dated June 28, 2013, to accelerate clearance of capital expenditure arrears and report to the National Assembly the current status and solutions going forward. Dropped prior action 2011 PIR 2 The Borrower has submitted, for consideration by the National Assembly, a draft law on Public Investment providing a common framework for all public projects. Fighting Corruption 2007 PRSC 6 Operationalize Steering Committee against corruption with power to suspend high level officials if suspected 71 APPENDIX B 2008 PRSC 7 Implementation of the asset declaration requirement with respect to senior officials and their immediate family members, and application of penalties for non-compliance. 2010 PRSC 9 The Recipient has: (a) carried out annual procurement compliance and performance audits of projects funded by the state budget; (b) made public findings of selected audits; (c) adopted a pilot procurement code of ethics for participants; and (d) launched an open access electronic bidding system. 2012 PRSC 10 The Recipient, through GI, has developed a framework for monitoring progress on the implementation and results of efforts to prevent and combat corruption. 2013 EMCC 1 The Recipient has issued Law Number 27/2012/QH13 dated November 23, 2012, amending Law Number 55/2005/QH11 dated November 29, 2005 on Anti-Corruption and including stricter transparency guidelines in areas and sectors most vulnerable to corruption. 2014 EMCC 2 Government has issued Decree Number 59/2013/ND-CP dated June 17, 2013, Decree Number 78/2013/ND-CP dated July 17, 2013, and Decree Number 90/2013/ND-CP dated August 8, 2013, to regulate and guide the implementation of the Amended Law on Anti-Corruption, including increased transparency, income and asset declaration of public officials, and accountability of public agencies and officials. Tax Reform Policy measures not included as prior action 2007- PRSC 6-  Establish single-window mechanism for businesses to cover registration, tax, and seal formalities in 12 10 selected provinces.  Simplify enterprise registration including by unifying tax and business identification numbers and streamlining seal procedures.  Rationalize incentives related to Enterprise Income Tax and simplify tax procedures for household businesses.  Consolidate administration procedures for all taxes into a single law, modernizing assessment and enforcement methods. 2013 EMCC 1 The Recipient has issued Law Number 21/2012/QH13 dated November 20, 2012, amending Law Number 78/2006/QH11 dated November 29, 2006 on Tax Administration, to streamline procedures, introduce advance pricing arrangements, increase risk-based management, and improve transparency. 2014 EMCC 2 Government through National Assembly, has adopted the Amended Law on Corporate Income Tax to establish competitive corporate income tax rates, clarify rules and regulations on transfer pricing, and introduce provisions on deductible expenses. 2014 EMCC 2 Government through National Assembly, has adopted the Amended Law on Value Added Tax to: (a) adjust the group of goods and services not subject to value added tax; (b) clearly specify the goods and services subject to 0% value added tax rate; and (c) apply thresholds as appropriate. 72 Appendix C. TABMIS Rationale for TABMIS In the early 2000s the Government of Vietnam’s Financial Management Information Systems were fragmented and had overlapping and conflicting functionality. Little attention was paid to critical flows of information between system components and beyond the system. The end result was lack of integrity in overall fiscal data, transparency and control. The problems could be traced to weaknesses at two levels: Institutional weaknesses (i) The lack of a common accounting standard; at least three parallel standards existed within Government in Vietnam: that of the State Treasury Department, that of the State Budget Department and that used by spending units (which in turn was inconsistently applied). Separate financial management information systems operated in the State Budget Department, the State Treasury Department and spending units. (ii) Extra-budgetary funds, on-lent official development assistance and much commune-level spending were not consolidated into the budget or recorded in the Treasury Department's main accounting system. The lack of a fully consolidated and integrated budget made it difficult to monitor total revenues and expenditures, distorting perceptions of the true fiscal position, with potentially serious consequences for fiscal stability. It made it impossible to assess how resources are being allocated as a whole and to poverty alleviation in particular. It made it hard to compare budget plans with budget out-turns. It also acted as an impediment to the successful management of further devolution and delegation. Technological weaknesses Although the Treasury databases were a reasonably effective tool for controlling and recording expenditure, the continued fragmentation and incompleteness of electronic data recording and reporting resulted in laborious semi-manual consolidation of data from multiple satellite databases. Consequently financial reporting was late and inaccurate. In addition, outdated ICT hardware, software and weaknesses in telecommunications infrastructure were also a major limitation. These deficiencies contributed to the poor flow of budgetary information between provinces, central financial agencies, central ministries, donors and the public. Central financial agencies and ministries that would like to take stock of often highly decentralized spending within their sectors were unable to obtain this information without great effort. Foreign donors who would like to give or lend more by way of fungible budget support and support for sector-wide programs found this hard to justify in the absence of reliable budget management information and reliable fiduciary systems. Although by 2003 Vietnam had made strenuous efforts to increase fiscal transparency, the availability of timely and high quality data was a constraint. 73 APPENDIX C Strengthening and integrating financial management information systems was therefore a critical next step for Vietnam and required interventions at both the institutional and the technological levels. Institutional Reforms The Government first moved to address the institutional weaknesses. The Public Administration Reform Master Plan for the period 2001 to 2010 stated that Treasury management will remain a central Government function and this was also clearly stated in the new State Budget Law. Article 61 of the new State Budget Law designated the State Treasury Department as the lead agency responsible for the financial management information system. In addition, the Government committed to integrate the Treasury accounting system, the budget accounting system and the spending units accounting system within an integrated chart of accounts. Work was undertaken to identify and consolidate extra-budgetary funds into the budget. Thus, although Vietnam's budget planning processes were decentralized, budget execution was firmly established as a responsibility of the State Treasury Department. The Treasury operated through a network of Treasury offices exist at provincial and district levels, which reported to the Central State Treasury Department. Improvements in the Technological Infrastructure In 1999, Vietnam's key fiscal agencies began to invest in a united information infrastructure in their various agencies and in modernizing the hardware and software in use. The purpose was to create a network backbone connecting individual departmental networks. In March 1999, the Minister of Finance approved the development of a State Budget database. As a result Local-Area Networks (LANs) connected the key financial agencies (the Treasury department, finance department, taxation department and customs department) in the provinces. Wide-Area Networks (WANs) were progressively established in some of the larger cities. Investment was undertaken to complete a national network backbone, which would link these LANs and WANs. By the mid 2000’s this backbone would link Hanoi, Danang and Ho Chi Minh City, and 58 of Vietnam's 63 provincial Treasury offices via leased lines to this backbone. The 600 district offices would mainly use dial-up connections to their respective provincial Treasury offices. Sufficient bandwidth was available for leasing as it became necessary. Upgrade of desk tops and LAN environments were also planned, and would be satisfactory for use with a new Treasury and Budget Management System. These initiatives provided the basis of a sound and stable institutional and technological infra-structure platform for implementing a modernized Treasury and Budget Management Information System – TABMIS -- across central government and all provinces and districts. PFMRP Component Related to TABMIS Description The first and the largest component of the PFMRP envisaged the implementation of an integrated Treasury and Budget Management Information System (TABMIS). A PHRD grant immediately preceding the project financed Technical Assistance to develop the conceptual design of the new system - TABMIS, including identification of functional user requirements APPENDIX C 74 and documentation of technical specifications. For this the Government through a competitive process contracted with Booz Allen and Hamilton (BAH) in January 2003 to develop the technical and functional specifications for the new TABMIS system during the preparatory phase of the project. The component included financing for: A turn-key contract covering:  The procurement of a new TABMIS, including hardware, software, implementation and integration services, training and change management services to be acquired through a two stage ICB process. The solution would be designed by bidders to meet functional user requirements and technical specifications precisely set out in the Bidding Document prepared by the BAH consultants.  Installation and configuration of TABMIS in Treasury head office, MOF, MPI and one pilot province: Treasury head office (which will house the central database and be the repository of the "core" system) will form the first production environment. The production environment will be tested here prior to the populating the system with live data. The chart of accounts and other control data (security profiles, banking controls, audit trails, data import and export capabilities, standard report configuration, etc.) will be established and live data entry enabled. The Ministry of Finance, Ministry of Planning and Investment and one pilot province would then be connected to the central Treasury system.  Roll out to all Treasury offices in provinces and districts: Implementation would then take place in a phased manner in all Treasury offices in provinces and districts. This sub-component will be considered complete when TABMIS is functioning in all Treasury offices and data communications between Districts and their Provinces and between Provinces and Central Treasury are operating successfully.  Roll-out to Finance Departments and planning departments at provincial and district levels: Implementation would then take place in Finance Departments and planning departments at provincial and district level. At the completion of this sub- component, all District and Provincial Offices of both the Treasury and the finance departments would be operating on the new TABMIS and the legacy systems will be de-commissioned.  Pilot implementation in selected large spending units: The Government, in consultation with its consultants and with the World Bank, concluded that it would be premature to attempt to integrate all spending units into TABMIS, so that they could transact electronically with the Treasury. There are many tens of thousands of spending units, and it would be a major further expense as well as a further logistical challenge to integrate them all. This sub-component would support the pilot integration of selected large spending units into TABMIS during the last phase of the project. Technical assistance for:  Procurement assistance and Independent Bid Evaluation: The component would also finance the continued assistance of the consultants with procurement and Independent Bid Evaluation. 75 APPENDIX C  Independent Validation and Verification (IV&V): The IV&V work would include validating and verifying progress against specific acceptance criteria and sign-off stages within the turn-key solution provider’s contract; advising the Government team on progress; conducting post-implementation reviews at sample locations; providing arbitration between the Government and the turn-key solution provider; and accepting responsibility for escalating issues that threaten the successful outcome of the project components and that local arbitration cannot resolve, through a contractually- stipulated escalation path, until resolution has been achieved.  Training and change management: This involved the development and implementation of a full and detailed strategy for change management and training. A “training of trainers“ approach would be used to deliver training to an estimated 14,000 trainees in the State Treasury Department, the Ministry of Finance, the Ministry of Planning and Investment and in provincial and district finance departments and line agencies. Supplementary technical assistance with financial management rationalization and reform: As the Government implements TABMIS, additional technical assistance needs would emerge. This sub-component would ensure that funds are available to finance such inputs as they emerge. A major input would be to support the establishment of a Treasury Single Account. The combination of TABMIS and the new inter-bank transfer system would make a Treasury Single Account technically feasible, but assistance with institutional development will be required to support its operationalization and the development of associated processes and procedures. The PAD for the project clearly identified the need for: (i) A Government TABMIS implementation team: Although much of the responsibility for delivery of the new TABMIS would be given to the tum-key contractor, it would still be essential to have a Government counterpart team in place at central, provincial and district levels to interact with the tum-key contractor's consultants. These costs will be financed 100 per cent by Government contribution. (ii) The need for recurrent operating costs: There are major recurrent operating costs associated with this component. These include the costs of hardware maintenance, software maintenance and connectivity (bandwidth purchase). These recurrent costs will build up during the implementation period, as the system is rolled out and additional software licenses are purchased. During the implementation period, these costs would be financed through a blend of IDA credit and Government contribution. In steady state, they were estimated to be some US$3.2 million per year, and the Government would meet these costs in full from the date of project closing. In addition, hardware replenishment costs will occur from time to time. These are estimated to be in the order of US$2.6 million approximately every five years and the Government will ensure that these funds too are made available if the system is to remain fully operational. APPENDIX C 76 After the project became effective, the TORs of BAH were enhanced (Feb 2005) to include activities envisaged under the categories:  Procurement assistance and Independent Bid Evaluation  Independent Validation and Verification (IV&V)  Training and change management. IBM Singapore was awarded in 2005 the contract for the implementation of TABMIS (after a Bank NOL issued on in June 23 2005) covering all implementation activities envisaged in after a two stage ICB based on tender documents and technical specifications prepared by BAH. IBM proposed Oracle EBS as the application software to be used as a basis for TABMIS implementation. Comments on the Preparatory and Design Policy and Technical Infrastructure Prerequisites. The preparatory and diagnostic work under taken by the Government and the Bank team had clearly identified the reasons for the fragmentation of the information systems which resulted in the lack of timely availability and integrity in overall fiscal data, transparency and control in financial operations. The Government had accordingly initiated activities that enabled improvements in the underlying policy and institutional environment that are essential pre-requisites for any FMIS implementation. These were: (a) Amendments to the legal framework to clarify institutional roles responsibilities for financial management; (b) Recognized the requirement for a uniform budget classification structure and Chart of accounts across all levels of Government that is in conformity of international standards such as the IMF’s GFS and started activities to put this in place; (c) Recognized the requirement of consolidation of Government bank accounts into a Treasury Single Account (TSA) at the Central Bank and started parallel initiatives to set this up. In addition the Government also undertook parallel investments to improve the technological infrastructure, including the required telecommunications network. System Design and Sequencing. Consultants (BAH) hired by the project developed focused functional specifications for TABMIS based on the World Bank-IMF Treasury Reference Model. 1 The design parameters for the systems ensured that:  The scope of the implementation would be restricted to the modules to cater to Core Budget Execution Processes, and processing of payments and receipts transactions, across government, before covering other non-core elements (fixed assests, HR management, etc.). Fixed assets were initially included, but subsequently dropped and not implemented. 1 World Bank Technical Paper 505, 2001; Ali Hashim and Bill Allan. 77 APPENDIX C  A Treasury Centric System would first be implemented and then decentralized to spending units as necessary - An attempt should be made to first capture payment / receipt transactions at Treasury offices then de-centralize to Spending units – if necessary/possible.  To start with, budget preparation will be done outside the system or by another system, and the final approved budget will and then be loaded in the system and used to control expenditure - However, all subsequent in year budget transactions, like budget releases, “virements,” transfers etc. would be done in the FMIS system.  Budget Department, Treasury, and Line Ministries would use the same system to process their transactions and would share databases; Budget Preparation and Budget Execution should use the same Chart of Accounts. The Economic classification segment should be a subset of the accounts line item segment.  Transactions should be captured in real time as they occur; Financial controls should be applied in ex-ante mode to all transactions processed by the system (e.g. funds availability checking on budgeted expenditures prior to committing funds or making payment); Ex-post transaction posting would be avoided; No expenditure transaction should be processed outside the system. Data should be captured only once as an accounting transaction progresses through the system. The scope of the system should include: Budget Funds - Payroll, Debt, Subsidies, Fiscal transfers, EBFs, Donor Funds and internally generated revenues.  The TABMIS databases should be treated as the primary source for financial reporting within Government; there should be no second set of books. Accordingly it was agreed that the initial functionality for the Vietnam FMIS would include the following elements: (a) Budget Management-Budget Apportionment, Budget Allotment, Budget Releases, Budget Transfers. (b) Commitment Management – Recording all commitments relating to intended government expenditures. (c) Payment Management- Processing all government expenditures relating to: (Procurement of goods and services from current/capital budgets, Salary Payments, Debt servicing payments, Subsidies/Fiscal transfers to sub-national levels or SOEs. (d) Receipts Management - Recording revenues and other receipts- Accounting (posting all transactions as they occur). (e) Cash Management. (f) Fiscal Reporting. In order to fulfill this functionality the system would need to: (a) Record- Initial budgets, budget revisions, budget releases; commitments; purchase orders, receipt of goods and services; vendor Invoices. (b) Authorize payments after checking for controls and give payment instructions to Bank- initially using checks printed from the system and then trough EFT arrangements. APPENDIX C 78 (c) Record revenues and other receipts paid into Government accounts, Reconcile with bank records. (d) Enable monitoring of balances in Government accounts. (e) Enable posting of all transactions, enforcement of controls, accounting and comprehensive reporting. (f) Ensure that these functions are performed by authorized staff only. (g) These design parameters conform to the Bank’s experience in implementing such projects in several other countries which also showed that implementation needs to be sequenced as above. Systems procurement was carried out through a competitive two stage ICB process based on well prepared detailed technical specifications which described the scope of the system in terms of the functional processes that would be covered and the agencies where the systems would be installed. Implementation envisaged a configuration/ parametrization phase in which the application software would be configured to Vietnam specific requirements and tested. It would then be piloted in selected sites and finally rolled to all sites. A treasury centric model was adopted but provision was made to also link selected spending units (SUs) towards the end of the project. System Procurement. The two main contracting assignments that were involved in project implementation were: a diagnostic and design consultancy and a contract for actual system implementation. Diagnostic and Design Consultancy. The design of the main consultancy package proved to be a significant bottleneck for systems implementation in Vietnam. The major consultancy required for the design, procurement and implementation phases is for diagnostic, design and project implementation and contract management. The work is to: (a) review the existing business processes, recommend changes and develop functional requirements and systems specifications for the system to be procured; (b) during the procurement phase, assist the government in the execution of the complex World Bank procurement processes; (c) in the implementation phase, help the government manage and supervise the work of the contractor hired t, and ensure that the system that is implemented according to specifications; and (d) help the government in contract management. Other subsidiary consultancies, for example, for change management and training, may be contracted separately. In Vietnam, the main consultancy was split into various contracts with the same consultant (BAH). However the consultancy for the supervision/implementation phase was called an Independent Validation and Verification (IV&V) consultancy.2The consultants misread the 2 Normally the IV&V process is carried out by the government with the help of Bank support where the consultant’s design is evaluated by the government and Bank team to ensure that it makes sense in the country specific circumstance. A consultant could also be hired for this process. 79 APPENDIX C purpose of the hiring and saw it as an audit type of exercise where they were to critique the initial design and the implementation work, instead of actually helping the government to ensure that the contractor implements the system in accordance with the design which had been specified and approved earlier. This caused major difficulties, and this consultancy did not achieve the purpose for which it was intended and the Government finally terminated the contract. As a result the government was left with limited support in the contract management and for supervising implementation. This resulted in a situation that the Government had to rely exclusively on the implementing contractor (IBM) advice during this phase. Configuring this consultancy assignment as a single package extending from the design to the implementation phases as was done in e.g. Pakistan, Kazakhstan, Russia may have mitigated these problems and ensured that the Government had consulting help and independent expert technical advice available throughout. System Implementation and Utilization Project Management Arrangements. Strong Government commitment, support from the MOF, High level project sponsorship, end user involvement and interagency coordination are critical for successful implementation. A set of project management structures are required to achieve this including:  A senior level project sponsor from the MOF to ensure that government commitment and support that will be required for introduction of policy changes and the decision making required is present.  A Steering committee with representatives of all major stake holders- MOF, Treasury, Budget, Central Bank, Line Ministries, Revenue collection agencies to provide policy guidance and ensuring consensus across all stakeholders.  A PMU to handle day to administrative aspects; Procurement of consultants and of the hardware and software necessary to implement the system.  A project manager who is a senior official from the functional side with stature within bureaucracy and adequate financial and administrative powers to cater to day to day operational administrative and financial requirements. It needs to be noted that FMIS projects should not be seen as information technology projects. The primary emphasis of the project needs to be on the functional objectives such as fiscal control, cash management etc. and this requires that management of the project is competent in these areas rather than technology. Project Management arrangements set up in Vietnam fulfilled these requirements. The Vice - Minister of Finance played the role of the project sponsor and showed continued commitment to the setting up of a FMIS. This was able to sustain Government interest in the project over its long execution period. The Vice Minister headed the steering committee which consisted of General Director level officials from the various MOF with representatives from other line ministries. The Steering committee met twice a year and APPENDIX C 80 provided policy guidance. A PMU oversaw activities in all project components and handled day to administrative aspects. A TABMIS implementation unit was set up in the Treasury to oversee TABMIS related activities. The TABMIS project manager was a senior official from the functional side. TABMIS Implementation and Status. The Vietnam TABMIS came into operation in phases over the period 2010-2011. It is now operating in stable mode. It is a mission critical system and is the backbone of the Government of Vietnams financial operations. The system serves as a common platform to all treasury offices and financial agencies at all four levels of the Government for budget allocation and control, payment and receipt processing, expenditure and revenue accounting and reporting. TABMIS is fully operational in all 1,500 treasuries and financial agencies across all 63 provinces, and 37 spending ministerial organizations, and 3 major departments of Hanoi city since October 2012. The Vietnam TABMIS has been implemented in a centralized architecture with the main servers located at the Treasury headquarters in Hanoi. All treasuries connect to the central servers over a telecommunications network to process their transactions. In addition Budget managers (vote controllers) at the Center i.e.; the MOF and the Line ministries HQ, and the Directors of Finance and Planning at the Provinces at the Districts are also connected to the system. These officials perform the budget allocation function and allocate the budget to the SUs under their jurisdiction and make any in-year changes and can monitor the budget execution process for the SUs under their jurisdiction, e.g. query the budget balances, etc. This has enabled the government to allocate, execute and monitor the state budget through a centralized financial management information system, on a transparent and real-time basis. Improved accuracy and consistency in budget control, payment processing, accounting and reporting is achieved through application of a unified TABMIS Chart of Accounts (COA) and a single database of TABMIS, replacing multiple COAs and multiple treasury legacy systems located throughout the country. Furthermore, data on government receipts and expenditures, cash and fund balances available in TABMIS in real-time and treasury and finance offices are able to produce instant reports for management purposes. Payment processing through TABMIS has helped enforced compliance and enhanced transparency in budget execution throughout all levels of government. The system operates in Treasury Centric mode which means that SUs at the Center and the provinces need to submit their transactions such as payment requests to the appropriate treasury office. Treasuries at these locations subsequently enter the transactions in the system. To illustrate this process: SUs in Hanoi submit these transactions to the Hanoi Treasury; sectoral departments in the provinces and the districts submit them to the relevant provincial and the district treasuries respectively; and the communes (about 10,000) submit their transactions to the district treasuries. A Government wide training program was carried out to train end users in the use of the system/ Around 15,000 end user staff were trained to use the system to perform their day to day work 81 APPENDIX C The TABMIS is a large country wide system. At present about 11,000 users connect to the system country wide3. Of these about 7,500 are Treasury officials and 3,500 are officials of the Finance and Planning Departments at the center, the provinces and the districts. The current transaction volumes range from about 2.2 million transactions per month in Jun2 2015 to around 3.8 million transaction per month in November/ December 2015, the close of the fiscal year. Thus approximately 30 to 40 million transactions are processed by the system in a year. The TABMIS systems deployment architecture is shown in figure. 1. The way TABMIS fits in to the Vietnam’s Public Financial Management Architecture is shown in in figure 2 which depicts TABMIS along with the other systems used in Vietnam to support PFM (not all of which are automated at this time). 3 Compared to about 1,000 users in Malawi, 3,000 in Ghana and 6,000-7,000 in Pakistan. APPENDIX C 82 The Use of a Web-Portal for Systems Access. A spending unit based system deployment is in principle preferable since it is desirable to pick up the transaction at its origination point. However there can be significant cost implications involved in such a deployment. The number of spending units in a country can be quite large, compared to typical Sub Treasury office numbers (In Vietnam the SUs are estimated to be around 300,000, while the Treasury and Finance offices are around 1,500). Connecting such a large number of end users directly to the system requires that end users have an end user license for the application software (typically US$500 to US$1,000 per user annually). The processing requirements on the central servers would also rise correspondingly. These incremental costs could become very high if all 300,000 end users are to be connected directly to the system. 83 APPENDIX C An additional issue is that the Treasury Law in many countries including Vietnam requires that the Treasury be responsible for payment control. Direct access to the system, even if it were to be provided, would therefore need to be restricted to recording the transaction but not clearing it for payment. With the advent of web based technologies, alternative means are now available to give spending units access to the system. Costs may thus be contained while the role of the treasury to exercise control on payments is retained. SUs can now have access to a web- based portal that would enable them to send a transaction to the system. SUs can create transactions in the public services provided by the web Portal to send the transaction information to the State Treasury and then interfaces it to TABMIS on the basis of SUs using digital signatures for authentication and security purposes. The public services in the web Portal do not yet allow look-up budget execution figures and reports. Line ministries access the reports directly in TABMIS. Figure 3 above shows the working of web-portal based access to the system. APPENDIX C 84 The Vietnam Treasury has now started a pilot to provide such access to SUs and other end users. It will enable them to send transactions to the system electronically via a web portal instead of being required to bring them physically to the Treasury offices. A pilot has been completed in 5 provinces with selected SUs in each province being given access. At present the access is limited to sending transactions such a payment requests, but there are plans to expand the functionality to include querying the system data bases and producing reports such as budget execution reports. An Assessment of the Capacity and Effectiveness of the TABMIS as a Budget Management Tool Methodology. The following features determine the effectiveness of a FMIS as a budget management tool: (a) Enabling environment: This includes the status of the TSA and an evaluation of the degree of consolidation of Government Cash Balances in a TSA (Government funds, Internally generated funds (IGF's), Donor funds). (b) FMIS Coverage: This includes an evaluation of the coverage of government financial transactions by the FMIS- comprehensiveness of transaction processing; (c) Core System Functionality: This includes functionality related to: Budget Management; Commitment Management; Payments Management and associated controls; Payroll related payments; Debt Service Payments; Fiscal transfers/ Subsidies; Evaluation of the comprehensiveness and timeliness of tax and non-tax receipts posted in FMIS; The nature of the Banking Interface; Quality of Fiscal reporting; Basis of Accounting; Use of Advanced Budgeting features-Line Item- Program based etc. (d) Non-Core functionality: This includes use of Other Systems Modules and interfaces with other systems, e.g. Budget Preparation; MTEF capability; Establishment control and its integration with Payroll payments; Debt Management; Fixed Assets; Auditing. (e) Some technical aspects: Nature of Information systems support for Budget Execution / Treasury Processes; Systems Architecture; Use of a Data Warehouse/and analytical tools. Some informational items: Name of Application software package (COTS) used with version; Capital Cost to date; Annual Recurrent cost (operating expenses) including License fees (Application Software, middleware); Telecommunications; Technical staff for systems operation and maintenance; Number of end-users connected to the system - Average, maximum. Coverage of TSA. Percentage of the Governments financial resources banked in the Central Bank and scope and comprehensiveness of the Treasury single Account (TSA): (a) The Mission was informed that a TSA4 had been set up at the Central Bank of Vietnam with the major part of Governmental financial resources being banked in it. A move to the TSA started in the third Quarter of 2013. TSA arrangements had been 4 The TSA related activities were not directly financed by the project but the TSA is a critical feature in terms of the capacity of a FMIS as a budget management tool. 85 APPENDIX C rolled out to all State Owned Commercial Banks (SOCB) by Mid-2014. Out of these four SOCBs provide the direct revenue collection and payment services through four accounts that have replaced the approximately 700 previously existing bank accounts. This has resulted in rationalization of banking arrangement for government cash management. (b) The revenue agencies deposit tax and customs receipts in deposit accounts in designated commercial banks which are cleared to the TSA. Non-tax receipts are also deposited in the TSA. (c) Line Ministries do not receive large advances at the start of the year which would be banked outside the TSA. (d) The two large extra budgetary funds (EBFs) are the Social security and the Health insurance fund and the Employment Insurance fund. They constitute about 95% of the total not covered by the TSA. This is normal practice in many countries which do not allow co-mingling of these funds with other government financial resources. Out of the remainder, some EBFs are part of the TSA and some are not – e.g. the disaster relief fund, is not. The total value of the resources outside the TSA is estimated to be small. (e) IGFs - revenues (fees etc.) are deposited in the Central Bank/TSA. The new 2015 budget law encourages devolution of financial and other powers to lower levels. IGFs will be managed by SUs in the future (starting 2017) and managed separately from the TSA. This aspect needs to be monitored as it evolves. It may become necessary to require SUs to open Bank accounts in designated commercial banks. These accounts would operate on a zero balance basis (ZBA) and be cleared to the TSA overnight. The SUs would still be able to execute transactions against these accounts without going through the Treasury and retain autonomy of expenditure. (f) Donor funds are not banked in the Central Bank /TSA. They are banked in accordance with agreements with the respective donors. However total resources only constitute 8% of the total government financial resources (compared to 25-45% in some African countries) so this is not a pressing issue. However, it may be useful to ascertain whether these resources are used by the commercial banks to buy any short or long term instruments floated by the government, and if so to consider other options. On the basis of the above it is estimated that around 85-90% of the total government resources are banked in the TSA or TSA linked arrangements. This level of coverage is considered high. Coverage of TABMIS. The mission collected information needed to establish the comprehensiveness of the coverage of the TABMIS for government financial transactions and to determine which payments payment and receipt transactions are routed through the TABMIS: (a) The TABMIS has been implemented at the Central, the Provincial and the District levels. All SUs at each level are required to send their transactions (such as payment requests) to the respective treasury offices. Budget controllers at each level have APPENDIX C 86 access to the system to allocate budgets to SUs under their jurisdiction and perform the budget monitoring function. It is therefore a country wide system5. Transactions against those EBFs which are part of the TSA are processed through the FMIS. (b) Transactions against IGFs are currently processed through the FMIS; (as noted above this may change in 2017). (c) Transactions against Donor funds are NOT processed through the FMIS. Grant transactions are processed through TABMIS. It is estimated that around 85-90% of the total government resources are processed through the TABMIS. Core Functionality and Controls. The mission tried to establish the quality of the core functionality provided by the system and the controls it incorporates: (a) Basis of accounting: TABMIS currently operates on a Modified Cash basis; however it has the capacity transition to full accrual accounting as required.6 (b) Budget Management: A comprehensive budget classification structure with capacity to also monitor expenditures on projects is in use. Budget execution is on a line item basis. Transactions related to both the capital and the recurrent budgets are processed through TABMIS. (c) A unified COA for TABMIS: This has been developed and applicable throughout all four levels of the government for treasury and budget management. This multi- dimensional COA with 11 segments captures all relevant information required for budget management such as source of fund, location, administrative, economic and functional budget classifications, targeted programs, projects, spending units etc. provides flexibility to analyze, consolidate and report for each segment and for a combination of segments. Relevant segments of the COA are consistent with the international functional and economic classifications of expenditures as provided in GFSM2001/COFOG allow for GFS reporting. (d) COA: COA used for budgeting is the same as that for accounting: This enables integration of budget and accounting data. The same COA is used across the various levels of Government. This enables consolidation across different levels of Government. 5 This contrasts with the case in many countries where the system operates only at the Central or the Central and provincial levels. Also the Treasury law in Vietnam makes it responsible for all budget transactions at each level of government. In other countries, e.g. Russia, the Central Treasury is responsible for execution of the Governments Central Budget only. (Spending units all across the country funded by this budget are required to use the Central Treasury system) Oblasts and Rayons or cities operate independent Treasuries may have their own systems. 6 TABMIS is supporting a move toward accrual accounting as large state budget expenditure commitments are being controlled and monitored through it. In addition, some accrual accounting information such debt obligations are accounted and recorded in TABMIS immediately as they are incurred. 87 APPENDIX C (e) Budget controllers at each level have direct access to the system: At the start of the year the budget controllers at the MOF load the initial budget at a high aggregate level in the system. The Line ministry budget controllers allocate the budgets to SUs under their jurisdiction, those at the provincial and the district levels do so for the SUs in the provinces, the districts and the communes. All in year budget transactions are also entered directly by the budget controllers at each level. Commitment Control: Treasury has implemented commitment thresholds for the Recurrent and the Capital budgets. These are set at US$5,000 and US$25,000 respectively. SUs are required to send commitment transactions for expenditures equal to or exceeding these values to the Treasury within three days of the commitment, e.g. issue of a Purchase Order for goods or services. The Treasury enters this as a commitment in the system. When an invoice is received from the SU and entered in the system by the Treasury, TABMIS checks for prior commitment in addition to budget availability before releasing it for payment. In this way Treasury manages to exercise commitment control over 85% of the budget or so without requiring the SU to raise commitment transactions for every little payment transition which would be counterproductive and clutter up the system with very high transition volumes without a corresponding benefit. (f) It may be noted that TABMIS has not been implemented in a full P2P mode as in Malawi or Ghana: It is a Treasury centric system and doing so would require the SUs to send very stage of the transaction e.g. request for services, POs GRNs Invoices etc. to the Treasury and also require the Treasury to generate the PO through the system which would not be possible in view of the very large volumes of transactions involved. The mode of implementation adopted in Vietnam is the same as that adopted in other countries which operate a Treasury centric system, such as Russia, Pakistan, etc. (g) Payment management: TABMIS handles all types of payments. These include goods and services payments, salaries, debt servicing; fiscal subsidies/ transfers. The system checks for budget availability and commitment (for transactions above the commitment threshold) prior to releasing for payment and therefore exercises Ex-ante control:  For payroll payments a payment request is sent from individual SUs based on a calculated payroll. The Treasury enters it as a payment request in the system. These transactions are subject to ex-ante budget control.  Debt service payments are sent to the Treasury by the Debt Management office; these are routed through TABMIS and subject to ex-ante budget control.  Fiscal Transfers subsidies are routed through TABMIS and subject to ex-ante budget control. (h) Tax and nontax receipts: Tax and customs receipts are deposited in Treasury Single Accounts (TSAs) of the State Treasury. Budget revenue information is exchanged among the State Treasury, Tax and Customs Departments and commercial banks through centralized Tax Collection systems/applications (TCSs), the Tax Collection e-Portal, Customs e-Portal. The State Treasury receives and controls budget revenue information and runs the automated interface between TCS and TABMIS GL. Non- APPENDIX C 88 tax revenues are recorded in the TABMIS GL on the basis of information received from TCS, Finance Departments or Spending units (i) TABMIS is connected with banking systems via the VST payment applications for both bilateral settlement and inter-bank clearance IBPS and then via enterprise service bus connects to banks under the two mechanisms of webservices and MQ. (j) TABMIS has a capacity to provide comprehensive fiscal and financial reporting. It has complete and timely information on all budget receipts and expenditures and the capacity to produce a comprehensive set of fiscal/BERs reports required by the MOF. However, at present the MOF does not use this facility directly and requests Treasury to run the required reports for them as necessary. A capacity exists to down load data from the system to EXCEL and to publish reports using XML publisher. On the basis of the above assessment TABMIS has the core functionality required for effective budget execution and incorporates key controls which are applied in ex- ante mode. Considering its coverage as described above, transactions related to 85-90% of the government financial resources are subjected to controls prior to payment. Ancillary Functional Features. The following assesses some ancillary features related to system functionality, such as use of other systems modules and their interfaces with other systems. (a) Nature of the Budget preparation system: The MOF operates a separate Excel based Budget preparation system. Budget figures from this system are loaded by the MOF and Budget controllers into TABMIS at the start of the year. (b) The MOF is piloting an MTEF in selected pilot ministries: MTEF is not integrated with TABMIS. (c) Human Resource management System (HRMIS) and establishment control: There is no Central HRMIS or establishment control system in place. SUs have position lists available to them and check before forwarding the payroll request to TABMIS for payment. (d) Debt Management- Government uses the Debt Management and Financial Analytical System (DMFAS) for external and Domestic Debt: DMFAS is now accessible by 90 users in all five relevant departments in the MOF. An interface between DMFAS and TABMIS has been developed and is operational. Through this automated interface, receipt from debt issues or borrowings for on-lending purposes or for projects is posted directly into the TABMIS General ledger (GL). Payment for interest or repayment of principal for both domestic and external debts is interfaced into TABMIS Accounts Payable (AP). (e) Fixed Assets management is carried out separately from TABMIS. (f) The External auditor does not have direct access to the system data bases and requests Treasury for reports/data as may be required. Some Technical Features and Costs. TABMIS is based on a commercial off- the-shelf application Software Oracle EBS v 11.5. It is a full function system; currently the modules that have been implemented are modules for (i) Budget Allocation, (ii) Accounts, Payable, (iii) Accounts Receivable, (iv) General Ledger, (v) Purchase order, and (vi) Cash Management. The system has been implemented in a centralized architecture and in a 89 APPENDIX C Treasury centric mode. However, web portal access is being piloted for the System from selected provinces and SUs. A data warehouse is also available but not used extensively at present. The Government is considering migration to version 12 of the ORACLE in the coming years but no firm plans have been made as yet. Number of Users7: Treasury has 10,730 named user licenses. At present the maximum number of concurrent users is around 7,700 and on average this number is 4,000. TABMIS Costs: The initial contract with IBM Singapore for the TABMIS was for US$49.02 million. The BAH contract had a ceiling of $2.69. In addition variation orders were signed for the supply of additional Oracle licenses and additional server capacity. According to the VST8 the cost breakdown is as under: (i) Oracle licenses US$14.51 million; Implementation services US$40.69 million; HW and System Software US$11.62 million; Design and IV&V consultancies US$2.09 million; for a total of US$68.91 million. The cost of the Telecom network which is shared with other Government applications was borne by the MOF and is not included. Training costs are also not included. Therefore according to these figures the total cost of TABMIS is around US$70-75 million9. This is in line with Implementation per user costs for other large budget execution systems projects e.g. Indonesia, Pakistan. The time taken for implementation, 10 years is also comparable. Ongoing Operations and Maintenance. In addition to the investment costs that are incurred in first setting up the IFMIS a provision also needs to be made to cover costs that will be required to be incurred on an on-going basis each year to keep the system operational. The main recurring cost elements that need to be funded are: (a) Hardware Maintenance / Replacement: Maintenance can be estimated to cost between 10–15 percent of the installed hardware costs per annum. Hardware life cycles vary between 3–5 years. (b) Application Software and DBMS/ Tools License Fees: This is a very significant element and may reach between 20–22 percent of the initial license fees per annum (c) Telecommunications Costs: These are associated with use of the wide area network (WAN) for communicating between the remote and central sites. (d) Stationary, Utilities including POL for generators etc. Office Premises. (e) Staff Costs. (f) On Going Training costs: These continue throughout the life of the system as trained end- users rotate out of their positions in most governments after a few years and these positions are filled up with new, untrained staff. Technical skills may also require upgrading as new versions of the application software and hardware are implemented. 7 Response to a diagnostic survey for PEMNA countries distributed at a PEMNA conference in December 2014. 8 As above. 9 According to the PFMRP ICR the increase in the cost of TABMIS due to increase in its capacity to accommodate the substantially higher number of users and transactions resulting from huge jump in public expenditure. Between design and TABMIS full roll out, the number of system users has increased nearly three times and the total budget expenditure processed by the system has gone up by five times. APPENDIX C 90 Vietnam Treasury has allocated some US$3.2 million dollars per year for maintenance. However this is not the complete figure as other departments (e.g. the MOFs central IT Department) are responsible for Hardware and Application Software maintenance costs and for Telecom costs over the country wide network. The mission was informed that an amount of US$5 million per year has been allocated for Telecom costs across the network and another US$5 million has been used for contracting for two years recurrent Oracle license fees maintenance costs. Additional amounts are budgeted for hardware maintenance and support. The major categories of spending that needed to ensure continued satisfactory operations and maintenance of the TABMIS have been budgeted for. Technical Staffing. Once operational a TABMIS type system becomes a mission critical system. Availability of technical expertise is critical for the uninterrupted operation and maintenance of the system and can be a source of significant vulnerability for the continued operational stability of the system. Usually, a central organization is responsible for maintenance of the integrity of the databases and for ensuring smooth and uninterrupted operation of the systems network. End-user technical support is also provided through this organization. In Vietnam Treasury Staff have successfully taken over operation and maintenance activities from the contractor after project completion and are satisfactorily performing this function for the last two years of so. The IT Department is responsible for hardware and WAN support and maintenance. Treasury has 70 certified Oracle specialists on its staff and other specialties required for systems operation and maintenance. Some of these staff are posted in the provinces. They also have access to external consultant assistance as ay be required to supplement in-house facilities. Conclusions. On the basis of the above assessment the mission is of the view that TABMIS is an effective tool for effective budget management for the Government of Vietnam. Its scope covers 85-90 % of the financial resources available to the government. Its geographic coverage extends across the country and across all four levels of Government. The system functionality implemented includes effective ex-ante budget control for government financial resources (and commitment control above specified thresholds). Systems functionality covers core budget execution processes. Some 10,000 Treasury and Finance officials use it for the Governments day to day financial operations. The technology employed is state of the art and adequate numbers of staff are available for its continued operation and maintenance. The Government has made provisions to cover recurring costs related to system operations and maintenance and the system is currently operating in stable mode. Continuing Risks and Vulnerabilities Whist TABMIS is the system of record for Vietnam’s Financial Management Information its use is still restricted to the Treasury staff and the Budget Controllers at the financial agencies. It provides a stable environment for processing and recording transactions related to Government financial operations. However, end users in line ministries and elsewhere do not at present have access to the system and its databases. Therefore they need to maintain 91 APPENDIX C parallel systems in their own agencies to be able to respond to queries regarding the status of their budgets from their management. The Mission visited two line ministries: the Ministry of Agriculture and the Ministry of Health. Both these ministries are maintaining parallel systems. In addition to the fact that the budget related information required by the line ministries is available in the central system, the additional problem with the approach of maintaining parallel systems is that it requires double entry of transaction data and the expenditure data in the satellite systems, which is not reliable since it is not based on actual payments. The budget department at the MOF also stated that to get information on the status of the budget for one or more line ministries they need to call the Treasury who will run the required reports for them. While it is understood that it would not be cost effective to give direct connections to all end users, web-portal based connections which are being piloted in selected provinces and SUs to enable SUs to send payment requests could be enhanced to enable access to budget execution and other reports required by end users. The Treasury assured the mission that this was planned. Also at present a set of 16 reports which included the most commonly required types of information are already accessible to budget controllers in line ministries who already have access to the system. In addition to provision of access to the SU level there is also a training issue. Treasury needs to ensure that staff in Line ministries and financial agencies who are already connected get the required training to enable them to access the system and print reports. It is important that the above issues be tackled promptly so that end users can make full use of the detailed information available in the TABMIS system databases and fuller use is made of this expensive system. While Centralized payment systems are an efficient means to ensure budgetary control and good cash management, they are resisted by line ministries which perceive that the Central Treasury delay payment requests. It is therefore necessary that Treasury implement a system to monitor the time taken for it to approve and make payments for various types of transactions. They need to develop standards on the percentage of payment requests that are approved and paid within a specified time period and maintain data on the time taken to pay all requests so that they can monitor those requests which are taking excessive time. During initial systems implementation in 2011 it was observed that the system response time slowed as the volume of transactions became large (e.g. towards the close of the year). It was found that while performing the budget check at a given provincial Treasury in a given province the system was locking out all other users even in other provinces. IBM and Oracle had to put in a patch to the application software to fix this problem. The Treasury informed the IEG mission that the problem has now been fixed. System response times need to be monitored to ensure that this problem does not surface again. Also as the Treasury moves to the newer version of the software (version 12) this patch will need to be ported to the new version. APPENDIX C 92 The COA used in Vietnam has eleven segments. This is good in so far as it allows transaction data to be analyzed in multiple ways. However, too elaborate a COA can be cumbersome at transaction entry and can also degrade system performance due to the additional load on the servers. The Treasury may examine simplification and rationalization of the CoA in the light of actual reporting requirements. Profile of Expenditure Transactions The mission asked the Treasury staff to run a profile of expenditure transactions. This is tabulated below for one full year of transactions (about 30 million) for the year 2015. It indicates that only 7.8% of the transactions (above VnD 200 million) account for 88% of the budget processed through TABMIS. % of % of total amount in # Range Amount of Transaction in VnD Transactions VnD 1. <200,000,000 92.33 11.90 2. >=200,000,000 and < 1,000,000,000 5.86 12.77 3. >=1,000,000,000 and < 2,000,000,000 0.87 6.36 4. >=2,000,000,000 and < 10,000,000,000 0.81 16.93 5. >=10,000,000,000 and < 20,000.000.000 0.07 4.96 6. >=20,000,000,000 0.06 47.08 Total 100 100 The mission asked the Treasury to also try to determine which of the 300,000 SUs units that they serve are responsible for generating much of these transactions. (This number would be very small compared to the total and could of the order of 5000 SUs). These are the big spenders in the system akin to large tax payers on the revenue side. Identifying these SUs would enable the Treasury to focus control on their transactions above this threshold instead of wasting efforts across the whole network. Also this would make it easier to determining future cash requirements and developing borrowing strategies. 93 Appendix D. Updated Results Matrix of MDTF II Project Development Objective Intermediate Outcomes Strengthened capacity for: increased coordination Increased coordination and transparency across the and transparency in public financial management; national PFM system through a clear reform strategy, improved revenue mobilization; and effective and greater disclosure, improved oversight capacity, more efficient public expenditure management secure IT systems, and compliance with PFM legislation. Improved revenue mobilization through equitable and efficient tax policies; and efficient customs administration. Strengthened expenditure management through higher accounting standards, and improved capacity in financial reporting, cash management, and state asset management Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present 1) Increased 1.1) Completed - MOF has completed the - PEFA report was used coordination and Public Expenditure PEFA report. Having extensively to inform transparency across and Financial approved by the MOF and new Government the national PFM Accountability Government, the report has policy including the system through a (PEFA) Review. been disclosed/ recent issuance of the clear reform disseminated through a revised State Budget strategy, greater workshop held in Nha Law in 2015, which disclosure, Trang (4-5/7/2013); among other things, (i) improved oversight builds a robust fiscal capacity, more - Currently, benchmarks under the PEFA report are policy framework with secure IT systems, the medium-term and compliance informing the formulation of PFM reforms and fiscal framework that with PFM aligns revenue and legislation serving as baseline data for monitoring and evaluation expenditure with the system under Financial development Sector Strategy’s Medium objective; (ii) Term Action Plan 2014- increases fiscal 2016 (MTAP) transparency by disclosing the executive budget proposal to the public when it is submitted to the National Assembly; and (iii) aligns recurrent budget classification with capital budget classification. - PEFA report served as the baseline result framework to monitor the PFM Analytical and Advisory Assistance (AAA) TA provided to the APPENDIX D 94 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present Government during 2015-2020. 1.2) Action Plan for - With the support of - The medium-term implementation of MDTF2, MOF has action plan 2015-2017 FDS (2011-2020) completed developing its was issued by the approved by the Action Plan for MOF Decision Ministry of Finance implementation of 704/QD-BTC dated 17 (MOF) and the FDS2011-20; accordingly, April 2015, which Monitoring and this has been approved by comprised 8 pillars Evaluation the Minister of MOF on and 45 activities to be framework of FDS 30/1/2013 through his implemented during fully developed. Decision 224/QĐ-BTC 2015-2017. 36 of 42 regarding granting activities were approval to the FDS2011- completed in 2015. 20’ Action Plan to One activity was implement the set adjusted and one more objectives and solutions/ activity was added to measures under the the action plan. Some FDS2011-20. results are highlighted - The draft MTAP2014-16 below. and the M&E framework - Pillar on revenue has been sent to relevant mobilization: the agencies, Government has departments/units inside already made progress the financial sector, as well on several tax policy as to the developing and tax administration partners, for their including increasing comments. It is expected tax revenue from that the MTAP will be domestic market and improved/finalized within collection of tax 12/2013 with aim to arrears. concretize and implement - Pillar on the tax and the objectives of the custom administrative MTAP2014-16 in a procedure reform has prioritized and sequenced been progressing very manner and with specific well. The Government project for each recently issued departments/units of MOF resolution 19 to headquarter and in the substantially shorten financial sector as a whole. the tax compliance time. - Pillar on public debt management was being consciously managed by MOF. Although public debt has been under the debt threshold, debt servicing and interest payment are rising, leading to the need to 95 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present restructure the debt portfolio. 1.3) Developed - MDTF2 Project has - GOV issued Decision regulations on provided support for the Number 689/QD-TTg, disclosure of development of Circular dated May 4, 2013, information on public 56/2011/TT-BTC dated approving a medium- and publicly 29/4/2011 providing term debt management guaranteed debt. guidelines on method to program for the period calculate the supervision 2013-2015. criteria and organize for - The Ministry of supervision of public debt Finance has prepared and national foreign debt. an updated medium- - Public Debt Bulletin is term debt management published on bi-annualy program for the period basis to provide additional 2016-20, with information on domestic strengthened risk debt. The DMFAS system, analysis based on which initially included additional scenarios only public debt and and updated government guaranteed macroeconomic debt, has been extended to assumptions incorporate also information on domestic debt. - MDTF2 provided support to organize a workshop to provide input information to develop the PM Decision 958/QD-TTg dated 27/7/2012 regarding granting approval to the Public Debt and National Foreign Debt Strategy 2011-2020, with Vision to 2030, with aims to strengthen the institutional framework on public debt management and to establish objectives to ensure the financial security and the medium- term indicators. The decision approving the long-term public debt management strategy 2010-2020 has, for the first time, consolidated both foreign debt, domestic debt and local authority’s debt, apart from the budget and SOEs’ debt. The Decision includes the approval to APPENDIX D 96 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present the following policies: (i) norms on budget deficit of medium to long term and the borrowing limits; (ii) structure/ composition of the debt toward an increase in domestic debt proportion; (iii) medium to long-term thresholds of public debt and repayment ability. 1.4) Developed - The result of MDTF2’s - The Government has indicators and support has an direct or issued Decree Number monitoring system indirect impact to the 87/2015/ND-CP, dated for oversight of SOE development of the Decree October 6, 2015 which financial 61/2013/ND-CP regarding replaced Decree performance. promulgation of the Number 61/2013/ND- regulations on financial CP, dated June 25, supervision, financial 2013, adopting the performance evaluation audited reports on the and disclosure of financial public dissemination information of the of the key financial enterprises owned by the performance of all State and of the enterprises State Economic with State capital; the Groups Decree 71/2013/ND-CP regarding regulations on investmetn of State capital in enterprises and financial management of enterprises which State hold 100% of charter capital; and the circulars providing implementing guidelines such sas Circular 153, 158, 171 and 173. - The draft criteria/indicators and processes for evaluation of financial situation of SOEs and enterprises with State capital in the designated areas of transportation, industry and financial organization has been developed and is expected to be finalize in 12/2013. - The result and experience drawn from the overseas study tour which is funded by MDTF2 will incorporated in the draft Law on Management and 97 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present Utilization of State Capital Invested in Enterprises, which is expected to submit to the National Assembly in May/2014. 1.5) Completed audit - Completed the review of - Based on the security of IT security security of TABMIS review results, the arrangements in system, in 8/2013. IBM (contractor) has MOF with action made corrective plan for follow up - Completed the evaluation of current status of measures to improve implementation. the security of financial sector information safety and TABMIS. security, in 11/2013 - MOF issued a decision - Completed the 3317/QD-BTC dated development of roadmap to 24/12/2014 on implement and ensure regulations on financial sector financial information information safety and security for public security to 2020, in financial information 11/2011. systems. - It is expected that the proposed contents of the Policy on financial sector information safety and security and Feasibility Study Report for the project on financial sector information safety and security will be completed within 12/2013. The report will serve as a prerequisite for the implementation of the Project on financial sector information safety and security in the years to come. 1.6) Piloted and - Carried out pilot evaluation applied Regulation of the situation on PMF on evaluation of legislation implementation implementation of in the field of (i) State PFM legislation in budget management; (ii) Vietnam. corporate finance; (iii) tax administration; and (iv) social security. - Pilot evaluation reports of international and national consultants in the above- mentioned fields have been completed in 5/2013; 11/2013; 4/2013 and 4/2013, respectively. APPENDIX D 98 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present - During the process of pilot evaluation, MOF has developed: the method, the criteria and the processes to evaluate the legislation implementation in PFM, and then, after the pilot evaluation exercises have been completed, such evaluation method and processes have been improved/finalized, and turned it to the Regulation on evaluation of legislation implementation situation in PFM; - The Regulation has been approved by the minister of MOF through his Decision 2247/QĐ-BTC dated 06/9/2013, which serves as the legal basis for carrying out evaluating the legislation implementation situation in PFM across the country. 1.7) Adopted legal - Completed drafting of - The Government framework for the Decree on Internal Audit adopted the establishment of which was submitted to Accounting law 2015, internal audit N.A in 2011 in seeking for which mandates mechanism in its adoption, however, due establishment of the Vietnam. to the fact that it was Internal Audit function planning to have the in government Constitution and the agencies. The Budget Law revised, so a accounting and audit number of relevant legal policy department of documents, including the MOF is tasked with Decree on Internal Audit, drafting the internal would be issued after the audit regulation adoption of the revised pursuant to the Constitution and Budget Accounting law. The Law. internal audit function will be managed by the supervision department at the line ministries. 2) Improved 2.1) Reviewed With the MDTF2 support, - The revised law on revenue implementation of 11 MOF has completed import, export tax was mobilization tax policy measures implementing 3 proposals through: equitable with on development of the tax issued in April 2016 and efficient tax recommendations laws (under IP1); and that promulgates the policies; and submitted to MOF. reviewed/ evaluated some new contents and See appendix G for 11 types of taxes (under 99 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present efficient customs borrower comments IP2). Accordingly, it has revised tax rates in administration. re natural resource submitted to the MOF, accordance with the tax; environmental Government and N.A for recently signed trade protection tax; promulgation of the agreements. personal income tax; following legislations/legal and import and documents: - Law on fees and export duty tax law. 1. Tax Law on charges was issued in Environmental November 2015 that Protection (2010); promulgates the list of Decree 67/2011/NĐ- fees and charges to be CP providing guidelines to collected as well as the implement the Tax authorities responsible Law on Environmental for setting the price of Protection; Circular user fees and charges. guiding the implementation of - Revised law on special Decree 67/2011/DN- consumption was CP; Decree 69 to issued in November supersede the Decree 2014 that applies 67/2011; Circular 159 revised tax rates to revise the Circular towards selected guiding the special consumption implementation of Decree 67/2011. goods. 2. Tax Law on Utilization of Non- agricultural Land (2010); 3. Law on Amendment and Supplementation of a Number of Articles of the Law on CIT (09/6/2013); 4. Law on Amendment and Supplementation of a Number of Articles of the Law on PIT (23/1/2012); 5. Law on Amendment and Supplementation of a Number of Articles of the Law on VAT (19/6/2013). - Based on assessment of the achievements of the existing tax policy on land, the consolidation of the constraints and obstacles faced during the implementation of each land tax policy, MOF has studied and then proposed APPENDIX D 100 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present some solutions regarding amendment and supplementation of policies and contents which is more appropriate with the practical situation and contributed to the revision and amendment of the Law on Land which has been recently adopted by the NA. - MOF has completed the report on evaluation of the implementation situation of natural resources tax policy, with recommendations on what contents need to be revised or amended toward making the natural resource tax as an effective tool to manage, protect and promote the effective and efficient use of national resources, particularly the non-renewable resources; promoting exploitation of natural resources in association with deep processing and limit export of unprocessed resources; recommendations on revision and amendment of regulations on tax calculation, tax rates and tax collection method for appropriate with practical situation of natural resource exploitation in each period. NA issued its Resolution 928 regarding new tax rates for natural resources, and this is an important result from the review of this tax policy area. - With the technical assistance provided by MDTF2, MOF has carried out review and evaluate the situation of implementation of special consumption tax policy 101 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present area, and recommended what need to be revised or supplemented. It has completed the Report on summary of special consumption tax policy implementation, with recommendation on what to be revised or supplemented. The report recommended a number of new contents such as calculation of tax liabilities on tobacco, etc, and inclusion of new products like electronic games. The report provide usable input for the amendment of the Special Consumption Tax Law. At present the draft Revised Special Consumption Tax Law has been submitted to the MOF leader, and is planned to be submit to the NA for issuance in 2014. - Through the MDTF2’s support, MOF understands the current situation on implementation of the policies on fees and charges at the agencies, units and localities, at the same time it completed the Report on review of fees and charges implementation situation, with recommendation on what need to be revised or amended toward: + Research and consider to include more agencies who should have authority to issue legal documents regarding the collection rates, regimes, payment, management and utilization of fees and charges to ensure the consistency with the specialized legislations and suitable with practical situation. APPENDIX D 102 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present + Research to further delegate the authority to local authority to make decision on fees and charges. + Research to make amendments in a manner that facilitate the agencies and units those who have revenue from fees and charges can take the initiative in using this revenue in order to implement the mechanism of autonomy and socialization. The report has contributed to the issuance of legal documents on the rates of fees and charges (each year, MOF issues some 20- 30 Circulars regarding fees and charges) and will contribute to the development of the Law on Fees and Charges which is planned to submit to the NA for issuance in 2015. - MOF has completed the Report on review and evaluation of the current implementation situation of import/export tax/duties policy from different angles (from the view point of various objects) from which it come up with recommendation on revisions and amendments to optimize the current tax policy of this kind and at the same time to meet the requirements regarding international economic integration. According to the NA’s working agenda, in 2015, MOF will submit the NA for its adoption of the Revised Law on Import/ Export Tax. - Apart from reviewing 11 tax policy areas and issuance of the legal 103 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present documents as above- mention, the Project has also granted approval to review the twelfth tax policy area that is “evaluation of implementation situation of real-estate tax policy.” This is expected to complete within 12/2013. This is considered as an important content that help the MOF in drafting and submitting for issuance of the Law on Real-estate in 2015 in accordance with the working agenda registered with the NA. 2.2) Developed - MOF’s leaders decided not National Single to use funds from MDTF2 Window System for this proposal, but consistent with another source of funds to ASEAN implement this requirements for customs administration. 3) Strengthened 3.1) Developed - Completed developing - The Government expenditure roadmap for Public Roadmap for adopted the management Sector Accounting implementation of Public Accounting Law through higher Standards with Sector Accounting (2015) to develop the accounting finalized gap Standards in Vietnam. consolidated the standards, and analysis. Accordingly, the Government financial improved capacity implementation will be statement in which in financial splited into many phases, public accounting reporting, cash with a number of standards standards will be management, state per each phase, based on applied. asset management the principles of (i) those and debt international standards that management. are with clear contents and suitable with the conditions of Vietnam, requiring little revision or supplementation – will be published first; (ii) those that are more complicated and require more time to study, research, revise or supplement (or even, depending on the budget law, the financial policy of Vietnam) will be published later on. Specifically, it is planned that, in 2013-2015, APPENDIX D 104 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present Vietnam will update some 15 Public Sector Accounting Standards; in 2016-2017, Vietnam will update 8 PSASs; and 2018 it will update another 9 PSASs - MOF has also clearly determined the objects to apply the PSASs, accordingly, the administrative and public service delivery units will apply the accrual based accounting standards while the revenue-creating and spending units will apply the cash based accounting standards. - On that basis, it has clearly determined who will be to prepare the report and aggregate the Government report, the format of Government report, and developed a set of general report for the whole nation to improve the accuracy and integrity of financial reporting as well as aligned with international accounting standards. 3.2) Reviewed the Under the project, it has - The Accounting Law existing corporate reviewed 13/26 corporate 2015 requires MOF to accounting standards accounting standards and is in Vietnam with expected that, by 12/2013, it prescribe accounting recommendations for will submit for the issuance by standards on the basis update. the MOF 8/26 corporate of international accounting standards, namely: standards. The most 1. General standard re the recent Circulars issued framework for development by MOF are Circular and presentation of financial 200/2014/TT-BTC statements 2. Standard re. inventories applicable from 3. Standard re. tangible fixed January 201510, assets Circular 210/2009/TT- 4. Standard re construction BTC and Circular contracts 202/2014/TT-BTC. 5. Standard re borrowing costs These Circulars 6. Standard re revenue; 7. Standard re price difference; provide guidance on 10 This supersedes Decision 15/2006/QD-BTC. 105 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present 8. Standard re investment, joint the Vietnamese ventures and associate Corporate Accounting System and try to bring VAS more closely align with international accounting standards 3.3) Developed - State Treasury has - The State General model and completed draft Project on Accounting function implementation State General Accounting was established roadmap for State (incl. model and roadmap pursuant to Decision Accounting General to implement the SGA 1188/QD-BTC in function. model). This will be 2014. The SGA will submitted for the approval be responsible for of MOF in 12/2013. preparing the consolidated Government financial report in accordance with public accounting standard. 3.4) Developed - Completed drafting the - The Ministry of roadmap for the bond market roadmap in Finance issued development of end of 2012, which Circular 111/2015/TT- Vietnam’s bond focusing on, for the first BTC regulating the markets. hand, the government bond Government bond market, including method issuance in the of primary issuance and domestic market. It secondary transactions, also provides buyback, development of procedure for bond market service buyback. infrastructure, and development of organized investor system. 3.5) Developed - With the support of - Decree 24/2016/ND- modern cash and risk MDTF2, under IP1, in CP on cash management system supporting the management policy toward security and Government’s policy that was issued on April 5, effectiveness with is to manage non-cash 2016. The State improved revenue and expenditure Treasury has coordination between (Circular 164 and Circular implemented TSA cash and debt 33), State Treasury arrangements, management. developed the inter- consolidating cash banking and bilateral balances from more payment processes, making than 700 accounts. it the basis for the ST to Cash balance implement inter-banking consolidation has and bilateral payment, and helped optimize the to develop the safety and use of cash balance, security processes in minimized borrowing bilateral payments requirements and minimized the risk of APPENDIX D 106 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present - Under IP2, thanks to the temporary cash Project support through the shortages. consultants’ reports of high quality and the experience learnt from other countries, ST has completed developing the cash flow forecast system and developing the cash fund control and risk management system. These outputs have contributed to the development of the Circular guiding the cash flow forecast process, as well as the Decree on cash fund management, which includes the content regarding cash fund risk management. The draft of Circular and Decree has been developed, but the issuance of these will be made upon the revised Budget Law has been issued. 3.6) Developed - With the technical - The Ministry of model for asset assistance provided by the Finance is drafting the management system Project, MOF has selected revised State Asset with reviewed an appropriate asset Management Law business processes management model to following the and finalized gap implement asset Constitution 2013. analysis. management in proactive The output of MDTF manner. MOF has provided helpful conducted assessment of background and the information technology analyses to the applications in process revision of the state management of current asset management assets, reviewing data in law. The draft Law is the national database of expected to be State Assets; assess gaps submitted to the and thereby deliver a legal Government in May and recovery process to 2016 and the National consolidate existing Assembly in late 2016. databases. On that basis, It is expected to be the experts helped build passed by the NA by standardized process data the first half of 2017. in the national database; - Conducted overseas study tour and engaged the international consultant to provide international experience in application 107 APPENDIX D Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present of information technology in managing the public assets and recommend appropriate solution for application in Vietnam. The standardized business processes on public asset management have been piloted in 2 ministries and 2 provinces for the first time, and then the additional training has been delivered to another 3 ministries and 32 provinces. - After completed implementing the original approved plan (i.e. development of model, standardized business processes on public asset management, piloting and provision of training for selected ministries and provinces). The Public Asset Management Department requested additional resources to extend the pilot review and standardization of state assets to increase the impact of the project. Accordingly, 10 additional provinces and then 3 further provinces were covered. Thus making a total of 5 ministries and 45 provinces have been trained on standardization of data on state assets, including 2 ministries and 15 provinces of direct piloting. 3.7) Developed the - Evaluated the - The Prime Minister policy on centralized implementation of the pilot has issued Decision procurement of state centralized procurement of 08/2016 dated assets state asset. February 26, 2016 to - Learned experience from institutionalize the the UK study tour on the centralized implementation of procurement centralized procurement. mechanism adopting the framework contract concept that APPENDIX D 108 Intermediate Indicators Results achieved as of Results achieved from Outcomes December 2013 2014-present - Drafted Decision and the MOF learned from circular providing the UK experience implementation guidelines, through the study visit including the procedures to to UK under the carry out centralized MDTF-2. procurement of state assets of MOF 109 Appendix E. Review of PFMRP ISRs ISR Interval ISR Comment # Date Mission Mission ISR SM CD Remarks 1 6/30/2003 5/22/2003 1.3 0 0 Initial PSR. Well detailed. Original TL remains involved. 2 12/19/2003 10/15/2003 2.2 4.9 1 Minimal 0 Well detailed. Original TL remains involved and sectoral experts part of mission. Risks cited but not discussed (to ISR4) 3 5/26/2004 4/16/2004 1.3 6.1 1 Helpful 0 Well detailed. Original TL remains involved. Sign of early slippage in CoA. 4 8/2/2004 7/2/2004 1.0 2.6 0 0 Well detailed. Original TL remains involved. Good intensity of supervision. Sign of early slippage in CoA and TSA. Implicit questions over adequacy of Bank procurement procedures for IT systems. 5 6/15/2005 5/14/2005 1.1 10.5 0 1 Detailed Change of reporting format approved from PSR to ISR. Many of no the required fields left empty comment and failure to provide basic -- e.g.; contact, team composition, disbursements and covenants -- or critical information -- notably issues and pending actions. PDO indicators entered as per PAD but not intermediate ones -- which had been formulated in PAD. Disbursement lag of 9 months ignored. CD comments provide specific feedback on ISR quality and realism. 6 12/28/2005 5/14/2005 7.6 0.0 0 1 Detailed No supervision for close to 8 months. Status write-up improved from ISR 5 but other issues remain. Disbursement lag has risen to 15 months which contradicts APPENDIX E 110 S IP rating. Earlier CD comments not taken into account and additional ones provided. 7 7/15/2006 3/15/2006 4.1 10.2 0 1 Detailed Less detailed status report approved with none of the problems no identified earlier addressed. comment All IP and other ratings S despite 18 months disbursement lag. 8 3/29/2007 12/15/2006 3.5 9.2 0 1 Detailed Slightly more detailed status approved write-up following MTR. no None of the reporting comment problems identified earlier addressed. All IP (except component 3) and other ratings S despite 26 months disbursement lag. Notable delays include CoA (initial completion date Fall 2004). Proposed project restructuring limited to extension and additional financing, even though other areas that continue to be pointed out by CD (notably overhaul of indicators). CD comments quite comprehensive. 9 2/20/2008 11/15/2007 3.2 11.2 0 1 Detailed Complete write-up of key approved issues, including a handful no that should have been comment identified at the MTR. Other shortcoming in previous ISRs still present, including unrealistic IP ratings. Required extension still not undertaken -- due to apparent push back Country Management level. Original disbursement lag of 34 months and revised of 4 months after less than a year highlighted as a "great source of concern" in CD comments which are extensive (2 111 APPENDIX E pages). Request for immediate reissue of ISR. 10 4/23/2008 3/14/2008 1.3 4.0 0 1 ISR reissued less than 2 approved months after this was no requested by CD. Adequate comment write-up of key issues and actions -- including continued problems with CoA. Other shortcoming in previous ISRs still present, including unrealistic IP ratings. Required extension of closing date still not undertaken. - due to apparent push back Country Management level. Original disbursement lag of 37 months and revised of 7 months suggests the proposed extension of less than 2 years will be insufficient. I. 11 12/17/2008 10/29/2008 1.6 7.6 1 1 Adequate write-up of key issues -- including problem with UNCTAD contract under component 3. Other shortcoming in previous ISRs still present, including unrealistic IP ratings. Required extension of closing date still not undertaken -- due to apparent push back Country Management level. Original disbursement lag of 42 months and revised of 10 months suggests the proposed extension of less than 2 years will be insufficient. IP rating now MS (performance would justify MU). CD comments summarizes previous -- still no CoA. APPENDIX E 112 12 1/30/2010 5/15/2009 8.7 6.6 0 1 Detailed Adequate write-up of key approved but late issues by component but not no by objectives. TABMIS comment implementation problems well documented. Extension of 2 years approved, but insufficient given 50 months original disbursement lag. IP rating remains MS. CD comments on target, even if sent after 20 days, and question close relevance om an ISR issued 9 months after the mission. 13 10/13/2010 6/15/2010 4.0 3.0 1 0 NB: No ISR prepared after March 2010 mission and ISR based on June mission under new format including ORAF. Adequate write-up of key issues and actions. Indicators not revamped and some information not provided. Further extension envisaged in conjunction with additional financing. ISR not submitted to CD. 14 1/4/2011 10/15/2010 2.7 4.1 1 0 Adequate write-up of key issues and actions. Previous issues still present. Further extension envisaged in conjunction with additional financing. Narrow supervision team with no sectoral experts (expect for second supervision). ISR not submitted to CD. 15 4/1/2012 5/31/2011 10.2 7.6 1 1 Risk rating increased to substantial without explanation -- medium in previous ISR. M&E rated substantial in all ISRs despite evident issues repeatedly noted in CD comments. Additional financing approved and closing date extended by a total of four years. Team also supervised mission on March 2010. ISR issued late. 113 APPENDIX E 16 2/23/2013 11/26/2012 3.0 18.2 1 1 Evidence of SM comments being provided on draft ISR. Indicators revised through additional financing. Risk appropriately reduced to moderate. 17 10/16/2013 11/26/2012 10.8 0.0 0 1 Government request to approved amend indicators noted and no CD requests this be reflected comment in Final ISR -- there was none filed prior to closing. No record of formal supervision after November 2012. 114 Appendix F. List of Persons Met Government Officials Dept. of Public Assets Management: 1. Mr NGUYEN Tan Thinh, Deputy General Director 2. Ms PHAM Thi Tuyet, Deputy Head of Infrastructural Assets Division 3. Ms NGUYEN Thi Phuong Hao, Deputy Head of National Center of Public Assets Database 4. Ms HOANG Thu Quynh, Deputy Head of Administrative Division of the Dept. General Dept of Taxation (GDT): 1. Mr DANG Tuan Hiep, Deputy Director of International Taxation Department under GDT 2. Ms NGUYEN Thi Huyen My from Transfer Pricing Audit Division 3. Ms NGUYEN Minh Huong from Policy Dept under GDT. Tax Policy Department: 1. Ms Tran Thi Phuong Nhung - Deputy Division Head (I believe she was the lady who did most of the talking) 2. Tran Thi Nguyet Tu - Officer 3. Dao Thi Huyen Trang - Officer 4.Tran Thi Ngoc Linh - Officer 5. La Thi Noi – Officer Vietnam State Treasury: 1. Ms. DANG Thi Thuy, Deputy General Director of Vietnam State Treasury, former project director 2. Ms. VU Thanh Huyen, Director of International Cooperation Department, former TABMIS project component manager 3. Mr. BUI The Phuong, Director of Information Technology Department 4. Mr. DUONG Van Tuan, Deputy Head of IT support Division – IT Department 5. Mr. DANG Viet Cuong, official of IT Department 6. Ms. NGUYEN Thi Hoai, Deputy Director of State Accounting Department 7. Mr. LUU Hoang, Director of Legal & General Affairs Department 8. Ms. TRAN Thi Hue, Director of State Cash Management Department 9. Ms. NGUYEN Thi Ngoc Hieu, Head of Capital Mobilization Division – State Cash Management Department 10. Ms. BUI Thi Bao Quynh, Head of Accounting Department, Treasury Transaction Centre 11. Ms. NGUYEN Thu Trang, official of International Cooperation Department 12. Ms. DO Thi Huyen Trang, official of International Cooperation Department Ministry of Agriculture and Rural Development (MART): 1. Mrs PHAM Thanh Huyen, Deputy Director of finance dept. (MART) 2. Mr PHAM Van Hiep, staff of finance dept. (MART) 3. Mr NGUYEN Tung Lam, staff of finance dept. (MART) 4. VU Thanh Liem, staff of planning dept. (MART) 115 APPENDIX F 5. Ms VU Thanh Huyen, Tamis project component manager, Director of international cooperation, Vietnam State Treasury 6. Mr DUONG Tuan, deputy head of division, IT dept., Vietnam State Treasury. Ministry of Health: 1. Ms PHAM Thi Minh Nga, deputy head of finance career division, finance and planning dept. 2. Ms VO Minh Hai, deputy head of aid division, finance and planning dept. 3. Mr TA Quang Thien, staff of capital expenditure division, finance and planning dept. 4. Mr NGUYEN Viet Hong, staff of finance career division, finance and planning dept. 5. Ms NGUYEN Thi Bich, head of finance and accounting division of Administrative Dept. 6. Ms VU Thanh Huyen, Tamis project component manager, Director of international cooperation, Vietnam State Treasury 7. Mr DUONG Tuan, deputy head of division, IT dept., Vietnam State Treasury. Legal Department MOF: 1. Ms HO Thi Hang, Deputy Director of legal dept. 2. Ms NGUYEN Ban Mai, staff of legal dept. State Budget Dept.: 1. Mrs TRAN Thi Kim Hien, head of statistics division 2. Ms NGUYEN Minh Tam, staff of statistics divisions 3. Mr DUONG Tien Dung, head of budget management 4. Mr NGUYEN Tri Phuong, deputy head of budget estimates division PFMRP and MDTFII PMU 1. Ms. DANG Thi Thuy, Deputy General Director of Vietnam State Treasury, former project director 2. Mr. HOANG Trung Luong, Deputy Director of State Accounting Department - VST, former project deputy director 3. Mr. NGUYEN Ngoc Duc, official of Inspection Department - VST, former project chief accountant. 4. Mr. Nguyen Ba Toan Development Partners Ms. Lien DFID Ms. Chau CIDA Ms. Nguyen Hong Giang SECO World Bank Staff Ms. Victoria Kwakwa Mr. Minh Van Nguyen Ms. Tran Thi Lan Huong Ms. Quyen Hoang Vu Mr. Alwaleed Fareed Alatabani APPENDIX G 116 Appendix G. Borrower Comments 117 APPENDIX G APPENDIX G 118 Borrowers Comments Attachment SUMMARY OF FEEDBACK FROM THE MINISTRY OF FINANCE ON THE DRAFT PROJECT PERFORMANCE ASSESSMENT REPORT ON THE PUBLIC FINANCIAL MANAGEMENT REFORM PROJECT AND THE MULTI- DONOR TRUST FUND PHASE II (MDTF II) I. MDTF-2 1. Experience and lessons learnt: In the period of preparation for project completion, the Project worked with WB experts to conduct studies and prepared a project completion report, which recommended the PFM reforms to be followed up after the completion of the Project. The draft report specifies this as an encouraging feature. This is practically the starting point for the discussion on the preparation of the AAA Program that the Ministry of Finance is implementing. Therefore, the Ministry of Finance would like to include this into the part on "experience and lesson learnt" - an important outcome of the mission in addition to the assessment of the performance and efforts. II. PFMRP: 1. Refer to Table 2.2 (Project cost planned and actual, by component and financing course): The data in the "planned" column only reflects the original budget of the project, and is not updated with the additional financing, exchange rate gain/loss and increasing counterpart funding of the Government. Therefore, it is suggested to update the report with data and disbursement rates for financing sources as follows: Cost % By component: Planned Actual Planned Strengthening Treasury and Budget Management 83.63 77.94 93 Strengthening State Budget and Investment Planning 3.58 3.13 88 Strengthening Management of Public Debt and Monitoring 3.28 2.43 74 of SOE Fiscal Risks Project Management Support 5.63 4.22 76 Other (recurrent operating costs and contingencies) 0.00 NA Total 96.13 87.79 91 By financing source: 119 APPENDIX G Borrower 10.14 8.17 81 DFID 11.32 10.5 92 IDA 74.68 69.15 93 Total 96.13 87.79 91 2. Refer to Table 2.4 (Principle changes and dates): Please update the date of DFID grant closing on June 30, 2011. 3. Refer to Para. 2.19: The statement “Project management was also strengthened after the MTR, through the appointment of a senior MOF official as a full time coordinator..." is not accurate because right at the beginning of the Project, the Ministry of Finance already appointed a departmental director to act as the full time PMU Director in compliance with the Agreement. 4.Refer to Para. 3.7: The statement in this paragraph “The scope of the MTFF was a bit narrower than in the original indicator" is not accurate. The piloting of the Medium-term Fiscal Framework and the Medium- term Expenditure Framework from 2005 - 2007 was launched in 04 sub-national governments including Hanoi city, Ha Tay, Binh Duong and Vinh Long. In 2008, Ha Tay province was merged into Hanoi city, then the pilot was in 3 provinces. Please include this information in the draft report. 5. Refer to Para. 3.9: The statement “The revised State budget Law promulgates the preparation of the five-year medium-term budgetary framework" is not accurate and complete. Please revise it into "The revised State Budget Law promulgates the preparation of the five-year financial plan and the 3- year financial-budgetary plan”. 6. Refer to Para. 6.3: Point (i) states that: “Lack of on-going support for cross-departmental working, particularly between MOF, State Bank of Vietnam and MPI. As noted in the previous section, this risk has not been eliminated." In practice, the risk is being reduced through the support of the Ministry of Finance (the State Treasury) for the use of TABMIS by sector ministries, including: training, user training; central help desk to support via email, hotlines; regular direct phone calls to understand obstacles and to provide on-site supports (for difficult issues) or off-site support (for simple issues). 7. Refer to Para. 6.5: APPENDIX G 120 Point (iii) is not accurate about the process of budget revenue collection as follows: "Tax and customs receipts are deposited in bank accounts controlled by Customs and Tax Departments and deposited twice a day in TSA (Treasury Single Account). The Treasury receives this information via an interface with the Central Tax System, which enters it directly into the General Ledger module. Non-tax receipts are recorded in the account receivable module by the Treasury on the basis of information received from the Spending units.” Please revise the above as follows: "Tax and customs receipts are deposited in Treasury Single Accounts (TSAs) of the State Treasury. Budget revenue information is exchanged among the State Treasury, Tax and Customs Departments and commercial banks through centralized Tax Collection systems/applications (TCSs), the Tax Collection e-Portal, Customs e-Portal. The State Treasury receives and controls budget revenue information and runs the automated interface between TCS and TABMIS GL. Non-tax revenues are recorded in the TABMIS GL on the basis of information received from TCS, Finance Departments or Spending units” 8. Refer to Para. 6.5: The statement in point iv that “TABMIS is connected to the Banking system by an automated interface (SWIFT)." is not accurate and complete. Please revise it into "TABMIS is connected with banking systems via the VST payment applications for both bilateral settlement and inter- bank clearance IBPS and then via enterprise service bus connects to banks under the two mechanisms of webservices and MQ” 9. Refer to Para. 8.7: The statement “the main issue that the PMU had to deal with was related to the non-performance of the IV&V consultant. Following the change in team members, performance was no longer deemed satisfactory... The Government thus relied on the advice of the TABMIS provider, which lacked independence." is not accurate. In practice, upon the terminatio of the IV&V contract, the Government continued to use several individual and firm consultants to support in the assessment of System functionality and in TABMIS contract management. 10. The list of persons met (page 116) has many names repeated, please revise and revise as in Annex 2. 11. Specific comments on Appendix C of the draft report is provided in Annex 3. 121 APPENDIX G Annex 2 LIST OF VST AND PMU OFFICIALS MET WITH THE IEG ASSESSMENT MISSION 1. I/ Vietnam State Treasury 2. 1. Ms. DANG Thi Thuy, Deputy General Director of Vietnam State Treasury, former project director 3. 2. Ms. VU Thanh Huyen, Director of International Cooperation Department, former TABMIS project component manager 4. 3. Mr. BUI The Phuong, Director of Information Technology Department 5. 4. Mr. DUONG Van Tuan, Deputy Head of IT support Division – IT Department 6. 5. Mr. DANG Viet Cuong, official of IT Department 7. 6. Ms. NGUYEN Thi Hoai, Deputy Director of State Accounting Department 8. 7. Mr. LUU Hoang, Director of Legal & General Affairs Department 9. 8. Ms. TRAN Thi Hue, Director of State Cash Management Department 10. 9. Ms. NGUYEN Thi Ngoc Hieu, Head of Capital Mobilization Division – State Cash Management Department 11. 10. Ms. BUI Thi Bao Quynh, Head of Accounting Department, Treasury Transaction Centre 12. 11. Ms. NGUYEN Thu Trang, official of International Cooperation Department 13. 12. Ms. DO Thi Huyen Trang, official of International Cooperation Department 14. II/ PMU OF PFMRP 15. 1. Ms. DANG Thi Thuy, Deputy General Director of Vietnam State Treasury, former project director 16. 2. Mr. HOANG Trung Luong, Deputy Director of State Accounting Department - VST, former project deputy director 3. Mr. NGUYEN Ngoc Duc, official of Inspection Department - VST, former project chief accountant. APPENDIX G 122 Annex 3 Comments to the draft Appendix C. TABMIS As in the draft report VST comments Appendix C. TABMIS (page 85) With the advent of web based Please revise this paragraph as SU can technologies, alternative means are now create transactions in the public services available to give spending units access to provided by the web Portal to send the the system. Costs may thus be contained transaction information to the State while the role of the treasury to exercise Treasury and then interfaces it to TABMIS control on payments is retained. SUs can on the basis of SUs using digital signatures now have access to a web-based portal that for authentication and security purposes. would enable them to send a transaction to The State Treasury already demonstrated the system. However, this transaction this functionality to the Assessment would not update the system data bases Mission in March 2016. and would only create a file that would subsequently be used by the Treasury office to review the transaction and update and process it for payment. Appendix C. TABMIS (page 85) Web portals can also be set up to enable Please revise this paragraph because the public budget administrators to have direct access services in the web Portal do not allow look-up to the system to carry out their budget budget execution figures and reports. Line management responsibilities and for line ministries access the reports directly in ministries to get access to budget execution TABMIS. figures and reports. Figure 3 above shows the working of web-portal based access to the system. Appendix C. TABMIS (page 86) The Vietnam Treasury has now started a Please revise this paragraph because now the pilot to provide such access to SUs and State Treasury already completed the pilot of other end users. It will enable them to send public services at the end of Quarter I, 2016 in transactions to the system electronically 5 cities and a selected number of SUs. In via a web portal instead of being required addition to allowing SUs to send payment to bring them physically to the Treasury requests, the web Portal will develop on-line offices. Currently the pilot is being tested balance look-up. The budget execution reports in 5 provinces with selected SUs in each can be accessed on TABMIS. province being given access. At present the access is limited to sending transactions such a payment requests, but there are 123 APPENDIX G plans to expand the functionality to include querying the system data bases and producing reports such as budget execution reports. Appendix C. TABMIS (page 84) The Use of a Web-Portal for Systems Please revise the number of SUEs in TABMIS Access. A spending unit based system to be about 300,000 units. deployment is in principle preferable since it is desirable to pick up the transaction at its origination point. However there can be significant cost implications involved in such a deployment. The number of spending units in a country can be quite large, compared to typical Sub Treasury office numbers (In Vietnam the SUs are estimated to be around 100,000, while the Treasury and Finance offices are around 1,500). Appendix C. TABMIS (page 83 and page 91) Please revise it into about 30 million The TABMIS is a large country wide transactions to be consistent between two system. At present about 11,000 users sections. The text on 3.8 million transactions connect to the system country wide68. Of can be revised into only in December 2015. these about 7,500 are Treasury officials and 3,500 are officials of the Finance and Planning Departments at the center, the provinces and the districts. The current transaction volumes range from about 2.2 million transactions per month in Jun 2015 to around 3.8 million transaction per month in December 2015, the close of the fiscal year. Thus approximately 30 million transactions are processed by the system in a year. The mission asked the Treasury staff to run a profile of expenditure transactions. This is tabulated below for one full year of transactions (about 30 million) for the year 2015. It indicates that only 7.8% of the transactions (above VnD 200 million) APPENDIX G 124 account for 88% of the budget processed through TABMIS. Appendix C. TABMIS (page 91) Number of Users72: Treasury has 11,500 Please revise it into 10.730. named user licenses. At present the maximum number of concurrent users is around 7,700 and on average this number is 4,000.