MONGOLIA Governance Assistance Project Report No. 145083 JANUARY 29, 2020 © 2020 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2020. Mongolia—Governance accuracy of the data included in this work. The Any queries on rights and licenses, including Assistance Project. Independent Evaluation boundaries, colors, denominations, and other subsidiary rights, should be addressed to Group, Project Performance Assessment Report information shown on any map in this work do World Bank Publications, The World Bank 145083. Washington, DC: World Bank. not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 145083 PROJECT PERFORMANCE ASSESSMENT REPORT MONGOLIA GOVERNANCE ASSISTANCE PROJECT (IDA H2220) January 29, 2020 Human Development and Economic Management Independent Evaluation Group Currency Equivalents (Fiscal Year) Currency Unit = Mongolian tughrik (MNT) $1 = MNT 2,743 (January 8, 2020) SDR 1 = $1.38 Government: January 1–December 31 Abbreviations CMCS Computerized Mining Cadastre System ECBTAP Economic Capacity Building Technical Assistance Project GAP Governance Assistance Project GDP gross domestic product GDT General Department of Taxation GFMIS Government Financial Management Information System HRMIS Human Resources Management Information System IAAC Independent Authority against Corruption IAD income and asset declaration ICR Implementation Completion and Results Report IDA International Development Association IEG Independent Evaluation Group IMF International Monetary Fund ISR Implementation Status and Results Report IT information technology MSISTAP Mining Sector Institutional Strengthening Technical Assistance Project NSO National Statistical Office PAD project appraisal document PCU Project Coordination Unit PDO project development objective PEFA Public Expenditure and Financial Accountability PFM public financial management PMMIS Procurement Management and Monitoring Information System PPAR Project Performance Assessment Report SDR special drawing rights TAIS Tax Administration Information System Director-General, Independent Evaluation Ms. Alison Evans Director, Human Development and Economic Management Mr. Oscar Calvo-Gonzalez Manager, Country Programs and Economic Management Mr. Jeffrey Chelsky Task Manager Mr. Konstantin Atanesyan ii Contents Preface .............................................................................................................................................................. viii Summary ............................................................................................................................................................. ix 1. Country and Project Background...........................................................................................................1 2. Mongolia Governance Assistance Project ..........................................................................................4 Objectives .......................................................................................................................................................4 Project Components and Costs...............................................................................................................4 Related Operations......................................................................................................................................6 Implementation ............................................................................................................................................ 7 3. Relevance of the Objectives and Design.......................................................................................... 11 Relevance of the Objectives ....................................................................................................................11 Relevance of Design...................................................................................................................................11 4. Achievement of Objectives ................................................................................................................... 13 Objective 1: Improving the Efficiency and Effectiveness of Governance Processes in the Management of Public Finances ........................................................................................................... 13 Debt Management .................................................................................................................................................. 13 Budget Management .............................................................................................................................................. 14 Public Investment Management ......................................................................................................................... 15 Civil Service Expenditure Management ............................................................................................................ 16 Objective 2: Promoting Transparency and Accountability in the Performance of Public Sector Functions ......................................................................................................................................... 17 Accountability in Civil Service and Anticorruption Framework ................................................................ 17 Budget Transparency and Disclosure............................................................................................................... 20 Public Access and Reliability of Poverty Statistics ......................................................................................... 21 Strengthening Audits and Accounting in the Public Sector ..................................................................... 22 Objective 3: Fostering the Investment Climate in Mongolia ...................................................... 23 Tax Administration .................................................................................................................................................. 23 Public Procurement ................................................................................................................................................ 25 Administration of Mining Licenses .................................................................................................................... 26 5. Efficiency ...................................................................................................................................................... 28 6. Ratings .......................................................................................................................................................... 30 iii Outcome ...................................................................................................................................................... 30 Risk to Development Outcome ............................................................................................................. 31 Bank Performance .................................................................................................................................... 33 Quality at Entry ........................................................................................................................................................ 33 Quality of Supervision ........................................................................................................................................... 33 Borrower Performance ............................................................................................................................ 35 Government Performance.................................................................................................................................... 35 Implementing Agency Performance................................................................................................................. 36 Monitoring and Evaluation .................................................................................................................... 36 Design ......................................................................................................................................................................... 36 Implementation ....................................................................................................................................................... 37 Use................................................................................................................................................................................ 38 7. Lessons.......................................................................................................................................................... 38 Bibliography..................................................................................................................................................... 43 Figure Figure 2.1. Governance Assistance Project Structure .............................................................................5 Tables Table 1.1. Selected Governance Indicators.................................................................................................3 Table 1.2. Selected Country Policy and Institutional Assessment Ratings, 2005–17.....................3 Table 2.1. Major World Bank Technical Assistance Investment Lending Projects, 1998–2015 ..... 7 Table 4.1. Perceptions and Incidence of Corruption Measured by Household Surveys .......... 19 Table 4.2. Perceptions of Corruption Measured by Private Business Surveys............................. 19 Table 4.3. Registered Taxpayers, 2005–15 .............................................................................................. 24 Appendixes Appendix A. Basic Data Sheet ................................................................................................................... 45 Appendix B. Governance Assistance Project Implementation Status and Results Reports .. 48 Appendix C. Prior Actions Related to Technical Assistance........................................................... 49 Appendix D. Detailed Project Data ......................................................................................................... 51 Appendix E. Relevant Public Expenditure and Financial Accountability Findings, 2015 ..... 54 Appendix F. Results Framework ............................................................................................................... 56 iv Appendix G. Mongolia’s Macroeconomic and Fiscal Performance After the Global Financial Crisis................................................................................................................................................. 62 Appendix H. List of Persons Met.............................................................................................................. 67 This report was prepared by Konstantin Atanesyan (senior evaluation officer) and Lev Freinkman (consultant). Armen Sahakyan (consultant) provided research assistance. The report was peer reviewed by Ismail Arslan (senior evaluation officer) and panel reviewed by Soniya Carvalho (lead evaluation officer). Patricia Acevedo provided administrative support. v Principal Ratings Indicator ICR ICR Review PPAR Outcome Satisfactory Moderately satisfactory Satisfactory Risk to development Modest Significant Significant outcome Bank performance Moderately satisfactory Moderately satisfactory Moderately satisfactory Borrower performance Moderately satisfactory Moderately satisfactory Moderately satisfactory Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of the ICR. PPAR = Project Performance Assessment Report. Key Staff Responsible Management Appraisal Completion Project Team Leader Saiyed Shabih Ali Mohib Zahid Hasnain Sector Manager or Practice Manager Barbara Nunberg Robert R. Taliercio Sector Director or Senior Global Homi Kharas Mario Marcel Practice Director Country Director David R. Dollar Bert Hofman vi IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About This Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which executive directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrower’s comments are attached to the document sent to the World Bank’s Board of Executive Directors. After an assessment report is sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing toward the achievement of development outcomes). The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation and complied with covenants and agreements toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vii Preface This Project Performance Assessment Report (PPAR) evaluates the Governance Assistance Project (GAP) of the International Development Association (IDA) in Mongolia (P170780). The project was approved in March 2006 in the amount of special drawing rights 9.7 million (equivalent to US$14 million), funded by an IDA grant. Following two extensions, it closed on December 31, 2014. This assessment aims to review whether and how the operation achieved its intended objectives. The PPAR also examines the long-term sustainability of GAP support, such as the extent to which the GAP’s main achievements have been sustained more than four years since the project’s closure. This report provides additional evidence and analysis of relevant data for a more complete picture of the project outcomes and the factors that influenced them. By reviewing developments from 2014 to 2019 (after the project closed), it offers an opportunity for a longer-term perspective on the factors affecting outcomes. This report presents findings based on a review of project appraisal documents, the Implementation Completion and Results Report, Implementation Completion and Results Report Review, aide-mémoire, International Monetary Fund and World Bank reports, and other relevant materials, including several publicly available studies by various donors. The report was prepared by Konstantin Atanesyan (task team leader and senior evaluation officer) and Lev Freinkman (consultant) under the supervision of Jeff Chelsky (manager). Armen Sahakyan (consultant) provided research assistance. Patricia Acevedo provided administrative support. An Independent Evaluation Group mission visited Mongolia in June 2019 to interview government officials, World Bank staff, and other development partners and stakeholders (see appendix H for a list of persons interviewed). The Independent Evaluation Group is grateful to the numerous representatives of the government of Mongolia, private sector entities, and nongovernmental organizations who provided valuable insights into this evaluation. The team is also thankful to the World Bank Group management and country team members, including both previous and current staff working on the Mongolia program, who provided valuable time, information, and feedback. Copies of the draft PPAR were sent to government officials and implementing agencies for their review. No comments were received. viii Summary This Project Performance Assessment Report (PPAR) evaluates the Governance Assistance Project (GAP, 2006–14, $14 million) of the World Bank in Mongolia. The GAP’s project development objectives were to (i) improve the efficiency and effectiveness of governance processes in the management of public finances, (ii) promote transparency and accountability in the performance of public sector functions, and (iii) foster the investment climate. The project was designed around five principal intervention areas: debt management, budget execution, tax administration, public procurement, and mineral resource management. The PPAR focused on the following evaluation questions: (i) How and to what extent did the GAP fit into the broader World Bank strategy to support governance reforms in Mongolia? (ii) How effective and efficient was the project in achieving results? and (iii) How sustainable are the achievements of the GAP? This PPAR concludes that sustained technical assistance provided to the government of Mongolia through several International Development Association projects and grants, including the GAP, contributed to building capacity on core government functions. The ongoing problems in macroeconomic management relate more to the lack of consensus on sustainable policies and associated populist pressures than to capacity constraints or managerial competences in the Ministry of Finance and the Central Bank of Mongolia. The overall outcome of the GAP is rated satisfactory. These key project outputs contributed to important development outcomes in terms of improved investment climate, reduced corruption, and increased government accountability. Although progress under two of the project’s subcomponents (public investment management and civil service expenditure) was limited, these subcomponents were small. In the assessment of the overall project’s outcome, the Independent Evaluation Group has given greater weight to the key components and achievements. Relevance of objectives was high and relevance of design was substantial. The GAP’s objectives were linked directly to the priorities of the government of Mongolia’s development strategy, which called for “promoting good governance.” The project’s objectives were in line with World Bank Group strategies for Mongolia for fiscal year (FY)05–08 and FY13–17, which considered support for improvements in public sector effectiveness among their priorities. The 2018 Mongolia Systematic Country Diagnostic reconfirmed the relevance of the GAP agenda. ix The project’s design reflected lessons from earlier technical assistance projects, which emphasized the importance of streamlined design and client ownership. The project’s theory of change was sound. It pointed to a reasonable causal chain among project inputs across the main government agencies, which would result in improved capabilities of these agencies and better governance, transparency, and accountability. The project-funded investments and technical assistance activities were relevant to achieving the project development objectives. Achievement of all three main objectives was substantial. The GAP achieved 15 of 16 specific targets in the project’s results framework. Most of the systems introduced under the GAP have been in continuous operation and appear to be sustainable. Moreover, steady progress was made in recent years through government of Mongolia–funded upgrades of the core e-governance systems, initially supported by the GAP, such as the Government Financial Management Information System (GFMIS) and Debt Management and Financial Analysis System. These key project outputs contributed to important development benefits (outcomes) in terms of improved investment climate, reduced corruption, and increased government accountability. This is reflected in improved perceptions of corruption in tax administration and public procurement, expanded taxpayer registration, better public access to budget information, and increased competition in procurement. Efficiency was substantial. Core complex e-governance systems supported by the GAP were delivered mostly on schedule and within the allocated budget. Quantitative analysis of the investments under the project into the GFMIS extension suggests a high rate of return. Benefits from the implementation of both the Tax Administration Information System and the Computerized Mining Cadastre System have been significant and exceeded expectations. Risk to development outcomes is considered modest. The government of Mongolia retained required technical skills, managed to provide necessary maintenance and training budgets, and funded several system upgrades. Despite high staff turnover, institutional memory of the project was preserved in many places. However, there are still two major risks: (i) lack of fiscal discipline and a high level of public debt are a macroeconomic risk; and (ii) volatile political environment adversely affects building a broad political consensus. Overall Bank performance is rated moderately satisfactory (satisfactory at entry and moderately satisfactory during implementation). The project design was built on solid analytical work, and lessons learned from previous technical assistance projects in x Mongolia and other countries were taken into account. The World Bank identified relevant risk factors and proposed broadly adequate mitigation efforts. The quality and volume of implementation support were appreciated by government counterparts. The task team succeeded in maintaining a productive engagement with the client despite personnel changes. However, weaknesses in the results framework could have been addressed much earlier. Overall borrower performance is rated moderately satisfactory (moderately satisfactory for the government and satisfactory for the implementation agency). The government of Mongolia’s ownership of reform remained uneven because the commodity boom weakened incentives to pursue governance reforms. However, the government of Mongolia has shown a strong commitment to the sustainability of two key components of the project, the GFMIS and the Tax Administration Information System, by providing significant state budget financing (approximately $7.5 million) for a quick and efficient countrywide rollout of these systems after the initial GAP-funded pilots. The implementing agencies demonstrated their capacity to operate the new systems in a sustainable way after project closure. Moreover, the government of Mongolia implemented additional reforms and system upgrades that helped enhance the sustainability of GAP outcomes and expand project benefits. The Project Coordination Unit proved capable of providing the implementing agencies with adequate day-to-day implementation support. This PPAR offers the following lessons: • In a low-capacity environment, introduction of basic technical solutions and application of incremental step-by-step reforms can be an effective strategy. Large and comprehensive e-government solutions can be introduced gradually even in a low-capacity environment if there are enabling factors in place, such as strong client ownership, availability of basic infrastructure and adequately trained local staff, and a sufficient budget for local training and technical support. • Implementation risks related to project complexity and multiple government implementing agencies can be successfully managed if there is strong leadership from the core government agency (such as the Ministry of Finance) and an experienced and empowered Project Coordination Unit. Effective use of local talent and expanding local partnerships in project implementation can be beneficial for project outcomes. xi • Technical assistance projects with multisectoral coverage require significant supervision support. Lack of budget can limit the ability of the World Bank to provide the specialized technical inputs needed to help the client make better design and implementation choices. Oscar Calvo-Gonzalez Director, Human Development and Economic Management Department Independent Evaluation Group xii 1. Country and Project Background 1.1 Mongolia is a large, sparsely populated country in East Asia, richly endowed with mineral deposits, including coal, copper, gold, and uranium. The mining sector produces approximately 20 percent of the gross domestic product (GDP) and is responsible for more than 80 percent of foreign direct investment, approximately 90 percent of exports, and nearly 30 percent of government revenues. The dominant role of mining raises many developmental challenges and well-known associated risks, such as governance problems, revenue volatility, and boom-and-bust cycles. 1.2 Over the past two decades, Mongolia has experienced a period of intense political competition, resulting in fiercely contested elections and unstable governments, which were important factors undermining overall fiscal discipline. 1.3 The primary motivation behind the preparation of the Governance Assistance Project (GAP) was to assist the government of Mongolia in consolidating past reforms and advancing additional reforms in the public sector to improve efficiency in the use of public resources. The World Bank considered strengthening Mongolia’s public sector management the top priority for its assistance program. Maintaining growth momentum and reducing poverty required addressing critical governance issues, including (i) enhancing fiduciary controls and improving the investment climate, (ii) implementing transparency measures, and (iii) putting in place direct anticorruption reforms. 1.4 By the time of the GAP preparation in 2005, Mongolia had made great strides in transitioning to a functioning market economy. Macroeconomic and fiscal policies had been somewhat strengthened, with the budget achieving a modest surplus in 2005. Driven by rising commodity prices, Mongolia was one of the fastest-growing economies in the world. GDP growth increased from an average of 3.2 percent per year from 1997 to 2002 to 8.9 percent per year in 2004–05, which helped reduce the poverty rate from 61 percent in 2002–03 to 35 percent in 2007–08. 1.5 However, sustainability of progress in the public financial management (PFM) area was a major concern, given Mongolia’s high resource dependence. Budget planning remained weak, with low budget credibility. The Medium-Term Expenditure Framework did not provide a hard budget constraint, and corruption was a serious challenge. Transparency International’s Corruption Perception Index for Mongolia was low (28 out of 100 in 2006) reflecting significant corruption risk. In particular, at the time of appraisal, little information was available to the public on budget execution, public debt, and procurement awards, which fostered perceptions of corruption. Progress in investment project selection, public procurement, and external audit capacity was inadequate. According to several World Bank Enterprise Surveys, three of the five most 1 significant obstacles to doing business in the country were corruption, tax policy, and tax administration. Mining regulation remained another area of concern. 1.6 Macroeconomic developments during project implementation (2006–14). Economic growth was generally sustained throughout the life of the project (table G.1). From 2006 to 2014, the average GDP growth rate exceeded 10 percent per year (except for the years of the global financial crisis, 2009–10) and the annual inflow of foreign direct investment reached approximately 50 percent of GDP by 2011–12 (compared with 7 percent of GDP in 2005), reflecting the global commodity boom. The resulting large increase in government revenue and spending presented both great opportunities and challenges for the government of Mongolia in terms of how to spend public funds efficiently and effectively. 1.7 The 2008–09 global financial crisis, followed by the collapse of global mineral prices, hit Mongolia hard and underlined the importance of a robust framework for fiscal policy to avoid procyclical spending policy. By February 2009, global copper prices had dropped 65 percent compared with June 2008. The government of Mongolia spent too much during the boom, was unprepared for the price collapse, and had to borrow heavily to stabilize the economy in the aftermath of the crisis (table G.2). The severity of the crisis in Mongolia was a significant factor affecting project implementation because it diverted the attention of decision makers from longer-term institutional development and capacity building (see appendix G for details on Mongolia’s macroeconomic performance). 1.8 Governance performance over the lifetime of the GAP and beyond. Mongolia’s progress in strengthening governance over the lifetime of the GAP was mixed. International governance indexes suggest significant progress between 2006 and 2014 in several areas, especially tax administration, corruption perception, public access to budget information, and political stability (table 1.1). At the same time, little or no progress was registered in the areas of rule of law, voice and accountability, and government effectiveness. Progress mostly stalled after 2014, except for regulatory quality. 1.9 In contrast, Mongolia’s Country Policy and Institutional Assessment ratings over 2005–17 showed little improvement (table 1.2), except for the transparency rating. This reflects the gap between institutional and policy progress: recent improvements in capacity for economic and fiscal management did not lead to a significant shift toward more sustainable macroeconomic policy. The largely procyclical (at least until 2017) expenditure policy affected several areas of economic management, including debt and public investment management. 2 Table 1.1. Selected Governance Indicators (scores) Governance Indicator 2006 2010 2014 2016 2017 2018 Corruption Perceptions Index (0 = highly corrupt to 100 = very 28 27 39 38 36 37 clean) Open Budget Index Scorea (0 = scant or no information to 100 = 18 60 n/a 51 46 n.a. extensive information available) Control of Corruption (perceptions of the extent to which public −0.58 −0.76 −0.47 −0.49 −0.45 n.a. power is exercised for private gain, −2.5 to 2.5) Government Effectiveness (perceptions of the quality of public 0.19 0.19 0.21 0.19 0.19 n.a. services and civil service, −2.5 to 2.5) Regulatory Quality (perceptions of the government's ability to −0.33 −0.24 −0.27 −0.08 −0.12 n.a. formulate and implement private sector development policies and regulations, −2.5 to 2.5) Rule of Law (perceptions of the extent to which agents have 0.19 0.14 0.15 0.16 0.15 n.a. confidence in and abide by the rules of society, −2.5 to 2.5) Voice and Accountability (perceptions of the extent to which 0.14 0.13 0.13 0.13 0.13 n.a. citizens are able to participate in selecting their government, −2.5 to 2.5) Political Stability and Absence of Violence (perceptions of the 0.65 0.6 0.79 0.8 0.82 n.a. likelihood of political instability or political violence, −2.5 to 2.5) Doing Business—Paying Taxesb (score is the average of four 64.32 71.52 73.79 86.01 86.73 77.32 component indicators) Source: Worldwide Governance Indicators; Transparency International; International Budget Partnership; Doing Business Indicators. Note: n.a. = not applicable. a. The 2015 score was used because survey data for 2016 were unavailable. b. The 2018 data point uses Doing Business 2017–19 methodology. Table 1.2. Selected Country Policy and Institutional Assessment Ratings, 2005–17 CPIA Indicator 2005 2010 2015 2017 Debt policy rating 3.0 3.5 3.0 2.5 Efficiency of revenue mobilization rating 3.5 3.5 3.5 3.5 Quality of budgetary and financial management rating 4.0 4.0 3.5 3.5 Quality of public administration rating 3.5 3.5 3.5 3.0 Transparency, accountability, and corruption in the public sector rating 2.5 3.0 3.5 3.5 Economic management cluster average 3.5 3.3 2.8 2.8 Business regulatory environment rating 3.5 3.5 3.5 3.5 Source: Country Policy and Institutional Assessment database, World Bank. Note: Scores are on a scale of 1 to 5. CPIA = Country Policy and Institutional Assessment. 3 2. Mongolia Governance Assistance Project Objectives 2.1 The objective of the project was to “assist the government in (i) improving the efficiency and effectiveness of governance processes in the management of its public finances, (ii) promoting transparency and accountability in the performance of public sector functions, and (iii) fostering the investment climate in Mongolia” (World Bank 2006b). No revisions were made. Project Components and Costs 2.2 The project design was based on considerable diagnostic and analytical work to complement several earlier World Bank–supported operations. The design was informed by findings of the Mongolian Investment Climate Survey, the Public Expenditure and Financial Management Review, and the Mining Sector Report. 2.3 The GAP consisted of four main components (see figure 2.1; see appendix D for details on project costs):1 • Management of public finance: (i) strengthening debt management and the public investment program, (ii) decentralizing financial management, (iii) improving the efficiency of civil service expenditure, and (iv) consolidating public sector accounting reform. • Fostering public accountability and monitoring: (i) assisting the government in bringing Mongolian laws in compliance with the United Nations’ Convention against Corruption, (ii) strengthening public disclosure of Environmental Impact Assessments and other information on environment and natural resource management, (iii) building capacity for data transparency and data access for poverty monitoring, and (iv) upgrading the website of the Ministry of Finance to a dynamic portal. • Investment climate: (i) supporting reforms to strengthen tax administration, (ii) building capacity in public procurement, and (iii) strengthening the mining registry and cadastre system. • Project coordination and evaluation. 4 Figure 2.1. Governance Assistance Project Structure Source: Independent Evaluation Group. Note: HR = human resources; MoF = Ministry of Finance; PIP = public investment program. 2.4 The project appraisal document (PAD) outlined five main areas of project intervention (grouped under three main project components): (i) enhancing PFM, (ii) fostering public accountability and monitoring, (iii) upgrading tax administration, (iv) improving public procurement, and (v) strengthening the mining registry and cadastre system. 2.5 Revised components. Due to overlapping support under the Mongolia Mining Sector Institutional Strengthening Technical Assistance Project (P108768), the proposed GAP’s activities related to strengthening public disclosure of Environmental Impact Assessments and other information on environment and natural resource management under the original component B(ii) were canceled in 2007 (before any disbursement was made under this subcomponent). Two major project subcomponents—A(ii) Government Financial Management Information System (GFMIS) and C(i) Tax Administration Information System (TAIS)—were implemented on a much larger, nationwide scale than 5 originally envisioned because of an additional contribution from the government of Mongolia’s budget. 2.6 Project cost and financing. The project was financed by a $14 million (special drawing rights 9.7 million) International Development Association Technical Assistance Grant, which was fully disbursed. Due to special drawing rights appreciation over the life of the project, the dollar equivalent increased from $14 million at project approval to approximately $15 million (107 percent of appraisal estimate) by project closing (table D.2). No contribution from the borrower was expected. 2.7 The total project costs did not cover significant government investments made in countrywide rollover of the information technology (IT) systems designed and piloted under the GAP. In two major cases—GFMIS and TAIS—such investments, funded by the government of Mongolia’s regular budget, were significant (estimated $7.5 million). Related Operations 2.8 The GAP was prepared and implemented as part of the long-term World Bank program of technical assistance to the government of Mongolia in public sector management (table 2.1). The program focused on strengthening PFM but also covered tax administration, civil service reforms, and regulatory governance in the mining sector.2 For four years (2010–13), four of these projects were implemented in parallel, and three were managed by the same Project Coordination Unit (PCU) based in the Ministry of Finance. This helped generate considerable synergies. There was evidence of productive project complementarity in the integrated financial management information system development and in launching the Computerized Mining Cadastre System (CMCS). 2.9 The latest World Bank technical assistance project to enhance economic management in Mongolia, the Strengthening Fiscal and Financial Stability Project (P161048), was approved in June 2017. It further advances the original GAP agenda by supporting institutional and capacity building in budget planning, government accounting, auditing and reporting, debt management, public investment management, and budget transparency. 2.10 Several GAP activities were directly linked to the implementation of the government of Mongolia’s policy reform program supported by two development policy operations (DPOs; P115737 and P117421) disbursed in 2009–10. The GAP’s link with the DPO program was most prominent in PFM (including improving capital expenditure management) and mining regulations (reforming the mining cadastre) areas (see appendix C).3 6 2.11 In addition, the GAP was complemented by technical assistance support provided under several trust funds, including the Institutional Development Fund’s Strengthening Corruption Prevention and Monitoring grant, which funded the development of the Mongolia Corruption Index by the Independent Authority against Corruption (IAAC) and helped upgrade the online system for managing income and assets declarations (IADs). 2.12 All World Bank technical assistance to improve public sector governance under the GAP was coordinated with other development partners. For instance, the United Nations Development Programme and the Swedish International Development Agency provided parallel assistance to advance reforms in public administration. The Asian Development Bank provided a governance reform loan, and an International Monetary Fund (IMF) treasury adviser assisted in cash management. Table 2.1. Major World Bank Technical Assistance Investment Lending Projects in Mongolia, 1998–2015 FTAP ECBTAP GAP MSTAP MSISTAP Project number P051855 P077778 P098426 P119825 P108768 Implementation period 1998–2005 2003–13 2006–14 2010–17 2008–15 Amount ($, millions) 5.3 8.0 15.0 11.7 8.7 Project outcome rating by IEG MS U MS MS MU Main areas of support PFM/IFMIS Yes Yes Yes Yes - Public accountability - - Yes - - Tax administration Yes - Yes - - Public procurement - - Yes - - Mining regulations - - Yes - Yes Source: Independent Evaluation Group. Note: Project outcome ratings were moderately satisfactory (MS), moderately unsatisfactory (MU), and unsatisfactory (U). ECBTAP = Economic Capacity Building Technical Assistance Project; FTAP = Fiscal Accounting Technical Assistance Project; GAP = Governance Assistance Project; IEG = Independent Evaluation Group; IFMIS = Integrated Financial Management Information System; MSISTAP = Mining Sector Institutional Strengthening Technical Assistance Project; MSTAP = Multisector Technical Assistance Project; PFM = public financial management. Implementation 2.13 Project dates. The GAP was approved by the Board of Directors on May 23, 2006, and became effective on July 21, 2006. The original closing date was January 15, 2012. It was extended twice (in December 2011 and December 2013). The actual closing date was December 31, 2014, almost three years later than originally planned (table D.1). The extensions were primarily because of the slow implementation of the TAIS, caused by 7 delays in contract signing, poor performance of the supplier, and weak client ownership during early implementation. 2.14 Project restructuring. The project was restructured three times to extend the closing date, make minor reallocation of funds, and adjust the results framework (see appendix A). 2.15 Implementation experience. For a country with relatively weak institutional capacity, Mongolia has had a comparatively strong track record of implementing technical assistance projects supported by the World Bank. Of eight recent projects, six had moderately satisfactory or above ratings for development outcomes, based on IEG Implementation Completion and Results Report (ICR) Reviews. The GAP implementation capitalized on the availability of relevant experience by selecting the same PCU in the Ministry of Finance in charge of implementing earlier technical assistance projects. This proved to be beneficial for project implementation, reducing risks of procurement delays, and ensuring relatively smooth operation from a very early stage. Disbursement lags were modest (since 2010 only) and mostly related to TAIS implementation. 2.16 Implementation arrangements. It was agreed that the project would be led by a high-level steering committee comprising the minister of finance (chairperson), the state secretary for finance (project director), members representing key participating agencies, and representatives from the private sector and civil society. The steering committee was established before the GAP was approved by the World Bank. According to the PCU, it met regularly and reviewed project developments. However, IEG’s mission was not able to validate this claim (no documentation existed to that account). Moreover, a steering committee member representing civil society informed the IEG team that he was invited only once to participate in the committee’s meetings. 2.17 Partnerships. Donor collaboration on governance has strengthened over the lifetime of the GAP. Various multilateral (Asian Development Bank, European Union, United Nations Development Programme) and bilateral (United States, Germany, Japan, Korea) donors have expanded governance-related assistance programs in Mongolia and demonstrated a willingness to complement several initiatives that originated under earlier World Bank–supported technical assistance projects, including the GAP. The ongoing multidonor effort in this area has been reasonably well coordinated. 2.18 Project ratings during implementation. The project team rated project implementation fully satisfactory until April 2012 despite significant delays in project implementation. It subsequently downgraded the implementation progress and development objective ratings to moderately unsatisfactory primarily because of slow 8 progress with TAIS implementation. For approximately six months in 2013–14, both project ratings stood at moderately unsatisfactory. Appendix B summarizes project Implementation Status and Results Reports (ISRs). 2.19 Other important factors that influenced project implementation. Beyond the factors already discussed (effects of the mining boom and global financial crisis, synergy with other technical assistance projects, and availability of the experienced PCU), the following factors also had a considerable impact on project implementation: • Government ownership of structural reforms supported under the GAP remained uneven, with significant policy reversals that made implementation more complicated. Despite strong leadership from the Ministry of Finance, implementation was affected by vested interests in several areas, including budget investment planning, public procurement, mining licenses, and management of civil service. • Parliamentary elections in 2008 and 2012 triggered a massive replacement of government officials at senior and middle levels. In several cases (for example, the mining subcomponent), it disrupted project-related decision-making and delayed finalization of procurement. Fortunately for the GAP, the Ministry of Finance was least affected by this staff turnover, which helped preserve institutional memory and provided for a quick recovery in the pace of implementation. • Creation of the Ministry of Economic Development (in 2012) and Development Bank of Mongolia (in 2011) brought additional institutional fragmentation and took over a significant part of the Ministry of Finance’smandate. This created additional coordination challenges for policy development and reform implementation. Debt management, public procurement, and public investment management subcomponents of the GAP were the most affected. The dissolution of the Ministry of Economic Development in 2015 helped return several functions to the Ministry of Finance. • The expected link with the DPO on governance (originally planned for 2007– 08) did not materialize. The government felt that it did not need additional budget support in 2006–08 when mining prices remained high, and only requested World Bank budget support in 2009 when it was facing a severe macroeconomic crisis (a series of two DPOs was approved in 2009–10). At that time, to facilitate DPO preparation, which focused on macroeconomic stabilization and less on the traditional governance agenda, World Bank management decided to restructure the ongoing Economic Capacity Building 9 Technical Assistance Project (ECBTAP) and prepare a new technical assistance project (Multisector Technical Assistance Project), although the link between these new DPOs and the GAP was not as strong as originally envisioned. • Implementation experiences of the Fiscal Accounting Technical Assistance Project and ECBTAP were reflected in the GAP design. In light of implementation problems and insufficient client ownership of major reforms experienced with earlier technical assistance projects, the design of the GAP was less ambitious, with less emphasis on reforms that would require considerable cross-ministerial ownership and cooperation. In terms of implementation capacity, the project design relied more on the central PCU in the Ministry of Finance and less on the capabilities of other implementing agencies. 2.20 Fiduciary issues and safeguards. The project did not trigger any social issues or safeguards. It was categorized “C” for environmental assessment purposes. The project risk related to financial management was rated modest, which was justified by the considerable experience of the PCU, which had implemented similar World Bank projects in the recent past. Preparation of the project’s financial management manual, which had been approved before effectiveness, was identified as the primary risk mitigation measure. 2.21 The ICR states that the project had an adequate financial management system that provided reasonable accuracy and timely information on the implementation progress and use of project funds. The PCU was effective in facilitating compliance with the World Bank fiduciary and reporting requirements across participating agencies. It also effectively used the Ministry of Finance’s Internal Audit Department as an enforcement tool to ensure compliance. The financial management rating in the project’s ISR was satisfactory through the life of the project. According to the ICR, the closing audit report of the project was submitted on time and had an unqualified opinion. 2.22 However, since 2008, the World Bank's Integrity Vice Presidency has been investigating three substantiated allegations of fraud and corruption within the project. One entity was debarred for following fraudulent practices in World Bank procurement; three others were sanctioned for providing corrupt benefits in the course of procurement tenders under the GAP. Two of these cases were not GAP-specific problems; they were related to the highly problematic performance of specific international consultants, who followed the same business practices in several World Bank client countries. 2.23 The Project Performance Assessment Report (PPAR) mission found that the procurement process had been managed well. The GAP had an unusually large number of small contracts (for hiring national consultants on a short-term basis), and this was 10 organized quite efficiently. The only serious procurement delays were related to the case of TAIS procurement (not unusual for procurement of a large and complex IT system in the low-capacity environment) and, to a smaller degree, to the CMCS. 3. Relevance of the Objectives and Design Relevance of the Objectives 3.1 The relevance of the objectives is rated high. The GAP’s objectives were linked directly to the priorities of the government’s development strategy, which called for promoting good governance. The government of Mongolia’s action plan aimed at enhancing public transparency and improving justice and public services to reduce actual and perceived corruption. The project’s objectives were also in line with the World Bank–supported Country Assistance Strategy for Mongolia (FY05–08), which included among its priorities support for improvements in public sector effectiveness through strengthened public expenditure management, improved quality of infrastructure investments, enhanced fiscal sustainability, and a reformed civil service. It underlined a need to reduce fiscal deficits and improve the use of limited public resources. The subsequent Mongolia Country Program Strategy FY13–17 further developed these themes, explicitly placing the challenge of robust economic management at the center of World Bank assistance.4 3.2 The 2018 Mongolia Systematic Country Diagnostic reconfirmed the relevance of the GAP agenda. The Systematic Country Diagnostic identified governance as a cross- cutting root issue underpinning all key challenges the country faces. It highlighted three broad governance issues: budget credibility, quality and predictability of public investments, and capability and capacity of the civil service. Risky macroeconomic management was considered the most urgent governance problem. Relevance of Design 3.3 The relevance of design was rated substantial. The project tried to address governance issues in Mongolia in three interrelated areas: (i) implementing transparency measures, (ii) advancing anticorruption reforms, and (iii) improving selected aspects of the investment climate. This approach was designed to tackle both the perception of corruption (through increasing transparency and streamlining administration) and actual corruption (through fiduciary and anticorruption programs). 3.4 The project’s design reflected lessons from earlier technical assistance projects, such as the Fiscal Accounting Technical Assistance Project and ECBTAP, which emphasized the importance of streamlined design and client ownership for successful 11 implementation of core reforms in public sector management. Compared with two earlier World Bank–supported technical assistance operations, the GAP was more modest in its development objectives. At the same time, GAP design featured the same very broad (and thin) coverage of core public sector functions. The list of activities supported by the GAP looked like a “big tent” covering debt management, investment planning, accounting, tax administration, public payroll management, decentralization, public procurement, and an anticorruption strategy. Such broad project coverage was partially motivated by World Bank staff’s expectation that the GAP would support the preparation and implementation of a DPO series to enhance major governance reforms. This series did not materialize in the expected format because the 2009 DPOs focused more on postcrisis macroeconomic stabilization and less on a traditional governance agenda. 3.5 The project’s theory of change was sound (appendix F). It described a reasonable causal chain among project inputs (various investments in IT systems, new laws and regulations, advisory services, and training) across the main government agencies responsible for economic management, which, if properly and promptly implemented, would have resulted in improved capabilities of these agencies (outputs) and better governance, transparency, and accountability (outcomes). The project-funded investments and technical assistance activities were substantially relevant to achieving the project development objectives (PDOs). The weak point in this design logic was the underestimation of the time needed to build the capacity of local institutions to make them capable of fully using new systems and implementing new regulations. Similar to earlier technical assistance projects in Mongolia (and elsewhere), project design was excessively ambitious and the original timeline was not feasible, which contributed to the implementation delays that extended the project. 3.6 The GAP was a multisector governance operation. The project design was complex because it included 14 subcomponents, which were implemented by 8 different government agencies. However, the responsibility for project implementation was more concentrated: two main agencies (Ministry of Finance and General Department of Taxation; GDT) were responsible for the implementation of activities that made up two- thirds of the total project cost, whereas three main IT systems supported by the GAP (GFMIS, TAIS, and CMCS) made up just over half the total. 3.7 To address capacity constraints, the GAP project provided for contracting a large number of residential advisers, many Mongolian nationals, who were expected to work on a long-term basis with regular government staff. This worked well and indeed helped enhance implementation capacity, especially in the Ministry of Finance. The provision for a relatively large training budget5 reduced implementation risk. To a large extent, the project’s impact on capacity came from training interventions. The rationale 12 for massive local training programs was clear. The introduction of new e-governance systems covering the entire country required training of future users, especially in local governments outside of Ulaanbaatar. The GAP delivered needed training and also supported the development of various training manuals, many of which are still in use. 3.8 The design of the results framework showed some weaknesses, which were partially addressed during the implementation. 4. Achievement of Objectives 4.1 This section assesses achievement of project objectives, based on the review of the project’s ICR, ICR Review, aide-mémoire, IMF and World Bank reports, and other relevant materials, including several publicly available studies by various donors and the findings of the IEG mission to Mongolia in June 2019. Objective 1: Improving the Efficiency and Effectiveness of Governance Processes in the Management of Public Finances The achievement of objective 1 is rated substantial. Debt Management Main Outputs • The first medium-term debt management strategy for 2007–09 was developed and adopted in May 2007. It has been updated regularly, approximately every two years (latest in May 2019). The Debt Management Law was adopted in February 2015, providing a legal framework for updating the debt management strategy. • The Debt Management and Financial Analysis System was initially installed in the Ministry of Finance in 2002 and was successfully upgraded in 2013 under the GAP to version 6.0 (project target).6 The Debt Management and Financial Analysis System is used for the quarterly production and disclosure of comprehensive reports on government and publicly guaranteed debt. Public debt reporting is timely and meets the IMF’s standards. The project’s target to introduce comprehensive monitoring of public debt was achieved. Main Outcomes 4.2 GAP-supported technical assistance helped strengthen the quality of public debt management, and this facilitated improvements in transparency, predictability, and soundness of public finance. Debt strategy documents were used by the government to 13 inform decisions on the issuance of new public debt and improve coordination between the Ministry of Finance and the Central Bank of Mongolia. In particular, within the framework of the Medium-Term Government Debt Management Strategy for 2016–18,7 the government of Mongolia halted the issuance of short-term high-interest domestic government securities, took measures to amortize some outstanding government securities before maturity, and borrowed externally only on concessional terms for the implementation of development projects. These actions resulted in the repayment of a significant portion of the most expensive public debt and, from 2017, a reduction in overall public debt burden. Budget Management Main Outputs • The GFMIS was launched in large budget entities and district governments (soums); 50 soums and 10 large budget entities were connected in 2007–10, meeting the project target. By 2015, the GFMIS became fully operational in all 333 soums. The implementation of the GFMIS took less time to complete than predicted. • The Integrated Budget Law was approved in 2011 and provided a legal framework for modernizing budget management, including modern treasury management. Together with the Fiscal Stability Law, the Integrated Budget Law introduced new fiscal rules aimed at addressing the procyclicality of the Mongolian budget by limiting both the pace of annual expenditure growth and the size of the annual budget deficit. These laws were drafted with technical assistance provided under both the GAP and ECBTAP; their adoption was supported by DPO lending. Main Outcomes 4.3 The successful countrywide rollout of GFMIS allowed for real-time monitoring of budget expenditure and helped improve the reliability of budget management, reduce delays in budget reporting, and enhance government control over budget execution. According to the Public Expenditure and Financial Accountability (PEFA) report, “Thanks to the regularity of the annual budget process, and the functional TSA [treasury single account] and GFMIS, schools and basic health units receive resources on time and as per the budget plan, and this information is accurately accounted for and audited, and made publicly available, representing a significant achievement of Mongolia’s PFM reforms over the past decade” (World Bank 2015b, 14). In addition, the GFMIS generated considerable budget savings (reduced debt cost8 and eliminated commercial bank fees), drastically reduced delays in transferring budget funds to local governments, and 14 improved predictability in local public finance. The GFMIS contractor (FreeBalance) used Mongolia as a model of successful implementation and brought officials from other countries (Kyrgyz Republic, Vietnam) to Ulaanbaatar to learn from their experience. GFMIS operations were robust, and sufficient just-in-time technical support was available for users across Mongolia. GAP funding provided adequate initial training for remote users and helped develop training manuals. At the same time, the GFMIS operates as a nonintegrated, stand-alone IT system. Although initially adequate, this system created issues of integration with other e-governance systems (debt management, fixed asset accounting, procurement, financial reporting, and so on) that have to be addressed at the next stage of system development. 4.4 There has been considerable synergy between ECBTAP and GAP in the development of GFMIS. The division of labor between these two projects was clear: the ECBTAP funded the design and installation of the initial, central part of the system to cover management of central government expenditure (moving the national treasury online) and the GAP funded its nationwide rollout, linking the central treasury with local governments’ budget departments and training of these remote users. 4.5 Effectiveness of the new legislation (Integrated Budget Law and Fiscal Stability Law) declined drastically after the parliament decided (in 2011) to keep public expenditures funded by the Development Bank of Mongolia outside of the regular budget. After amendments to the Development Bank of Mongolia Law (2016), the scope of off-budget operations was reduced, and the legal limits for budget deficit adjusted upward. Public Investment Management Main Outputs • Preparation of a medium-term public investment program was supported through GAP and approved in 2011. It has been amended regularly in compliance with the requirements of the Integrated Budget Law. The Ministry of Finance produces a public investment program implementation report annually. • Development of the manual on preparation, appraisal, and selection of public investment projects (2010). The manual contained a reasonable set of recommendations on cost-benefit analysis and project selection. However, according to the PPAR mission’s interviews, the government of Mongolia could not implement the manual’s recommendations owing to a lack of necessary data and staff trained in cost-benefit analysis. 15 Main Outcomes 4.6 Public investment efficiency in Mongolia remains low. According to the Public Investment Management Assessment jointly conducted by the IMF and the World Bank in 2016, Mongolia performs poorly relative to the global average and the average for a sample of similar emerging market economies (IMF 2018b). The main challenges within public investment management relate topoor planning quality that inhibits the effective screening of projects for strategic relevance, weak project appraisals, and continuous political interference into decisions on allocations from the investment budget. Overall, the system faces the same challenges as a decade ago: no reliable investment planning, project database, or working procedures for project selection. The legal framework in place is quite advanced, but there has been little compliance with its requirements. The lack of traction in this area has been disappointing, especially considering the heavy emphasis on the public investment management agenda under the DPO program in 2009–10 (appendix C). The 2011 public investment program, similar to many other public investment programs in Mongolia, was not realistic in light of actual budget constraints. It contained too many projects, was too political, and was not backed by the necessary implementation infrastructure, such as an integrated project database. Since 2018, there has been a renewed attempt to upgrade public investment management in Mongolia, jointly supported by the World Bank, Asian Development Bank, and Korean government. Civil Service Expenditure Management Main Outputs • A wage bill modeling tool was developed (with GAP technical assistance) and used in 2011–14 by the Ministry of Finance to conduct an impact analysis of different civil service recruitment options. It was dropped after 2014 when the Ministry of Finance started to use an alternative, locally developed model. • A private sector pay comparator survey for the civil service (benchmarking) was successfully completed in July 2010. It was used as a basis for the 2011 report that the Civil Service Council submitted to parliament to justify a comprehensive reform in civil service pay. The proposal was rejected because of its cost implications. • An annual civil service census was introduced in 2007 based on the newly developed methodology. The census is administered by the Civil Service Council, and the census results are used as a primary input to the annual Civil Service Council report to parliament on public sector employment (publicly available). The census defines and verifies the actual size of the civil service. It 16 helped establish and regularly update a civil service–wide Human Resources Management Information System (HRMIS) database. • The Training Research and Advisory Center was established at the Civil Service Council in 2012 with GAP funding. The center was expected to take the lead on policy analysis in civil service management. However, the center was abolished by a parliamentary decision in 2014 as part of the budget savings policy package. The recently revised Civil Service Law (adopted in December 2017) provides for the reestablishment of the center, but no action had been taken at the time of the PPAR mission. Main Outcomes 4.7 GAP-supported interventions provided for some improvements and additional transparency in public human resources expenditure management. For the first time, the Civil Service Council obtained verified estimates of the actual size of the civil service, was able to determine the number of “ghost workers,” and created a government-wide human resources registration system. The HRMIS database became a good reference for policy analysis to estimate the expected number of civil servants over the short and medium term and has provided helpful input to budget planning. The data from the 2014 survey were used by parliament to inform the legal framework to control the overall size and cost of the civil service. 4.8 However, overall progress in human resources expenditure management has been limited. The HRMIS database is not linked to the payroll management system and GFMIS, which is the primary source of significant weaknesses in payroll control. The government of Mongolia still lacks a modern web-based human resources management system. Furthermore, the quality of HRMIS data remains problematic owing to frequent changes in the structure of the government, poor data, and irregular and low-quality reporting arrangements. Payroll audits take place infrequently.9 Objective 2: Promoting Transparency and Accountability in the Performance of Public Sector Functions The achievement rating of objective 2 is substantial. Accountability in Civil Service and Anticorruption Framework Main Outputs • The law on prevention of conflict of interest in the public sector was developed with help from the GAP and approved by the parliament in January 2012. The 17 National Anticorruption Program was approved by the parliament in November 2016. • The government’s system for IADs was established by the Anticorruption Law of 2006. The IAAC was set up to run the system of collecting and analyzing IADs and to impose sanctions for noncompliance. In 2012, 99.9 percent of 41,000 public officials declared their property and income to the IAAC. The GAP supported various training and capacity building activities for the IAAC staff to enhance the ability to conduct ongoing monitoring and analysis of the declarations. • The GAP supported the introduction (in 2013) of a relatively simple IT system (Personal Income and Assets Disclosure Information System) to help the IAAC to scan, store, and analyze the IADs, which were originally collected in paper format. The 6,000 designated ethics officers received additional training to become focal IAAC contacts in the field and facilitate the entry of IAD data into the system. • With support from the GAP and an Institutional Development Fund grant, a methodology was developed for the establishment of the Corruption Perception Index, as required by Mongolia’s Anticorruption Law. The index formed the initial baseline (2009) by which the IAAC has been measuring its progress in fighting corruption.10 • The IAAC consolidates the survey results and other analyses into its annual report to parliament on progress with the implementation of the National Anticorruption Program. Main Outcomes 4.9 There has been measurable improvement in government accountability in Mongolia, which is reflected in changes in public perceptions of governance and corruption incidence shown by national indexes against the 2006 baseline (see tables 4.1 and 4.2). International indexes, such as the Control of Corruption Index (World Bank Governance Indicators) and the Transparency Index by Transparency International, place Mongolia below the global average (indicating that corruption remains a serious issue) yet acknowledge the progress that has been made during the past decade (table 1.1). Although available data show some recent deterioration in accountability (since 2014), the situation is better than at the time of GAP initiation (2006). 4.10 More detailed results on trends in corruption perception and actual corruption, based on the regular national surveys, confirm this broad-based improvement, especially concerning control of petty corruption. The results of the regular Household 18 Survey on Perceptions and Knowledge of Corruption (SPEAK), administered by the Asia Foundation, suggest that the incidence of reported corruption cases decreased significantly over the 2006–18 period (tables 4.1 and 4.2).11 4.11 One of the main drivers of change in government accountability in Mongolia has been the considerable expansion in independent monitoring of transparency by local civil society organizations that are generously funded by the international community. At the same time, there has been less progress with respect to grand corruption, which is deeply rooted in the political economy and may require fundamental political reforms. A 2019 Organisation for Economic Co-operation and Development report acknowledged the recent strengthening of the government of Mongolia’s enforcement efforts to address grand corruption but concluded that more political will was needed (OECD 2019). Table 4.1. Perceptions and Incidence of Corruption Measured by Household Surveys (percent) Perception 2006 2010 2014 2016 2018 Respondents who believe that corruption has recently 75 64 34 52 63 increased (either a little or a lot) Respondents who claim that their household paid a 26 16 8 7 4 petty bribe in the last three months Respondents who “often” hear about grand corruption — — 22 16 26 recently Respondents who are confident that the IAAC is fighting — 27 39 30 19 corruption (either “confident” or “rather confident”) Source: Asia Foundation 2018b, 26, 29, 44–45. Note: — = n.a. IAAC = Independent Authority against Corruption. Table 4.2. Perceptions of Corruption Measured by Private Business Surveys Perception 2012 2014 2016 2018 Respondents who personally know about corrupt transactions 28 23 18 15 with the government in the last three months Respondents who believe in high corruption in public 75 64 56 61 procurement (answered either “often" or “always”) Source: Asia Foundation 2018a, 37–38. 4.12 Improvements in governance could not be fully attributed to the GAP; however, it is plausible that GAP-supported activities made direct contributions to the increased transparency of government operations and helped improve the government of Mongolia’s capacity to monitor the situation, introduce stricter accountability rules for civil servants, impose sanctions for noncompliance with these rules, and raise public awareness on corruption matters. The IAAC’s capacity for policy coordination and prevention work has increased as a result of GAP support, and its management and 19 analysis of IADs was broadly recognized as helpful for imposing a more stringent accountability framework within the civil service. The IAD management system enhanced the agency’s capacity to produce its reports in a timely manner. But overall, the IAAC’s contribution to improved control of corruption has been only marginal so far. The IAAC is seen as excessively politicized, its public profile is affected by frequent leadership changes, and public confidence in the agency’s impartiality remains low (table 4.1).12 The government of Mongolia’s anticorruption laws and action plans have been only partially implemented. Many more accountability benefits were realized through government actions to expand transparency in the execution of its core functions, such as public spending, mining license administration, and property registration. Public access to the IADs remains limited. Only summary data are compiled and published in newspapers and on the IAAC’s website. Budget Transparency and Disclosure Main Outputs • The GAP funded Ministry of Finance efforts to establish a new comprehensive budget information portal to enhance public access to budget data, launched in 201213. It allowed designated non-IT staff of the Ministry of Finance to upload budget information regularly. Since its launch, the Ministry of Finance provided for the regular maintenance and update of the portal to expand its functionality and make it more user-friendly. The latest update was completed in 2018. • The Integrated Budget Law, drafted with assistance under the GAP, adopted fiscal principles to enhance public access to budget information and ensure public participation in budget planning, execution, and monitoring. The Integrated Budget Law provided a legal framework for the design and subsequent operation of the Ministry of Finance’s web portal. • In July 2014, parliament passed a separate law on Glass Accounts (budget transparency law). It requires all government agencies and legal entities with state involvement to make budgetary information available to the public to ensure the disclosure of all public contracts above Mongolian tughrik (MNT) 5 million. The government of Mongolia established a separate portal to host all relevant information related to the implementation of the Glass Accounts law.14 The portal operates in parallel with the Ministry of Finance‘s website as an alternative source of various information on public finance. Main Outcomes 4.13 With technical assistance from the GAP and other World Bank–supported technical assistance projects, Mongolia has made good progress in increasing the 20 comprehensiveness of budget information as measured by the types of documents that are submitted to parliament for deliberation and approval as well as by the scope of public disclosure of this information. The annual budget package meets eight of nine documentary requirements of PEFA. From March 2008, the treasury department has published monthly and annual budget execution reports on its website. With the introduction of the GAP-supported new portal in 2012, the Ministry of Finance has expanded coverage and regularity of its data disclosure. In 2015, PEFA found that public disclosure of fiscal information had been steadily increasing since 2006 and included annual budget documentation at the time of submission to parliament, in-year budget execution reports, year-end financial statements, external audit reports, contract awards, and information on resources available to primary service units. The project’s target for budget disclosure (to achieve PEFA’s A rating for public access to key fiscal information, measured by the PEFA indicator PI-10)15 was achieved. See also appendix E. 4.14 Since 2014, there has been a further effort to enhance budget transparency, including an expansion of functionality and data coverage at the Ministry of Finance and Glass Account portals. Mongolia’s Open Budget Index score increased significantly over the period—from 17 in 2006 (out of 100) to 46 in 2017. The 2017 Open Budget report16 provides several priority recommendations (timelier online publication of the Prebudget Statement; preparation of the Citizens Budget), which have since been implemented. 4.15 Despite this progress, the current Open Budget Index score is still relatively low, indicating opportunities for further improvement in public access. One immediate concern is the limited disclosure of the annual budget audit report. Another issue raised by the civil society groups is that the Ministry of Finance publishes its budget information only in PDF, which makes it hard to analyze. Public Access and Reliability of Poverty Statistics Main Outputs • The GAP helped the National Statistical Office (NSO) to upgrade its basic tool for poverty monitoring by introducing the new comprehensive Households Social and Economic Survey, undertaken regularly since 2007. • World Bank staff provided regular technical assistance to the NSO aimed at further strengthening the survey methodology and poverty data analysis. • A database with the results of previous household surveys (Living Standards Measurement Survey) was created and made publicly available through the NSO website. 21 • A separate database was created with raw microdata and main results for all household surveys and censuses conducted by the NSO for the past 10 years. It is updated regularly. However, the NSO website does not provide an English version for the most recent surveys, which limits outreach to donors and international organizations. Main Outcomes 4.16 The GAP succeeded in strengthening the NSO’s capacity to generate high- quality, timely, and policy-relevant poverty data, as well as improving public access to poverty information. The administration of the new Households Social and Economic Survey has been assessed by the World Bank as efficient and robust. As a result, public confidence in Mongolia’s poverty statistics has increased and their use in day-to-day government policy making has expanded. In particular, the updated poverty survey methodology has been used in the administration of poverty benefits (means testing). The NSO’s poverty estimates are also regularly used in budget planning (allocation of social protection budget across various subnational governments, depending on their poverty incidence). The detailed household survey data are regularly updated and are available for independent poverty analysis from the NSO website.17 Strengthening Audits and Accounting in the Public Sector Main Outputs • The GAP provided assistance to the Mongolian Institute of Certified Public Accountants in developing their first manual on quality assurance, which helped local audit firms establish internal control systems. • A legal and regulatory framework for asset valuation was strengthened. A Law on Asset Valuation was drafted with support under the GAP and was approved in January 2010. The new methodology on asset revaluation was also approved with a focus on revaluation of intangible and immovable assets. • A training program in methodology for local appraisers and accountants was conducted, and immovable assets were revalued for more than 350 government entities, which helped close some accounting gaps within the public sector. Main Outcomes 4.17 This was a small self-standing subcomponent without clearly identified outcomes. 22 Objective 3: Fostering the Investment Climate in Mongolia The achievement rating of objective 3 is substantial. 4.18 The GAP sought to enhance the investment climate in Mongolia through support in tax administration, public procurement, and administration of mining licenses. The GAP did not intend to deliver any systemwide improvements in the investment climate. Therefore, the formulation of this objective was excessively broad. Tax Administration 4.19 According to several enterprise surveys, weaknesses in tax administration, such as high compliance costs, corruption, and nontransparency in tax audits, were perceived as significant administrative barriers for business development and negative factors affecting the investment climate. Main Outputs • The TAIS was developed to improve the efficiency and transparency of the entire tax administration cycle. TAIS became fully functional in late 2014. It covers the entire territory of Mongolia (33 regional tax offices) and all 19 main taxes. The countrywide coverage was funded by the government of Mongolia from its budget, whereas the GAP funded only the initial pilot (two offices) and the development of the software. The TAIS supports all core functions of tax administration, including tax registration, invoicing, tax payment, and compliance. • The GAP provided the GDT with detailed recommendations on tax administration reform. Most of these recommendations have been implemented. There was a major effort to advance core business process reengineering that included upgrading human resources management practices, developing taxpayer services, strengthening the monitoring and evaluation function, and improving the regulatory framework for dispute resolutions. Several internal policy documents and regulations were developed at the GDT with assistance under the GAP, including risk management framework, and audit policies and procedures. • More than 2,000 GDT staff received training on the new functional environment. Main Outcomes 4.20 Although the tax administration reforms supported under the GAP did not have specific fiscal policy objectives (formally they aimed at investment climate improvements), they still brought considerable benefits by broadening Mongolia’s tax 23 base, as reflected in the increased number of registered taxpayers (table 4.3). This improved the robustness of public revenue and made the government budget less dependent on volatile mining prices. Table 4.3. Registered Taxpayers, 2005–15 2005 2010 2015 Growth over Period Taxpayer Type (no.) (no.) (no.) (percent) All taxpayers 511,683 743,805 1,440,091 281 Value-added tax payers 5,453 14,236 32,512 596 Corporate income tax payers 2,776 5,850 10,069 363 Source: General Department of Taxation website. 4.21 The coverage of the TAIS project was broader than originally envisioned under the GAP. As a result, it brought about more benefits to the country’s tax administration, despite significant delays with its completion. 4.22 The GAP was supposed to pilot new IT-based business processes in two tax offices and automate the administration of the three main taxes (corporate income tax, personal income tax, and value-added tax). It was expected at appraisal that these pilots would be scaled up later (after the GAP original closing date in 2012), but no specific timetable and funding sources for such an expansion were identified. In practice, the TAIS had been running countrywide (33 tax offices) by 2015 and all 19 major taxes were automated. This occurred primarily because the government of Mongolia provided an additional $4.5 million to complement GAP funding to ensure nationwide rollout of the system. 4.23 Implementation of the TAIS resulted in a comprehensive modernization of Mongolia’s tax administration, which brought about numerous benefits for taxpayers and helped enhance the country’s investment climate. These benefits include the following: • Reduced costs of tax compliance, including availability of online tax registration and tax payment for all types of taxpayers across the country. Almost all taxes are currently paid online. Compared with 2005, in 2015 the number of taxpayers increased almost threefold, a broad improvement in tax compliance (table 4.3). • Reduced corruption risks and better administration of tax audits owing to the introduction of risk-based auditing and the adoption of modern human resources management practices within the GDT. Perceptions of corruption in the GDT have improved. According to the Asia Foundation, in 2012, 35 percent of private businesses considered the GDT to be the agency that created the most obstacles to doing business. This declined to 23 percent in 2017. The 2018 24 household survey revealed that tax administration has become the government function least affected by corruption when compared with a wide range of other government functions. • Better availability of taxpayer services, especially outside the capital city of Ulaanbaatar, including through online tools. Mongolia’s score for the Doing Business Paying Taxes indicator, the integrated measure of tax compliance costs, improved from 64.3 (out of 100) in 2006 to 86.7 in 2017. • Improved transparency of taxpayer obligations. According to the 2015 PEFA assessment (World Bank 2015b), good progress has been made with both transparency in taxpayer obligations and tax collections. 4.24 GDT disclosure practices improved, but the associated GAP target was only partially met. Although there has been an expansion in the publication of information on taxpayers’ rights and available services, no regular taxpayer satisfaction survey has been administered since GAP closure. Public Procurement Main Outputs • The legal and regulatory framework for procurement was upgraded with technical assistance provided under the GAP. Parliament adopted several amendments to the Public Procurement Law in 2011, which aimed at making procurement processes more transparent and participatory, as well as strengthening the framework for a complaint review. The revised law transferred many responsibilities for public procurement to a newly established independent Government Procurement Agency. In addition, several pieces of core implementation regulations were drafted and adopted, including competitive bidding and bid evaluation procedures. A National Program for Procurement Capacity Development was also prepared and approved by the government in January 2007.18 • The Procurement Management and Monitoring Information System (PMMIS) was developed with GAP support and officially launched in March 2010.19 During 2010–13, it was a primary tool for public procurement planning and monitoring. It also allowed for significant expansion in access to procurement information and facilitated competition. However, the system design was not sufficiently flexible, and the contractor failed to upgrade the system to meet the full requirements of the revised Public Procurement Law (2011). The PMMIS was 25 eventually shut down and replaced by a new system, funded by the government of Mongolia.20 Main Outcomes 4.25 Since 2005, the monitoring and evaluation function of the national procurement system has improved and transparency and competition in public bidding have increased, supported by progress in e-procurement. In 2014, 92 percent of public procurement funded from the government budget was based on open competition, exceeding the project target of 90 percent.21 Since GAP closure, the competitive tendering in public procurement has further expanded. Direct contracting was reduced from MNT 174 billion in 2007 to MNT 132 billion (8.3 percent of total) in 2018. The government of Mongolia reports that in 2018 approximately 90 percent of all procurement activities subject to the Public Procurement Law were handled by the e- procurement system (OECD 2019, 79). The remaining procuring entities use e- procurement for the publication of their procurement notices. Perceptions of corruption within the public tendering system have declined (table 4.2). 4.26 Attribution of the preceding achievements to the GAP is modest. The main direct contributions of the project (2011 amendments to the Public Procurement Law, PMMIS) were unsustainable and largely replaced by 2016.22 According to IEG mission interviews, the GAP can be credited for indirect contributions that facilitated changes in the mind- set of government officials and made the policy environment more supportive for later successful adoption of a modern e-procurement solution backed by an adequate regulatory framework. In addition, the main procurement reform principles introduced in the government of Mongolia’s procurement regulations in 2011 were the basis for more recent updates in the regulatory framework. Administration of Mining Licenses Main Outputs • The mining cadastre system (CMCS) was computerized and has been fully functional at the Mineral Resources Authority of Mongolia since April 2010. It records and manages all mineral licenses (more than 3,000 for both exploration and production). The web-based version of the CMCS was installed and made fully operational in February 2013. The cadastre information is publicly available online.23 The upgraded version can support the allocation of new licenses through public tendering. • The CMCS was designed and installed through a joint effort of the two parallel World Bank projects: the GAP and the Mining Sector Institutional Strengthening Technical Assistance Project (MSISTAP). The GAP funded the underlying 26 diagnostic and system design, whereas the actual installation was funded mainly by the MSISTAP. • A detailed user manual describing the operation and maintenance of the system was developed and published. Main Outcomes 4.27 With support received under the GAP and MSISTAP, the government of Mongolia has made considerable improvements in the administration of its mining licenses. In particular, the CMCS provided for a more efficient and less time-consuming process of license administration (issuing of original licenses, their renewal and cancellation), more efficient fee collection (reduced payment delays and budget losses), and more transparency in license ownership (who owns what, what is available for future licensing, and so on). The system helps identify cases of overdue license fees and charges interest on late payments. The time to obtain a cadastre license was reduced from three to four months in 2009 to approximately 20 days in 2014 (below the 45-day limit imposed by law). At the same time, the application process was made more transparent because it allows applicants the opportunity to monitor their application online. 4.28 The system operations helped address some major inefficiencies under the previous license management regime. It allowed the government of Mongolia to identify and address cases of overlapping mineral rights. The system also helped reduce the area coverage of licenses that expired without use by more than half—from 14.2 million hectares in 2011 to 6.2 million hectares in 2014. 4.29 The installation of CMCS was part of broader progress made by the government of Mongolia during the past decade toward a more transparent regime for managing developments in the mining sector, as confirmed by two additional sources: • Mongolia became only the second country in the world to meet all the requirements of the new 2016 Extractive Industry Transparency International standard. • The Resource Governance Index 2017 Report rated the governance regime in Mongolia’s mining satisfactory and ranked Mongolia 15th globally (among 89 mining countries). 4.30 Notwithstanding this progress, the existing legal and regulatory framework in the sector still needs enhancement, in particular through better enforcement. According to the PAD, the project team expected that the new system would bring considerable benefits in terms of additional foreign direct investment inflows in mining and an 27 associated increase in budget revenues from additional license fees. These expectations have not been realized. During 2010–14, the issuing of new exploration licenses was on hold, waiting for the adoption of amendments to the country’s Mining Law. This legal drafting was affected by serious delays because there was no consensus among major interest groups about several fundamental revisions. Since the amended law was adopted in 2015, the licensing process has resumed, but it remains at a depressed level (approximately 30 new exploration licenses per year). 5. Efficiency 5.1 The efficiency is rated substantial. At appraisal, World Bank staff argued that a quantitative economic analysis was not justified or feasible for this project. However, at completion, the ICR provided a detailed set of quantitative estimates for project rates of return for the four largest outputs developed with GAP financing (GFMIS, PMMIS, TAIS, and CMCS). These estimates were based on bold assumptions and generated high estimates for project efficiency. As a result, efficiency was rated high in the ICR. These efficiency estimates were rejected by IEG’s validation of the ICR, which rated overall project efficiency modest because of implementation delays, cost overruns,24 and identified cases of fraud. The PPAR mission reviewed the efficiency of implementation for the four largest GAP subcomponents—GFMIS, TAIS, PMMIS, and CMCS (approximately 60 percent of the total project costs)—and identified the following findings. 5.2 GFMIS. This was the only system supported by the project (albeit the most important one) for which IEG was able to undertake quantitative efficiency analysis. The main channel of direct budget savings for GFMIS was associated with reduced government borrowing costs. The GFMIS allowed for the consolidation of the balances of various local government accounts in commercial banks. Annual budget savings through this channel were estimated based on the following assumptions: • Average daily balances in budget accounts of local governments in 2011: MNT 35 billion ($28.6 million). • Average interest rate for government domestic borrowing in 2011: 11.8 percent. • Total costs of GFMIS rollover (linking all aimags [regions] to the system) were $4.5 million, of which $1.5 million was provided under the GAP (International Development Association grant), and the government of Mongolia budget funded the rest. 28 5.3 Under the preceding assumptions, annual budget savings were estimated at $3.4 million. Therefore, the cost recovery period for investments made in the GFMIS rollover was less than 1.5 years, indicating high efficiency. Moreover, this estimate reflects only partial benefits from the new system and does not reflect other important benefits, such as (i) reduced commercial bank fees and (ii) faster and more predictable transfers of budget funds to local governments. In addition, the rapid expansion in local government expenditures after 2011 would imply considerable growth in daily cash balances. Between 2011 and 2015, annual local government spending almost doubled. This suggests even larger budget savings compared with the “no project” scenario and a higher rate of return from the consolidation of local government accounts under the GFMIS after 2011. 5.4 TAIS. The system brought considerable benefits in terms of reducing tax compliance costs, improving efficiency in tax collection, and raising transparency of tax obligations. Considerable evidence shows improved tax compliance and reduced perception of corruption in the GDT. According to the ICR, the overall cost of the TAIS was lower than the cost of equivalent systems in other countries. Although TAIS implementation experienced considerable delays, the final product has much greater scope with countrywide coverage instead of the originally planned version that was to be piloted in two GDT offices. Overall, the efficiency of TAIS implementation is rated substantial. 5.5 CMCS. The computerized system for the Mineral Resources Authority of Mongolia was introduced on time, and it delivered the expected benefits, including more transparency in the allocation of licensing rights and greater efficiency in license allocation and license fee administration. The CMCS succeeded in clarifying ownership rights with respect to major licensed plots, which is commonly seen as a major enabling condition to encourage longer-term investment. However, the system did not generate the expected increase in license revenues because of a failure to amend the country’s Mining Law. The costs of the system were reasonable. The core IT equipment and software, provided under the GAP, remained in daily use as of June 2019. The efficiency of CMCS implementation is rated substantial. 5.6 PMMIS. The implementation of the e-procurement system was less successful. Because of design problems, the system was shut down after approximately three years and replaced by a new system, funded by the government of Mongolia’s budget. However, this subcomponent was not a complete failure. For 2010–13, it was the primary government tool for public procurement planning and monitoring that helped facilitate competition within the entire government public procurement portfolio (approximately $1.2 billion a year), whereas the cost of launching the system was relatively small ($1.2 million). The efficiency of PMMIS implementation is rated modest. 29 5.7 There were several other factors that reduced the overall efficiency of GAP implementation: • Two subcomponents (public investment management and payroll management) did not generate tangible benefits. But these were quite small, with a total cost of approximately $800,000 (approximately 5 percent of total project costs). • Allocations for foreign study tours were not well justified and were excessive. • Integrity Vice Presidency investigations did not point to any specific misprocurement but suggested some inefficiencies in the organization of project tendering. 5.8 Overall, GAP implementation efficiency is rated substantial. The three main e- governance systems were implemented well, brought significant cross-cutting governance improvements, and remained fully operational 4.5 years after the project closure. Project efficiency was further boosted by cost savings from the synergies across several World Bank projects implemented in parallel. 6. Ratings Outcome 6.1 Overall outcome is rated satisfactory, a composite rating reflecting a high rating for relevance of objectives, substantial rating for relevance of design, substantial achievement of all three projects objectives (efficacy), and substantial rating for efficiency. 6.2 The project’s theory of change was solid overall, but the project was overly complex, with many subcomponents that were not adequately prioritized. This deficiency was largely mitigated through the strong leadership of the Ministry of Finance. 6.3 In terms of efficacy, GAP achieved 15 of 16 specific targets in the project’s results framework (appendix F). No backsliding has been recorded for any of these targets since GAP closure (2014). Most of the systems introduced under the GAP have been in continuous operation and appear to be sustainable. Moreover, steady progress was made in recent years through the government of Mongolia–funded upgrades of core e- governance system that were initially supported by the GAP. This was the case, for instance, for GFMIS and the Debt Management and Financial Analysis System. Two project subcomponents (public investment management, payroll management) failed to 30 gain traction, but these were relatively small and related shortcomings are considered minor. 6.4 These key project outputs contributed to important development benefits (outcomes) in terms of improved investment climate, reduced corruption, and increased government accountability. This is reflected in improved perceptions of corruption in tax administration and public procurement, expanded taxpayer registration, better public access to budget information, and increased competition in procurement. Although progress under two project subcomponents (public investment management, civil service expenditure) was limited, these subcomponents were small. In the overall assessment of the project’s outcomes, IEG gave greater weight to the most important project elements. The GAP has made a significant contribution to enhancing the core government capacity for economic management. The PPAR mission confirmed the earlier findings of IEG’s Mongolia Country Program Evaluation (2015): the core institutional structures and most of the e-governance systems that the GAP helped put in place have been working effectively and sustainably—the mechanics of budgeting improved, the treasury system and the associated GFMIS were well used, and the TAIS helped reduce tax compliance costs and improved transparency. 6.5 Efficiency is rated substantial. Core complex e-governance systems supported by the GAP were delivered mostly on schedule and within the allocated budget. Quantitative analysis of the investments made under the project in the GFMIS extension suggests a high rate of return. Benefits from the implementation of both TAIS and CMCS have been significant and exceeded expectations. Risk to Development Outcome 6.6 The overall risk to development outcome rating is moderate. At appraisal, the overall project risk was rated significant. This was the result of problematic implementation experience of several past governance projects, primarily ECBTAP, mostly owing to (i) lack of government ownership, (ii) inadequate clarity on roles and responsibilities of implementing agencies, and (iii) lack of capacity in implementing agencies. 6.7 At closure, the risk to development outcome was kept at significant. This was justified by the risk of not being able to retain technical staff, who were hired under the project to maintain complex e-governance systems after the project funding was no longer available. IEG agreed with this rating in its ICR Review. 6.8 In retrospect, this risk did not materialize. The government of Mongolia did not experience serious difficulties in retaining required technical skills,25 helped a certain degree by assistance under follow-up World Bank projects and support from other 31 development partners. The government managed to provide the necessary maintenance and training budget and funded several system upgrades. It succeeded in delivering the training and retraining programs that are especially needed in Mongolia because of frequent staff changes linked to the political cycle. Despite high staff turnover, institutional memory was preserved. 6.9 The overall risk rating for the most recent World Bank technical assistance project (P161048), approved in 2017, which continues to support the governance reform agenda in Mongolia, was moderate. This is a recognition by the World Bank that the institutional environment for capacity building interventions has improved in recent years. 6.10 This PPAR concludes that the main GAP achievements have been largely sustainable and the risk to their future sustainability is modest. This is justified by the following factors: • Sustainable operation of the large e-governance systems for at least the past four years since GAP closure. • Provision of necessary funding for maintenance, training, upgrades, and expansion of these systems by the government of Mongolia. • Demonstrated ability of the Ministry of Finance and other economic management agencies to protect key technical staff from political risks. • Availability of a considerable amount of donor support to build on the existing systems in an incremental way. • Improved general quality of dialogue on governance reforms after 2016 (according to IEG mission interviews). 6.11 There are still two major risks that, if realized, may disrupt the positive trends to gradual governance improvement. First is the macroeconomic risk related to the lack of fiscal discipline and the high level of public debt. A major budget crunch is possible, which might have a disruptive effect on core government operations. Second is the volatile political environment, which makes building a broad political consensus a daunting task. This complicates the implementation of comprehensive administrative and civil service reforms that are needed to secure the long-term sustainability of governance initiatives undertaken during the past decade. Lack of real progress with civil service reform could block further opportunities to enhance governance and even compromise some of the existing achievements. 32 Bank Performance 6.12 Overall Bank performance is rated moderately satisfactory. Quality at Entry 6.13 The quality at entry is rated satisfactory. Project design was built on several pieces of analytical work, including political economy analysis (Mongolia was a pilot country under the governance and anticorruption program), a comprehensive demand assessment for public disclosure, and diagnostics of investment climate, debt management, and tax administration. Lessons learned from previous technical assistance projects in Mongolia and experience with implementing complex government IT-based management systems in other countries were taken into account. For instance, in contrast to the ECBTAP (rated unsatisfactory by IEG) and Fiscal Accounting Technical Assistance Project,26 the GAP’s design better reflected existing political and local implementation capacity constraints in Mongolia and did not include the objectives that proved to be unrealistic under the earlier projects (introduction of a Medium-Term Expenditure Framework, comprehensive reform of social transfers, and adoption of new civil service rules). Still, GAP design was overly ambitious (see the Relevance of Design section). Similar to earlier technical assistance projects, the GAP’s coverage was too broad, and the timetable for implementing several subcomponents (primarily, TAIS) proved to be unrealistic. 6.14 The overall project risk was rated substantial, which in light of experience with earlier similar projects, was appropriate. The World Bank identified several relevant risks such as insufficient government ownership, inadequate clarity on roles and responsibilities of implementing agencies, insufficient capacity in implementing agencies, and difficulty in maintaining trained personnel. The proposed mitigation efforts were broadly adequate. Risk analysis at approval was focused primarily on capacity constraints and understated political risk. At appraisal, there was no explicit mention of the risk of Mongolia becoming a resource-rich and highly resource- dependent country in the short term. The commodity boom of the late 2000s and early 2010s weakened the government of Mongolia’s incentives to pursue governance reforms. This risk was not anticipated at the time of project preparation, and the World Bank could not realistically mitigate it through the GAP. Quality of Supervision 6.15 The quality of supervision rating is moderately satisfactory. GAP supervision was undertaken simultaneously with the supervision of the related projects (ECBTAP and the Multisector Technical Assistance Project) by a joint team. ISRs were prepared and filed at least once every year, and twice a year in 2010, 2013, and 2014. A midterm 33 review was carried out in November 2009 but did not bring about any significant changes. The midterm review did not identify the main obstacles to project implementation, in particular, for the tax administration subcomponent. 6.16 In general, the quality and volume of implementation support and dedication shown by World Bank staff were appreciated by government counterparts. The team succeeded in maintaining productive engagement with the Ministry of Finance throughout the entire project, despite numerous key personnel changes on the client’s side. The availability of very capable national staff in the country office helped maintain effective working relationships with key counterparts. The World Bank showed a lot of flexibility during implementation by agreeing, for instance, to use a large number of local consultants to augment government implementation capacity and to increase the training budget to accommodate massive training needs (especially for remote users). The World Bank’s flexibility in implementation was also reflected in the complementary way in which GAP funds were mixed with funding available under other World Bank projects to accelerate project implementation and support scaling-up efforts. This was seen in GAP synergies with the ECBTAP and MSISTAP, as well as with an Institutional Development Fund grant. 6.17 At the same time, supervision requirements for the GAP were relatively high. The project had multisectoral coverage, which provided for the design and procurement of several large IT systems for various government agencies that did not have enough experience in handling projects of such complexity. In this situation, the normal supervision budget available for the GAP was inadequate. Although some cost savings were realized during implementation owing to higher reliance on the country-based staff and synergy in supervision of several ongoing technical assistance projects in the portfolio, the World Bank could not afford to bring in sufficient specialized headquarters-based technical expertise. This reduced the effectiveness of World Bank supervision in several cases, including in the design of the PMMIS and formulation of requirements for the GFMIS contractor (when the World Bank did not insist that the contractor had to have a permanent presence in Mongolia during implementation). 6.18 The project ratings produced by the team in ISRs showed a lack of realism. Ratings were kept at the satisfactory level for an extended period. The first downgrade to moderately satisfactory was made only in April 2012, after the first extension of the project, despite well-known problems with TAIS implementation since 2010. Also, the financial management rating was kept satisfactory through the entire life of the project despite several Integrity Vice Presidency investigations. 6.19 There was also a lack of proactivity in addressing weaknesses in the results framework, which happened only during the latest project restructuring in May 2014 34 (six months before project closure) but still did not resolve all the problems with the project monitoring and evaluation system. The midterm review could have been used to address the shortcomings of the original results framework. Borrower Performance 6.20 Overall borrower performance is rated moderately satisfactory. Government Performance 6.21 The rating of government performance is moderately satisfactory. The government of Mongolia’s ownership of reform remained uneven, with several incidences of backsliding, which complicated project implementation. The commodity boom of the late 2000s and early 2010s weakened incentives to pursue governance reforms. For several years, off-budget operations of the Development Bank of Mongolia undermined progress in strengthening public procurement.27 Long delays in the adoption of the new Mining Law reduced the benefits associated with the introduction of the CMCS. Frequent changes to the legal framework for public procurement complicated and increased the costs of the introduction of the e-procurement system. Another example of costly policy reversal relates to the elimination and then reestablishment of the Training and Advisory Center at the Civil Service Council. 6.22 Turnover of senior government officials became a significant problem for project implementation in several instances, causing delays with decision-making and procurement clearances. This was the case, for example, with respect to the introduction of an IT-based licensing management system in the mining cadastre: many key staff in both the Mineral Resources and Petroleum Authority of Mongolia and cadastre office were replaced after the 2008 elections. 6.23 Many of these problems reflected fundamental weaknesses in Mongolia’s political economy and underlying institutions. Major unresolved issues in the areas of public investment management and civil service reform could not be addressed without a broader political consensus including a better-defined role for the parliament to intervene in the implementation of economic policy. 6.24 On the positive side, despite being led by six different ministers of finance over the GAP’s lifetime, the Ministry of Finance exercised strong leadership in project implementation. The government of Mongolia also showed a strong commitment to the sustainability of both the GFMIS and TAIS by providing significant state budget financing (approximately $7.5 million) for a quick and efficient countrywide rollout of these systems after the initial GAP-funded pilots. The implementing agencies demonstrated their capacity and political will to operate the new systems in a 35 sustainable way after project closure. Both the Ministry of Finance and GDT succeeded in retaining key technical staff critical for sustainability of the GAP-supported systems. 6.25 Moreover, after project closure, the government of Mongolia implemented additional reforms and system upgrades that helped enhance the sustainability of GAP outcomes and expand project benefits. These actions included a GFMIS upgrade in 2017; harmonized economic classification across the budget and reporting systems; a revised Tax Package Law (2019) aimed at simplifying the tax regime, reducing the frequency of tax reporting and enhancing accountability mechanism in tax administration; a CMCS upgrade (2019); an upgrade of the IAAC’s management system; revised Statistics Law and strengthened mandate of the NSO to disseminate poverty data; and amendments to the Law on Auditing (2016). 6.26 Although the Project Steering Committee was duly established, there was little evidence that its meetings were held regularly, as the project was de facto run by the Ministry of Finance. Implementing Agency Performance 6.27 The rating of implementing agency performance is satisfactory. Fourteen subcomponents and eight implementing agencies introduced significant implementation risks for the project, but these risks were managed relatively well through strong leadership by the Ministry of Finance and effective coordination by the PCU. The PCU had accumulated considerable experience implementing earlier similar World Bank– supported technical assistance projects. This experience was used effectively, as the PCU proved to be capable of providing implementing agencies with day-to-day support. Most implementation issues were addressed in a timely manner; procurement and disbursement delays were modest on average. The PCU also performed well in collecting information from implementing agencies. The response of the Ministry of Finance and PCU to the Integrity Vice Presidency’s findings was timely and adequate. Monitoring and Evaluation 6.28 The rating of monitoring and evaluation is modest. Design 6.29 The original results framework was quite weak. Annex 3 of the PAD presented the project’s results framework, which included three PDO-level results indicators and 12 intermediate results indicators. However, most indicators, especially those at the PDO level, were qualitative and did not have quantitative baselines and targets. As was found during implementation, one of the PDO indicators (costs of compliance for business taxpayers) was impossible to monitor owing to a lack of data and had to be 36 replaced. Moreover, the original set of indicators showed gaps in coverage. For instance, there was no indicator to monitor the establishment of an automated TAIS despite this being the largest activity funded under the GAP (approximately 30 percent of total allocation). 6.30 The original results framework was never used. From early implementation, the team used simpler, more practical, and generally more adequate indicators to monitor progress under the project. This included three PDO-level indicators (which were similar to the original PDO indicators but more precisely defined) and eight intermediate results indicators. Because several of these had multiple targets, the project’s results framework had 16 specific targets corresponding to 11 indicators. 6.31 The mismatch between the results framework in the PAD and the actual results framework was officially reconciled through a retrospective restructuring in 2014, only six months before the project closure. The results framework restructuring included two changes. First, the results framework that was used for project monitoring was made the official results framework for the project. Second, one of the PDO-level indicators was replaced by a new one to track developments of TAIS. 6.32 Changes introduced to the original results framework can be summarized as follows: (i) four original intermediate indicators were dropped, (ii) three indicators were either significantly restructured or replaced, and (iii) eight were continued, although several of them were clarified to make them more measurable and relevant to project activities. The final results framework is presented in appendix F. Some of the changes were motivated by a desire to streamline the results framework, but others were more substantive. Although results framework restructuring helped strengthen the project monitoring and evaluation arrangements considerably, it did not address all the weaknesses in the original framework. For instance, the baseline was still missing in several cases (for example, access to budget information, B5, and competition in procurement, B6). The TAIS indicator was part of the results framework only for the last six months of the project life. 6.33 Several important project outcomes were not reflected in the results framework. These include growth in taxpayer registration and use of poverty data in the administration of social assistance. A separate design weakness was the absence of a mechanism to assess the effectiveness and efficiency of training programs funded by the GAP. Implementation 6.34 Monitoring and evaluation implementation was conducted by the PCU, which gathered necessary information through its regular communication with implementing 37 agencies. The monitoring and evaluation system was finalized during the early implementation and formally restructured in 2014, as discussed previously. Most indicators were quite simple, and collecting data was generally nonproblematic. The only exception was the Corruption Perception Index, which was survey-based and supposed to be regularly updated by the IAAC. The IAAC’s decision to change the Corruption Perception Index methodology made it impossible to monitor this indicator after 2013. Although this indicator was part of the results framework approved in the course of the 2014 restructuring, it was not presented in the 2015 ICR and was dropped without due discussion. Use 6.35 The data collected were used in the World Bank’s supervision reports (ISRs) and the project ICR. The monitoring and evaluation framework presented and discussed in the ICR had some discrepancies with the framework officially adopted by the 2014 restructuring paper. However, these discrepancies did not have a material impact on the ICR’s ratings. No effort was made during project supervision (and later in the ICR) to conduct a formal assessment of the overall training effort, including analysis of retaining the trained staff and their job responsibilities. This was a missed opportunity to learn more about the actual project impact on staff development and capacity strengthening. 7. Lessons 7.1 This PPAR suggests the following lessons: • In a low-capacity environment, major benefits can be realized from relying on base-level technical solutions and incremental step-by-step reform strategies. The implementation of large, modern, integrated IT-based systems often fails because they exceed local absorptive capacity. The GAP strategy of first introducing stand-alone, relatively simple e-governance solutions for fiscal and economic management has demonstrated effectiveness and sustainability. • Several large and comprehensive e-government solutions can be introduced gradually, even in a low-capacity environment, if there are enabling factors in place, such as strong client ownership, good fiberoptic infrastructure, adequately trained local IT staff, and a sufficient budget for local training and technical support, especially for remote users. • Implementation risks related to the project’s complexity and multiple government implementing agencies can be successfully managed if there is strong leadership from the core agency (such as the Ministry of Finance) and an experienced PCU that is empowered to make day-to-day decisions and impose 38 project discipline across participating agencies. In the case of the GAP, the leadership of the Ministry of Finance and the robust operations of the PCU made a significant contribution to successful project outcomes. • Technical assistance projects with multisectoral coverage require significant supervision support from the World Bank and a larger than normal supervision budget. Lack of budget can limit the ability of the World Bank to provide the specialized technical support needed to help the client make better design and implementation choices. In the case of the GAP, with more resources, the World Bank might have provided better advice in several cases related to the implementation of large IT systems (design of PMMIS, contracting requirements for GFMIS). • Effective use of local talent and expanding local partnerships in project implementation can be beneficial for project outcomes. The GAP was quite successful in its mobilization and flexible use of local consultants. At the same time, a stronger partnership with local civil society organizations (especially under the accountability component) could have helped the government of Mongolia better use the newly built e-governance infrastructure. • In technical assistance projects with a considerable training component, more attention to formal assessments of efficiency and effectiveness of project investment in training might be appropriate. In the case of the GAP, lack of respective indicators in the monitoring and evaluation framework prevented learning more about the actual impact of project training on capacity development. 1 The project appraisal document also contained a contingency provision for $600,000 (see table D.2; World Bank 2006b). 2The Fiscal Accounting Technical Assistance Project (P051855, approved in 1998, closed on September 30, 2005) and Economic Capacity Building Technical Assistance Project (ECBTAP, P077778, approved in 2003, closed in 2013) preceded the approval of the Governance Assistance Project (GAP) in 2006; the Mining Sector Institutional Strengthening Technical Assistance Project (P108768, 2008–15) and Multisector Technical Assistance Project (P119825, 2010–17) followed the GAP and were intended to complement its activities. 39 3 Most technical assistance to support the development policy operation program was delivered under other technical assistance operations (the Economic Capacity Building Technical Assistance Project and the Multisector Technical Assistance Project). 4 Two Interim Strategy Notes for Mongolia also covered FY09–12. 5A considerable share of project funds ($2.7 million or 18 percent) was spent on training, including foreign study tours. See table D.6 for more details on the training cost. 6 Appendix F presents the full list of project targets and provides evidence of their achievement. 7 In the preceding period, at the time of the operations of the Ministry of Economic Development (2012–15), the government’s responsibility for issuing public debt was split and the central overview mechanism was weak, which resulted in a significant deterioration of debt profile during that period. With the dissolution of the Ministry of Economic Development in early 2015, the Ministry of Finance regained the sole authority on debt matters. 8The conservative estimate for interest cost savings is $3.4 million a year. This is a significant amount, given that the total costs of the Government Financial Management Information System rollover was $4.5 million. See section 4, Efficiency, for details. 9 World Bank. 2016. “Mongolia Governance and Public Sector Management Mission. ” Back to Office Report April 10–21, 2016, World Bank, Washington, DC. 10In December 2015, the Independent Authority against Corruption changed its survey methodology, consolidated its two survey instruments (the Mongolia Corruption Perception Index survey and Integrity Assessment of Public Institutions), and stopped producing Corruption Perception Index data in the original format. The new survey format does not provide for any nationwide consolidated measure of corruption perception in Mongolia, focusing instead on the performance trends at the individual agency level. In fact, the Corruption Perception Index data are not available after 2013, which makes it impossible to properly assess progress against one of the GAP targets (Corruption Perception Index was supposed to improve from 0.65 to 0.80). 11 Although the respondents remained unsatisfied with government anticorruption actions in general, public perceptions have improved among both households and private businesses over the long term. For instance, the number of respondents who agreed that some level of corruption is acceptable declined from 37 percent in 2006 to 34 percent in 2014 and 23 percent in 2018, a drop of nearly 14 percentage points. In 2018, 12 percent of respondents believed corruption to be the most important problem in Mongolia and 21 percent held this view in 2006 (Asia Foundation, 2018b). 12 According to the Organisation for Economic Co-operation and Development and various local observers, the Independent Authority against Corruption still lacks independence, resources, and necessary support from other agencies to fully exercise its mandate. Its verification mechanisms remain underdeveloped, leaving considerable room for corrupt officials to “game the system.” 13 See the official website for the Ministry of Finance: https://mof.gov.mn/en/ 40 14Information related to the implementation of the Glass Account Law could be found at http://www.smart.gov.mn/en/ 15 There was no baseline for this indicator because the 2015 Public Expnditure and Financial Accountability (PEFA) was the first undertaken in Mongolia. However, there was a shared understanding that at the time of GAP preparation, public access to budget data was limited (which was reflected in the very low Open Budget score). 16 For more information see https://www.internationalbudget.org/open-budget-survey/results-by- country/country-info/?country=mn#transparency. 17 See the website for the National Statistical Agency www.1212.mn 18 However, some of these legal changes were partially reversed after the change of government in 2016: the Government Procurement Agency became part of the State Property Policy Regulation Agency and the Procurement Policy Department in the Ministry of Finance was integrated as a division under the Ministry of Finance’s Legal Department. Furthermore, in 2019 the Public Procurement Law was amended again, and the Government Procurement Agency was reestablished as an independent agency. These numerous legal changes have brought considerable instability to the procurement regulatory framework and slowed down institutional development. 19 Access to the public procurement portal is through http://pmmis.e-procurement.mn 20Currently, the government of Mongolia operates e-procurement portal (http://www.tender.gov.mn), which was launched in December 2016. This e-procurement system covers the whole procurement process, from the publication of a procurement notice to a contract award. The revised Public Procurement Law (2019), as well as the country’s Anticorruption Strategy, made the use of the e-procurement system mandatory. 21 According to the PEFA, the regulatory framework for budget procurement is generally considered comprehensive and transparent. 22 The currently used e-procurement system seems to be quite successful and, compared with the Procurement Management and Monitoring Information System, expanded benefits for users in both the public and private sectors, but it was not supported by the GAP. 23 The cadastre information is available from the website https://www.mrpam.gov.mn 24 The Project Performance Assessment Report (PPAR) mission did not find any evidence of serious cost overruns. 25The PPAR mission’s interviews with the implementing agencies provided consi derable evidence that a significant proportion of the staff, who participated in the GAP’s foreign training, are still working in senior technical positions in their respective agencies. This was true not just for the Ministry of Finance (which generally has much less staff turnover) but also for the General Department of Taxation, National Statistical Office, and Mineral Resources Authority of Mongolia. 41 26 The Independent Evaluation Group review of that Fiscal Accounting Technical Assistance Project Implementation Completion and Results Report (prepared in 2006) highlighted its unsatisfactory quality at entry, in part because of (i) inadequate assessment of local capacity and (ii) an unrealistic timetable and overoptimistic assumptions concerning the time frame for reform implementation. 27 In March 2019, the parliament adopted additional amendments to the Public Procurement Law that finally brought Development Bank of Mongolia investments under the law. 42 Bibliography Asian Development Bank. 2017. Mongolia. Procurement Risk Assessment. Project 41254-012. ADB, Manila. Asia Foundation. 2018a. Study of Private Sector Perceptions of Corruption. San Francisco, CA: Asia Foundation. ———. 2018b. Survey of Perceptions and Knowledge of Corruption. San Francisco, CA: Asia Foundation. Hashim, Ali, and Moritz Piatti-Fünfkirchen. 2018. “Lessons from Reforming Financial Management Information Systems.” Working Paper 8312, World Bank, Washington, DC. IMF (International Monetary Fund). 2012. “Mongolia: Second Post-Program Monitoring Discussions.” Mongolia IMF Country Report 12/52. Washington, DC: International Monetary Fund. ———. 2018a. “Fifth Review Under the Extended Fund Facility Arrangement and Request for Modification and Waiver of Applicability of Performance Criteria.” Staff Report. Washington, DC: International Monetary Fund. ———. 2018b. “Mongolia: Improving the Efficiency of Public Investment Spending.” Technical Assistance Report, February. International Monetary Fund, Washington, DC. ———. 2018c. “Mongolia: Strengthening Financial and Statistical Reporting Systems.” Technical Assistance Report, December. International Monetary Fund, Washington, DC. Ministry of Finance of Mongolia. 2015. Project Completion Report for Governance Assistance Project (P098426). Ulaanbaatar. Mongolia Extractive Industries Transparency Initiative. 2018. Mongolia Twelfth EITI Reconciliation Report 2017. Ulaanbaatar, Mongolia: Mongolia Extractive Industries Transparency Initiative Secretariat. OECD (Organisation for Economic Co-operation and Development). 2019. Anti-Corruption Reforms in Mongolia: Fourth Round of Monitoring of the Istanbul Anti-Corruption Action Plan. Paris: OECD. USAID (United States Agency for International Development). 2014. “Survey on Perceptions and Knowledge of Corruption.” USAID Strengthening Transparency and Governance in Mongolia Program. World Bank. 2004. Country Assistance Strategy for Mongolia, FY05–08. Washington, DC: World Bank. 43 ———. 2006a. “Mongolia: Fiscal Accounting Technical Assistance.” Independent Evaluation Group, Implementation Completion and Results Report Review ICCRR12469, World Bank, Washington, DC. ———. 2006b. “Project Appraisal Document on a Proposed Grant to Mongolia for a Governance Assistance Project.” Report No: 35212-MN, Washington, DC: World Bank. ———. 2012a. Mongolia: Country Partnership Strategy for the Period FY2013 –2017. Washington, DC: World Bank. ———. 2012b. “Review of the 2005–2008 Mongolia Country Assistance Strategy Completion Report and of the 2009–2011 Interim Strategy Note Completion Report.” World Bank, Washington, DC. ———. 2013. “Restructuring Paper on a Proposed Project Restructuring of Governance Assistance Project.” Grant H222-MOG. Report No: 83127. World Bank, Washington, DC. ———. 2014a. “Mongolia: Economic Capacity Building Technical Assistance.” Independent Evaluation Group, Implementation Completion and Results Report Review ICRR14308, World Bank, Washington, DC. ———. 2014b. Mongolia: Country Program Evaluation FY05–13. Independent Evaluation Group. Washington, DC: World Bank. ———. 2015a. “Implementation Completion and Results Report on a Grant to the Government of Mongolia for a Governance Assistance Project.” Report No: ICR00003480. Washington, DC: World Bank. ———. 2015b. “Mongolia: Public Financial Management Performance Report. ” Washington, DC: World Bank. ———. 2015c. “Mongolia: Governance Assistance Project.” Independent Evaluation Group, Implementation Completion and Results Report Review ICCRR14884. World Bank, Washington, DC. ———.2016. “Mongolia: Performance and Learning Review of the Country Partnership Strategy for the Period FY2013–2017.” Washington, DC: World Bank. ———. 2017. “Mongolia: Mining Sector Institutional Strengthening TA Project.” Independent Evaluation Group, Implementation Completion and Results Report Review ICRR0020708. World Bank, Washington, DC. ———. 2018a. “Multi-Sector Technical Assistance Project.” Independent Evaluation Group, Implementation Completion and Results Report Review ICRR0021216. World Bank, Washington, DC. ———. 2018b. “Mongolia: Growing without Undue Borrowing. Enhancing Efficiency of Spending and Revenue.” World Bank, Washington, DC 44 Appendix A. Basic Data Sheet Governance Assistance Project (IDA H2220, P098426) Table A.1. Key Project Data Actual or Current Appraisal Estimate Estimate Actual as Percent of Financing ($, millions) ($, millions) Appraisal Estimate Total project costs 14.00 14.98 107 IDA grant amount 14.00 14.98 107 Cofinancing — — — Note: IDA = International Development Association. Table A.2. Cumulative Estimated and Actual Disbursements Disbursements FY07 FY08 FY09 FY10 FY11 FY12 FY13 FY14 FY15 Appraisal estimate ($, 0.30 3.30 7.30 11.0 13.00 14.00 - - - millions) Actual ($, millions) 1.33 4.39 7.50 9.93 11.50 12.60 13.18 13.30 14.98 Actual as percent of 443 133 103 90.3 88.5 90.0 - - - appraisal Note: Date of final disbursement was 12/31/2014. Table A.3. Project Dates Event Original Actual Concept review - October 6, 2005 Board approval - May 23, 2006 Effectiveness - July 21, 2006 Closing date January 15, 2012 December 31, 2014 45 Table A.4. Staff Time and Cost World Bank Budget Only Staff time Costa Stage of Project Cycle (no. weeks) ($, thousands) Lending FY08 78.6 414.5 FY09 0.3 1.3 Total 78.9 415.9 Supervision or ICR FY08 23.3 117.7 FY09 15.5 92.5 FY10 14.5 104.8 FY11 13.0 77.9 FY12 9.0 95.2 FY13 11.7 105.3 FY14 11.8 126.5 FY15 3.6 36.9 Total 102.3 756.9 Note: FY = fiscal year; ICR = Implementation Completion and Results Report. a. Including travel and consultant costs. Table A.5. Task Team Members Name Titlea Unit Lending Charles Abelmann Lead Program Coordinator LLILM James Anderson Country Manager EACMF Maria Caridad Araujo Economist EASHS Genevieve F. Boyreau Senior Economist SACBT Giovanna Dore Environmental Specialist EASRE Carlos Ricardo Escudero Consultant LEGLA Christopher Finch Senior Social Development Specialist GSURR Chorching Goh Lead Economist GMFDR Sudarshan Gooptu Lead Economist GMFDR Sameer Goyal Sr Financial Sector Specialist GFMDR Charles A. Husband Consultant GEEDR David I Sr Financial Management Specialist GGODR Xiaoping Li Senior Procurement Specialist GGODR Yi-Ling Liu Finance Analyst WFAFO Stephen N. Ndegwa Manager, Operations SACKB Lhagvasuren Ochir Operations Officer EACMF 46 Name Titlea Unit Altantsetseg Shiilegmaa Economist GMFDR Gerelgua Tserendagva Procurement Specialist GGODR Anthony J. Whitten Sr Biodiversity Specialist AFTN1—HIS L. S. Christine Wong Shui Wan Operations Officer GGODR Supervision or ICR Genevieve F. Boyreau Senior Economist SACBT Yi Dong Sr Financial Management Specialist GGODR Giovanna Dore Environmental Specialist EASRE—HIS Michael Engelschalk Senior Private Sector Development GTCDR Chorching Goh Lead Economist GMFDR Jianjun Guo Senior Procurement Specialist GGODR Xiaowei Guo Senior Procurement Specialist GGODR Ronnie W. Hammad Senior Operations Officer GPSOS/GGODR Graeme Eric Hancock Senior Energy Specialist, Mining SEGOM—HIS Zahid Hasnain Sr Public Sector Specialist GGODR Charles A. Husband Consultant GEEDR David I Sr Financial Management Specialist GGODR Lars Jessen Lead Financial Officer FABDM Michael S. Kubzansky Program Manager FPDVP—HIS Tuan Minh Le Senior Economist GMFDR Xiaoping Li Senior Procurement Specialist GGODR Angkanee Luangpenthong Program Assistant EACTF Lhagvasuren Ochir Operations Officer EACMF Seda Pahlavooni e-Government Specialist GTIDR Jinan Shi Senior Procurement Specialist GGODR Altantsetseg Shiilegmaa Economist GMFDR Marinus Verhoeven Lead Economist GGODR L. S. Christine Wong Shui Wan Operations Officer GGODR Note: ICR = Implementation Completion and Results Report. a. At time of appraisal and at time closure, respectively. 47 Appendix B. Governance Assistance Project Implementation Status and Results Reports Table B.1. Review of Governance Assistance Project Implementation Status and Results Reports ISR Sequence No. Archived Date Remarks August 15, 2006 Initial ISR. Not many details, except for noting that key personnel are in place and implementation of certain components is under 1 way. February 24, 2011 Format changed from Sequence 1 ISR. Well-detailed. The only issue raised is the slower progress on tax administration reform 7 and computerization. April 5, 2012 Well-detailed. The first ISR to have lower than satisfactory overall ratings. Notes that TAIS is the last outstanding subproject, which 8 has been facing some difficulties and delays. March 6, 2013 Further downgraded the project rating for implementation 9 progress (to MU). October 22, 2013 Further downgraded the project rating for progress toward 10 achievement of PDO (to MU). April 11, 2014 Upgrades overall ratings from MU to MS. Notes again the delays 11 and difficulties in TAIS rollout. December 17, 2014 New ISR format. Well-detailed. Overall implementation progress rating upgraded to satisfactory. Notes some shortcomings in the subcomponents related to human resource expenditure management and debt management, in part owing to factors external to, and beyond the control of, the project. 12 Source: World Bank Operations Portal. Note: Implementation Status and Results Reports (ISRs) Sequences 2–6 are not available from the World Bank system. MS = moderately satisfactory; MU = moderately unsatisfactory; PDO = project development objective; TAIS = Tax Administration Information System. 48 Appendix C. Prior Actions Related to Technical Assistance Table C.1. Development Policy Operation Prior Actions Related to Technical Assistance Funded under the Governance Assistance Project DPL Prior Action Fiscal policy and management DPL 1 Improve budget planning and screening, as demonstrated by the exclusion of 30 projects without feasibility studies from the 2009 budget, as amended in March 2009. DPL 1 Protect capital expenditures for the maintenance of basic infrastructure in the transport, education, and health sectors, as evidenced by (i) the increase in the ratio of capital repairs to new investments from 2.5 percent (in the original approved budget for 2009) to 3.4 percent (in the amended budget for 2009) and (ii) the allocation of MNT 12.4 billion for that purpose. DPL 2 The recipient has amended the FY10 budget to improve the public investment plan by reducing by 197 (amounting to MNT 40 billion) the number of projects that (i) do not have the necessary feasibility studies (technical drawings and accurate cost estimates) and (ii) do not meet national priorities and account for the contingent liabilities from contractor-funded projects by specifying these in the amended FY10 budget. DPL 2 The recipient has prioritized the level of capital expenditures for maintenance of basic infrastructure (including the Road Fund), as evidenced by (i) the increase in the ratio of capital repairs to new investments from 6.1 percent (in the FY09 revised budget) to 6.5 percent (in the FY10 Budget) and (ii) the allocation of MNT 28.2 billion. Social protection DPL 1 Prepare (Ministry of Social Welfare and Labor) a time-bound action plan to clean its roster of beneficiaries of the Child Money Programme, to reduce leakage to the nonpoor and improve its targeting on the poor. DPL 2 The National Statistical Office has approved the Proxy Means Test methodology as the official targeting mechanism for the provision of social welfare benefits in Mongolia. DPL 2 The Ministry of Social Welfare and Labor has completed a proposal defining the parameters of the new poverty-targeted benefit program and has submitted said proposal to the Cabinet for approval. Mining sector DPL 1 Develop and adopt (Ministry of Mineral Resources and Energy) model investment agreements for the mining sector, clarifying Mongolia’s mining policy, including taxation and the state’s capital participation, and incorporating responsible mining practices consistent with the Equator Principles. DPL 1 Transfer sufficient resources to the Mongolia EITI Secretariat to finance an independent validation of EITI compliance. 49 DPL Prior Action DPL 2 The recipient has revised the regulation on operations and processes of the mining cadastre. DPL 2 The recipient has submitted the documentation required by the International EITI Board to assess the compliance of the recipient’s national EITI program. DPL 2 The recipient has placed a searchable database online of Environmental Impact Assessments for mining and other projects. Source: World Bank. Note: Refers to DPL1 (P115737) and DPL2 (P117421) approved in 2009 and 2010, respectively. DPL = development policy loan; EITI = Extractive Industry Transparency International; MNT = Mongolian tughrik. 50 Appendix D. Detailed Project Data Table D.1. Key Dates for the Mongolia Governance Assistance Project Event Date Approval May 23, 2006 Effectiveness July 21, 2006 Midterm Reviews May 11, 2010 Closing Original January 15, 2012 First extension December 12, 2011 Second extension December 17, 2013 Closing (actual) December 31, 2014 Table D.2. Project Costs by Component and Financing Source Appraisal Estimate Actual Cost Percentage of Cost Source ($, millions) ($, millions) Appraisal By component A. Management of public finance 3.59 4.34 121 B. Fostering public accountability and 1.52 1.99 131 monitoring C. Investment climate 7.51 7.62 101 D. Project coordination and evaluation 0.78 1.03 132 Contingency 0.60 — — Total baseline cost 14.00 14.98 107 By financing source Borrower 0.00 0.00 — IDA grant 14.00 14.98 107 Sources: World Bank (2006b), Ministry of Finance of Mongolia (2015). Note: IDA = International Development Association. 51 Table D.3. Project Costs by Implementing Agency Cost, Share of Total Agency ($, thousands) (percent) Ministry of Finance 5,313.7 35.5 General Department of Taxation 4,614.6 30.8 Civil Service Council 621.5 4.1 Independent Agency against Corruption 757.6 5.1 Mongolian Institute of Certified Public Accounting 124.9 0.8 National Statistical Commission 878.6 5.9 Mineral Resource Authority 1,640.6 10.9 Project Implementation Unit 1,032.9 6.9 Total 14,984.4 100 Source: GAP Completion report by the Project Implementation Unit, January 2015. Table D.4. Project Costs by Procurement Category (SDR, millions) First Second Appraisal Restructuring Restructuring Actual May Description Estimate 2006 December 2011 December 2013 2015 Goods 3.82 3.62 3.74 3.78 Consultancy services 4.14 4.23 4.07 4.01 Training and workshops 1.56 1.71 1.74 1.74 Incremental operating costs 0.21 0.18 0.19 0.19 Designated account −0.02 Total disbursed 9.74 9.74 9.74 9.71 Unused funds 0.035 Total grant 9.74 9.774 9.774 9.774 Sources: Ministry of Finance of Mongolia (2015); World Bank (2006b, 2011, 2013). Note: SDR = special drawing rights. 52 Table D.5. Project Training Cost by Component Training cost Training as Total Cost ($) Percentage of Components ($) Total Local Foreign Total Cost A. Management of public finance A1. Debt management and PIP A1.1 Debt management 740,302 299,269 28,512 270,757 40.4 A1.2 PIP 400,335 52,997 13,013 39,984 13.2 A1.3 Budget legal 146,792 0 0 0 — framework A2. Decentralizing financial 1,561,970 313,200 74,503 238,697 20.1 management A3. Improving human resource expenditure management A3.1 Wage bill modeling 398,051 35,089 1,971 33,118 8.8 A3.2 Effective CSC 621,541 105,892 70,533 35,359 17.0 A4. Auditing, accounting, and quality assurance system A4.1 Asset valuation 347,376 156,275 152,775 3,500 45.0 A4.2 Quality assurance 124,868 63,401 46,404 16,997 50.8 system B. Fostering public accountability B1. Improving the 757,620 254,488 140,570 113,918 33.6 anticorruption framework B3. Capacity building for data 878,610 151,678 85,593 66,086 17.3 transparency and data access for poverty B4. Upgrading MOF website to 353,849 91,421 32,550 58,871 25.8 a dynamic portal C. Investment climate C1. Tax administration 4,614,561 403,067 170,817 232,240 8.7 C2. Public procurement 1,365,011 498,012 94,117 403,895 36.5 C3. Mining registry and cadastre 1,640,577 155,371 11,848 143,522 9.5 system D. Project management 1,032,931 105,846 9,525 96,322 10.2 Total 14,984,394 2,686,007 932,731 1,753,276 17.9 Source: Ministry of Finance of Mongolia (2015.) Note: CSC = Civil Service Council; MOF = Ministry of Finance; PIP = public investment program. 53 Appendix E. Relevant Public Expenditure and Financial Accountability Findings, 2015 The Public Expenditure and Financial Accountability (PEFA) report (World Bank 2015b) notes that Mongolia’s annual budget package meets eight of nine documentary requirements for public disclosure of fiscal information. This has been steadily increasing since 2006. The PEFA also points to the good progress in tax administration, particularly with regard to the transparency in taxpayer obligations and tax collections. However, it also mentions weaknesses in taxpayer registration arising from the multiple poorly integrated systems, increased use of ad hoc audits, and a weak regulatory framework for dispute resolution. The absence of a medium-term perspective in budgeting, particularly with regard to the rapidly increasing capital spending (both on- and off-budget), and weak coordination between the Ministries of Finance and Economic Development in the preparation of the recurrent and capital budgets, respectively, compromises the strategic allocation of resources, as exemplified by the absence of costing of future recurrent expenditures for capital projects. The accounting, recording, and reporting practices were reported as generally strong. Good progress was made in the functioning of the treasury single account and in imposing the expenditure limits exercised through the Government Financial Management Information System (GFMIS), with the notable exception of payroll controls, which were weak. The accounting basis was consistently disclosed. However, reporting was not in full compliance with the international public sector accounting standards. The GFMIS was not used to prepare financial reports, which were prepared using desktop systems instead, resulting in task duplication and inefficiencies. Thanks to the regularity of the annual budget process, and the functional treasury single account and GFMIS, schools and basic health units received resources on time and in keeping with the budget plan. This information is accurately accounted for and audited, and made publicly available, representing a significant achievement of Mongolia’s public financial management reforms over the past decade. The regulatory framework for public procurement was viewed as generally comprehensive and transparent, with open competitive procurement as the default method of procurement coupled with adequate public disclosure. However, at the time of PEFA preparation, this framework did not apply to projects financed by the Development Bank of Mongolia, which undermined the comprehensiveness and transparency of the procurement system. Another area of weakness highlighted was the 54 lack of representation from the private sector and civil society on the two government procurement complaints bodies. Table E.1. Relevant Scores from the 2015 Mongolia Public Expenditure and Financial Accountability Report Topic Score PI-10. Public access to key fiscal information A PI-13. Transparency of taxpayers’ obligations and liabilities B PI-14. Effectiveness of measures for taxpayer registration and tax assessment C+ PI-15. Effectiveness of tax collection C+ PI-17. Recording and management of cash balances, debt, and guarantees C PI-19. Competition, value for money and controls in procurements D+ PI-22. Timeliness and regularity of accounts reconciliation B+ PI-23. Availability of information on resources received by service delivery units A PI-24. Quality and timeliness of in-year budget reports C+ PI-25. Quality and timeliness of annual financial statement C+ Source: World Bank 2015b. Note: PEFA = Public Expenditure and Financial Accountability; PI = PEFA indicator. 55 Appendix F. Results Framework Table F.1. Results Framework Objective and Most Recent Component Indicator Baseline, 2006a Target Values, 2014b GAP Closure, 2015c Information, 2019d 1. Improve the efficiency and effectiveness of governance processes in the management of its public finances A1.1. Debt Improved efficiency There is no formal debt A public debt strategy The Debt Management The latest medium-term management and effectiveness of management strategy in is developed, adopted, Law was adopted in debt management A1.3. Budget legal management of public place and no and used as a critical February 2015; it strategy update was framework finance framework as comprehensive tool for the government includes a requirement approved in May 2019 evidenced by more monitoring of debt to make sound fiscal to produce a debt and provided a efficient monitoring of situation. policy decisions; management strategy. framework for debt the debt situation comprehensive The first medium-term policy decisions. The based on the adoption monitoring of the debt debt management DMFAS remains fully of the debt strategy. management to ensure strategy for 2007–09 operational and is used sustainability. was developed and to produce quarterly adopted by finance reports on public debt. minister order no. 144 The system of public on May 24, 2007, and debt reporting is more updated every two comprehensive because years. The Miinistry of it now reflects the Finance produces publicly guaranteed regular comprehensive debt issued by the reports on government Development Bank of and publicly Mongolia. guaranteed debte using the DMFAS. A1.1: Debt Effective debt No comprehensive DMFAS upgrade DMFAS upgrade from DMFAS remains fully management recording system is monitoring of the debt completed; DMFAS 6.0 5.2 to 6.0 was operational and is used developed. situation. is functional. completed in 2013 and for production of captures all foreign and quarterly public debt domestic government reports. The latest 56 Objective and Most Recent Component Indicator Baseline, 2006a Target Values, 2014b GAP Closure, 2015c Information, 2019d debt and government- upgrade of the system guaranteed debt.e was completed in 2018. GFMIS operations are sustainable. The government of Mongolia has financed the upgrade to the latest web-based version of the software. A2. Decentralizing GFMIS rolled out to No soums are connected 50 soums are 50 soums and 10 large GFMIS operations are financial additional budget to the GFMIS. connected. budget entities were sustainable. The management entities and soums. connected to the government of GFMIS under the GAP. Mongolia financed the By 2015, the GFMIS upgrade to the latest became fully web-based version of operational in all 333 the software. soums of Mongolia. The latter expansion was funded by the government of Mongolia. 2. Promote transparency and accountability in the performance of public sector functions B1. Improving the Improved public MCI = 0.64 (2009). MCI = 0.8. MCI = 0.65 (2013).f MCI data are not anticorruption accountability as No legal framework in Conflict of Interest Law Law on Preventing available after 2013. framework evidenced by (i) place to regulate conflict and related regulations Conflict of Interest was Data from alternative reduced perception of of interest. are adopted and being enacted in January regular corruption corruption and (ii) an implemented. 2012. surveys suggest steady effective conflict of improvement in interest framework perceptions of that complies with the corruption. For UN Convention example, the share of against Corruption. respondents in the 57 Objective and Most Recent Component Indicator Baseline, 2006a Target Values, 2014b GAP Closure, 2015c Information, 2019d private sector who had personal knowledge of corrupt transactions with the government in the preceding three months declined from 28% in 2012 to 15% in 2018. The Law on Preventing Conflict of Interest remains fully effective. B1. Improving the Functional system of No incomes and assets Detailed assets and Assets and income of The capabilities of the anticorruption income and asset are disclosed by civil income declarations of all public officials PIADIS have improved framework declaration and servants. civil servants are submitted and since the transition to disclosure. published. disclosed through the the Web-based version Improved analysis of online PIADIS with in 2016. Compliance the declarations 99.9% compliance. remains high. through a functional Over 38,000 civil About 41,000 civil Asset and Income servants submit servants submit the Declaration Information annually their Income declarations annually. System. and Assets Declarations The capacity for analysis and Conflict of Interest has improved, but more statements. benefits could be Limited capacity for the realized through closer analysis of the collaboration with civil declarations. society groups. B3. Capacity Raw LSMS data made LSMS data are accessible LSMS data are readily The results of the new The detailed household building for data publicly available. in theory, but access is available to the public. Households Social and survey data are transparency and difficult in practice and Economic Survey and continuously available data access for not advertised. all raw microdata from for independent poverty the household surveys poverty analysis from from the past 10 years the NSO website and 58 Objective and Most Recent Component Indicator Baseline, 2006a Target Values, 2014b GAP Closure, 2015c Information, 2019d conducted by the NSO are regularly updated. are publicly available at Public confidence in the the NSO website and national poverty data disseminated in paper has increased. format. B4. Upgrading Improvement in public Public access to budget PEFA PI-10 score: A. PEFA PI-10 score per Further effort to Ministry of access to key fiscal data are limited. Open the PEFA (2015) Report. enhance public access FinanceSPE website information (measured Budget Index score was to budget data, to a dynamic portal by the PEFA indicator 18 (out of 100) in 2006. including through the PI-10). upgrade of the Ministry of Finance’s portal. Open Budget Index score was 46 in 2017, reflecting substantial progress made since 2006. But all these gains were realized before 2010. 3. Foster the investment climate C1. Tax TAIS is fully functional Multiple and inconsistent The TAIS is The TAIS is successfully The TAIS has been administration in two pilot sites. tax administration operationally accepted deployed in two pilot operating successfully systems resulting in data by the GDT and sites (three years later with nationwide errors, lack of successfully deployed in than originally coverage (three offices) transparency, and two pilot sites. planned). providing a inefficiency in tax considerable reduction registration, assessments, in tax compliance cost, collections, and audits. especially for SMEs. C1. Tax Improved and The GDT publishes The GDT publishes and The GDT’s disclosure The GDT’s disclosure administration expanded disclosure taxpayer assistance disseminates (i) practices improved practices improved of key GDT data and information and taxpayer assistance taxpayer’s access to tax information on policies. information about taxes, information, (ii) the legislation, info on taxpayers’ services and results of annual taxpayer rights, and is easily available on the 59 Objective and Most Recent Component Indicator Baseline, 2006a Target Values, 2014b GAP Closure, 2015c Information, 2019d its rulings, and taxpayer satisfaction GDT’s performance info GDT’s website. There guidelines. survey, and (iii) through the GDT has been no regular comprehensive website and the public taxpayer satisfaction statement of taxpayers’ service desk of tax survey since the GAP rights. offices, PEFA (2015) PI- closure. 13 (transparency of The statement on taxpayers’ liabilities) taxpayers’ rights is score: A. published online. C2. Public Increased use of open Sound legal framework: Use of open 92% of procurement Direct contracting procurement competition in public the 2006 Procurement competition for more funded from the stayed at the level of procurement. Law is based on than 90% of public government budget 8% of total government international best procurement by value. uses open competition. procurement in 2018. practice, but weak However, there is The government of implementation significant use of direct Mongolia completed practices. contracting for off- the transition to the budget public new e-procurement procurement, including system, which boosts public investment competition and financed by Chinggis transparency. bond proceeds. The 2019 amendments to the Public Procurement Law brought Development Bank of Mongolia investments under the law. C3. Mining registry Efficient, timely, and No computerized A CMCS is fully The CMCS was The CMCS remains fully and cadastre system independent operation cadastre system; functional. launched in April 2010. operational. The MRAM of the mining registry management of mineral the Web-based version has the capacity to and cadastre system. licenses arbitrary and of the CMCS was made maintain and regularly nontransparent. operational in February update the system. The 2013 and remained CMCS operations 60 Objective and Most Recent Component Indicator Baseline, 2006a Target Values, 2014b GAP Closure, 2015c Information, 2019d fully functional at the contributed to the time of GAP closure. It improvements in global is used to record and perceptions of the manage all mineral mining regime in licenses. Cadastre Mongolia. information is available online for the public. Sources: World Bank (2006b, 2014c, 2015a), PPAR Mission’s findings. Note: CMCS = Computerized Mining Cadastre System; DMFAS = Debt Management and Financial Analysis System; GAP = Governance Assistance Project; GDT = General Department of Taxation; GFMIS = Government Financial Management Information System; LSMS = Living Standards Measurement Survey MCI = Mongolia Corruption Index; MRAM = Mineral Resources Authority of Mongolia; NSO = National Statistics Office; PEFA = Public Expenditure and Financial Accountability; PIADIS = Personal Income and Assets Disclosure Information System. a. Based on relevant info from the project appraisal document, Implementation, Completion and Results Report, and Restructuring Paper. b. Per Restructuring Paper, May 2014. c. At the time of Implementation, Completion, and Results Report preparation, early 2015. d. As of time of Project Performance Assessment Report mission, June 2019. e. Off-budget public debt contracted by the Development Bank of Mongolia was not captured in DMFAS at the time of GAP closure as it did not meet the definition of government or government-guaranteed debt adopted in Mongolia’s legislation. This issue has been resolved through recent legal amendments. f. The website of the National Statistical Office (http://www.1212.mn). 61 Appendix G. Mongolia’s Macroeconomic and Fiscal Performance After the Global Financial Crisis The 2008–09 global financial crisis, followed by the collapse of global mineral prices, hit Mongolia hard and required considerable additional support from international development partners. In 2009–10, the World Bank, in close collaboration with the International Monetary Fund (IMF) and other development partners, provided budget support through two development policy operations (DPOs) of $75 million (1 percent of gross domestic product). The DPOs aimed at addressing the country’s core fiscal management challenge, which was related to procyclical fiscal management. However, according to the Country Program Evaluation for Mongolia, prepared by the Independent Evaluation Group in 2014, progress under the Country Partnership Strategy pillar of Macroeconomic and Public Financial Management Policies was rated moderately unsatisfactory. The Country Program Evaluation claimed that the key reforms, including those to improve fiscal transparency, were only partially implemented, although the government’s reform ownership remained weak. As such, the DPO program did not succeed in preventing Mongolia from returning to a nonsustainable fiscal path. The overall stance of macroeconomic and fiscal management has deteriorated substantially during and immediately after the DPO program implementation (table G.1.). As commodity prices recovered in 2010, there was a massive increase in public expenditures in 2010–12, largely driven by increases in universal cash transfers, public wages, and (not well-prioritized) capital spending. Swift price recovery undermined incentives for structural reforms and complicated implementation of projects, such as the GAP, which aimed at enhancing the institutional environment for sustainable economic management. The 2012 parliamentary elections brought in a new coalition government that embarked on an expansionary fiscal program. In September 2012, a new Ministry of Economic Development was established with responsibility for planning the capital budget. The biggest step backward was the exponential increase in extrabudgetary financing of public investments through the Development Bank of Mongolia, which operated outside of the budget framework and, until early 2019, was not subject to government procurement requirements. As reported by the IMF, by 2012, the fiscal policy was back to its boom-and-bust pattern with a substantially widened budget deficit, high inflation, and negative interest rates. Despite high growth rates and some regulatory and budgetary reforms, the postcrisis period of 2011–15 was turbulent, with several reform reversals. Strengthening of macroeconomic and fiscal discipline had been insufficient, falling behind a steadier pace of institutional progress (table G.2). As a result, in 2016 (the election year), the 62 consolidated overall budget deficit widened sharply to more than 19 percent of the gross domestic product, with total public expenditures increasing by more than a quarter despite declining revenues. This had a negative impact on both inflation and current account balance. The 2016 parliamentary elections resulted in another change of government and a subsequent shift in the government’s strategy. This time, the new government was able to follow a more responsible macroeconomic path. The IMF’s November 2018 Staff Report pointed to significant improvements in Mongolia’s fiscal performance in 2017–18, driven by large revenue overperformance, saving of most of these revenue gains, and considerable compression of off-budget expenditure. At the same time, the IMF expressed concern that the ongoing political pressure for policy changes could put these recent achievements at risk. With public debt still high and the economy exposed to global commodity price fluctuations, the report stressed the critical importance of further strengthening fiscal discipline and improving the budget expenditure structure. 63 Table G.1. Key Economic Indicators, 2005–15 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Output employment and price GDP growth (annual 7.3 8.6 10.2 8.9 −1.3 6.4 17.3 12.3 11.6 7.9 2.4 percent) Unemployment, total 7.6 7.0 7.2 5.6 5.9 6.5 4.8 3.9 4.2 4.8 4.9 (modeled ILO estimate) (percent of total labor force) Inflation, consumer prices 12.7 5.1 9.6 28.0 7.6 10.1 8.4 14.3 10.5 12.2 6.6 (annual percent) Fiscal balance Budget balance (percent of 2.4 3.1 2.7 −4.5 −5.2 0.4 −3.9 −6.4 −0.9 −3.7 −5.0 GDP) Foreign trade, BOP, and debt Goods exports (BOP, 1.1 1.5 2.0 2.5 1.9 2.9 4.1 3.8 3.8 5.5 4.4 current $, billions) Goods imports (BOP, 1.2 1.4 2.0 3.1 2.1 3.1 6.7 6.8 6.4 5.3 3.9 current $, billions) Current account balance 3.5 10.9 4.1 −12.3 −7.5 −12.3 −43.3 −43.8 −37.6 −15.8 −8.1 (percent of GDP) Foreign direct investment, 187.6 245.5 372.8 844.7 623.6 1,691.4 4,571.2 4,272.5 2,059.7 337.8 94.3 net inflows (BOP, current $, millions) Total PPG debt (percent of 56.6 44.3 36.1 31.0 43.3 30.8 27.9 39.0 46.3 54.9 56.0 GDP) Financial markets Net domestic credit (% 26.6 20.0 28.1 31.4 29.4 25.7 33.8 25.7 52.0 61.8 62.1 GDP) 64 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Total reserves of imports 2.5 4.6 4.6 2.0 5.6 6.1 3.1 4.9 2.9 2.4 2.5 (months) Deposit interest rate 13.0 13.0 13.5 11.4 13.3 11.9 10.5 11.3 12.0 12.3 13.0 (percent) Official average exchange 1,205.2 1,179.7 1,170.4 1,165.8 1,437.8 1,357.1 1,265.5 1,357.6 1,523.9 1,817.9 1,970.3 rate (MNT) per $, period average) GDP (current $, billions) 2.5 3.4 4.2 5.6 4.6 7.2 10.4 12.3 12.6 12.2 11.7 Source: World Bank Databank; Mongolian Statistical Information Service; IMF Country Reports 08/200, 10/52, 11/76, 14/64, and 15/109. Note: BOP = balance of payments; GDP = gross domestic product; ILO = International Labour Organization; PPG = public and publicly guaranteed; MNT = Mongolian tughrik. 65 Table G.2. Mongolia: Fiscal Trends, 2005–15 (percent of gross domestic product) 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Total revenue and grants 27.5 33.8 37.9 33.1 30.3 32.0 34.0 29.7 31.2 28.4 25.8 Tax revenue 22.8 28.0 30.3 28.8 24.6 27.6 27.8 25.2 26.5 23.2 22.1 Total expenditure and net 25.1 30.7 35.3 37.6 35.5 31.6 37.9 36.1 32.2 32.1 30.8 lending Current expenditure 19.7 24.4 27.6 26.9 28.0 23.8 25.1 27.2 24.4 24.5 25.4 Goods and services 12.7 17.2 13.6 16.1 15.6 12.6 12.0 13.0 13.2 12.6 12.8 Wages and salaries 4.7 4.9 6.0 8.3 8.7 6.6 6.1 7.2 7.3 7.1 7.4 Social security 0.0 0.0 0.0 0.0 0.8 0.6 0.6 0.7 0.7 0.7 0.7 contributions Other expenditures 8.0 12.3 7.6 7.8 6.0 5.3 5.3 5.1 5.2 4.8 4.7 Interest payments 0.7 0.4 0.4 0.3 0.4 0.4 0.3 0.8 1.4 2.3 3.2 Subsidies and 6.3 6.7 13.6 10.5 12.0 10.7 12.9 13.5 9.9 9.6 9.4 transfers Capital expenditure 3.0 4.4 6.0 9.5 7.0 6.1 9.7 9.1 7.8 8.0 6.0 Domestically financed 2.6 4.1 5.6 9.2 6.6 5.3 8.4 8.6 6.4 8.0 5.0 Foreign financed 0.3 0.2 0.5 0.3 0.4 0.3 1.3 0.6 1.3 0.0 1.1 Budget balance 2.4 3.1 2.7 −4.5 −5.2 0.4 −3.9 −6.4 −0.9 −3.7 −5.0 Source: IMF, World Bank 2018c. 66 Appendix H. List of Persons Met Government of Mongolia Mr. Amartuvshin, Network Engineer, Mineral Resource and Petroleum Authority of Mongolia Mr. Z. Baasannyam, Director, Administration Department, Independent Agency against Corruption (IAAF) Mr. Batbayar, Head of the Information Technology Division, General Department of Taxation Mr. M. Batkhuyag, Director-General of Treasury Department, Ministry of Finance Mr. D. Batmunkh, Project Coordinator, Strengthening Fiscal and Financial Stabiliity project of the World Bank, former senior staff member for the Government Assistance Project Project Coordination Unit Mr. T. Batzul, Head of the Procurement Policy Division, Ministry of Finance Mr. K. Bilguun, Sr foreign affairs officer, IAAC Ms. S. Bolormaa, Sr Statistician, Households Income and Expenditure, National Statistical Office (NSO) Ms. N. Doljinsuren, Sr Statistician, Social Statistics Unit, NSO Ms. D. Enkhtuya, Senior Official, Tax Administration and Cooperation Department, General Department of Taxation Mr. T. Enkhjavkhlan, Division Head, Tax Administration and Cooperation Department, General Department of Taxation Mr. Ganbold, Director of the Taxpayer Service Center, General Department of Taxation Mr. Khashbaatar, Head of the Inspection and Analysis Division of the IAAC Mr. B. Lkhagva-Ochir, Head of the Financial Information Technology Division, Ministry of Finance Mr. Otgontsetseg, Chief Accountant, Ministry of Food, Agriculture and Light Industry, Former Head of the Payment and Settlement Division, Treasury, Ministry of Finance Ms. Oyunbaatar, Consultant, Fiscal Policy Department, Ministry of Finance 67 Ms. Soyolmaa, Specialist, Public Investment Department, Ministry of Finance Mr B. Tamir, Sr Officer, IT department, NSO Mr. S. Tsedendamba, Member of the Civil Service Council Mr. Tserensambuu, Head of IT Division, State Property Policy and Coordination Agency Provincial Government Officials, Tuv Province Ms. L. Ariunzaya, Integrity Officer, local coordinator for Income and Assets Declarations Mr. Z. Amgalan, Head, Regional Tax Department Mr. Batjargal, Director, Authority for Local Property Mr. Y. Byambatsogt, Head, Regional Finance and State Fund Department Ms. D. Byambadulam, Head, Internal Control, Treasury Department Ms. D. Myagmarsuren, Treasury officer for Zuunmod soum (local government) Ms. B. Odonchimeg, Chief Accountant, Treasury Department Nongovernment Organizations in Mongolia Mr. Y. Amarbat, Compliance Manager at the State Bank, former Acting Head of the Debt Management Division, Ministry of Finance Mr. O. Batbayar, Executive Director, Transparency International Mongolia Mr. A. Bayanmunkh, Governance Program Manager, The Asia Foundation Mr. L. Enkh-Amgalan, Deputy Director, Mongolian Institute of Certified Public Accounting (MonICPA) Mr. Demid, IT Consultant, former Head of the Financial Information Technology Division, Ministry of Finance Mr. Khuyagtsogt, Director-General, Public Administration Department, Ministry of Education, Culture and Sports; former Head of the Budget Expenditure Division, Ministry of Finance 68 Mrs. D. Oyunbadam, Manager for the Economic Policy Program, Open Society Forum Mrs. J. Oyunchimeg, Director of the Policy and Coordination Department, the Chamber of Commerce Mr. L. Tur-Od, Chair, Transparency International Mongolia World Bank Jim Andersen, former Mongolia Country Manager Davaadalai Batsuuri, Economist, Mongolia Country Office Badamchimeg Dondog, Public Sector Specialist, Mongolia Country Office Tserendagva Gerelgua, Procurement Specialist, Mongolia Country Office Zahid Hasnain, Senior Public Sector Specialist, former Governance Assistance Project task team lead Ronnie W. Hammad, Consultant, Governance Assistance Project Implementation, Completion, and Results Report Primary Author Jean-Pascal Nganou, Senior Economist, Acting Mongolia Country Manager Other Agencies Guohua Huang, Sr Economist, Fiscal Affairs Department, International Monetary Fund (IMF) Geoff Gottlieb, Mongolia Mission Chief, IMF Gantsogt Khurelbaatar, Adviser to the ED, IMF; former Deputy Minister, Ministry of Finance Irina Kravchenko, Resident Representative, European Bank for Reconstruction and Development Declan Magee, Deputy Country Director, Asian Development Bank Resident Mission in Mongolia 69