Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) Report Number: ICRR0021835 1. Project Data Project ID Project Name P111942 Tax Administration Modernization Project Country Practice Area(Lead) Armenia Governance L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-51140 30-Apr-2016 10,966,927.82 Bank Approval Date Closing Date (Actual) 03-Jul-2012 31-Jan-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 12,000,000.00 0.00 Revised Commitment 12,000,000.00 0.00 Actual 10,966,927.82 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Judyth L. Twigg Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (page 4), the Project Development Objective (PDO) was to reform and modernize the tax administration to: (a) increase voluntary tax compliance; (b) reduce tax evasion; (c) reduce compliance costs; and (d) increase administrative efficiency. The Project Appraisal Document (PAD) had the same PDO. Page 1 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project had four components: Component 1: Institutional development (estimated cost at appraisal - US$2.2 million; actual cost - US$1.0 million). The component was to strengthen Ministry of Finance (MOF) capacity to formulate tax policy and help the State Revenue Commission (SRC) improve strategic planning, strengthen management, and increase efficiency in tax inspectorates at the field level. To do so, the project was to provide consulting services for workflow analysis and organizational restructuring, and equipment for tax administration training centers. The United States Agency for International Development (USAID) was to provide support for strategic planning. Component 2: Strengthened operations (estimated cost at appraisal – US$1.0 million; actual cost – US$1.3 million). The component was to strengthen compliance management by re-engineering business processes for more effective self-assessment, routine processing of returns and calculation of taxes, automated risk management, stronger risk assessment, better audit skills, better detection of tax fraud, and improved taxpayer services and appeals structures. Project financing was to procure equipment, while technical assistance and training was to be provided by USAID. Component 3: Information technology (IT) infrastructure and system modernization (estimated cost at appraisal – US$11.0 million; actual cost - US$10.0 million). The component was to upgrade information and communications technology in the SRC for a comprehensive and integrated tax management system that was to unify functions at all levels, improve telecommunications and web connectivity, strengthen information exchange with other agencies, and augment other technological infrastructure. The aim was to create an efficient and effective environment for tax operations and management that would improve reliability and availability of information, better target compliance activities, lower administrative costs, and improve revenue collection. Component 4: Project management (estimated cost at appraisal – US$0.9 million; actual cost US$0.9 million). The component was to strengthen the Borrower’s capacity to implement the project by financing consulting services to support procurement, financial management, and technical implementation of project components, as well as operating costs for project implementation. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost and financing. At appraisal, total project costs were estimated at US$15.1 million, with US$12.0 million in International Development Association (IDA) financing and US$3.1 million in domestic resources. US$6.0 million in parallel financing was provided by USAID to finance complementary technical assistance. Actual costs at closing were US$13.2 million, with US$10.9 million in IDA financing. The difference in IDA financing between appraisal and actual is mainly due to shifts in the SDR/US$ exchange rate. Page 2 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) Dates. The project was approved on July 3, 2012. It became effective on December 28, 2012. The project underwent five Level 2 revisions during implementation. A first revision was introduced in March 2014 to accommodate an administrative adjustment in project management, giving responsibility for implementation to a project management board, which avoided the need to establish a separate Steering Committee for project management. Subsequent revisions consisted of extensions of the Closing Date that were driven by government change, changes in the institutional framework (tax legislation), and slower-than-expected implementation. The original Closing Date, April 30, 2016, was extended twice, a first time on November 4, 2015, to December 31, 2017; followed by three additional extensions - to April 15, 2018, then to September 30, 2018, and finally to January 31, 2019, when the project closed. 3. Relevance of Objectives Rationale The project objectives were relevant to addressing public finance challenges that Armenia was facing in the aftermath of the international economic and financial crisis in the latter part of the last decade, notably significant fiscal deficits coupled with low tax collection rates. The objectives were aligned with Bank strategy as set out in its Country Partnership Strategies (CPS) at appraisal (CPS 2009-2013) and throughout implementation (CPS 2014-2017). The former emphasized tax reform, and the latter reduction in tax compliance costs. The current CPS, covering the period 2019-2023, also raises revenue collection efficiency as a tool for addressing limited fiscal space for public spending. All three CPSs support the government’s ongoing tax administration reform strategy. To achieve the objectives, the project mainly focused on investments in information and communications technology (ICT) infrastructure and equipment and, selectively, software development. Related institutional changes and human resource requirements, also essential for achieving the project’s objectives, were financed by USAID. In the ICR, USAID-financed activities were not taken into account in assessing project outcomes. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To increase voluntary tax compliance Rationale Page 3 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) The objective was to be achieved by equipping the SRC with modern ICT infrastructure. This was to expand capacity in tax revenue management, notably allowing the expansion of modules essential for electronic service delivery (taxpayer registration, VAT refunds, e-invoicing, electronic tax declarations). This in turn was to contribute to lower compliance costs, and increased taxpayer registration and filing, resulting in an increase in voluntary tax compliance. The objective of an increase in voluntary tax compliance was met:  The number of active VAT payers increased by 32 percent between January 2011 (11,800) and end- January 2019 (15,590). This exceeded the original target of 12,400 that had been set for 2016 (based on an annual 1 percent increase set out in the indicator). The actual result, 15,590, attained in end- January 2019, exceeded the 1 percent per annum growth target.  The number of active corporate income tax (CIT) payers increased from 13,100 to 16,000 (a 22 percent increase) between 2014 and 2018. The original baseline was 29,000, with a target of 30,400 (again assuming an annual increase of 1 percent), but legal changes to the classification of taxpayers in 2014 reduced the number of active CIT payers. Maintaining the annual growth target, the 2018 number exceeded the target. Rating Substantial OBJECTIVE 2 Objective To reduce tax evasion Rationale The objective was to be achieved through the introduction of new systems that were to reduce CIT evasion. They included a new tax audit case management system to raise audit efficiency and transparency; an analytical tools system to analyze business transactions; and sub-systems for tax revenue management, notably for data cross-checking and for applying third-party information. The objective to reduce tax evasion was met:  CIT productivity (reduced tax evasion) increased from 0.10 in 2011 to 0.14 in 2018, achieving the original target; no revised target was set to take into account the extension of the project.  Average adjustments following audit results more than doubled, from a 2011 baseline of AMD10.1 million to AMD22.1 million in 2018, compared to an original target of a 20 percent increase over the period 2012 to 2016. Page 4 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) Rating Substantial OBJECTIVE 3 Objective To reduce compliance costs Rationale The objective was to be achieved by investments in ICT infrastructure to introduce electronic filing, e- invoicing, and electronic payments; and by improving taxpayer services, including the establishment of a call center, a queue management system, and a monitoring center, as well as establishing and equipping service center points. The objective was met, based on survey results. Surveys on perceptions among taxpayers indicated that professionalism and honesty in the tax administration has risen from 23.7 percent in 2013 to 62.5 percent in 2018. However, the 2013 and 2018 surveys are not strictly comparable, as the former was a household survey while the latter focused on a representative sample of taxpayers. On average, 83 percent of respondents thought that the tax submission process as a whole had become much simpler, and also stated that the introduction of automation had reduced administrative costs for taxpayers. An increase in tax returns filed electronically, from 20 percent in 2011 to 97.5 percent in 2018, may have mainly reflected the introduction in 2013 of a legal obligation to file electronically. Rating Substantial OBJECTIVE 4 Objective To increase administrative efficiency Rationale The objective was to be achieved through change management and capacity building. The objective was met. Capacity building was provided through a newly established Training Center and taxpayer service centers, used for training and distance learning. Here, training capacity extended to 3,000 trainees per year. Several study visits abroad were undertaken. A printing house linked to the Training Center allowed essential documentation and learning materials to be made available. Page 5 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) Achievement of this objective was measured by the evolution of the ratio of revenue collected by the SRC to its operating costs, with a target of a 10 percent increase by the end of the project (2016). That ratio increased from 74 in 2011 to 107 in 2018, a 69 percent increase from 2011. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale All the objectives are rated substantial; they all indicate substantial improvements over the project period, extrapolating from the original targets, as none of the indicators was adjusted to take into account the much longer implementation period. That said, there are challenges of attribution in assessing achievement of the individual objectives. The achievement of these objectives is interrelated; for example, changes in compliance costs influence voluntary compliance and vice versa, and both influence administrative efficiency. In addition, USAID played an important role in providing technical assistance, which directly influenced project outcomes but was not assessed in the ICR. Overall Efficacy Rating Substantial 5. Efficiency An economic analysis was undertaken in the PAD and the ICR. However, the two analyses are not comparable, as the former was based on the total increase in tax revenues, while the latter more correctly assessed only the effects of increases in revenue over and above the trendline. Using a discount rate of 5 percent, and based on reasonable assumptions about GDP growth, increase in the number of taxpayers, and inflation, the ICR found a financial internal rate of return of 219 percent and net present value of US$275 million. The economic analysis indicated a net present value of US$14.2 million, and an internal rate of return of 31 percent. Efficiency was also influenced by qualitative factors. The positive side included technical assistance provided by USAID that drove the application of much of the infrastructure and software of the project; institutional changes during implementation, notably the relocation of project management under the MoF; financial management arrangements that were placed under an experienced fiduciary agency; and interplay with other Bank projects that were supporting better governance, i.e. a series of Public Sector Modernization projects. The extension of the project by 33 months is likely to have provided better value for money by allowing the project to surpass the Page 6 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) original targets, essentially at no additional investment costs. However, there were some minor shortcomings driving that extension, including complications with procurement, notably non-responsive bids and excessive bids that required additional time to complete bidding processes. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is rated high, as the project directly addressed and significantly contributed to reducing concerns over fiscal imbalances, as these no longer had a prominent role in the latest CPS. Efficacy is rated substantial, as the project mostly achieved its four objectives: it provided IT infrastructure, upgraded ICT, and developed key modules for the tax management system, leading to lower tax evasion and gains in revenue, and ultimately, contributing to gains in efficiency of tax administration. Efficiency is rated substantial. Taken together, these ratings indicate only minor shortcomings in the project's preparation and implementation, leading to an Outcome rating of satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome With the investments in IT systems and infrastructure in the SRC completed, training and capacity building undertaken, and a strategic plan for the SRC in place to further strengthen systems and maintain staff skills, the risk to the development outcome appears low. Potential risks are further diluted by a Bank development policy operation about to come on-stream (a fourth Armenia DPO, P169624) that includes the roll-out of a tax audit case management system as a prior action. Page 7 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) 8. Assessment of Bank Performance a. Quality-at-Entry The project was strategically relevant in supporting efforts at sustainable deficit reduction and focusing on revenue collection to do so. The components could reasonably have been expected to achieve project objectives. M&E design included difficulties in attribution in applying indicators (see Sections 4 and 9). At the government’s request, preparation (and later, implementation) was closely coordinated with USAID, with the former emphasizing ICT and the latter related technical assistance, emphasizing the importance of proper sequencing. Project design drew on lessons learned in supporting tax administration reform, focused on anchoring investments in government reform plans, linking ICT to business processes (and not vice-versa), and applying incremental reforms. At the same time, project preparation envisaged a relatively short implementation period that may not have been realistic based on experience with other tax reform projects. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank undertook an intensive supervision effort, including headquarters and locally based task team leaders (TTL), and experts in IT and tax administration. Following a slow start due to rotating TTLs, the team subsequently operated proactively to address implementation issues and move the project onto a more realistic time frame. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design A detailed results framework and monitoring plan were developed during preparation. Six outcome indicators were to be tracked, all with baseline data, milestones, and targets covering a five-year implementation period. All of the indicators were measurable and relevant to their respective objectives. While the formulation of the project's outcome targets did permit extrapolation to measure the validity of outcomes when introducing the almost three-year extension of project implementation, it would Page 8 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) have been reasonable to introduce adjustments during implementation. Now the M&E is limited to estimates of outcomes: were they achieved or not, albeit on a relatively solid basis, because of the way the indicators were formulated. Moreover, attribution is challenged by the fact that some indicators influence each other, as noted in Section 4. Likewise, parallel financing of technical assistance by USAID made it difficult to determine the precise impact of the Bank’s IT investment on SRC performance. b. M&E Implementation The SRC collected data on outputs and outcomes for quarterly and annual reports, focusing on results- based accountability and accomplishments against milestones. c. M&E Utilization Progress reports were reviewed on a quarterly basis, and SRC coordinators met regularly to discuss implementation. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was assessed as a Category C project, and therefore no environmental assessment was required. b. Fiduciary Compliance Financial management. The fiduciary function was carried out by the Foreign Financing Project Management Center. It had experience and capacity to implement projects in accordance with the World Bank’s policies and procedures. Financial management arrangements were adequate and satisfactory to the Bank throughout the project implementation period. All disbursement procedures were in compliance with the Bank’s disbursement guidelines. The auditors issued unmodified (clean) opinions on the project’s annual financial statements, which were received by the due dates. Page 9 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) Procurement. Procurement was carried out in accordance with the Bank’s Procurement and Consultant Guidelines. Some packages had to be cancelled and re-bid primarily due to lack of competition. Rebidding required more time to complete key project activities and was the main reason for the last two extensions of the project closing date. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory IEG notes shortcomings in QAE sufficiently serious to influence overall project progress during implementation. Moreover, the Bank Performance Satisfactory Moderately Satisfactory ICR rated QAE as MS, and thus, according to the harmonized guidelines, overall Bank Performance should also be rated MS in the ICR. IEG notes that outcome targets were not adjusted when the Quality of M&E Substantial Modest project was extended, as well as challenges of attribution. Quality of ICR --- Substantial 12. Lessons Three factors contributed to good design. First, project design was informed by several pieces of analytical work. Second, the project built on existing SRC operations and facilities, which ensured that changes were incremental and therefore manageable. Finally, the project design reflected an integrated approach, including the technical assistance on institutional changes and process re- engineering provided by USAID and IT investments financed by the Bank. Page 10 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tax Administration Modernization Project (P111942) The efficiency of an integrated approach may be difficult to assess. In this project, the economic and financial analysis exclusively focused on the efficiency of Bank-financed components, including the benefits of the USAID-financed activities but not their costs. There are key features to developing complex ICT systems. The upgrading to a comprehensive and integrated tax management system is usefully accompanied by a series of key considerations: (i) vision and strong leadership to ensure consistency in IT strategy and priorities; (ii) the right sequencing of technical assistance and IT investments to help ensure smooth and timely implementation; (iii) realistic timelines when choosing between a customized system (the project option) and a commercial off-the-shelf solution; (iv) the local IT market’s capacity when opting for customized systems; and (iv) the extent of technical assistance needed to support business process reengineering and change management. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a thorough assessment of the project. It covered the relevant sections in sufficient detail, providing sufficient evidence-based information and analysis to allow the ICR Review to be completed. It drew important lessons that also were covered in the text. a. Quality of ICR Rating Substantial Page 11 of 11