Public Sector Governance Indicators for EU Regions Table of Contents Acknowledgments ....................................................................................................................................... ii Introduction ................................................................................................................................................. 1 I. Analytical Framework and Methodology ............................................................................................. 2 Figure 1. A country’s governance system ................................................................................ 3 II. Developing Indicators for Seven Key Aspects of Public Sector Governance Performance ............ 5 Public Financial Management (PFM) Indicators ..............................................................6 Procurement Indicators ....................................................................................................6 Tax Administration Indicators ..........................................................................................7 Human Resource Management (HRM) Indicators ............................................................7 Justice System Indicators ..................................................................................................8 Indicators of Public Accountability Mechanisms (PAM) ...................................................9 Anti-Corruption Indicators ............................................................................................. 10 Figure 2. Manifestations of poor governance ........................................................................ 11 III. Next Steps—Possible Uses and Extension of the PSG Indicators .................................................. 12 IV. References ........................................................................................................................................... 14 Annex 1: Definitions of Governance Variables ...................................................................................... 15 Table 1A.1. Governance Principles in Public Financial Management (PFM) Systems..... 15 Table 1A.2. Governance Principles in Procurement Systems .......................................... 15 Table 1A.3. Governance Principles in Tax Administration Systems ................................ 16 Table 1A.4. Governance Principles in Human Resource Management (HRM) Systems .. 16 Table 1A.5. Dimensions of Justice System Performance .................................................. 17 Table 1A.6. Governance Principles in Public Accountability Mechanisms (PAM) .......... 18 Annex 2: Public Sector Governance Indicators List .............................................................................. 19 Table 2A.1. Public Expenditure and Financial Accountability (PEFA) ................................ 19 Table 2A.2. Public Investment Management Assessment (PIMA) ....................................... 23 Table 2A.3. PEFA-Style Public Investment Management Indicators (PEFA-PIMI) ............ 24 Table 2A.4. Debt Management Performance Assessment (DeMPA) .................................... 25 Table 2A.5. Open Budget Survey .......................................................................................... 28 Table 2A.6. Description of PFM Indicator Sets..................................................................... 30 Table 2A.7. Shortlist of Procurement Indicators ................................................................... 35 Table 2A.8. Short List of Tax Administration Indicators ...................................................... 43 Table 2A.9. Description of Tax Indicator Sets ...................................................................... 47 Table 2A.10. Short List of Human Resource Management (HRM) Indicators ..................... 50 Table 2A.11. Description of HRM Indicator Sets ................................................................. 52 Table 2A.12. Shortlist of Justice System Performance Indicators ......................................... 54 Table 2A.13. Justice Sector Performance Indicators - Datasets and Methodologies ............. 64 Table 2A.14 Shortlist of Public Accountability Mechanism Indicators ................................ 67 Table 2A.15. PAM Indicators – Datasets and Methodology ................................................. 72 Shortlist of Corruption Indicators................................................................................... 73 Table 2A.16. Indicators of State Capture ............................................................................... 73 Table 2A.17. Indicators of Nepotism and Patronage ............................................................. 75 Table 2A.18. Indicators of Embezzlement of Budget ............................................................ 77 Table 2A.19. Indicators of Administrative/Petty Corruption................................................. 78 Table 2A.20. Indicators of Corruption in Public Procurement .............................................. 80 i Table 2A.21. Indicators of Purchase of Public Positions ....................................................... 81 Table 2A.22. Additional Indicators/Control Variables about the Institutional Environment 82 Table 2A.23. Additional variables on respondent’s characteristics ....................................... 86 Table 2A.24 Corruption Indicators: Sources and Methodologies .......................................... 87 Acknowledgments This report was prepared in response to a request by the Economic Analysis Unit of the European Commission’s Directorate-General for Regional and Urban Policy (DG REGIO) to the World Bank. The report was prepared by a World Bank team led by David Bernstein, Task Team Leader, and comprised Francesca Recanatini and Elena Georgieva-Andonovska from the Governance Global Practice. The team would like to thank Adrian Fozzard and Roberto Senderowitsch, Practice Managers, Global Governance Practice for Europe and Central Asia, and Rogier van den Brink, Program Leader, Europe for their guidance and advice. The Bank team thanks the following World Bank experts and consultants for their assistance with the identification of specific sets of indicators: Oleksii Balabushko (PFM), Miki Matsuura (PFM, tax), Rajul Awasthi (tax), Hunt La Cascia (procurement), Zahid Hasnain (HRM), Daniel Rogger (HRM), Georgia Harley (justice), Eva Melis (justice), Svetozara Petkova (justice), and Stephanie Trapnell (PAM, corruption). The team would also like to thank the following peer reviewers and commenters for their helpful guidance on improving the final report: Steven Knack, Edouard Al-Dahdah, Luis-Felipe Lopez-Calva. The team also benefited from comments and guidance provided by the members of the European Commission’s Inter-Service Group on Public Administration Quality and Innovation. Finally, the team would like to thank Lewis Dijkstra, Head, Economic Analysis Sector, DG REGIO for providing the trust funds to support the Actionable Governance Indicator Project and for his guidance and leadership in the development of the Public Sector Governance Indicators and this final report. ii Introduction With a budget of 351.8 billion euros ($430.5 billion) for 2014–2020—about one-third of the entire EU budget—the European Union’s Cohesion Policy is its biggest investment instrument. The Cohesion Policy is implemented primarily through investments in EU regions and cities. Governance constraints have been identified as causing delays in the implementation of Cohesion Policy programs, and in reducing the impact of investments. The European Commission’s 6th Cohesion Report stipulates that there is a “growing understanding that without good governance high growth rates and regional economic convergence cannot be achieved” (European Commission, 2014: 3). The report devotes an entire chapter to the importance of good governance for economic and social development. The Development of EU Governance Indicators Project, funded by the Directorate-General Regional and Urban Policy (DG REGIO) under an Administrative Agreement signed in October 2015, includes three work streams. One of these is the development of actionable Public Sector Governance (PSG) indicators. The PSG indicators build on the Indicators of the Strength of Public Management Systems (ISPMS) Initiative, a multi-stakeholder effort coordinated by the World Bank seeking to develop a consensus among practitioners around a set of cross-national performance indicators of the strength and quality of governance and public management systems. The ISPMS Initiative proposed a set of criteria to identify governance indicators relevant for policy making. The World Bank team took stock of the universe of indicators that met ISPMS criteria and identified a subset that is responsive to the objective of measuring the strength and capacity of public sector management based on the World Bank’s operational experience in supporting public sector reforms in EU Member States and candidate countries at the subnational level. This report summarizes the work of the Bank team in collaboration with DG REGIO to identify and curate an initial set of PSG indicators that can measure the performance of public sector institutions at the national and subnational levels. The report focuses on the process used to identify and agree upon the seven sets of PSG indicators contained in Annex 2: public financial management (including public investment management), public procurement, tax administration, human resource management, the justice sector, public accountability mechanisms, and anti-corruption. Additional deliverables produced during the implementation of the project are not discussed here. The work began with the production of a Consultation Note by the Bank team to develop and explain a methodology for identifying those public administration functions that are critical to the operation of good governance in EU Member States at the national and subnational levels.1 Once agreement was reached with DG REGIO on the seven public administration functions listed above, the Bank team identified the governance features that would be measured for each area and then developed seven PSG indicator sets. The report is divided into four parts and includes two annexes. It starts by presenting a brief discussion of the analytical framework developed to define public sector governance and the methodology used to identify the aspects of public sector performance to be measured (Part I). The report then describes the Bank team’s process in working with DG REGIO to identify, analyze, and curate a final list of actionable indicators to measure performance in each of the seven public administration areas with a potential application at the subnational level. The specific steps taken by the team to develop each of the seven sets of indicators are also outlined (Part II). The final part identifies some potential uses for the PSG indicators and some ideas for expanding the World Bank’s work to collect data and expand some of the indicator sets with a particular focus on applicability at the subnational level (Part III). Detailed definitions of the governance variables that each set of indicators is intended to measure are included in Annex 1. Annex 2 1 According to the current Nomenclature of Territorial Units for Statistics (NUTS) 2013 classification, there are 276 EU regions at the NUTS 2 level (Eurostat 2013). The regions are a mix of administrative and purely statistical entities, which as such do not necessarily correspond to functional labor markets, functional economic urban areas or political jurisdictions. 1 aggregates the seven sets of PSG indicators with additional information on the methodology, data availability, and coverage for each of the data sources used to generate the indicator sets. I. Analytical Framework and Methodology We broadly define “governance” as the way the state acquires and exercises its authority to provide public goods and services. This functional definition highlights governance’s intrinsic complexity and the challenge that practitioners face when attempting to measure it. The World Bank’s World Development Report 2017 highlights the role of power, rules, and actors in shaping governance relations. In particular, it defines governance as “the process through which state and non-state actors interact to define and implement policies within a given set of formal and informal rules that shape and are shaped by power ” (World Bank, 2017). As defined, governance is a multi-dimensional phenomenon that can affect different sectors and groups of citizens in a country, and that can operate through both informal and formal rules. This complexity and the multi-sectoral breadth of governance have led to the development of various methods to measure it, producing a sizeable amount of quite dissimilar information and data. Figure 1 provides a simplified illustration of a typical country’s governance system. The circular figure highlights the relationships among state institutions and between those institutions and citizens. A governance system includes the institutions in charge of the management of public resources and service delivery—that is, the public administration and the executive of a country—which is shown in the center of the figure. These institutions work closely with subnational institutions in the management of resources and service delivery at the local level (subnational government and communities). Governance systems also include institutions responsible for the distribution of power, such as political parties (political actors and institutions) and those institutions responsible for the formation of laws and regulations and the oversight of the functioning of the state (formal oversight institutions). Governance, however, is about more than just the institutions of a country—it is also about the relationships between different parts of the state machinery and between the state and its citizens. The way in which these relationships are established and sustained has an impact on the quality of governance in a country. The ability of a citizen to provide feedback to the public office responsible for the provision of a specific service, for example, can affect both the quality of the service provided and the quality of the government agencies providing that service. 2 Figure 1. A country’s governance system While governance encompasses the institutions and relationships among different public entities and stakeholders (figure 1), the PSG indicators focus only on the public sector and public administration, in particular on governance systems at the subnational level. The PSG indicators discussed in Part II include those indicators that can capture (i) the performance of the public administration while managing public resources and delivering services and (ii) the performance of key public sector institutions, especially at the subnational level. The PSG indicators complement the two other work streams that comprise the EU Governance Indicators Project. The Citizen-Centric Governance Indicators capture the relationship between the private and public sectors and the delivery of services at the local/subnational and national levels, while the Regulatory Governance Indicators capture the design, coordination, implementation, and monitoring of regulations by regulatory agencies (these two baskets of governance indicators are identified in figure 1 by the grey boxes on the right and in the center). To develop the PSG indicators, the World Bank identified the features of the public administration that can affect its performance and the quality of the services delivered by public institutions. The performance of the public administration is a function of a key set of public sector management characteristics, as captured by the following equation: = { , , } The PSG indicators seek to identify and measure these characteristics across a series of public administration functions. A country’s institutional arrangements (both formal and informal) form the foundation of the structure and shape of the public administration and, as such, can affect its ability to operate and function effectively. These arrangements define the rules under which the public administration operates: Who delivers which public service? To whom? How is the budget allocated? How is accountability defined? Who is rewarded and when? And who can change the rules and under what conditions? 3 These arrangements alone, however, do not guarantee that the public administration will function well. They must be supported by the capacity—that is, the underlying skills and resources—of the public administration, including both staff technical capacity and the organizational and leadership capacity of public institutions. Capacity also includes the availability of adequate resources (financial, personnel, equipment, facilities/buildings, etc.) to carry out the public administration’s work and the quality of those resources (types of technology and staff qualifications). Finally, the specific culture and environment of the country or region where the public administration or agency operates affects its performance. A public administration’s institutional arrangements may be structured and implemented differently depending upon the culture present in the system. When reviewing existing governance indicators, the Bank team narrowed its focus to “actionable” indicators of governance—that is, indicators that measure a dimension of governance that can be directly affected by a specific policy intervention and that can be used to measure the impact of that policy intervention. Actionable governance indicators (AGIs) focus on specific and narrowly-defined aspects of governance, rather than broad dimensions. These indicators are defined clearly, providing information on discrete elements of governance reforms and often capturing data on the “missing middle” in the outcome chain—institutional arrangements, organizational capacity, and system performance. Because of their specificity, AGIs can help establish the link between policy actions and indicator values. In short, AGIs provide greater clarity regarding the actions that governments can take to achieve better governance outcomes and effectively monitor policy implementation. “Actionability” implies greater clarity and specificity regarding what may be wrong (or right) in terms of policy interventions and why, and helps to monitor progress more clearly as governments implement specific policies. AGIs are not limited to objective measurements; subjective indicators can be actionable. Subjective indicators, however, may more often be generated in less precise ways, thereby undermining their reliability and credibility. In general, indicators with narrower and more explicit definitions are more actionable. While actionability is the critical characteristic of the indicators identified for the PSG indicator basket (it is also important for the other two baskets), it is not the only feature on which researchers and policy makers should focus. Two additional features—replicability and contestability—are equally important for reliable and robust indicators of governance. Reliable indicators should be replicable across studies, requiring independent and informed observers that arrive at similar scores (replicability). In the case of the PSG indicators, the Bank team identified indicators that can be meaningful and potentially replicable at the subnational level. The indicators should also be based on precise definitions, allowing for tractable and meaningful deliberations regarding the meaning and implications of particular indicator values (contestability). Actionable governance indicators can facilitate the design and implementation of reforms targeting the improvement of specific elements of a particular governance dimension. The Bank team selected the sets of PSG indicators to (i) improve the design of governance reforms; (ii) improve the implementation and monitoring of specific governance reforms; and (iii) strengthen understanding of what works under what circumstances in addressing critical governance challenges. 4 II. Developing Indicators for Seven Key Aspects of Public Sector Governance Performance Using this framework and methodology—and following consultations with the European Commission’s Inter-Service Group on Public Administration Quality and Innovation—the Bank team and DG REGIO identified seven areas of public sector governance where the PSG indicator sets would measure performance. These areas are public financial management (including public investment management), public procurement, tax administration, human resource management, the justice sector, public accountability mechanisms, and anti-corruption. The Bank team collected and reviewed the set of indicators that satisfied the ISPMS Initiative criteria (namely that they be actionable, action-worthy, behavioral, replicable and feasible). The ISPMS Initiative was limited to five public administration functions: public administration and civil service, tax administration, public financial management, procurement, and public information systems. While important, these functions are insufficient to encompass the broader range of public administration functions and public sector institutions captured by the World Bank’s definition of governance (figure 1). Also, ISPMS datasets were limited to those that covered at least 20 countries; the Bank team felt this excluded some newer and innovative approaches and indicator sets that could provide indicators relevant for DG REGIO’s cohesion policy and the monitoring of its implementation. The Bank team conducted a review of the European Commission’s work on governance to identify themes that DG REGIO and other Directorates-General have recognized as critical to the implementation of the European Union’s cohesion strategy, EU Structural and Investment Funds, the European 2020 Agenda and, more generally, themes that are important to promoting economic development throughout EU Member States. Modernizing public administration, reducing the costs of inefficient administration for businesses, and improving the quality of the judiciary were identified as key requirements for the success of the Europe 2020 Agenda and have thus become one of the five priorities of the European Semester and the EU’s Cohesion Policy. In addition, European Commission services (DG REGIO, DG GROW, ECFIN, and DG EMPL, among others) agree that for institutions governing economic and social interactions within a country to be successful they need to fulfil several key criteria: (i) absence of corruption; (ii) a workable approach to competition and procurement policy; and (iii) an effective legal environment and an independent and efficient judicial system. In reviewing these criteria, the Bank team also considered the range of assessments and “scorecards” developed by the various European Commission services to tie this analysis of drivers of economic development to public sector management systems and institutions (for example, DG GROW’s Indicators of Public Administration Efficiency and European Public Sector Innovation Scoreboard and the Directorate-General for Justice and Consumers’ (DG JUST) annual Justice Scoreboard). As a result of this broadening of its review of public sector indicators and datasets, the Bank team identified the additional public sector areas of justice/the rule of law, anti-corruption, innovation, and competitiveness as possible additions to ISPMS’s five original public administration functions. Following a series of exchanges between the Bank team and DG REGIO, it was agreed that the justice sector and corruption would be added to public financial management, procurement, tax administration, human resource management, and public accountability mechanisms to comprise the seven public sector governance areas where performance was to be measured. In each of these seven public sector governance areas, the Bank team—working with additional World Bank subject-matter experts— applied a standard methodology to identify, assess, and curate indicator sets that could measure the performance of each area. That methodology was as follows: 5 1) The Bank team reached consensus on a set of governance variables that would be measured by specific indicators for each performance area (Annex 1). 2) The Bank team identified, reviewed and created a longlist of indicators covering the performance area working with World Bank subject-matter experts and practitioners in the area. 3) DG REGIO (and, in some cases, other European Commission services) reviewed and provided comments on the longlist and held follow up discussions with the Bank team. During this review process DG REGIO often requested that the Bank team include additional information on the characteristics of potential indicator datasets (for example, subnational coverage, cost to collect data, frequency of data collection, etc.) or examples of the data from member states for specific indicators, where this was available. 4) The Bank team revised the governance variables and further filtered the longlist indicators based on DG REGIO input and further review by World Bank subject-matter experts to produce a final draft shortlist for DG REGIO’s review and approval. The final, approved shortlists also include detailed information on the coverage, methodology, cost, and frequency of data collection for many of the datasets included in each shortlist (Annex 2). The application of this methodology to each of the seven public sector governance areas is summarized in the following sub-sections. Public Financial Management (PFM) Indicators To identify indicators of public financial management performance, the Bank team worked with World Bank PFM and public investment management experts to identify a longlist of indicators from internationally-accepted indicator sets developed by the IMF and the World Bank—the Public Expenditure and Financial Accountability (PEFA) indicators and Public Investment Management Assessment (PIMA) indicators. Indicators on debt management performance and open budget practices were also reviewed and included. The PFM indicators were selected to measure the efficiency, effectiveness, reliability, transparency, and accountability of the PFM system (table 1A.1.). Indicators included in the final shortlist of PFM indicators are those that measure budget performance and public investment management quantitatively as well as process indicators that cover the basic PFM cycle and subnational government PFM systems. The PEFA, the PIMA, Open Budget Survey and the Debt Management Performance Assessment indicators are third-party expert assessments that have been used primarily in developing economies (except the Open Budget Survey, which covers 102 economies including some EU Member States at the national level). PEFA, Open Budget Survey and Debt Management Performance Assessment have developed subnational methodologies. The cost of expert assessments—which can be high—can be reduced if countries conduct self-assessments accompanied by third-party verification of data and scores. Procurement Indicators Before identifying indicators of procurement performance, the Bank team worked with World Bank procurement experts and identified the principles that are important to a well-functioning procurement system—efficiency, effectiveness, integrity, and transparency (table 1A.2.). A review was then conducted of World Bank procurement documents and the World Bank/OECD-DAC Methodology for Assessment of Procurement Systems (MAPS) and the ISPMS list of procurement indicators. The Bank team also reviewed reports produced by DG GROW (for example, the EU’s Public Procurement Strategy and annual public procurement indicators), DG REGIO (a study on administrative capacity on public procurement in the European Union and guidance for practitioners on avoidance of the most common errors in public procurement of projects funded by ESIF), the European Court of Auditors, the European Anti-Fraud Office (OLAF) as well as the European Commission’s Action Plan on Public Procurement. The final shortlist of 6 procurement indicators (table 2A.7.) identifies those specific indicators that World Bank procurement experts consider the most important in identifying problematic procurement systems. Most of the procurement indicators can be collected as administrative data through the normal operation of national or subnational procurement systems. Many of the indicators on the shortlist are available through the EU’s Tender Electronic Daily (TED) database. If collected through existing administrative systems— particularly through e-procurement systems—the marginal cost for adding the shortlist indicators would be minimal. However, the cost would be significantly higher if the indicators are collected separately (and manually). Tax Administration Indicators The tax administration indicator set focuses on process. These indicators evaluate the performance of tax administrations around the core governance variables of effectiveness, integrity, efficiency, accountability, and transparency (table 1A.3.). The tax administration indicators build on the indicators in the Tax Administration Diagnostic Assessment Tool (TADAT), developed by the World Bank, International Monetary Fund (IMF), and other partners to assess central government tax administration systems. The Bank’s tax experts identified the TADAT indicators that could be applied at the subnational level; these are included in the final tax administration indicator set. The World Bank’s tax experts also incorporated key indicators from other sources in the final list including Doing Business 2017, the World Economic Forum’s Global Competitiveness Report 2016– 2017, and the Business Environment and Enterprise Performance Survey (BEEPs). These additional indicators measure tax compliance from the perspective of firms and assess central government tax administration systems; they were not adapted for subnational application by the Bank team. TADAT and most of the other indicators (except for BEEPS) are collected through third-party assessments. These indicators focus on measuring the performance of national tax administrations—only the Doing Business paying taxes indicator set has been applied at the subnational level—and would need to be adjusted to apply to subnational tax administrations. Subnational Doing Business indicators have been completed for some EU Member States, with assessments underway in some others. TADAT assessments can be expensive, but self-assessments with outside verification can reduce costs. Human Resource Management (HRM) Indicators A government’s ability to implement policies effectively is, in large part, a function of the incentives and skills of the personnel employed in government bureaucracies. The HRM indicator set aims to capture the skills, motivation and working arrangements of government employees to gain a better understanding of the personnel dimension of the public sector and its impact on public sector performance. There are multiple definitions of public sector employment summarized by the organizational classification of the public sector in the IMF’s Government Finance Statistics. The variables in the HRM indicators cover either the public sector, general government, or central government. Within each of these categories, a key distinction is between organizations and individuals whose primary employment is focused on the administration of the functions of government. The variables used in the HRM indicators are intended to focus on public administration within central government, which is defined as those organizations and individuals whose primary employment is focused on the administration of institutional unit(s) of the central government plus those non-market, non-profit institutions that are controlled by the central government (IMF 2014). The Bank team used the World Bank’s ongoing work to develop a dataset of public employment and wages—called the Worldwide Bureaucracy Indicators (WWBI)—as a basis for the HRM indicators. The 7 WWBI, which provides comparative, country-level, descriptive statistics, covers some EU Member States. Given the inadequacy of existing data sources, most of the HRM indicators are based on primary data collection using surveys of public employees to develop perception-based measures of organizational and human resource practices in a bureaucracy. The World Bank’s Governance Global Practice, which has carried out such surveys in a number of countries, plans to make these a core analytical tool in its public administration work. The potential survey-based indicators are large, and all of the indicators can be disaggregated by gender, age, and other demographic characteristics. Surveys tend to take a government organization as the unit of observation, though some surveys have been conducted in subnational jurisdictions. The HRM indicators (table 1A.4.) incorporate the following set of governance variables: • Descriptive statistics on the number and demographic characteristics of public sector employees; • The merit orientation and attractiveness of the public administration for recruitment and career advancement; • The extent of public service motivation and the mission orientation of public employees; • The quality of management along the dimensions of organizational goal setting, monitoring achievement of those goals and performance management of staff; • The competitiveness and transparency of wages and benefits; and • Capacity development of staff. Together, these indicators provide an overall measure of the quality of organizational and human resource management, which in turn is a key determinant of state capacity and public sector performance. Justice System Indicators The use of indicators to measure justice system performance is well established globally. The list of justice sector indicators curates and builds upon this extensive body of work and contains indicators that can reliably measure system performance. The justice system indicators are actionable in that they can inform policy and reforms to improve performance. The list includes indicators that the World Bank has identified or developed through its justice reform work as well as indicators identified in the EU’s Justice Scoreboard or collected by other organizations. The final shortlist includes indicators that court systems should collect as part of their management systems. Justice system performance is defined as the ability to deliver justice services to people and businesses to the standards expected within the European Union. By taking a service delivery perspective to this aspect of governance, the indicators seek to understand the experiences that people and businesses have when they navigate the justice system. The final list of justice system indicators focuses on the actual implementation and day-to-day functioning of justice institutions and justice outcomes, rather than the form of the institutions or the law on the books. The indicator list focuses on court performance because courts are the main institution for providing justice and the primary public sector justice institutions in EU Member States and their subnational governments. The final shortlist focuses primarily on civil and administrative justice. Litigious and non-litigious aspects of civil, commercial, administrative, and criminal justice were reviewed in developing the initial indicator longlist. Aspects of these case types are included among the final indicators to the extent that either indicators disaggregate by case type or where the Bank team felt that performance in these areas contributes directly to economic development. The justice system indicators were developed through a consultative process with both DG REGIO and DG JUST. The Bank team adjusted and filtered the original longlist of indicators based on inputs received through video conferences and written comments. As a result, the final indicators are organized into three 8 dimensions of justice performance that correspond to the structure of the EU Justice Scoreboard: efficiency, quality, and independence (table 1A.5.). The World Bank recognizes that there is no universally-agreed upon or authoritative model for understanding justice system performance. Different countries, donors, and researchers may group justice dimensions somewhat differently. The following filters were applied to pare down the original longlist of justice system indicators (in order of importance and weight assigned during the curating process): • Meaningfulness of the indicator in assessing justice system performance, especially at the subnational level (some national-level indicators would not be meaningful at the subnational level); • Actionability of the indicator in driving justice system performance; • Feasibility of data collection; and • Availability of data. None of the final justice system indicators are currently compiled systematically at a subnational level across the European Union or compared across subnational levels to drive improvements in justice system performance. Nonetheless, much of the data needed to measure the indicators should be collected by courts in their information and administrative systems (such as court registers, case management systems, or case tracking systems). However, the Bank team found that this data often is not tracked systematically. Other sources of justice indicators include perception-based surveys, expert assessments or a combination of both. As a result, there is no existing comparable dataset that can be analyzed or compared in a way that can drive performance enhancement. Annex 2 contains a summary of the available datasets and methodologies used to compile the justice system indicators. Indicators of Public Accountability Mechanisms (PAM) The World Bank’s PAM Initiative served as the starting point for identifying a longlist of indicators measuring public accountability mechanisms. The PAM Initiative focused on the presence and implementation of the following public accountability mechanisms that should be incorporated into well- functioning governance systems for civil servants: (i) financial disclosure systems; (ii) conflict of interest restrictions; (iii) freedom of information/access to information; and (iv) immunity provisions. These mechanisms focus on two aspects of public sector performance—transparency and accountability—rather than on a specific public sector function (table 1A.6.). The European Research Centre for Anti-Corruption and State-Building (ERCAS) at the Hertie School of Governance in Berlin used the PAM Initiative indicators to develop a set of EuroPAM indicators under an EU-funded project. The EuroPAM indicators add procurement and political financing indicators to the PAM Initiative’s indicators on financial disclosure, conflict of interest, and freedom of information. The EuroPAM indicators cover all EU Member States, the European Commission, and the European Parliament. Transparency and accountability—new areas of assessment and analysis in the governance sphere—are significantly less established than other areas of measurement of public sector performance. The PAM Initiative focused mostly on developing indicators that measure the de jure dimension of transparency and accountability as a necessary condition for a well-functioning governance system. The de facto implementation of such mechanisms, while recognized as critical by the PAM team and practitioners in general, has received less coverage. Although the PAM Initiative developed tools to measure de facto implementation of these mechanisms, those for freedom of information were the only ones piloted. The PAM indicator set in Annex 2 includes some of these newer de facto indicators (namely the PAM implementation indicators and Right to Information Drivers of Effectiveness (RIDE) indicators). The selected indicators focus on transparency and accountability mechanisms for civil servants and elected officials only and are applied at the national level. Applying them at the subnational level may require some 9 adaptation and additional resources. Indicators measuring transparency and accountability associated with procurement and PFM functions are not included in the PAM shortlist; they are covered in the procurement and PFM indicator lists. The selected indicators should be viewed as a starting point since several areas of accountability and transparency are not currently covered. Transparency and accountability in campaign financing, for example, is included using only the currently available indicators whose coverage is limited. Media ownership indicators are not included but could be evaluated to capture a full assessment of public sector performance. The indicators in the PAM shortlist are collected through third-party expert assessments. As such, it can be expensive to gather data. While these indicators provide coverage of most EU Member States, they do so at the national level only (for example the Open Data Barometer and the Open Data Index). Anti-Corruption Indicators Unlike the previous indicator sets, which measure the performance of a public administration function (procurement or tax administration) or a sector (justice), the anti-corruption indicators measure the outcome of poor public sector governance. As a result, the list of anti-corruption indicators is organized around specific types of corruption rather than governance principles. For the purpose of these indicators, “corruption” is defined as the abuse of public power (by elected politicians or appointed civil servants) for private/personal gain. This definition expands on the original one by Susan Rose-Ackerman (Rose- Ackerman 1999) and captures a wider set of manifestations of corruption, from petty corruption to corruption in procurement processes and state capture. Corruption tends to be found where governance is weak, and there is limited implementation of principles like transparency, accountability, and participation, rather than as a specific public sector function. Figure 2 illustrates the manifestations of poor governance and their associated forms of corruption. The Bank team identified the following types of corrupt transactions to include in the indicator set: (i) state capture; (ii) nepotism and patronage; (iii) embezzlement of budget; (iv) administrative/petty corruption; (v) corruption in public procurement; and (vi) purchase of positions/jobs within the public administration. While there is a well-established body of knowledge on measuring petty corruption and corruption in public procurement, this is less so for other manifestations of corruption like state capture. Phenomena like nepotism have been studied, but few indicators with cross-country coverage exist. Such indicators have been included in the indicator set provided their approach and methodology are robust (even if they may have limited coverage). The indicators included in the final list focus on corruption that occurs between civil servants or elected officials and private citizens. It does not include any misconduct that may occur between private actors (businesspeople, citizens) such as collusion or tax evasion. There are a wealth of approaches and indicators to measure corruption. The Bank team focused on indicators that capture actual experiences—as opposed to generic perceptions—as much as possible. Thus, the indicators within each corruption category are organized by a hierarchy based on the strength of methodology. Data based on experiences is listed first, followed by evidence-based expert assessments, expert assessments and, finally, broader survey data. The team also tried to focus on methodologies and approaches that could be employed at the subnational level and did not include indicators that only capture corruption at the national level (the Worldwide Governance Indicators or the Bertelsmann Transformation Indicators, for example). Several indicators are not currently applied at the subnational level (for example, the World Justice Project, Global Competitiveness Index, and regional barometers). Adaptation of existing methodologies and additional resources might be required to apply those indicators at the subnational level. The shortlist of anti-corruption indicators does not include indicators of corruption associated with procurement and PFM functions; those are included in the respective indicator lists. 10 Figure 2. Manifestations of poor governance Nepotism Patronage Politicians/ State capture policymakers Embezzlement of budget Client power/ Public Citizens/ social accountability servants businesses Administrative corruption Corruption in public procurement Purchasing of HR positions Annex 2 includes information on the data sources, methodologies, frequency of measurement, and coverage for each of the datasets used. The shortlist of anti-corruption indicators uses datasets from different stakeholders and actors to triangulate the information available and reduce imprecision and margin of errors in the measures for the different types of corruption. 11 III. Next Steps—Possible Uses and Extension of the PSG Indicators The actionable PSG indicators included in Annex 2 provide DG REGIO and EU Member States with the tools to measure the capacity and performance of key public administration functions (PFM, procurement, tax administration, justice systems, and human resource management) and public sector characteristics (public accountability mechanisms and anti-corruption) at the national and subnational levels. These indicators can be used to identify public administration capacity and performance constraints that hinder the ability of EU Member States and their regions to use European Structural and Investment Funds to promote investment and economic growth. In this section, the Bank team identifies some specific ways that the PSG indicators can be used by DG REGIO, other Directorates-General, and EU Member States, and provides suggestions for possible extensions of some of the indicators. Monitoring Reforms under the EU Budget. The actionable PSG indicators could be used by the European Commission to identify public administration functions and areas that EU Member States may commit to reforming under the EU’s next budget cycle. The indicators also could be used to monitor the implementation of any public administration reform commitments. The actionable nature of the PSG indicators can help EU Member States measure results of their reform programs. If a Member State is planning to improve the efficiency of its judiciary or the accountability of its tax administration for example, the Member State could be encouraged to use the indicators for this area to monitor performance as reforms are implemented. Alternatively, civil society organizations could use the PSG indicators to provide third- party monitoring of a Member State’s public administration reforms. More broadly, the PSG indicators could be used to identify specific national or subnational reforms under the Cohesion Policy’s thematic objective of improving the efficiency of public administration. Assuming that a portion of the next cycle of Structural and Investment Funds will continue to target institutional capacity building and improving the efficiency of public administrations through country-specific recommendations, the PSG indicators could help to measure the outputs and impact of this capacity building assistance. Eurostat and National Statistical Agencies. Using the leverage or financing of the European Commission, data for the construction of some PSG indicators could be collected directly through Eurostat’s community- wide statistics. Alternatively, Eurostat could require the national statistical agencies of EU Member States to collect data for specific PSG indicators. Doing so would allow Member States to have more evidenced- based policy debates. Survey-based indicators from the anti-corruption PSG indicator set, for example, were included in a crime victimization survey conducted by Italy’s national statistical agency (ISTAT). Similar questions could be included in surveys carried out by other national statistical agencies or in Eurostat’s Eurobarometer Flash or special surveys focused on corruption or other public administration functions such as procurement or justice services. Questions from the HRM indicators could also be used individually or as a set to collect information on the civil service practices through national surveys of civil servants or in broader EU-wide efforts. Data Collection by EU Member States. Several individual PSG indicators are based on administrative data that national or subnational institutions may already collect through existing monitoring and evaluation systems. Indicators on the efficiency of procurement procedures and outcomes from public procurement activities, for example, should be generated by existing public procurement systems at the national and subnational levels. The same applies for indicators on the timeliness of hearings or appeal ratios, which should be captured by justice sector administrative or case management systems. The absence of the collection of this administrative data could be taken as a signal of the weakness in a country’s justice or procurement systems. Member States may need to be incentivized to collect data for these administratively- based PSG indicators—possibly through the EU budget process (as noted above), the European Semester 12 monitoring process, or through requirements set by national statistical agencies—if they are not already collected under their standard monitoring processes. Also, civil society organizations could collect data based on the PSG indicators to hold governments accountable for their performance. For justice system indicators to be meaningfully developed and monitored at a subnational/NUTS 2 level across the European Union and within EU Member States, it will be crucial for all participating EU Member States and subnational units to take measures to enable them to report reliably on indicators. For some countries, doing so will require adding new categories to their information systems; for others, it will be considerably less onerous. Ultimately, such improvements in statistics should be encouraged, as they generate diagnostics that can be used to drive performance improvements by participating countries, NUTS 2 statistical regions, policymakers, and even court presidents, judges, and court administrators. Data Collection by the World Bank Team. Several PSG indicator sets include indicators that have not typically been applied to EU Member States or collected at the subnational level; this is the case for the PEFA and PIMA-based PFM indicators and many of the anti-corruption and HRM survey-based indicators. For some of these PSG indicator sets, the World Bank could collect data at the national level or could pilot data collection at the subnational level to test the feasibility of the indicator sets. The Bank team could conduct surveys of public employees (at the national and subnational levels) to develop perception-based measures of organizational and human resource practices in a bureaucracy using the full set of HRM indicators. Similarly, the Bank team could conduct a PFM assessment—of EU Member States or various subnational/NUTS 2 regions within a Member State—using the PEFA and PIMA indicators in the PFM indicator set to test the applicability of these indicators for subnational financial systems. Also, the Bank team could use the anti-corruption indicators to conduct a targeted assessment of a sector, ministry or subnational government to identify specific transparency and accountability weaknesses that could be vulnerable to exploitation and corruption. Expansion of PSG Indicators. Some of the PSG indicator sets include relatively new indicators that have only been piloted or applied in non-EU countries (and often not at the subnational level); this is particularly true for the public accountability, HRM, and anti-corruption indicator sets. The Bank team could expand these indicators by applying them to EU Member States and their subnational administrations. For example, the team could expand the PAM indicators by piloting the newer, de facto indicators developed under the World Bank’s work on the PAM implementation and RIDE indicator sets. The Bank team could also expand the PAM indicator set by developing new indicators focused on public sector ethics systems and media ownership—two areas where additional research and consensus building would be needed to develop actionable indicators that can measure the implementation of these accountability and transparency tools. 13 IV. References European Commission. 2014. Investment for Jobs and Growth—Sixth Report on Economic, Social and Territorial Cohesion. Directorate-General for Regional and Urban Policy. Brussels: European Commission. IMF (International Monetary Fund). 2014. Government Finance Statistics Manual 2014. Washington, DC: IMF. Available at http://www.imf.org/external/Pubs/FT/GFS/Manual/2014/gfsfinal.pdf. Rose-Ackerman, Susan. 1999. Corruption and Government: Causes, Consequences and Reform. United Kingdom: Cambridge University Press. World Bank. 2017. World Development Report 2017: Governance and the Law. Washington, DC: World Bank. Available at https://www.worldbank.org/en/publication/wdr2017. 14 Annex 1: Definitions of Governance Variables Table 1A.1. Governance Principles in Public Financial Management (PFM) Systems Governance variable Governance sub-category Definition/scope of coverage Efficiency Efficiency of budget system The budget process including formulation, approval, execution, internal control, monitoring, and audit is carried out in an efficient way Efficiency of public Public investment projects are managed, planned, investment management appraised, executed, monitored, and reviewed in an system efficient way Effectiveness Effectiveness of budget Effective management of fiscal strategy, budget system process (formulation, approval, execution, internal control, monitoring, and audit), assets and liabilities Effectiveness of public Effective management of public investment strategy, investment management appraisal process, independent review, budget, implementation, assets and liabilities Reliability Reliability and predictability The government budget is realistic and is implemented of budget as intended Transparency Budget transparency Information on PFM (budget classification, published information on budget documents) is comprehensive, consistent, and accessible to users Accessibility to budget Budget documents are publicly made available and documents citizens access to these documents Comprehensiveness in Information provided in the budget documents is budget document comprehensive Accountability Effectiveness of external Review of public finances and the follow-up on the scrutiny and audit implementation of recommendations made by audit Effectiveness of accounting Maintenance of accurate and reliable records, and reporting system production and dissemination of information for decision-making, management, and reporting needs Public engagement in the Engagement of citizen in the budget process including budget process the access to information, and the institutions’ provision of opportunities for the citizens to engage Table 1A.2. Governance Principles in Procurement Systems Governance variable Definition/scope of coverage Efficiency Procurement process uses the minimum resources (time and money) necessary to complete the contracting process successfully Effectiveness Procurement process results in the specific need being met or satisfied through the procurement contract and at the best available price; procurement contract meets expectations of usefulness and cost; includes cost benefit analysis Integrity Well-functioning procurement system requires that all participants observe and adhere to the highest standards of ethics during the procurement and implementation of contracts; participants should refrain from fraudulent or corrupt practices; systems should be in place to limit, detect and deter such practices; procurement system should provide fair and equal treatment to all participants in the procurement and complaints process Transparency All relevant aspects of the procurement process are available for public scrutiny; documentation is disclosed and sufficient information is made publicly available to all interested parties in a consistent and timely manner 15 Table 1A.3. Governance Principles in Tax Administration Systems Governance variable Governance sub- Definition/scope of coverage category Effectiveness Effective revenue The ability of the tax administration to forecast revenue management and to adequately account tax revenue Effective risk management The ability of the tax administration to manage taxpayer compliance risks Effective tax dispute The existence, relevance and the ability of tax dispute resolution resolution mechanism Effective internal The existence, relevance and the ability of a sound audit/internal control internal control and audit mechanism Ensuring accurate The existence, relevance and the ability of verification reporting in declarations system in tax declarations and monitoring of inaccurate reporting Integrity Integrity of the registered The extent to which tax administration holds adequate taxpayer base and reliable taxpayers’ information Corruption The numbers of firms expected to give gifts in meetings with tax officials Efficiency Supporting voluntary The extent of tax administrations’ initiatives to reduce compliance taxpayer compliance costs and using taxpayer feedback at the lowest possible costs and in a timely manner Timely filing of tax The extent of taxpayers’ timely filing of tax declarations declarations and the use of electronic filing at the lowest possible costs and in a timely manner Timely payment of taxes Efficient tax collection systems, actual payments, and the use of electronic payments at the lowest possible costs and in a timely manner Efficient tax arrears Collection and management of stock and flow of tax management arrears information at the lowest possible costs and in a timely manner Timely tax refund Time to comply with and to obtain tax refund, and time to comply with and complete tax audit Effect of taxation on The extent fiscal measures (subsidies, tax breaks, etc.) investment distort market competition Labor market efficiency The effect of taxation on incentives to work Regulation and taxes The frequency of visits or required meetings with tax officials Accountability, Accountability and The extent of internal and external oversight of tax Transparency Transparency on tax administration, public perception of integrity, and administration publication of plans and results Making information on tax The degree of the public to have the access to expenditures public information related to tax administration Table 1A.4. Governance Principles in Human Resource Management (HRM) Systems Governance variable Definition/scope of coverage Characteristics of Basic characteristics of public employment, including total employment, share of public employment public sector employment in the labor market, and gender, age, and educational profiles of public employees. These indicators are meant to provide some broad brush descriptive statistics of the public sector that are useful for comparative purposes. 16 Governance variable Definition/scope of coverage Merit-based recruitment The underlying assumption is that access to public sector employment and and career advancement advancement on merit and competition, rather than on status or political connections, is necessary both to ensure equal access to public employment as a fundamental citizenship right and to ensure that those who are the best qualified get the job. Public service These indicators try to capture the levels of intrinsic motivation of public employees, motivation their levels of satisfaction in their jobs, and their perceptions of advancement along markers of career value. Quality of management These indicators draw on the methodology used in the World Management Survey to measure management practices in organizations and modify it for the public sector. The indicators capture a number of management practices, such as goal setting and communication of targets to staff, performance monitoring and people management. Competitiveness, The assumption is that wages should be attractive enough to ensure that qualified fairness, and incentives candidates seek employment to begin with, remain in the public sector and are of wages and benefits adequately motivated. Wages therefore, need to be sufficiently competitive with the broader labor market; wages also need to progress sufficiently over an employee’s career to motivate effort and public service. Capacity development Learning and development, including both formal and on-the-job training, is necessary to ensure that staff remain qualified to perform their jobs. It can also serve to motivate and engage employees. Table 1A.5. Dimensions of Justice System Performance Performance Definition/scope of coverage dimension Efficiency Timeliness of dispute resolution, including various steps in case flow, reduction/prevention of delay and backlog Ability of the justice system to produce a type/standard of service for the least amount of resources Timeliness of enforcement Use and effectiveness of Alternative Dispute Resolution (ADR) mechanisms Quality Ability of potential court users to access information about the law, courts and procedure Affordability of justice services Provision of legal aid to the poor/vulnerable especially in civil/administrative justice Availability of ICT tools for court users (e-justice) Perceived fairness in decision making Professionalism and competency of personnel (judges, associates, and bailiffs) Quality of decision making (analysis of appeal and overturn rates) Predictability of processes and outcomes across the jurisdiction (for example, presence of excessive variation at subnational level between the same type of courts in different regions of a country undermines principle of equality and predictability of decisions) Independence Corruption in the judiciary, especially reported corruption Integrity of judiciary and the extent of undue influence over the judiciary 17 Table 1A.6. Governance Principles in Public Accountability Mechanisms (PAM) Governance variable Definition/scope of coverage Transparency Access to information (rights; rules and regulations; monitoring) Transparency in managing of public resources by elected officials and civil servants (financial disclosure) Transparency in policy and decision making Open government Accountability Accountability in managing public resources by elected officials and civil servants (conflict of interest) Accountability in decision making Accountability in allocation of public resources 18 Annex 2: Public Sector Governance Indicators List Table 2A.1. Public Expenditure and Financial Accountability (PEFA) Governance variable Governance sub- Indicator What it measures Source variable Reliability Reliability of PI-1. Aggregate expenditure outturn This indicator measures the extent to PEFA is a budget • PI-1.1. Aggregate expenditure outturn which aggregate budget expenditure third-party outturn reflects the amount originally expert approved, as defined in government assessment, budget documentation and fiscal and each reports. dimension is scored A, B, C, or D PI-2. Expenditure composition outturn This indicator measures the extent to PEFA • PI-2.1. Expenditure composition outturn which reallocations between the main by function budget categories during execution • PI-2.2. Expenditure composition outturn have contributed to variance in by economic type expenditure composition. • PI-2.3. Expenditure from contingency reserves PI-3. Revenue outturn This indicator measures the change PEFA • PI-3.1. Revenue aggregate outturn in revenue between the original • PI-3.2. Revenue composition outturn approved budget and end-of-year outturn. Transparency Transparency of PI-6. Subnational operations outside financial This indicator measures the extent to PEFA public finances reports2 which subnational government • PI-6.1. Expenditure outside financial revenue and expenditure are reported reports outside subnational government. • PI-6.2. Revenue outside financial reports financial reports and/or consolidated • PI-6.3. Financial reports of extra general government financial report. budgetary units 2 Indicators highlighted in bold apply specifically to subnational government and their budgets and finances. 19 Governance variable Governance sub- Indicator What it measures Source variable PI-7. Transfers to subnational governments This indicator assesses the PEFA • PI-7.1. System for allocating transfers transparency and timeliness of • PI-7.2. Timelines of information on transfers from central government to transfers subnational governments with direct financial relationships to it. It considers the basis for transfers from central government and whether subnational governments receive information on their allocations in time to facilitate budget planning. Efficiency, Management of PI-10. Fiscal risk reporting This indicator measures the extent to PEFA Effectiveness, assets and • PI-10.1. Monitoring of public which fiscal risks to central Transparency liabilities corporations government are reported. Fiscal risks • PI-10.2. Monitoring of subnational can arise from adverse governments macroeconomic situations, financial • PI-10.3. Contingent liabilities and other positions of subnational governments fiscal risks or public corporations, and contingent liabilities from the central government’s own programs and activities, including extra budgetary units. PI-10.2 can apply to subnational government level below the subnational governments monitored. PI-12. Public asset management This indicator assesses the PEFA • PI-12.1. Financial asset monitoring management and monitoring of • PI-12.2. Nonfinancial asset monitoring government assets and the • PI-12.3. Transparency of asset disposal transparency of asset disposal. PI-13. Debt management This indicator assesses the PEFA • PI-13.1. Recording and reporting of debt management of domestic and foreign and guarantees debt and guarantees. It seeks to • PI-13.2. Approval of debt and guarantees identify whether satisfactory • PI-13.3. Debt management strategy management practices, records, and controls are in place to ensure efficient and effective arrangements. PI-18. Legislative scrutiny of budgets This indicator assesses the nature and PEFA • PI-18.1. Scope of budget scrutiny extent of legislative scrutiny of the annual budget. It considers the extent 20 Governance variable Governance sub- Indicator What it measures Source variable • PI-18.2. Legislative procedures for budget to which the legislature scrutinizes, scrutiny debates, and approves the annual • PI-18.3. Timing of budget approval budget, including the extent to which • PI-18.4. Rules for budget adjustments by the legislature’s procedures for the executive scrutiny are well established and adhered to. The indicator also assesses the existence of rules for in- year amendments to the budget without ex-ante approval by the legislature. Reliability Predictability and PI-21. Predictability of in-year resource allocation This indicator assesses the extent to PEFA control in budget • PI-21.1. Consolidation of cash balances which the central Ministry of Finance execution • PI-21.2 Cash forecasting and monitoring or local government department of • PI-21.3. Information on commitment finance (depending on the PFM ceilings model used) is able to forecast cash • PI-21.4. Significance of in-year budget commitments and requirements and adjustments to provide reliable information on the availability of funds to budgetary units for service delivery. PI-22. Expenditure arrears This indicator measures the extent to PEFA • PI-22.1. Stock of expenditure arrears which there is a stock of arrears, and • PI-22.2. Expenditure arrears monitoring the extent to which a systemic arrears problem in this regard is being addressed and brought under control. PI-26. Internal audit This indicator assesses the standards PEFA • PI-26.1. Coverage of internal audit and procedures applied in internal • PI-26.2. Nature of audits and standards audits. applied • PI-26.3. Implementation of internal audits and reporting • PI-26.4. Response to internal audits Accountability Accounting and PI-27. Financial data integrity This indicator assesses the extent to PEFA reporting • PI-27.1. Bank account reconciliation which treasury bank accounts, • PI-27.2. Suspense accounts suspense accounts, and advance • PI-27.3. Advance accounts accounts are regularly reconciled and • PI-27.4. Financial data integrity processes how the processes in place support the integrity of financial data. 21 Governance variable Governance sub- Indicator What it measures Source variable Accountability External scrutiny PI-30. External audit This indicator examines the PEFA and audit • PI-30.1. Audit coverage and standards characteristics of external audit. • PI-30.2. Submission of audit reports to the legislature • PI-30.3. External audit follow-up • PI-30.4. Supreme audit institution independence 22 Table 2A.2. Public Investment Management Assessment (PIMA) Short-listed indicators are those that can measure PIM performance quantitatively. Governance variable Governance sub- Indicator What it measures Source variable Efficiency Planning sustainable Fiscal principles or rules Measures existence of permanent PIMA is a levels of public • 1.a. Is fiscal policy guided by one or more fiscal principles or rules that support third-party investment permanent fiscal principles, or rules? sustainable levels of capital expert • 1.b. Do fiscal principles or rules protect spending. assessment. capital spending over the short term or (Details of the medium term? methodology • 1.c. Is there a target or limit for are being government liabilities, debt, or net worth? finalized through discussions between IMF and the World Bank) Efficiency Ensuring public Budget comprehensiveness and unity Measures the extent to which capital PIMA investment is • 7.a. Is capital spending mostly undertaken spending is undertaken through the allocated to the right through the budget? budget. sectors and projects • 7.b. Are externally funded (including donor funded projects and PPPs) capital projects shown in the budget documentation? • 7.c. Are capital and recurrent budgets prepared and presented together? Efficiency Delivering Availability of funding Measures whether capital projects are PIMA productive and • 12.a. Are ministries/agencies able to plan well managed and controlled during durable public assets and commit expenditure on capital the execution stage. projects in advance on the basis of reliable cash flow forecasts? • 12.b. Is cash for project outlays released in a timely manner? • 12.c. Is external (donor) financing of capital projects integrated into cash 23 Governance variable Governance sub- Indicator What it measures Source variable management and the Treasury Single Account (TSA)? Table 2A.3. PEFA-Style Public Investment Management Indicators (PEFA-PIMI) Short-listed indicators are those that can measure PIM performance quantitatively. Governance variable Governance sub- Indicator What it Measures Source variable Efficiency, Budget outturn PIM-11 Development and capital budget execution The ability to implement the PEFA-PIMI is Effectiveness performance rates: Aggregate expenditure outturn compared to development or investment budget a third-party original budget on a commitment basis including donor funds. expert • Budget execution rate assessment, • Donor budget execution rate which follows PEFA methodology. It assesses each dimension and assign a score of A, B, C, or D. PIM-12 Composition of development and capital The degree of variance in the PEFA-PIMI expenditure outturn compared to adjusted original composition of capital spending budget on a commitment basis across ministries/agencies. • Composite variance of capital expenditures across ministries, departments and agencies (MDAs) PIM-13 Project completion time and cost variances Cost and time overruns of projects. PEFA-PIMI of completed projects • Project completion time and cost variances of completed projects 24 Governance variable Governance sub- Indicator What it Measures Source variable PIM-14 Stock and monitoring of capital expenditure The existence of overdue PEFA-PIMI arrears obligations to pay suppliers or • Relative size and growth in arrears contractors for capital expenditures. • Availability of data Table 2A.4. Debt Management Performance Assessment (DeMPA) Short-listed indicators are those that cover the debt management cycle. Governance variable Governance sub- Indicator What it measures Source variable Effectiveness Governance and DPI-1 Legal framework Ensures that the legal framework DeMPA is a strategy • The existence, coverage, and content of the clearly sets out the authority to third-party development legal framework on authorization to borrow (in both domestic and expert borrow and undertake other debt foreign markets), undertake debt- assessment, management activities and to issue loan related transactions (such as debt which follows guarantees exchanges and currency and interest PEFA rate swaps, where applicable), and methodology. issue loan guarantees. It assesses each dimension and assign a score of A, B, C, or D. DPI-4 Debt reporting and evaluation Ensures that (i) the central DeMPA • Publication of a statistical bulletin on debt, government periodically prepares loan guarantees and debt-related operations and publishes a debt statistical • Reporting to parliament or legislature bulletin (or its equivalent; (ii) the government is accountable for its debt management operations to the parliament or legislature; and (iii) accountable to the country’s citizens by providing frequent reports on debt management and debt-related operations and making these reports publicly available. 25 Governance variable Governance sub- Indicator What it measures Source variable DPI-5 Audit Ensures that (i) the debt DeMPA • Frequency and comprehensiveness of management activities, policies, and financial audits, compliance audits, and operations are subject to scrutiny by performance audits (of the effectiveness the national audit bodies; and (ii) and efficiency of government debt the relevant government decision management operations, including the makers are committed to address the internal control system and its outcomes from the audits. effectiveness) as well as publication of the external audit reports • Degree of commitment to address the outcomes from the audits Efficiency, Borrowing and DPI-8 Domestic borrowing Ensures that (i) domestic borrowing DeMPA Transparency related financing • The extent to which market-based activities of the central government activities mechanisms are used to issue debt; the are conducted through market-based preparation of an annual plan for the instruments and that the borrowing aggregate amount of borrowing in the is undertaken in a transparent and domestic market, divided between the predictable manner; and (ii) written wholesale and retail markets; and the procedures are prepared for all publication of a borrowing calendar for domestic borrowing operations, that wholesale securities the terms and conditions of the • The availability and quality of instruments issued are made public, documented procedures for borrowing in and that regular interaction with the the domestic market and interactions with market participants takes place. market participants Effectiveness Cash flow DPI-11 Cash flow forecasting and cash balance Ensures that (i) reasonably reliable DeMPA forecasting and management forecasts of the central government cash balance • Effectiveness of forecasting the aggregate cash balance are produced and management level of cash balances in government bank available to the principal debt accounts management entity (or entities); and • Decision of a proper cash balance (ii) the cash balance is actively (liquidity buffer) and effectiveness of managed and short-term issuances managing this cash balance in are planned according to the cash government bank accounts (including the balance forecast. integration with any domestic debt borrowing program, if required) 26 Governance variable Governance sub- Indicator What it measures Source variable Accountability Debt recording and DPI-12 Debt administration and data security Ensures that (i) there are DeMPA operational risk • Availability and quality of documented documented procedures for the management procedures for the processing of debt- processing of debt-related related payments payments; (ii) there are documented • Availability and quality of documented procedures for the debt data procedures for debt and transaction data recording (new debt, disbursements, recording and validation, as well as and repayments) and validation as storage of agreements and debt well as for storage of agreements administration records and debt administration records; (iii) • Availability and quality of documented there are documented procedures for procedures for controlling access to the controlling access to the debt central government’s debt data recording recording and management system; and management system and audit trail and (iv) debt recording and • Frequency and off-site, secure storage of management system backups are debt recording and management system made frequently and that the backups backups are stored in a separate and secure location. 27 Table 2A.5. Open Budget Survey Short-listed indicators are those that focuses on accessibility, transparency, and citizen engagement. Governance variable Governance sub- Indicator3 What it measures Source variable Transparency Accessibility of key Public availability of key budget documents Survey asks a series of questions to Open Budget budget documents • Pre-Budget Statement must be released determine whether the key budget Survey is a at least one month before the Executive’s documents are made publicly third-party Budget Proposal is submitted to the available according to international assessment legislature for consideration. practices. conducted by Indicators highlight the key researchers in • Executive’s Budget Proposal must be transparency issues and accepted the civil released while the legislature is still timelines for accessibility of budget society in each considering it and before it is approved. information. country. In no case would a proposal released after the legislature has approved it be considered “publicly available.” • Enacted Budget must be released no later than three months after the budget is approved by the legislature. • Citizens Budget must be published within the same timeframe as the underlying document. For example, a Citizens Budget for the Executive’s Budget Proposal must be released while the legislature is still considering the Executive’s Budget Proposal and before it is approved. • In-Year Reports must be released no later than three months after the reporting period ends. • Mid-Year Review must be released no later than three months after the reporting period ends. • Year-End Report must be released no later than 12 months after the end of the fiscal year (the reporting period). 3 The following questions apply to Pre-Budget Statement (PBS), Executive’s Budget Proposal and its supporting Documents (EBP), Enacted Budget (EB), Citizens Budget (CB), In-Year Reports (IYR), Mid-Year Review (MYR), Year-End Report (YER), and Audit Report (AR). 28 Governance variable Governance sub- Indicator3 What it measures Source variable • Audit Report must be released no later than 18 months after the end of the fiscal year (the reporting period). Transparency Comprehensive-ness Comprehensiveness of the Executive’s Budget This assesses comprehensiveness of Open Budget in budget documents Proposal (EBP) and supporting documents the Executive’s Budget Proposal Survey and supporting documents, whether • EBP must present expenditures for the the information is provided in the budget year by administrative, economic, EBP upon its submission to the functional and program classification and legislature for consideration and for at least two years after the budget how that information is presented. year by one of the classifications. EBP must present revenue estimates by source as well as provide an estimate for additional two years by at least category (tax/non-tax). • EBP must present amount and composition of debt structure at the end of budget year; new borrowing and interest. • EBP must provide assumptions and policies on which estimates are based including macroeconomic forecast, new policy proposals, and explain how the proposed budget is linked to policy objectives of the government. • EBP must provide non-financial targets (outputs and outcomes) to be achieved. • In addition, EBP must provide information on extra-budgetary funds; intergovernmental transfers, transfer to public corporations; tax expenditures; earmarked revenues; quasi-fiscal activities; financial and non-financial assets, and contingent liabilities • EBP should provide a consolidated view on government finances (both budgetary 29 Governance variable Governance sub- Indicator3 What it measures Source variable and extra-budgetary) for at least the budget year. Accountability Public engagement Public engagement in the budget process It assesses whether there are Open Budget in the budget • The budget process must provide opportunities for the public to Survey process mechanisms for the public to engage engage during each of the four with executive, legislature and SAI in phases of the budget process with budget formulation, monitoring all the relevant institutions (the implementation of the budget and executive, the legislature, and the external audit program planning. Supreme Audit Institution). • Information on such mechanisms should be clearly communicated to the public in a timely manner. • Relevant institutions should provide feedback on how citizens’ inputs were used throughout budget process. Table 2A.6. Description of PFM Indicator Sets Indicator Background Methodology Coverage and subnational Current status and cost application Public Public Expenditure and Financial PEFA is a third-party expert PEFA assessments have PEFA is widely used in Expenditure and Accountability (PEFA) is a assessment. The 31 indicators are focused on developing both developed and Financial framework for assessing the grouped into seven pillars and each countries and have included developing countries. Accountability strengths and weaknesses of public indicator includes one or more very few OECD and Assessments have been (PEFA) financial management (PFM) using performance dimensions. Each developed countries. A PEFA done periodically and at Framework 31 quantitative indicators to dimension is scored separately on a assessment has been non-standard intervals. 2016 Indicators measure performance. PEFA is four-point ordinal scale: A, B, C, or completed in one Member designed to provide a snapshot of D, according to precise criteria. The State (Croatia). An expert assessment PFM performance at specific highest score is warranted for an Assessments have also been costs for a full PEFA points in time using a methodology individual dimension if the core PFM done in Norway, and Turkey assessment average that can be replicated in successive element meets an internationally at the national level and $130,000. Costs can be assessments, allowing the recognized standard of good Switzerland at the subnational reduced if countries comparison of changes over time. performance. In order to justify a level. conduct a self-assessment PEFA assessments provide a particular score for a dimension, The 2016 update of the PEFA with more limited third- foundation for reform planning, every aspect specified in the scoring framework remains applicable party verification of data dialogue on strategy and priorities, requirements must be fulfilled. If the to subnational governments. and scores. and progress monitoring. requirements are only partly met, the PEFA Secretariat has 30 Indicator Background Methodology Coverage and subnational Current status and cost application Governments can use PEFA as a criteria are not satisfied and a lower published Supplementary tool to achieve sustainable score reflecting lower performance is guidance for subnational improvements in PFM practices by given. Dimension-specific scores are PEFA assessments, which measuring and monitoring aggregated to reach an overall score provides additional guidance performance against a set of for each indicator using an for applying the PEFA indicators across a range of appropriate method based on the methodology at the important PFM institutions, degree of linkage between the subnational level. Countries systems, and processes. individual dimensions. A score of C which have conducted reflects the basic level of subnational PEFA using the performance for each indicator and 2011 framework in Eastern dimension, consistent with good Europe and Central Asia international practices. There may be include Albania, Armenia, two situations in which no score can Croatia, Georgia, Kosovo, be allocated to an indicator or a Serbia, and Switzerland. dimension: when it is ‘not applicable’ to the government system assessed or the indicators are ‘not used’. The PEFA Secretariat has issued detailed guidelines on conducting PEFA assessments. Public The Public Investment PIMA is a third-party expert PIMA indicators are targeted PIMA is a new set of Investment Management Assessment (PIMA) assessment. Details of the to assess central government indicators developed by Management is a diagnostic tool for evaluating methodology are still being discussed PIM functions. The IMF the IMF in 2016. The IMF Assessment the quality of a country’s public and agreed between the IMF and the intends on piloting the PIMA and the World Bank (PIMA) investment management practices. World Bank. As proposed, PIMA indicators in 2017. Slovakia is established a joint working Indicators Drawing on cross-country datasets includes 23 indicators divided the only EC Member State group to harmonize the covering advanced, emerging, and grouped into 8 core PIM functions. listed for as a possible PIM indicators. A tentative low-income economies, the PIMA It is likely that each dimension will candidate for a pilot. agreement on the indicator evaluates: be scored based on the performance No guidelines have been set was reached in • trends in public investment of the system. developed yet for subnational November 2016. and the value of the public assessments. capital stock; Costs for expert • the efficiency of public assessments can be high investment in improving depending on data infrastructure coverage and availability. Costs can be quality; and reduced if countries • the quality of public conduct a self-assessment investment management at with more limited third- 31 Indicator Background Methodology Coverage and subnational Current status and cost application each stage of the public party verification of data investment cycle. and scores. Based on this evaluation, PIMA indicators identify a country’s relative strengths and weaknesses in public investment management and make a series of recommendations for improving the efficiency and impact of their investments. PEFA-style With the growing recognition that This is a third-party expert PEFA-PIMI are targeted to The PEFA-PIMI are still Public the quality of public investment assessment. Under the 2011 revision, assess central government in draft and have not been Investment management (PIM) determines the PIMI includes 30 indicators broken PIM functions. There is no officially launched. Management performance of the public sector, into the same categories as the PEFA guideline developed for As noted above, a joint Indicators the World Bank developed a set of assessments. These PEFA-style PIMI subnational government. World Bank-IMF working (PEFA-PIMI) PEFA-PIM Indicators (PIMI). In follow the PEFA methodology and group is currently 2008 PIMI identified eight “must assigns scores A, B, C, and D to each discussing the have” or core functions of the PIM indicator. consolidation of PEFA- subsystem of a PFM system PIMI with IMF-led PIMA measured by 17 indicators. A indicators. diagnostic set of PIMI was developed and tested on governments’ PIM systems by 2011. The PIMI had no indicators covering the core functions of project adjustment and service delivery. In 2011, a preliminary revised draft of a new indicator framework for the PIM systems of government was devised that essentially applied a PEFA style framework to the PIMI. Debt DeMPA is a tool for assessing This is a third-party expert A Subnational DeMPA DeMPA was applied in 22 Management performance through a assessment, which follows PEFA methodology has been countries, including Performance comprehensive set of performance methodology. developed and updated in Albania, Armenia, indicators spanning the full range 2016. The SBN DeMPA Kazakhstan and Moldova. of government debt management assesses subnational debt 32 Indicator Background Methodology Coverage and subnational Current status and cost application Assessment (DeM) functions such as issuance A set of 14 Debt Management management performance at Costs for expert (DeMPA) of loan guarantees, on-lending, Performance Indicators (DPIs) aim to the level of local government assessments can be high cash flow forecasting and cash measure government debt through a comprehensive set depending on data balance management. However, management performance and of indicators spanning the full availability. Costs can be the DeMPA does not assess the capture the elements that are critical range of subnational debt reduced if countries ability to manage the wider public to achieving sound DeM practices. management functions. The conduct a self-assessment debt, including debts of state- Each indicator in turn comprises revised SBN DeMPA tool with more limited third- owned enterprises that are not dimensions for assessment that comprises five core areas, 13 party verification of data guaranteed by the central reflect established sound practice. A DPIs and 31 dimensions, and scores. government. score of A, B, or C is assigned to which are applied to evaluate DeMPA is modeled after the PEFA each dimension depending on the the capacity of the subnational indicators. It can be considered a criteria listed. If the minimum borrower to manage the debt more detailed and comprehensive requirements set out in C are not met, portfolio. This methodology is assessment of government debt then a score of D should be assigned. applied starting September management than is currently 2016. reflected in the PEFA indicators. The two frameworks are complimentary: the DeMPA can be used to undertake a detailed assessment of the underlying factors leading to poor PEFA ratings in the DeM area. Alternatively, if the DeMPA exercise precedes a PEFA assessment, the latter can use the DeMPA results to inform its assessment of the relevant indicators. Open Budget The Open Budget Survey is led by The Open Budget Survey is a third- The focus of the Open Budget Open Budget Survey the International Budget party expert assessment that assesses Survey is central government Questionnaire is applied in Partnership (IBP). The Open the three components of a budget budget processes in countries 102 economies including Budget Questionnaire is designed accountability system: public of every level of income. Albania, Bulgaria, Croatia, to assist independent civil society availability of budget information; The IBP Open Budget the Czech Republic, researchers in understanding select opportunities for the public to Initiative implemented two France, Georgia, international good practice participate in the budget process; and rounds of subnational budget Germany, Italy, Norway, benchmarks for budget the strength of formal oversight transparency studies that were Portugal, Romania, Serbia, transparency and accountability institutions, including the legislature completed in 2011 and 2013. Slovakia, Slovenia, Spain, 33 Indicator Background Methodology Coverage and subnational Current status and cost application and applying these standards to the and the national audit office or (Croatia was included in the Sweden, Turkey, and the practices the researchers observe in supreme audit institution. 2011 round of subnational United Kingdom. IBP their countries. In addition, the The majority of the Survey questions assessments.) Each round of published country reports questionnaire is intended to link assess what occurs in practice, rather the subnational studies used a in 2010, 2012, 2015 and civil society research efforts on than what is required by law. different approach to develop 2017. these issues across countries by The data is collected through the the transparency assessment Costs for expert providing a common methodology Open Budget Questionnaire which methodologies and assessments can be high for investigating budget comprises of 140 questions instruments. IBP used these depending on data transparency and accountability. completed by researchers typically experiences to publish a availability. Costs can be The Open Budget Questionnaire based in the country surveyed and questionnaire and a reduced if countries forms the basis of the Open Budget from academic institutions or civil methodology in 2013 to aid conduct a self-assessment Index and Open Budget Survey society organizations. Each civil society organizations in with more limited third- results. questionnaire is reviewed by an conducting budget party verification of data anonymous peer reviewer who has transparency studies at the and scores. substantial working knowledge of the subnational level. budget systems in the relevant country. Most of the Survey questions require researchers to choose from five responses. Responses “a” or “b” describe best or good practice, with “a” indicating that the full standard is met or exceeded, and “b” indicating the basic elements of the standard have been met. Response “c” corresponds to minimal efforts to attain the relevant standard, while “d” indicates that the standard is not met at all. 34 Table 2A.7. Shortlist of Procurement Indicators Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application Efficiency Average duration Timeliness of Excessively long Study on administrative All MS Annual High (unless Both between the process procedures are often a capacity in the EU using e- (TED data at deadline for signal for inefficiency Administrative data procurement national submission of and high administrative from procurement system then level only) offers and the burden. Very short system low) announcement of procedures may TED database for the award indicate inaccuracy in contracts above EU decision execution, as proper procurement directives implementation of thresholds4 procurement procedures can be time-consuming Percentage of Procurements Reveals how well Core sector indicator5 Used on Annual Low Both failed that are cancelled system plans and Administrative data project-by- (TED data at procurements before the final implements from procurement project national award is made procurement process. system basis. Could level only) Larger the % failed be applied to more questions raised any about the procurement procurement process. system. E-procurement The degree of Extent of adoption and E-procurement uptake All MS Annual Low Both uptake:6 adoption of e- use of e-procurement report (Uptake • Uptake of e- procurement system can be indicator Administrative data report data Notification systems for of more sophisticated from procurement only at (online different procurement system. system national publication procurement level) of tender processes notices) 4Data from TED can be broken down by NUTS region (NUTS 1-3) of the buyer. 5Core sector indicators are those indicators identified by the World Bank to help measure the performance of Bank-funded activities across all Bank operations. These indicators were identified through a rigorous peer review process within the Bank. 6 When referring to the uptake percentages for e-Notification, e-Access, e-Submission and e-Invoicing in the e-Procurement Uptake Report, the report considers data measured in value of the procedures for 24 EU Member States while for three EU Member States (Belgium, Bulgaria, and Spain) data refers to the uptake measured in volume; in one EU Member State (Sweden) data refers to the estimation of the usage of specific functions. 35 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application • Uptake of e- Access (online access to tender documents 24/7 in an automatic manner) • Uptake of e- Submission • Uptake of e- invoicing Number and The extent to The greater the use of E-procurement system All MS Annual Low Both value (in $ and which a an e-procurement administrative data using e- (TED data at %) of contracting system—measured in procurement national procurement authority is able number of procurement TED database for level only) activities to use an e- activities or volume of contracts above the EU conducted procurement activities (in $)—the procurement directive through e- system. greater the efficiency thresholds procurement (and transparency) of systems relative the procurement to the total system. number of procurement activities conducted Effectiveness Share of Amount of Contracts with high Administrative data Unknown7 Annual High if done Both contracts with variance between price amendments from procurement manually, amendments contracting price reveals weaker contract system Could be low with e- exceeding 15% and management and applied to procurement in value implementation possible loss of value any price. 7 Indicators identified by World Bank procurement experts can be applied to any procurement system. We do not have information on their application to specific EU Member States. 36 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application for money in procurement procurement process. system Percentage or Competitiveness Higher levels of Administrative data Unknown Annual High if done Both number of of procurement competition are from procurement manually, (TED data contracts process. associated with system; TED database Could be low with e- only at awarded through better/lower prices for for contracts above EU applied to procurement national open competitive goods and services. procurement directives any level) bidding thresholds. procurement system Percentage or Competitiveness Allows for better Procurement system All MS Annual Low Both number of of procurement understanding of data; TED database for (TED data contracts process. frequency of use of contracts above EU only at awarded through different procurement procurement directives national other (non- procedures. thresholds level) competitive) procedures (such as restricted, negotiated, competitive dialogue and electronic auction) Average number Level of Administrative data Unknown Annual High if done Both of bidders competitiveness. from procurement manually, (TED data system; TED database Could be low with e- only at for contracts above EU applied to procurement national procurement directives any system level) thresholds procurement system Retention rate of Procurement Higher turnover could Data from procurement Unknown Annual Low Both trained staff turnover be signal problems with agencies or civil service procurement procurement agency or data. staff process; higher turnover associated with greater weakness in procurement process 37 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application Percentage of Professionalism Defining a curriculum Study on administrative All MS Annual Medium Both procurement of procurement of competencies is a capacity in the EU. (study on specialists with staff good instrument to administrati defined foster the Procurement system ve capacity competences professionalization of data. did not procurement staff. collect data for MS) Integrity Existence of PEFA Limited Annual Low (with Both independent application self- administrative Expert assessment to MS – assessment complaints only and system Croatia. verification) Could get data from procurement system Percentage of Informs bidders that Guidance for All MS Annual Low Both contracts that there are tools available practitioners on the (guidance include to review and challenge avoidance of the most suggested provisions for the procurement common errors in indicator but dispute process public procurement of data was not resolution projects funded by the collected) mechanisms ESIF Procurement system data Existence of risk Reveals First step in auditing Public procurement – All MS Annual Low Both management procurement the procurement Study on administrative (guidance system to help system’s process. Automated capacity in the EU suggested identify “red readiness for risk identification indicator but flags” in the self-monitoring systems can improve Procurement system data was not procurement and existence of efficiency and data. collected) process processes for effectiveness as well as managing risky ensuring integrity. activities. 38 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application Existence of Anti-corruption policy Public procurement – All MS Annual Low Both anti-corruption reveals that contracting Study on administrative (guidance policy for authority has thought capacity in the EU suggested procurement through corruption indicator but prevention measures. Procurement system data was not data. collected) Percentage of Bribe-taking in Measures level of BEEPs/Enterprise New MS Annual High (part of Both firms saying that public integrity in the Surveys (covered in periodic giving gifts are procurement. procurement process 2013) regional needed to secure survey of public contracts firms) Percentage of Bribe-offering Measures level of BEEPs/Enterprise New MS Annual High (part of Both contract value integrity in the Surveys (covered in periodic that would be procurement process 2013) regional required to be survey of paid in informal businesses) payments in order to secure the contract Percentage of Commonly found error European Court of All MS Annual Low Both contracts by auditors indicating Auditors Special Report (indicator awarded without possible on procurement in based on following any corrupt/fraudulent cohesion expenditure Court of procurement practices Auditors procedure Procurement system Report but data no data collected) Number of Fairness of Higher figure implies Data from complaint Unknown Annual Medium Both complaints filed process as lack of fairness in the system as a share of perceived by procurement process contracts competing awarded bidders Conflict of Level of fairness Conflicts of interest Global Integrity Annual Medium Both interest rules or in the selection could lead to less Indicators (and part of Selected MS regulations are process effective and efficient ISPMS) 39 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application enforced in the procurement. (can be done procurement Perception of lack of Expert assessment as self- process. fairness where conflicts assessment are found. with verification to lower cost) Percentage of Same as above Same as above Guidance for All MS Annual Low Both tenderers that practitioners on the (Indicator have declared avoidance of the most based on any conflict of common errors in Guidance interest (also any public procurement of but no data conflict of projects funded by the has been interest with ESIF collected) tenderers' relatives) when Procurement system submitting their data tenders Number of Relative level of Number of protests ISPMS Unknown Annual Medium Both complaints/prote complaints with helps to identify sts filed as a the procurement fairness of procurement Data from complaints share of process (measure process. Larger the and procurement contracts by number or number more issues systems awarded value) raised with process. Percentage of Documentation of Guidance for All MS Annual Low Both procurements complete process practitioners on the (Indicator with complete allows for better avoidance of the most based on documentation monitoring and auditing common errors in Guidance of entire and allows agency to public procurement of but no data procurement justify all key projects funded by the has been process decisions. European Structural and collected) Investment Funds (DG REGIO) Procurement system data 40 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application Transparency Procurement Existence of Collection of Methodology for Broadly Annual Low (with Both system has statistical procurement statistics is Assessing Procurement applied self- procedures and capacity prerequisite for many Systems (MAPS) (revised assessment systems for other indicators. framework and collecting and Expert assessment. for verification) monitoring consultation procurement in 2016) statistics Processes to Openness of Ensures fairness of PEFA criteria. Limited Annual Low (with Both submit or appeal complaints complaints system. application self- procurement process Information from to MS – assessment complaints are procurement complaints only and clearly defined system. Croatia. verification) and publicly available Information on Openness of Greater access to MAPS Broadly Annual Low (with Both procurement procurement information facilitates Expert assessment. applied self- processes easily process competition and outside (revised assessment accessible in monitoring. framework and media of wide for verification) circulation and consultation availability in 2016) % of contracts Number of Greater publication ISPMS Unknown Annual Low Both publicly contracts broadly increases outside Administrative data disclosed publicized monitoring of from procurement (usually for procurement process. system. contracts above a Publication of contract threshold) awards in TED is required for those contracts above the EU procurement directive thresholds. However, responsibility is not often fulfilled by 41 Governance Indicator What it Reason for including Data Availability/ Recommended Cost of National or variable measures Sources/Methodology coverage frequency of collection subnational collection application national procurement authorities. 42 Table 2A.8. Short List of Tax Administration Indicators Governance Governance Indicator What it measures Source variable sub-variable Effectiveness Effective Soundness of government tax revenue The extent of tax administration input to TADAT is a Revenue forecasting process government tax revenue forecasting and estimating. third-party expert Management assessment, and each dimension is scored A, B, C, or D Adequacy of the tax revenue accounting Adequacy of the tax administration’s tax revenue TADAT system accounting system. Effectiveness Effective Risk Identification, assessment, ranking, and It measures (i) the extent of intelligence gathering TADAT Management quantification of compliance risks and research to identify compliance risks in respect of the main tax obligations; and (ii) the process used to assess, rank and quantify taxpayer compliance risks. Effectiveness Effective Tax Existence of an independent, workable, It measures (i) the extent to which an appropriately TADAT Dispute and graduated dispute resolution process graduated mechanism of administrative and judicial Resolution review is available to and used by, taxpayers; (ii) whether the administrative review mechanism is independent of the audit process; (iii) whether information on the dispute process is published and whether taxpayers are explicitly made aware of it. Effectiveness Effective Internal Existence of a sound internal audit It measures (i) assurance provided by internal audit; TADAT audit/internal mechanism in the tax administration and (ii) staff integrity assurance mechanisms. control Effectiveness Ensuring Scope of verification actions taken to It measures (i) the nature and scope of the tax audit TADAT Accurate detect and deter inaccurate reporting program in place to detect and deter inaccurate Reporting in reporting; (ii) the extent of large-scale automated Declarations crosschecking to verify information reported in tax declarations. Monitoring the extent of inaccurate The soundness of the method/s used by the tax TADAT reporting administration to monitor the extent of inaccurate reporting. Integrity Integrity of the Accurate and reliable taxpayer It measures (i) the adequacy of information held in TADAT Registered information respect of registered taxpayers and the extent to Taxpayer Base which the registration database supports effective 43 Governance Governance Indicator What it measures Source variable sub-variable interactions with taxpayers and tax intermediaries; and (ii) the accuracy of information held in the registration database. Efficiency Supporting Initiatives to reduce taxpayer compliance The extent of initiatives to reduce taxpayer TADAT Voluntary costs compliance costs. Compliance Efficiency Timely Filing of On-time filing rate It measures (i) the number of specific local tax 8 TADAT Tax Declarations (property tax or some other taxes) declarations filed by the statutory due date as a percentage of the number of declarations expected from registered taxpayers for the specific tax; (ii) the number of pay as you earn (PAYE) withholding declarations filed by employers by the statutory due date as a percentage of the number of PAYE declarations expected from registered employers. Use of electronic filing facilities The extent to which tax declarations are filed TADAT electronically. Efficiency Timely Payment Use of electronic payment methods The extent to which core taxes are paid TADAT of Taxes electronically. Use of efficient collection systems The extent to which withholding at source and TADAT advance payment systems are used. Timeliness of payments It measures (i) the number of specific local tax 9 TADAT (property tax or some other taxes) payments made by the statutory due date in percent of the total number of payments due; and (ii) the value of specific local tax payments made by the statutory due date in percent of the total value of the specific local tax payments due. Efficiency Efficient Tax Stock and flow of tax arrears It measures (i) the value of total core tax arrears at TADAT Arrears fiscal year-end in percent of total core tax revenue Management collections for the fiscal year; (ii) the value of collectible core tax arrears at fiscal year-end in percent of total core tax revenue collections for the fiscal year; and (iii) the value of core tax arrears 8 In TADAT, this indicator is measuring corporate income tax (CIT), personal income tax (PIT), and value-added tax (VAT). For the application of subnational governments, it should be modified as the local tax collected at the jurisdiction. 9 In TADAT, this indicator is measuring VAT. For the application of subnational governments, it should be modified as the local tax collected at the jurisdiction. 44 Governance Governance Indicator What it measures Source variable sub-variable more than 12 months’ old in percent of the value of all core tax arrears. Accountability, Accountability Internal assurance mechanisms It measures (i) assurance provided by internal audit; TADAT Transparency and Transparency (ii) staff integrity assurance mechanisms. of Tax External oversight of the tax It measures (i) the extent of independent external TADAT Administration administration oversight of the tax administration’s operations and financial performance; (ii) the investigation process for suspected wrongdoing and maladministration. Public perception of integrity The mechanism for monitoring public confidence in TADAT the tax administration. Obtaining taxpayer feedback on products It measures (i) the use and frequency of methods to TADAT and services obtain feedback from taxpayers on the standard of services provided; and (ii) the extent to which taxpayer input is taken into account in the design of administrative processes and products. Accountability, Making Publication of activities, results, and It measures (i) the extent to which the financial and TADAT Transparency Information on plans operational performance of the tax administration is Tax Expenditures made public, and the timeliness of publication; and Public (ii) the extent to which the tax administration’s future directions and plans are made public, and the timeliness of publication. Regular, annual public statements on the The extent to which tax revenues are foregone due WB staff input extent of tax revenues foregone (i.e., tax to tax exemptions and tax incentives are made expenditures) as a result of tax publicly available. exemptions and tax incentives Efficiency Timely Payment Tax payments for a manufacturing It measures the total number of taxes and Doing Business of Taxes (Tax company in 2015 (number per year contributions paid, including consumption taxes 2017 Payments) adjusted for electronic and joint filing (VAT, sales tax or goods and service tax); the and payment) method of payment, the frequency of payment, the • Total number of taxes and frequency of filing and payment. contributions paid, including consumption taxes (VAT, sales tax or goods and service tax) • Method and frequency of filing and payment Efficiency Timely Payment Time required to comply with three The time taken to prepare, file and pay three major Doing Business of Taxes (Time) major taxes (hours per year) types of taxes and contributions: CIP, VAT or sales 2017 45 Governance Governance Indicator What it measures Source variable sub-variable • Collecting information and tax, and labor taxes, including payroll taxes and computing the tax payable social contributions. • Completing tax return forms, filing with proper agencies • Arranging payment or withholding • Preparing separate mandatory tax accounting books, if required Effectiveness Effective Total tax rate (% of profit before all The amount of taxes and mandatory contributions Doing Business Revenue taxes) borne by the business in the second year of 2017 Management • Profit or corporate income tax operation, expressed as a share of commercial (Total Tax Rate) • Social contributions and labor taxes profit. paid by the employer • Property and property transfer taxes • Dividend, capital gains and financial transactions taxes • Waste collection, vehicle, road and other taxes Efficiency Timely Tax Post-filing index Time to comply with VAT or GST refund, time to Doing Business Refund (Post- • Compliance time of a VAT or GST obtain VAT or GST refund, time to comply with 2017 filing index) refund process corporate income tax audit and time to complete a • Time to receive a VAT or GST corporate income tax audit. refund • Compliance time of correcting an error in the corporate income tax return including compliance with an audit process if applicable Efficiency Effect of Effect of taxation on incentives to invest The extent fiscal measures (subsidies, tax breaks, World Economic Taxation on and so on) distort competition Forum Global Investment Competitiveness Report 2016-2017 Efficiency Effective Total tax rate Sum of profit tax and other taxes (percent of profits) World Economic Revenue paid by businesses Forum Global Management Competitiveness (Total Tax Rate) Report 2016-2017 Efficiency Labor Market Effect of taxation on incentives to work The amount of taxes and mandatory contributions World Economic Efficiency on labor paid by the business as a percentage of Forum Global commercial profits 46 Governance Governance Indicator What it measures Source variable sub-variable Competitiveness Report 2016-2017 Integrity Corruption Percent of firms expected to give gifts in Percent of firms expected to give gifts or an BEEPs meetings with tax officials informal payment in meetings with tax officials (Enterprise Survey) Efficiency Regulation and Number of visits or required meetings Number of visits or required meetings with tax BEEPs Taxes with tax officials, and if there were officials and the average number of visits over (Enterprise visits, average number of visits or period of time. Survey) required meetings with tax officials Table 2A.9. Description of Tax Indicator Sets Indicator Background Methodology Coverage and subnational Current status and application cost Tax TADAT provides a standardized The assessment of indicators follows a similar TADAT was launched in 2015 The assessment costs Administration means of assessing the health of key approach to that followed in the Public with the support from the for a full TADAT Diagnostic components of a country’s tax Expenditure and Financial Accountability (PEFA) development partners assessment average Assessment Tool administration system and its level of diagnostic tool. including the European Union, $100,000 to $165,000. (TADAT) maturity in the context of international TADAT is a third-party expert assessment. The 28 Germany, Japan, Netherlands, Costs can be reduced good practice. high-level indicators are grouped into 9 pillars, Norway, Switzerland, and the if economies conduct a with total of 47 measurement dimensions. Each United Kingdom along with self-assessment with TADAT can be used to assess the indicator has one to four measurement dimensions. the IMF and the World Bank. more limited third relative strengths and weaknesses of These dimensions are taken into account in Among EU countries, the party verification of countries’ tax administration system at arriving at the indicator scores. Each of TADAT’s assessment was piloted in data and scores. the central level. The focus is on the 47 measurement dimensions is assessed separately. Norway (2014) and conducted administration of the major direct and The overall score for an indicator is based on the in Romania (2016). indirect taxes critical to central/federal An assessment usually assessment of the individual dimensions of the TADAT indicators are starts with a request government revenues referred to as indicator. These are scored on a four-point ‘core taxes’: corporate income tax intended to measure central from a country. A ‘ABCD’ scale according to specific scoring government tax administration team of 3-5 experts (CIT), personal income tax (PIT), criteria prescribed throughout the field guide. The value added tax (VAT) and Pay-As- systems and have yet to be will conduct the interpretation of these scores is broadly as follows: applied at subnational level. assessment over the You-Earn (PAYE) amounts withheld ‘A’ denotes performance that meets or exceeds by employers (remittances of PIT). course of a two-week international good practice; ‘B’ represents sound visit. Social security contributions (SSCs) performance; ‘C’ means weak performance may also be included in assessments relative to international good practice; ‘D’ denotes Some assessments where they are a major source of inadequate performance, and is applied when the have been funded by requirements for a ‘C’ rating or higher are not met. the assessed country 47 Indicator Background Methodology Coverage and subnational Current status and application cost government revenue and are collected Furthermore, a ‘D’ score is given in certain (more likely for EU by the tax administration. situations where there is insufficient information Member States) while available to assessors to determine and score the donors have funded level of performance. assessments for TADAT assessments are conducted by an developing economies. assessment team typically comprising 3 or 4 trained assessors. The team must include at least one trained assessor(s) with at least five years of tax administration experience. TADAT Secretariat has issued detailed field guide on conducting TADAT assessments. Doing Business Doing Business measures regulations Doing Business ranks economies on the ease of Doing Business project was Doing Business report 2017 affecting 11 areas of the life of a paying taxes. The ranks are determined by sorting launched in 2002, looks at is published annually. business. Paying taxes is one of those their distance to frontier scores for paying taxes. domestic small and medium- areas included in the 2017 version. These scores are the simple average of the distance size companies and measures Doing Business records the taxes and to frontier scores for each of the component the regulations applying to mandatory contributions that a indicators, with a threshold and a nonlinear them through their life cycle. medium-size company must pay in a transformation applied to one of the component Tax indictors are under the given year as well as the indicators, the total tax rate. pillar ‘paying taxes’. Only administrative burden of paying taxes three indicators are related and contributions and complying with with tax administration. post-filing procedures. Taxes and The project was developed and implemented by contributions measured include the the World Bank in cooperation with PwC. Doing Business indicators profit or corporate income tax, social have been applied in a number contributions and labor taxes paid by of economies at the the employer, property taxes, property subnational level including transfer taxes, dividend tax, capital Spain (2015), Poland (2015) gains tax, financial transactions tax, and Italy (2013). waste collection taxes, vehicle and road taxes, and any other small taxes or fees. World Economic The Global Competitiveness Index The GCI includes statistical data from The World Economic Forum Global Forum Global (GCI) was developed by Xavier Sala- international organizations, notably the has published the Global Competitiveness Competitiveness i-Martín in collaboration with the International Monetary Fund (IMF); the World Competitiveness Index (GCI) Report is published Report 2016– World Economic Forum. Bank; and various United Nations’ specialized since 2005. annually. 2017 agencies, including the International 48 Indicator Background Methodology Coverage and subnational Current status and application cost The GCI combines 114 indicators that Telecommunication Union, UNESCO, and the Tax indictors are under pillars capture concepts that matter for World Health Organization. The Index also of ‘goods market efficiency’, productivity and long-term prosperity. includes indicators derived from the World and ‘labor market efficiency’. These indicators are grouped into 12 Economic Forum’s Executive Opinion Survey that Only three indicators are pillars: institutions, infrastructure, reflect qualitative aspects of competitiveness, or related with tax macroeconomic environment, health for which comprehensive and comparable administration. and primary education, higher statistical data are not available for a sufficiently These indicators are not education and training, goods market large number of economies. intended to be applied at efficiency, labor market efficiency, subnational level. financial market development, technological readiness, market size, business sophistication, and innovation. These pillars are in turn organized into three sub-indexes: basic requirements, efficiency enhancers, and innovation and sophistication factors. The Report this year covers 138 economies included based on data availability. Business Enterprise Surveys offer an expansive Private contractors conduct the Enterprise Surveys Firm-level surveys have been For most economies, Environment and array of economic data on 125,000 on behalf of the World Bank. Due to sensitive conducted since the 1990's by an Enterprise Survey Enterprise firms in 139 economies. An Enterprise survey questions addressing business-government different units within the is conducted about Performance Survey is a firm-level survey of a relations and bribery-related topics, private World Bank since 2005–2006. every 3-4 years. With Survey (BEEPs representative sample of an economy's contractors, rather than any government agency or Tax indictors are under the the exception of or Enterprise private sector. The surveys cover 12 an organization/institution associated with areas of ‘corruption’ and Africa, survey projects Surveys) areas of business environment topics government, are hired by the World Bank to ‘regulation and taxes’. Only are usually including corruption, crime, finance, collect the data. The Enterprise Survey is answered two indicators are related with implemented on a firm characteristic, gender, by business owners and top managers. Sometimes tax administration. regional basis (Europe informality, infrastructure, innovation the survey respondent calls company accountants and Central Asia and technology, performance, and human resource managers into the interview to These indicators have been not countries' BEEPs are regulation and taxes, trade, and answer questions in the sales and labor sections of intended to be applied at conducted workforce measures. the survey. Typically, 1,200-1,800 interviews are subnational level. simultaneously). conducted in larger economies, 360 interviews are conducted in medium-sized economies, and for smaller economies, 150 interviews take place. 49 Table 2A.10. Short List of Human Resource Management (HRM) Indicators Governance variable Indicator What it measures Data source Characteristics of 1.1 Public sector employment as a proportion of total Size of the public sector (GFS definition) ILOSTAT public employment employment 1.2 Public sector employment as a proportion of paid The size of the public sector in relation to World Bank employment the formal sector labor market WWBI 1.3 Wage bill of public sector employment as a The size of the public sector in relation to proportion of total government expenditures the scale of government activity 1.4 Age profile of public and private sector workers The relative age profiles of public and private sector workers 1.5 Gender profile of public and private employment The proportion of males and females working in the public sector as compared to the private sector Merit-based recruitment 2.1 Proportion of applicants who think that Transparency and the use of merit in Survey of and career advancement recruitment is based on merit recruitment applicants to public 2.2 Applicant views on the extent to which the Attractiveness of the public sector as an employment recruitment process was competitive (number of employer qualified candidates willing to accept each available position) 2.3 Number of promotions (point increases in grade Opportunities for promotion Survey of along a standardized salary scale) that staff have employees received in their public sector career 50 Governance variable Indicator What it measures Data source 2.4 Proportion of staff stating merit as core criteria Performance, rather than seniority or used for promotion (point increases in grade along political connections, as a basis for a standardized salary scale) promotions 2.5 Staff views on whether management of their Management of staff department have mechanisms for attracting and retaining talented people Public service 3.1 Proportion of staff who believe that Mission orientation of the ministry, Survey of motivation management/their organization is contributing to department, or agency employees a worthwhile mission 3.2 Proportion of staff who are satisfied with their Overall levels of motivation experience in public service 3.3 Distribution of staff assessments of how their Trends in motivation levels motivation levels are now compared to when they joined the public sector 3.4 Proportion of staff who state that they know what Goal orientation of the staff their individual roles and responsibilities are in achieving the organization’s goals 3.5 Views of staff on whether they prefer working in Preference for working in the public sector the public sector as compared to the private sector Quality of management 4.1 Proportion of staff who believe that their Goal setting in the agency Survey of department has a clear set of targets derived from employees the organization’s goals and objectives 4.2 Proportion of staff who state that there is Extent of inter-agency coordination to sufficient co-ordination within and across the achieve the government’s policy objectives departments/units of the organization 4.3 Staff views on the extent to which the organization Monitoring of organizational performance tracks the achievement of goals and targets 4.4 Assessment of whether under-performing staff are Management of staff effectively handled by managers 4.5 Staff views on whether their department uses Management of staff performance targets for tracking and rewarding (financially or non-financially) the performance of its employees Competitiveness, 5.1 Average public and private sector wages Pay competitiveness of the public sector World Bank fairness, and incentive 5.2 Relative wages across government occupations Relative wages within government across a WWBI of wages standard set of occupations with the aim of understanding pay progression in the bureaucracy 51 Governance variable Indicator What it measures Data source 5.3 Public-private wage differential (based on Mincer Pay competitiveness of the public sector regressions controlling for worker characteristics) 5.4 Degree of staff satisfaction with their overall level Pay competitiveness of the public sector Survey of of compensation (including net benefits) employees 5.5 Staff views on the fairness of the compensation Transparency and equity in pay system within the public service Capacity and 6.1 Profile of highest academic qualifications Skill levels in the public sector relative to World Bank development amongst staff relative to private sector the private sector WWBI counterparts 6.2 Staff views on whether their organization has the Skill levels in the public sector Survey of job–relevant knowledge and skills necessary to employees accomplish organizational goals 6.3 Staff views on the extent to which managers and Skill levels in the public sector supervisors try to use the right staff for the right job 6.4 Perceptions of staff on opportunities for learning Opportunities for technical and managerial relevant to their assigned tasks training for public employees 6.5 Proportion of staff satisfied with the in-service Quality of the technical and managerial training that they have received training Table 2A.11. Description of HRM Indicator Sets Indicator Background Methodology Coverage and subnational Current status and application cost Surveys of The World Bank has developed and The surveys are done in collaboration with the The World Bank has The WB plans to do public implemented a range of surveys of government and in partnership with a university or conducted recent surveys in 2-3 surveys each employees public sector institutions and workers. research organization. Some of the surveys take the Ethiopia, Ghana, Indonesia, year. The average Surveys build a more rigorous and government organization as the unit of observation Liberia, Nigeria, Pakistan, cost of a public contextually nuanced understanding of focusing on organization-level features and practices, and the Philippines. employee survey that human resource and organizational while others focus on the public official as the unit of Subnational jurisdictions the WB has management practices and their effects analysis. All the surveys except one (Pakistan) used were included in the Nigeria administered is on the quality and motivation of public in-person enumeration which ensured very high and Ethiopia surveys. $100,000 to employees as well as on public sector response rates (>95%). The sampling frame ensured $150,000. Costs can productivity. The surveys cover a representation at multiple levels (e.g. at the level of be reduced if the variety of public sector organizations the organization and different categories of survey is not face-to- 52 Indicator Background Methodology Coverage and subnational Current status and application cost and sectors to provide data that allows employees, such as managers and staff). Survey face (that is. sent out for diagnostic work in public enumeration was done through a combination of via email), but the administration reform. face-to-face interviews and classroom style written tradeoff is lower responses. percentage of respondents which can bias the results. The World The Worldwide Bureaucracy Indicators The WWBI data are drawn mainly from primary The WWBI cover about 80 The WWBI dataset is Bank’s (WWBI) is a new dataset on public sources, specifically the World Bank’s International countries over the time in its pilot test phase Worldwide sector employment and wages that can Income Distribution Database (I2D2) database, the period 2009–2015. There is and is being Bureaucracy help researchers and development Luxembourg Income Study (LIS) and the currently no sub-national validated. It will be Indicators practitioners gain a better International Comparisons Program (ICP) wage coverage. expanded in country (WWBI) understanding of state capacity, its survey. The I2D2 harmonizes nationally European economies and time coverage impact on the quality of governance representative household surveys—both welfare and covered include: Estonia over the next year. and, ultimately, on economic labor force surveys—from around the world, (2013), Finland (2013), development. The WWBI aim to fill presenting data using the same variables and coding France (2010), Germany the gap in information on the personnel in each country and survey. The LIS similarly (2013), Greece, (2013), dimension of the state by providing harmonizes household surveys from a number of Ireland (2010), Italy (2009), more objective measures drawing on mostly high-income economies. The ICP is a global Luxembourg (2013), Poland administrative data and household statistical partnership that calculates Purchasing- (2009, 2013), Romania surveys, thereby complementing Power-Parity (PPP) prices based on detailed (2009), and the United existing, expert perception-based comparative price data, including government wages Kingdom (2013). approaches. The WWBI dataset for a standard set of occupations, and GDP encompass three categories of expenditure values in 199 economies. The WWBI variables: the characteristics of public also capture some data from secondary sources, employment, average wages, and the namely the International Labor Organization’s (ILO) wage bill. employment database (ILOSTAT), and the International Monetary Fund’s (IMF) new dataset on the wage bill. 53 Table 2A.12. Shortlist of Justice System Performance Indicators Performance indicator Description of what it measures Data source availability and cost (short title) Efficiency of Justice Systems 1. Clearance rate Measures the number of resolved cases as a percentage of the Data source/coverage: CEPEJ and national statistics in many, number of incoming cases. if not all, EU MS. Identifies whether a court is keeping up with its incoming Cost: Low. Information systems that do not already collect caseload or generating backlogs. A clearance rate below 100% such data should start doing so, as this is standard practice. indicates that backlogs are being generated. For this purpose, the Some adjustments may be required to information systems in Bank suggests to measure the clearance rate only of civil, some EU MS to allow data to be generated routinely commercial and administrative litigious cases. This may be electronically/automatically at the court/subnational level. disaggregated by case type and court type. It is important to exclude non-litigious cases, which distort the rate. 2. Judicial Measures the average number of resolved cases per judge by Data source/coverage: Information systems in many, if not productivity court type in the reporting period, usually one year. The median all, EU MS collect the data that would enable this indicator could also be collected. to be generated. Not all systems currently disaggregate between litigious and non-litigious cases. Gauges roughly how many cases a judge disposes in a year. This could be disaggregated by litigious and non-litigious cases. Cost: Low. Systems that do not already collect such data (It would be important to define the number of judges consistently should start doing so, as this is good practice. Some to ensure comparability of the dataset (i.e. approved positions or adjustments may be required to information systems in some actual positions at desk), possibly by using the CEPEJ EU MS to allow data to be generated routinely definition.) electronically/automatically at the court/subnational level. Doing Business tracks the availability of judicial productivity reports in all countries and can provide links/copies of these reports for all countries that have such info. 3. Judge-staff ratio Measures the ratio of the number of staff to the number of judges. Data-source/coverage: CEPEJ and national court statistics. Useful in identifying sub-national differences in resourcing and Coverage: All MS at national level. Most EU MS also collect linking them to differences in productivity. at the court level, manually if not automatically. Thus, data exists that would enable this indicator to be generated. Cost: Low. Systems that do not already collect such data should start doing so, as this is standard practice. Some adjustments may be required to information systems in some 54 Performance indicator Description of what it measures Data source availability and cost (short title) EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. 4. Congestion rate Measures the total caseload (number of incoming and pending Data source/coverage: Information systems in many, if not cases) divided by the number of resolved cases in a given all, EU MS collect the data that would enable this indicator reporting period, usually one year. to be generated, both nationally and at the court level. Reflects the time it would take a court to resolve its pending Cost: Low. Information systems that do not already collect cases, given its current efficiency and clearance rates. High such data should start doing so, as this is good practice. congestion rates signal the need to take measures to reduce Some adjustments may be required to systems in some EU backlog and delay. MS to allow data to be generated routinely electronically/automatically at the court/subnational level. Doing Business tracks the availability of congestion rates reports in all countries and can provide links/copies of these reports for all countries that have such info. 5. Cost per case Measures the total budget, divided by the number of cases Data source/coverage: Computation of this indicator ideally resolved in a given jurisdiction in a given reporting period, requires data on executed budget disaggregated at the usually one year. court/subnational level. It is good practice to do this, and several EU MS are able to do it. (Currently, some EU MS, Gauges the average cost of resolving a case in the jurisdiction. e.g. Sweden, collect this data, but no comparable dataset exists.) (NB: in some EU MS, it can be difficult to disaggregate budget data at subnational level to generate a cost per case per Cost: Low. Information systems that do not collect such data court/statistical region.) should start doing so, as this is good practice. Some adjustments may be required to systems in EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. 6. Age structure of Measures the share of first instance pending cases that are older Data source/coverage: Information systems in many, if not civil, commercial than 1 year and older than 3 years. all, EU MS collect the data that would enable this indicator and administrative to be generated, both nationally and at the court level. cases Figures are usually shown in percentages and by case type and court type. This is useful because it requires courts to create an Cost: Low. Information systems that do not already collect Ageing List, which is a useful performance monitoring tool. such data should start doing so, as this is standard practice. Some adjustments may be required to systems in some EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. 55 Performance indicator Description of what it measures Data source availability and cost (short title) Doing Business tracks the availability of age structure of cases reports in all countries and can provide links/copies of reports for all countries that have such info. 7. Time needed to Measures a proxy for the disposition time using the SATURN Data source: EU Justice Scoreboard resolve civil, method in days disaggregated by case type (civil, commercial commercial and and administrative). It would be good to also collect the median. Coverage: All MS are requested to provide data at the administrative national level. However, several MS do not currently provide litigious cases According to the SATURN method, this indicator compares the such data, especially data disaggregated by case type. using the SATURN total number of pending cases at the end of the observed period method with the number of resolved cases during the same period and Cost: Low. Some adjustments may be required to systems in converts this ratio into a number of days using the following some EU MS to allow data to be generated routinely formula: Calculated Disposition Time = (Number of pending electronically/automatically at the court/subnational level. cases at the end of a period/Number of resolved cases in a period) x 365. (NB: this creates a proxy for how long it takes to resolve a case, but is more an indicator of congestion than of timeliness. Note also that this measure is subject to fluctuation with backlog reduction efforts, so reliance on it can be counter-productive in environments where delay and backlog is a problem.) 8. Disposition time of Measures the average and median disposition times of actual Data source/coverage: Information systems in many, if not resolved civil, resolved cases, disaggregated by case type (civil, commercial and all, EU MS collect the data that would enable this indicator commercial and administrative litigious cases). to be generated, both nationally and at the court level. administrative litigious cases (Note: If/when all justice systems are able to reliably measure Cost: Low. Information systems that do not already collect this indicator, the SATURN indicator above would not be such data should start doing so, as this is good practice. needed.) Some adjustments may be required to systems in some EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. 9. Expeditiousness of Measures the percentage of firms that report that courts are Data source/coverage: BEEPS, covering Bulgaria, Croatia, courts ‘quick’. Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Slovak Republic, and Slovenia at the national level. Cost: Moderate/High. The methodology would need to be extended to all EU MS with sample sizes that allow disaggregation at the sub-national level. 56 Performance indicator Description of what it measures Data source availability and cost (short title) 10. Effectiveness of the Measures successful service of process/summons as a share of all Data source/coverage: Few countries measure this, although system for service attempts at service/summons it is a good practice. Adjustments would need to be of process introduced to information systems to collect such data. and/or Cost: Moderate. Adjustments would need to be introduced to Average length of time needed to carry out service (to be information systems to start collecting such data. Doing so disaggregated by court region). would be good practice. As an alternative data source, Doing Business collects the median time from filing to service for a hypothetical case. 11. Timeliness of Measures the average time in days needed to issue a response to a Data source/coverage: Not currently available. Adjustments uncontested request for enforcement of uncontested claims (e.g. orders for would need to be introduced to information systems to enforcement claims payment, enforceable titles based on authentic documents). It collect such data. would also be good to collect the median. Cost: Moderate. Adjustments would need to be introduced to Depending on the particular national system, these enforceable information systems to collect such data. titles could be issued by different entities: courts, notaries, an enforcement authority. World Bank recently analyzed enforcement and collection of uncontested claims through expert surveys in Central and Eastern Europe. The report is available here; however the design of an indicator would require a more quantitative method that periodically collects data from national systems rather than reliance on expert assessments. 12. Timeliness of first For existing caseloads, this would measure the average length of Data source/coverage: Information systems in many, if not hearing (for various time (in days) from filing to first hearing for different categories all, EU MS collect the data that would enable this indicator categories of of litigious cases. to be generated, both nationally and at the court level. litigious cases) It would also be good to collect the median. Cost: Low. Information systems that do not collect such data should start doing so, as this is good practice. Some adjustments will be required to systems in several EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. 13. Number of Using caseload data, this indicator would identify the total Data source/coverage: Information systems in many, if not hearings needed to number of hearings in litigious cases and divide that by the total all, EU MS collect the data that would enable this indicator resolve a certain number of resolved litigious cases in a jurisdiction in a reporting to be generated, both nationally and at the court level. type of litigious period. (It would also be good to collect the median and case (civil, disaggregate by case type). 57 Performance indicator Description of what it measures Data source availability and cost (short title) commercial, Cost: Low/moderate. Information systems that do not already administrative) Alternatively, this data can be generated from surveys that ask collect such data should start doing so, as this is good parties how many hearings were necessary to resolve their case. practice. Adjustments will be required to systems in several EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. (The Bank has conducted surveys that ask court users how many hearings were necessary to resolve their case, and how many of them were useful in contributing towards the final resolution of the case. An index was then created of the share of all hearings that parties perceive as useful (a hearing efficiency index.) 14. Adjournments Measures the percentage of scheduled hearings that are adjourned Data source/coverage: Information systems in many, if not or cancelled (disaggregated by type of litigious case). all, EU MS collect the data that would enable this indicator to be generated, both nationally and at the court level. Gauges the effectiveness of calendaring and continuance practices in courts, and the culture of keeping commitments Cost: Low. Information systems that do not already collect among parties and court personnel. Reasons for adjournment may such data should start doing so, as this is good practice. be difficult to identify through statistical systems. Some adjustments may be required to systems in some EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. 15. Timeliness of the The timeliness of collecting on an enforceable title measured in Data source/coverage: Data is not currently available but can enforcement phase days based on either the SATURN methodology or on actual be collected from either the local Chamber of Private in days resolved enforcement cases. Enforcement Agents (or similar), the State Enforcement Authority or the courts, depending on the type of It would also be good to collect the median. enforcement system (private, public or combined). The methodology used can either be the SATURN methodology or methodology based on actual measurement of timelines (analogous to options for measuring length of court cases). Cost: Moderate. Measurement would require cooperation of the enforcement authorities. 16. Use of ADR Number of court settlements as a percentage of resolved first Data source/coverage: Information systems in several EU mechanisms (court- instance civil/commercial litigious cases MS collect the data that would enable this indicator to be annexed mediation) 58 Performance indicator Description of what it measures Data source availability and cost (short title) generated. However, few EU MS compile and compare this data. Cost: Low. Information systems that do not already collect such data should start doing so, as this is good practice. Some adjustments may be required to systems in some EU MS to allow data to be generated routinely electronically/automatically at the court/subnational level. Quality of Justice Systems 17. Unmet needs for Measures responses to the question: In the past three years, have Data Source/coverage: GAC survey, which currently does judicial/legal you or anyone in your household ever felt the need to use the not cover any EU MS. services court system but decided not to? Cost: High. The GAC survey (or at least these few questions) Gauges the extent of unmet needs for judicial/legal services in the would need to be introduced and conducted in EU MS, community. It may be good to also add responses to the follow including on a sub-national level. up question: Please indicate whether any of the following factors influenced your/their decision not to use the court system. 18. Access to justice Measures responses to the following questions: Data source/coverage: Not available. Could be collected as a for people with • Are court buildings wheelchair accessible? composite indicator with answers given at the level of disabilities individual courts and then aggregated sub-nationally. One • Do courts provide sign language interpretation for point could be assigned to each of the proposed questions. people with hearing loss? • Are guide dogs of people with vision loss admitted into Cost: Moderate. Introducing such indicator does not entail a court buildings? survey or in the national reporting systems, answers would • Are there alternative seating arrangements and locations be filled out administratively. Thus, costs are at the initial for people with disabilities? stage primarily. • Are there computer aided, real-time transcription services, text in alternative formats, such as Braille and assistive listening devices? 19. Use of court fee Measures the number of cases for which court fees were waived Data source/coverage: National statistical system could be waivers as a share all cases. required to collect such data at the national, sub-national and court level. 59 Performance indicator Description of what it measures Data source availability and cost (short title) Cost: Low. Once national statistical systems are adapted, data should be collected administratively. 20. Availability of ICT Composite index comprising responses to the following: Data sources/coverage: Some of these are measured by tools (service, • Are there any electronic case management tools in place Doing Business (not all information gathered in this regard is filing, case status) within the courts for use by judges? being made public by DB but can be used). EU Justice Scoreboard shows availability of some but not all tools. The • Are there any electronic case management tools in place available data sources cover MS at the national level. within the courts for use by lawyers? • Can the initial claim be filed electronically through a Cost: Moderate. The existing data sources would need to be dedicated platform within the courts? adapted to collect data at the sub-national level. • Is it possible to carry out service of process electronically for claims filed before the courts? (Note: It is expected that this indicator would have only • Can court fees be paid electronically within the courts? temporary significance and only for some countries, • Is it possible to check the status of a case electronically? depending on the development of e-justice and the level of • Is it possible for parties and lawyers to access the fragmentation of ICT tools within each respective member casefile electronically? state.10 ) 21. Legal aid provision Measures legal aid allocated on a per capita basis, controlling for Data source/coverage: CEPEJ and national court systems per capita relative poverty rates. collect data on legal aid provision. Eurostat collects poverty to poverty data. All MS are covered at the national level. Cost: Low. Adjustments would need to be made to information systems to allow EU MS to collect subnational data. From there, administrative data would be collected. 22. Legal aid granted Gauges the extent to which poorer segments of the community Data source/coverage: National statistics. Such data should for non-criminal have access to legal aid to resolve disputes and enforce rights in be available at the national and at the court level. If it is cases, as compared civil and administrative matters. missing, it should not be difficult to start collecting it to share of total through small adjustment to statistics. It would also need to number of cases be disaggregated for the sub-national level. granted legal aid (percentage) Cost: Low. Once the adjustments are made, data should be collected administratively. 23. Courts as an Measures the percentage of firms that report that the courts are an Data source/coverage: BEEPS covering Bulgaria, Croatia, impediment to the obstacle to business operations. the Czech Republic, Estonia, Hungary, Latvia, Lithuania, business climate 10In 2016 CEPEJ published a thematic report on the use of information technology in European courts (CEPEJ Study 24): https://www.coe.int/t/dghl/cooperation/cepej/evaluation/2016/publication/CEPEJ%20Study%2024%20-%20IT%20report%20EN%20web.pdf. 60 Performance indicator Description of what it measures Data source availability and cost (short title) Poland, Romania, the Slovak Republic, and Slovenia at the national level. Cost: High. The methodology would need to be extended to all EU MS with sample sizes that allow disaggregation at the sub-national level. 24. Fairness and Measures the percentage of firms indicating that court system is Data source/coverage: BEEPS covering Bulgaria, Croatia, impartiality of fair, impartial and uncorrupted. Czech Republic, Estonia, Hungary, Latvia, Lithuania, courts Poland, Romania, Slovak Republic, Slovenia at the national level. Cost: Moderate/High. The methodology would need to be extended to all EU MS with sample sizes that allow disaggregation at the sub-national level. 25. Appeal ratio Measures the percentage of resolved litigious first-instance cases Data source/coverage: National statistics11 should collect that have been appealed (disaggregated by case type). such data at the national and at the court level. Small adjustments would be necessary to collect it at the sub- Some caution should be exercised when using this indicator, national level. because high appeals rates do no per se indicate poor performance. Excessive variation of rates between subnational Cost: Low. Once the adjustments are made, data should be units, however, may be of interest. This indicator should be read collected administratively. together with the overturn ratio below. (Note: It is proposed to track this indicator only for first-instance cases since many countries limit by law the possibility to appeal the decision of the second-instance court and these legislative limitations may skew data.) 26. Overturn ratio Measures the percentage of appealed first-instance court Data source/coverage: National statistics should collect such decisions that have been overturned by the second-instance court data at the national and at the court level. Small adjustments (disaggregated by case type). would be necessary to collect it at the sub-national level. Some caution should be exercised when using this indicator, Cost: Low. Once the adjustments are made, data should be because high appeals rates do no per se indicate poor collected administratively. performance. Excessive variation of rates between subnational units, however, may be of interest. 11 CEPEJ reports such data only for some categories of cases (litigious divorce case, employment dismissal case and insolvency case). 61 Performance indicator Description of what it measures Data source availability and cost (short title) Independence of Justice Systems 27. Reports of paying a Measures response to survey question: Have you paid a bribe to Data source/coverage: TI Global Corruption Barometer bribe to the the judiciary in the last 12 months? (2013) covers MS at the national level. Not clear whether the judiciary in the last (Alternative formulation: Do you know of anyone who has paid a question is still being asked or whether it could be repeated 12 months bribe to the judiciary in the last 12 months?) in the future. Cost: Moderate. The methodology would need to be extended to the sub-national level. 28. No corruption in A value on a 0.0 to 1.0 scale (comprising 100 units) derived from Data source/coverage: Rule of Law Index of the World the judiciary surveys of households and expert surveys. Justice Project covers MS at the national level except for Ireland, Cyprus, Latvia, Lithuania, Luxemburg, Malta, This measurement is part of the Absence of Corruption factor Slovakia. measured by the Rule of Law Index, which comprises the following sub-factors: Cost: Moderate. The methodology would need to be • Government officials in the Executive Branch do not extended to the sub-national level and to the MS that are use public office for private gain currently not covered. • Government officials in the judicial branch do not use public office for private gain • Government officials in the police and the military do not use public office for private gain • Government officials in the legislative branch do not use public office for private gain 29. Judicial corruption Measures the percentage of firms reporting that bribery is Data source/coverage: BEEPS covering Bulgaria, Croatia, frequent in dealings with courts. Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Slovak Republic, and Slovenia at the national level. Cost: Moderate/high. The methodology would need to be extended to all EU MS with sample sizes that allow disaggregation at the sub-national level. 30. Judicial A value on a 1 to 7 scale, as well as a country ranking based on Data source/coverage: Global Competitiveness Index independence expert assessment. covering all EU MS at the national level. 62 Performance indicator Description of what it measures Data source availability and cost (short title) “Judicial independence” is measured as part of sub -factor “undue Cost: Moderate. The methodology would need to be influence”, contributing to factor “public institutions”. extended to the sub-national level. 31. Lower court Measures responses to the question: When judges not on the high Data source: V-Dem Index covering all MS at the national independence court are ruling in cases that are salient to the government, how level. often would you say that their decisions merely reflect government wishes regardless of their sincere view of the legal Cost: Moderate. The methodology would need to be record? extended to the sub-national level. 32. No improper A value on a 0.0 to 1.0 scale (comprising 100 units) derived from Data source/coverage: Rule of Law Index of the World governmental surveys of households and expert surveys. Justice Project covers EU MS at the national level with the influence in civil exception of Ireland, Cyprus, Latvia, Lithuania, Luxemburg, justice This measurement is part of the Civil Justice factor measured by Malta, and Slovakia. the Rule of Law Index, which comprises the following sub- factors: Cost: Moderate. The methodology would need to be • People can access and afford civil justice extended to the sub-national level and to the MS that are currently not covered. • Civil justice is free of discrimination • Civil justice is free of corruption • Civil justice is free of improper government influence • Civil justice is not subject to unreasonable delays • Civil justice is effectively enforced 33. Perception by firms Measures the percentage of companies reporting that the justice Data source/coverage: Flash Eurobarometer 448 covering all of judicial system is very good or fairly good in terms of independence EU MS at the national level. independence (alternatively, as very bad or fairly bad). Cost: Moderate. The methodology would need to be extended to the sub-national level. 34. General perception Measures the percentage of general population reporting that the Data source/coverage: Flash Eurobarometer 447 covering all of judicial justice system is very good or fairly good in terms of EU MS at the national level. independence independence (alternatively, as very bad or fairly bad). Cost: Moderate. The methodology would need to be extended to the sub-national level. 63 Table 2A.13. Justice Sector Performance Indicators - Datasets and Methodologies Dataset Organization Methodology Frequency Country coverage BEEPS World Bank/EBRD BEEPS is a firm-level survey of a representative sample of an Periodic 30 economies, economy’s private sector. including the following EU Member States: Bulgaria, Croatia, the Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, the Slovak Republic, and Slovenia. Doing Business World Bank Expert assessments for what it takes to resolve a commercial dispute Annual 190 economies, plus through a local first-instance court based on a hypothetical case some emerging scenario. subnational analyses European European Council of Europe members are invited to appoint national Every 2 years 45 Council of Europe judicial systems Commission for correspondents who are entrusted with the coordination of replying to (reporting member states efficiency and the Efficiency of the requests for information by the CEPEJ on the operation and includes a quality of justice Justice (CEPEJ) performance of national justice systems for their respective state or two-year lag entity. Often these correspondents are in Ministries of Justice. The data –e.g., 2016 they submit is largely based on self-reported national statistics. Every report two years, the CEPEJ circulates a detailed questionnaire to the reflects 2014 correspondents and produces a report based on data received. data) EU Justice The European The Scoreboard uses various sources of information. Large parts of the Annual EU MS Scoreboard Commission quantitative data are provided by the Council of Europe Commission for the Evaluation of the Efficiency of Justice (CEPEJ). The other sources of data are: the group of contact persons on national justice systems, the European Network of Councils for the Judiciary, the Network of the Presidents of the Supreme Judicial Courts of the EU, Association of the Councils of State and Supreme Administrative Jurisdictions of the EU, the European Competition Network, the Communications Committee, the European Observatory on infringements of intellectual property rights, the Consumer Protection Cooperation Network, the Expert Group on Money Laundering and Terrorist Financing, Eurostat, the European Judicial Training Network, the Council of Bars and Law Societies of Europe and the World Economic Forum. 64 Dataset Organization Methodology Frequency Country coverage Flash European The survey was designed to explore companies’ perceptions about the Annual EU MS Eurobarometer Commission, independence of the judiciary across EU MS. The results feed into the 448: Perceived Directorate- EU Justice Scoreboard. The survey explores how companies rate the independence of General for Justice independence of the courts and judges in their country, and the reasons the national and Consumers for these ratings. justice systems For the first survey, some 6,803 interviews were conducted among in the EU enterprises employing one or more persons in manufacturing, services among and industry. The sample was selected from an international database, companies with an additional sample from local sources where necessary. Interviews were conducted with key company decision-makers over the telephone in their mother tongue. The methodology used is that of Eurobarometer surveys as carried out by the Directorate-General for Communication. Flash Survey requested The survey was designed to explore respondents’ perceptions about the Annual EU Member States Eurobarometer by the European independence of the judiciary across EU MS. 447: Perceived Commission, independence of Directorate- The results feed into the EU Justice Scoreboard. The survey explores the national General for Justice respondents' perceptions of the independence of the courts and judges in justice systems and Consumers their country and the reasons for these perceptions. Respondents from in the EU different social and demographic groups were interviewed via telephone among the (mobile and fixed line) in their mother tongue. The methodology used is general public that of Eurobarometer surveys as carried out by the Directorate-General for Communication. Global World Economic Expert assessment. Annual 138 economies Competitiveness Forum (WEF) Report (GCI) Measures, among other things: irregular payments and bribes; Judicial Independence; and Efficiency of Legal Framework in Settling Disputes. Global Transparency The methodology involves face-to-face and telephone interviews with Every 2 years 95 economies Corruption International of a large number of citizens examining their perceptions of corruption Barometer in the public sector; how the government is tackling corruption; actual experiences of bribery; and social acceptability of reporting corruption. Rule of Law World Justice The Index measures constraints on government powers; absence of Annually 113 jurisdictions Index (RLI) Project (WJP) corruption; open government; fundamental rights; order and security; worldwide regulatory enforcement; civil justice; criminal justice; and informal justice. (without Ireland, Cyprus, Latvia, 65 Dataset Organization Methodology Frequency Country coverage Specifically, in respect of the judiciary, the Index measures, among Lithuania, Luxemburg, other things, bribery, improper influence by public or private interests, Malta, or Slovakia) and misappropriation of public funds or other resources by the judiciary. As regards civil justice the Index measures whether delivery of effective civil justice requires that the system be accessible and affordable, free of discrimination, free of corruption, and without improper influence by public officials; whether court proceedings are conducted in a timely manner, not subject to unreasonable delays, and are effectively enforced; the accessibility, impartiality, and efficiency of mediation and arbitration systems that enable parties to resolve civil disputes. The country scores and rankings for the WJP Rule of Law Index 2016 are derived from more than 110,000 households and 2,700 expert surveys in 113 countries and jurisdictions. GAC World Bank Experience and perception data from service users & providers through Ongoing Select economies multi-stakeholder surveys. worldwide; no EU Member States included. Varieties of V-Dem Institute V-Dem offers separate indices of five varieties of democracy: electoral, Ongoing, Worldwide Democracy (an independent liberal, participatory, deliberative, and egalitarian. There are some 350 with dataset Indices (V-Dem) research institute unique democracy indicators in the V-Dem dataset. The V-Dem using the based at “standard” dataset is in the country-year format, where date-specific country-year the Department of changes have been aggregated together at the year level. The 350+ V- format. Political Science, Dem specific indicators fall into four main types: (A) factual indicators University of coded by members of the V-Dem team, (B) factual indicators coded by Gothenburg, Country Coordinators, (C) evaluative indicators based on multiple Sweden) ratings provided by experts, and (D) composite indices. 66 Table 2A.14 Shortlist of Public Accountability Mechanism Indicators Governance Indicator What it measures Data source variable Transparency Freedom of Information (FOI) Aggregate measure of a country’s freedom of information framework EuroPAM De jure country score (comprehensiveness evaluation) Transparency FOI Scope and Coverage Aggregate measure of the scope of disclosure, coverage of public and private EuroPAM De jure sectors, and access to specific documents (e.g. laws, budgets, expenditures, annual reports) (comprehensiveness evaluation) Transparency FOI Information access and release Aggregate measure of procedural access and deadlines for public release of EuroPAM De jure information (comprehensiveness evaluation) Transparency FOI Exceptions and Overrides Aggregate measure of exemptions to disclosure and appeals process EuroPAM De jure (comprehensiveness evaluation) Transparency FOI Sanctions for non-compliance Aggregate measure of existence of administrative sanctions, fines, and EuroPAM De jure criminal sanctions (comprehensiveness evaluation) Transparency FOI Monitoring and Oversight Aggregate measure of enforcement, promotional, and implementation EuroPAM De jure monitoring bodies (comprehensiveness evaluation) Transparency Financial Disclosure (FD) country Aggregate measure of a country’s financial disclosure framework EuroPAM De jure score (comprehensiveness evaluation, includes Head of state, Ministers, Members of Parliament, and Civil servants) Transparency FD Disclosure items Aggregate measure of scope and extent of disclosure of income and assets, and EuroPAM De jure incompatibilities (conflicts of interest) (comprehensiveness evaluation) Transparency FD Filing frequency Aggregate measure of frequency of declaration filing (comprehensiveness EuroPAM De jure evaluation) Transparency FD Sanctions Aggregate measure of sanctions for failure to file, late filing, and false EuroPAM De jure disclosure (comprehensiveness evaluation) Transparency FD Monitoring and Oversight Aggregate measure of the institutional arrangements for receiving declarations, EuroPAM De jure enforcing disclosure, and verification of declarations (comprehensiveness evaluation) Transparency FD Public access to declarations Aggregate measure of availability, timing, location, and cost of access to EuroPAM De jure declarations (comprehensiveness evaluation) Transparency To what extent is there a well- Existence of open data policy Open Data De jure defined open data policy and/or Barometer strategy in the country? Transparency To what extent is there a robust Existence of data protection law Open Data De jure legal or regulatory framework for Barometer protection of personal data in this country? 67 Governance Indicator What it measures Data source variable Transparency Right to Information Drivers of Aggregate measure of the country drivers of RTI implementation success RIDE Indicators De facto Effectiveness (RIDE) country score Transparency RIDE Enabling Conditions Aggregate measure of the quality of the RTI legal framework, role of civil RIDE Indicators De facto society, and policy prioritization Transparency RIDE Demand for Information Aggregate measure of the public awareness of RTI and accessibility of RTI RIDE Indicators De facto system (requests, internet, language, assistance, etc.) Transparency RIDE Institutional Capacity Aggregate measure of RTI formal practices, staffing levels, staff capacity, and RIDE Indicators De facto staff incentives Transparency RIDE Oversight Aggregate measure of RTI implementation monitoring and disclosure RIDE Indicators De facto enforcement Transparency RIDE Transformative factors Aggregate measure of RTI state-society collaboration, technology, and intra- RIDE Indicators De facto governmental collaboration Transparency RTI Requests data RTI request and responses data, including total annual requests at national RIDE Indicators12 De facto level, and agencies targeted Transparency RTI Exemption data RTI exemption data, including total refusals and exemptions most often RIDE Indicators De facto invoked to deny disclosure of data Transparency RTI Appeals data Data on RTI appeals, including total numbers, reasons for appeal, and RIDE Indicators De facto responses Transparency Financial Disclosure (FD) country Aggregate measure of FD implementation practices PAM De facto score implementation indicators Transparency FD Management Evaluation of FD management practices, e.g., records, budget, staff PAM De facto implementation indicators Transparency FD Technology Evaluation of use of technology in FD practices PAM De facto implementation indicators Transparency FD Submission compliance Rates and quality of compliance with FD rules PAM De facto implementation indicators13 12 Data on RTI requests and appeals was collected as part of a pilot study. This is data that governments produce about their own performance, often required by law to be collected and released to the public. See: Lemieux and others 2015. “Transparency and Open Government: Reporting on the Disclosure of Information .” JeDEM - eJournal of eDemocracy and Open Government 7(2): 75-93. 13 Data on FD submission compliance, content audits, and sanctions was collected as part of a pilot study. This is data that governments produce about their own performance, often required by law to be collected and released to the public. 68 Governance Indicator What it measures Data source variable Transparency FD Content audits Rates and nature of FD audits and verification PAM De facto implementation indicators Transparency FD Sanctions and Enforcement Rates and type of FD enforcement PAM De facto implementation indicators Transparency FD Information dissemination Evaluation of FD content and method of information release PAM De facto implementation indicators Transparency National statistics – access Aggregate measure of national statistics in three areas: methodology, source Statistical Capacity De facto data, and periodicity (25 sub-indicators) Indicator Transparency To what extent is there a consistent Open data management approach by government Open Data De facto (open) data management and Barometer publication approach? Transparency To what extent is there a well- Country-level open data initiative Open Data De facto resourced open government data Barometer initiative in this country? Transparency To what extent are city or regional Sub-national open data initiatives Open Data De facto governments running their own Barometer open data initiatives? Transparency To what extent does the country Functioning of RTI law Open Data De facto have a functioning right-to- Barometer information law? Transparency To what extent are civil society and Civil society engagement with open government data Open Data De facto information technology Barometer professionals engaging with the government regarding open data? Transparency To what extent is training available Open data training availability Open Data De facto for individuals or businesses Barometer wishing to increase their skills or build businesses to use open data? Transparency To what extent is government Government support for open data culture Open Data De facto directly supporting a culture of Barometer innovation with open data through competitions, grants or other support? 69 Governance Indicator What it measures Data source variable Transparency Public Contracts – open data Aggregate measure of quality and availability of data sets (10 sub-indicators)14 Open Data De facto Barometer Transparency Health sector performance – open Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto data Barometer Transparency Primary or secondary education Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto performance data – open data Barometer Transparency National environment statistics – Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto open data Barometer Transparency Detailed government budget – open Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto data Barometer Transparency Detailed data on government Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto spending – open data Barometer Transparency Company registers – open data Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto Barometer Transparency Legislation – open data Aggregate measure of quality and availability of data sets (10 sub-indicators) Open Data De facto Barometer Transparency Transport timetables – open data Aggregate measure of quality and availability of data sets (9 sub-indicators)15 Open Data Index De facto Transparency Government budget – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency Government spending – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency Election results – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency Company register – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency National map – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency National statistics – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto 14 The 10 ODB sub-indicators are: Does the data exist? Is it available online from government in any form? Is the dataset provided in machine-readable formats? Is the machine- readable data available in bulk? Is the dataset available free of charge? Is the data openly licensed? Is the dataset up to date? Is the publication of the dataset sustainable? Was it easy to find information about this dataset? Are (linked) data URIs provided for key elements of the data? 15 The 9 ODI sub-indicators are: Does the data exist? Is data in digital form? Publicly available? Is the data available for free? Is the data available online? Is the data machine- readable? Available in bulk? Openly licensed? Is the data provided on a timely and up to date basis? 70 Governance Indicator What it measures Data source variable Transparency Legislation – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency Postcodes/zip codes – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Transparency Emissions of pollutants – open data Aggregate measure of quality and availability of data sets (9 sub-indicators) Open Data Index De facto Accountability Conflict of Interest Restrictions Aggregate measure of a country’s COI framework (comprehensiveness EuroPAM De jure (COI) country score evaluation) Accountability COI Restrictions Aggregate measure of restrictions on behavior (comprehensiveness evaluation) EuroPAM De jure Accountability COI Sanctions Aggregate measure of administrative sanctions, fines, and criminal penalties EuroPAM De jure (comprehensiveness evaluation) Accountability COI Monitoring and Oversight Aggregate measure of monitoring and enforcement arrangements EuroPAM De jure (comprehensiveness evaluation) Accountability Political financing (PF) country Aggregate measure of a country’s political financing framework EuroPAM16 De jure score (comprehensiveness evaluation) Accountability PF Bans and limits on private Aggregate measure of bans on donations from foreign interests, corporations, EuroPAM De jure income trade unions, anonymous sources, and donation limits (comprehensiveness evaluation) Accountability PF Public funding Aggregate measure of eligibility criteria, allocation calculations, and EuroPAM De jure earmarking provisions for direct and indirect public funding (comprehensiveness evaluation) Accountability PF Regulations on spending Aggregate measure of bans on vote buying, and spending (comprehensiveness EuroPAM De jure evaluation) Accountability PF Reporting, oversight, and Aggregate measure on reporting standards, oversight mechanisms, and EuroPAM De jure sanctions sanctions (comprehensiveness evaluation) Accountability COI mitigation Evaluation of COI review and remediation practices PAM De facto implementation indicators Accountability Ethics indicators (possible) Indicators may be developed from a World Bank study on the implementation World Bank De facto of ethics frameworks in 8 economies (2016). Topics include advisory services/guidance materials, organizational monitoring, training, technology, interagency collaboration, and outreach. 16 Political financing indicators were adapted from the Institute for Democracy and Electoral Assistance (IDEA). 71 Table 2A.15. PAM Indicators – Datasets and Methodology Dataset Organization Methodology Frequency Country coverage 17 EuroPAM Hertie School of Expert assessment Annual, ending in 2017 34 European economies plus Public Accountability Governance (Germany) de jure 2012, 2015, 2016, 2017 European Commission Mechanisms (PAM) PAM implementation World Bank (US) Expert assessment & Pilot study completed Pilot study completed (2012); 5 government statistics18 (2012) 5 countries economies de facto RIDE Indicators World Bank (US) Expert assessment & Pilot study completed Pilot study completed (2014); 6 Right to Information government statistics (2014) 6 countries economies Drivers of Effectiveness de facto (RIDE) Open Data Barometer World Wide Web Expert assessment Annual, ongoing 27 European economies Foundation (US) de jure & de facto 2013, 2014, 2015 Open Data Index Open Knowledge Crowdsourcing / Expert Annual, ongoing 30-35 European economies Foundation (UK) Assessment19 2013, 2014, 2015 de facto Statistical Capacity World Bank (US) Expert assessment Annual, ongoing 19 European economies (no high- Indicator (SCI) de facto 2004-2016 income) 17 Collected by the Hertie School of Governance and adapted from World Bank Public Accountability Mechanisms (PAM). 18 Government statistics are defined as data that governments produce about their own performance, often required by law to be collected and released to the public. 19 The assessment and review of the ODI datasets takes place in four steps. The first step is collecting the evaluation of datasets through volunteer contributors, and the second step is QA checks by the local coordinators, the third is verifying the results with paid expert reviewers. The fourth and the last step is a public review of the Index before it is published. 72 Shortlist of Corruption Indicators Table 2A.16. Indicators of State Capture20 Governance Indicator Data source Respondent profile Frequency of variable field applications State Capture It is often said that firms make unofficial payments/gifts, BEEPS 2005, GAC Diagnostics Business managers and BEEPS: 1 private payments or other benefits to public officials to (firms) firm owners round; 28 gain advantages in the drafting of laws, decrees, economies regulations, and other binding government decisions. To GAC: ad hoc, what extent have the following practices had a direct 17 economies impact on your business (no impact… decisive impact, don’t know): • Private payments/gifts or other benefits to Parliamentarians to affect their votes • Private payments/gifts or other benefits to Government officials to affect the content of government decrees • Private payments/gifts or other benefits to local or regional government officials to affect their votes or content of government decrees State Capture How often would firms like yours make unofficial BEEPS 2002, 2005; GAC Business managers and BEEPS: 2 payments/gifts for the following purposes: “To Diagnostics (firms) firm owners rounds; 28 influence the content of new legislation, rules, decrees, economies etc.?” GAC: ad hoc, 1 = never; 2 = seldom; 3 = sometimes; 4 = frequently; 5 17 economies = usually; 6 = always; State Capture Please tell me to what extent you agree or disagree with GAC Diagnostics Households; business ad hoc, 17 each of the following statements. Do you (1) totally managers and firm economies disagree, (2) partially disagree (3) are indifferent (4) owners partially agree OR (5) totally agree? • The judicial system is manipulated by economic interests 20Some of the indicators in this shortlist are taken from survey instruments designed for use in non-English speaking economies. In order to preserve the original structure of these survey questions, we have left them in a translated form with limited editing. These indicators would need to be adapted to fit a European context and to ensure appropriate language and phrasing is used. 73 Governance Indicator Data source Respondent profile Frequency of variable field applications State Capture Can you please tell me what is the capacity or ability of GAC Diagnostics Households; business ad hoc, 17 the following key players to influence, through bribes, managers and firm economies the decisions made by the top officials? Use the scale owners; public officials from 1 to 7, where 1 means high capacity,4 somewhat and 7 low capacity • Economic groups / business interests • Labor unions • Economic organizations • Local firms • Foreign firms • Drug trafficking groups • Other organized crime groups State Capture How frequently do you think that firms make donations GAC Diagnostics (firms and Households; business ad hoc, 17 like contributions to a political party or support public officials) managers and firm economies movement preparing for an election campaign? owners; public officials State Capture According to previous question, how important do you GAC Diagnostics (firms and Households; business ad hoc, 17 think it is for the private industry to make these public officials) managers and firm economies contributions to the political system, in terms of profits owners; public officials and revenues generated for the private sector? Use the scale from 1 to 7, where 1 means very important and 7 means non-important. State Capture Do major economic stakeholders influence economic Institutional Profile Database In-country French 3 rounds, 143 legislation? (score 0 to 4) economic and economies development officials State Capture Do major economic stakeholders influence tax policy? Institutional Profile Database In-country French 3 rounds, 143 (score 0 to 4) economic and economies development officials State Capture Can political authorities make independent decisions Institutional Profile Database In-country French 3 rounds, 143 vis-à-vis the various lobby and interest groups? (score 0 economic and economies to 4) development officials State Capture In law, there are restrictions for state civil servants State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states entering the private sector after leaving the government. State Capture In law, state civil servants must recuse themselves from State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states actions in which they may have a conflict of interest. 74 Governance Indicator Data source Respondent profile Frequency of variable field applications State Capture In law, there are limits on individual donations to State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states candidates and to political parties. State Capture In law, there are limits on corporate donations to State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states candidates and to political parties. State Capture In law, there are limits on lobbyists' donations to State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states candidates and to political parties. State Capture In practice, the limits on individual donations to State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states candidates and political parties are respected State Capture In practice, the limits on corporate donations to State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states candidates and political parties are respected. State Capture In practice, the limits on lobbyist donations to State Integrity Report Journalist-researchers 2 rounds, 50 (evidence-based) US states candidates and political parties are respected. State Capture In practice, state civil servants are protected from State Integrity Report Journalist-researchers 2 rounds; 50 political interference. (evidence-based) US states State Capture To what extent is the judiciary in your country Global Competitiveness Index Business managers and 10 rounds, 138 firm owners economies independent from influences of members of government, citizens, or firms? [1 = heavily influenced; 7 = entirely independent] Table 2A.17. Indicators of Nepotism and Patronage Governance Indicator Data source Respondent profile Frequency of Variable field applications Nepotism/ In the past twelve months, have elected officials, their GAC Diagnostics Public officials ad hoc, 17 Patronage appointees, or political party officials influenced any economies 75 Governance Indicator Data source Respondent profile Frequency of Variable field applications hiring decisions or promotions in your organization/department? [If yes] How often did it occur? 1. Never 2. Seldom 3. Frequently 4. Mostly 5. Always Nepotism/ In law, nepotism (favorable treatment of family State Integrity Report Journalist-researchers 2 rounds; 50 Patronage (evidence-based) US states members), cronyism (favorable treatment of friends and colleagues), and patronage (favorable treatment of those who reward their superiors) are prohibited within the state civil service. Nepotism/ In practice, state civil service management actions (e.g. State Integrity Report Journalist-researchers 2 rounds; 50 Patronage (evidence-based) US states hiring, firing, promotions) are not based on nepotism, cronyism, or patronage. Nepotism/ When granting licenses to start up private firms, public Quality of Government - Expert In-country experts 2 rounds, 159 Patronage Survey economies sector employees favor applicants with whom they have strong personal contacts Nepotism/ When recruiting public sector employees, the political Quality of Government - Expert In-country experts 2 rounds, 159 Patronage connections of the applicants decide who gets the job. Survey economies (Agree, rather agree, rather disagree, disagree) Nepotism/ When recruiting public sector employees, the personal Quality of Government - Expert In-country experts 2 rounds, 159 Patronage connections of the applicants (for example kinship or Survey economies friendship) decide who gets the job. (Agree, rather agree, rather disagree, disagree) 76 Table 2A.18. Indicators of Embezzlement of Budget Governance Indicator Data source Respondent profile Frequency of variable application Budget During the last two years, to what extent would you GAC Diagnostics Public officials ad hoc, 17 embezzlement agree that decisions relating to the budget administration economies (amounts assigned to the budget, services, programs which they were carried out, groups that received budget allocations) have been: 1. Done transparently (we know who received what and why) 2. Announced/open to public knowledge through various legal means 3. Subjected to regular audits by the internal control unit 4. Subject to regular external audits performed by professionals qualified and experienced in conducting audits 5. Based on specific criteria defined in writing 6. Influenced by regional ties 7. Based on political pressure 8. Based on influential connections within the institutions 9. Influenced by illegal payments 10. Planned with consideration given to institutional fulfilment (scale from 1 to 7 or 1 to 5 depending on the country of implementation) Budget During the last two years in the use of budget funds in GAC Diagnostics Public officials ad hoc, 17 embezzlement your institution, how frequently were there irregularities economies such as misappropriation or any other type of abuse of budget resources? 1. Never 2 Seldom 3. Frequently 4. Most times 5. Always Budget Members of the executive (the head of state, the head of Quality of Government - Expert In-country experts 2 rounds, 159 embezzlement government and cabinet ministers), or their agents, steal, Survey economies embezzle or misappropriate public funds or other state resources for personal or family use. (Agree, rather agree, rather disagree, disagree) 77 Governance Indicator Data source Respondent profile Frequency of variable application Budget Public sector employees steal, embezzle or Quality of Government - Expert In-country experts 2 rounds, 159 embezzlement misappropriate public funds or other state resources for Survey economies personal or family use. Agree, rather agree, rather disagree, disagree Table 2A.19. Indicators of Administrative/Petty Corruption Governance Indicator Data source Respondents profile Frequency of variable field application Petty Corruption On average, what percent of total annual sales do firms like yours BEEPS 2002 and Business managers and 2 rounds, 28 typically pay in unofficial payments/gifts to public officials? 2005 firm owners economies ______% Petty Corruption In the past fiscal year how much did all the inspections (by public BEEPS 2008 Business managers and 1 round; 28 officials) that took place at this establishment cost, including official firm owners economies fines and any unofficial payments? Petty Corruption Thinking now of unofficial payments/gifts that establishments like BEEPS 2005, 2008 Business managers and 2 rounds, 28 this one would make in a given year, please tell me how often would firm owners economies they make payments/gifts for the following purposes: • To deal with customs/imports • To deal with courts • To deal with taxes and tax collection Petty Corruption In reference to that application for an electrical connection, was an BEEPS 2008 and Business managers and 2 rounds, 28 informal gift or payment expected or requested? 2011 firm owners economies In reference to that application for a water connection, was an informal gift or payment expected or requested? In reference to that application for a telephone connection, was an informal gift or payment expected or requested? In order to obtain any of the compulsory certificates was a gift or informal payment expected or requested? In reference to any of those applications for permits, was an informal gift or payment expected or requested? In reference to that application for a construction-related permit, was an informal gift or payment expected or requested? 78 Governance Indicator Data source Respondents profile Frequency of variable field application Over the last year, how many times was this establishment either inspected by tax officials or required to meet with them? In any of these (tax) inspections or meetings, was a gift or informal payment requested? In reference to that application for an import license, was an informal gift or payment expected or requested? In reference to that application for an operating license, was an informal gift or payment expected or requested? Petty Corruption How frequently do households / firms like yours find themselves GAC Diagnostics Households; Business ad hoc, 17 obliged to make unofficial payments to public official associated managers and firm economies with the following? owners; and Public • Public service procedures officials • Request for licenses or permits • Update of fiscal situation • Obtain contracts with state institutions • Pursue legal proceedings in the judicial branch Petty Corruption During the last year has anybody in your household/firm sought to GAC Diagnostics Households; Business ad hoc, 17 obtain the following public service? If so, what was the official costs managers and firm economies of the service received? If so, did a public official ask for an extra owners payment when you sought that service? How much was on average the extra payment made? (question asked for a list of country- specific public services delivered to citizens and businesses, e.g. health, education, passport, permits, licenses, etc.) Petty Corruption If you or anybody in your household were involved in a lawsuit in GAC Diagnostics Households; Business ad hoc, 17 the past 3 years, did you or anyone in your household receive any managers and firm economies indication that you were expected to pay some gratification (to a owners judge, prosecutor, enforcement officials, or any other official involved in the case) in order to get a favorable decision in the case? Petty Corruption If you or anybody in your household were involved in a lawsuit in GAC Diagnostics Households; Business ad hoc, 17 the past 3 years, did you have to pay the following type of officials? managers and firm economies If so, how much was the gratification on average? owners [List of answers included: bailiffs, court clerks, judges, magistrates, public prosecutors, legal aid, local court chairmen, messengers, law enforcement officers] 79 Governance Indicator Data source Respondents profile Frequency of variable field application Petty Corruption Has a police officer / soldier / military officer asked you for a bribe LAPOP Citizens 6+ rounds, 28 in the last twelve months? economies Petty Corruption In the last twelve months, to process any kind of document in your LAPOP Citizens 6+ rounds, 28 municipal government, like a permit for example, did you have to economies pay any money above that required by law? Petty Corruption Level of “petty” corruption between citizens and the administrations Institutional Profile In-country French 3 rounds, 143 (score from 0 to 4) Database economic and economies development officials Petty Corruption Level of corruption between administrations and local businesses Institutional Profile In-country French 3 rounds, 143 (score from 0 to 4) Database economic and economies development officials Table 2A.20. Indicators of Corruption in Public Procurement Governance Indicator Data source Respondent profile Field variable application Corruption in How often would firms like yours make unofficial payments/gifts for BEEPS 2002 and Business managers and 2 rounds, 28 public the following purposes?” 1 = never; 2 = seldom; 3 = sometimes; 4 = 2005 firm owners economies procurement frequently; 5 = usually; 6 = always • “To obtain government contracts” Corruption in When firms in your industry do business with the government, what BEEPS 2005, 2008, Business managers and BEEPS: 3 public percent of the contract value would be typically paid in additional or 2011; GAC firm owners rounds; 28 procurement unofficial payments/gifts to secure the contract? ______% Diagnostics economies GAC: ad hoc, 17 economies Corruption in In many countries, it is common for enterprises to make additional GAC Diagnostics Public officials ad hoc, 17 public payments to win a procurement contract. economies procurement (i) What proportion of public procurement contracts in your organization involves any gratification? ______% of procurement contracts Corruption in When gratification is made for a procurement contract of your GAC Diagnostics Public officials ad hoc, 17 public organization, typically what percentage of the value of the contract economies procurement has to be paid in order to win the procurement contract? _______% of the value of contract. 80 Governance Indicator Data source Respondent profile Field variable application Corruption in Firms that provide the most favorable kickbacks to senior officials are Quality of In-country experts 2 rounds, 159 public awarded public procurement contracts in favor of firms making the Government Expert economies procurement lowest bid Survey Table 2A.21. Indicators of Purchase of Public Positions Governance Indicator Data source Respondent profile Field variable application Purchase of Please evaluate to what extent during the last two years personnel GAC Diagnostics Public officials ad hoc, 17 public positions management decisions (hiring, assignments, changes, promotions, economies salary increases) in your office were: 1. Made in a transparent manner (to know to whom were assigned, promoted, transferred, or received wage increase and why) 2. Position vacancies announced within the institution as well as announced publicly outside the institution (when appropriate) 3. Useful for the improvement of institutional efficiency 4. Subjected to regular audits by the internal unit of control 5. Subject to a formal procedure of appeal 6. Based on specific criteria defined in writing (as opposed to tacit – not written and informal rules) 7. Based on professional experience/merit/performance 8. Based on level of education 9. Based on seniority/age 10. Based on family ties or friendship 11. Influenced by business ties/associations 12. Based on political ties/political affiliation/political pressure 13. Based on influential connections within the institution 14. Influenced by illegal payments (purchase of positions or promotions) Purchase of We know that in many countries, public officials pay to “buy” GAC Diagnostics Public officials ad hoc, 17 public positions positions in the public sector. Based on your direct or indirect economies experience, how common is the practice of “purchasing jobs” in your organization among: 1. Your colleagues in higher positions (people at a higher level than you not including your superiors) 2. Co-workers (people at your same level not including yourself) 3. Your colleagues’ subordinates (people at a lower level than you, not including your subordinates) 81 Governance Indicator Data source Respondent profile Field variable application Purchase of Frequency of bribe paid for state employment PAPI Citizens 6 rounds, 63 public positions provinces in Vietnam Purchase of When recruiting public sector employees, the skills and merits of the Quality of In-country experts 2 Rounds, 159 public positions applicants decide who gets the job. Government Expert economies Survey Table 2A.22. Additional Indicators/Control Variables about the Institutional Environment Governance Indicator Data source Respondent Field application variable profile Control variable What services do you receive from the association or associations BEEPS 2005 Business managers 1 round; 28 economies to which you belong, and what is the value of these services to and firm owners your firm? “Critical value to your firm” should be placed on services that significantly reduce the costs of your firm or significantly increase the sales of your firm. (no value… critical value, not provided) • Lobbying government • Resolution of disputes (with officials, workers or other firms) • Information and/or contacts on domestic product and input markets • Information and/or contacts on international product and input markets • Accrediting standards or quality of products; reputational benefits • Information on government regulations Control variable Which of the following elements of the business environment, if BEEPS 2008, 2011 Business managers 2 rounds, 28 economies any, currently represents the biggest obstacle faced by this and firm owners establishment. • Corruption Control variable Over the last 12 months did your firm employ an outside BEEPS 2005 Business managers 1 round, 28 economies consultant to deal with public officials about the application and and firm owners 82 Governance Indicator Data source Respondent Field application variable profile interpretation of laws and regulations and to get or to maintain access to public services? (Y/N) Control variable If your firm did not apply for a loan, what were the main reasons? BEEPS 2005, Business managers 3 rounds, 28 economies • Does not need a loan 2008, 2011 and firm owners • Application procedures for bank loans are too burdensome • Collateral requirements for bank loans are too strict • Interest rates are too high • It is necessary to make informal payments to get bank loans • Did not think it would be approved • Others Abuse of power If you considered starting or completing a procedure at any of GAC Diagnostics Households; ad hoc, 17 economies these institutions [list of local agencies offering public services to Business managers citizens and businesses] and decided not to do it, why didn’t you and firm owners do it? 1. You knew they would not do what you needed 2. You didn’t have time 3. You couldn’t pay the official cost 4. You couldn’t pay the unofficial costs 5. You didn’t know who to turn to 6. It wastes a lot of time 7. Other reasons Mechanisms for In those cases where you had to make unofficial additional GAC Diagnostics Households; ad hoc, 17 economies abuse of power payment to receive or expedite a service: Business managers and firm owners - ............................................................................................................................................. D id the Public Servant ask or suggest discretely for the additional payment? - ............................................................................................................................................. D id you offer the additional payment or gift spontaneously? 83 Governance Indicator Data source Respondent Field application variable profile - ............................................................................................................................................. W as it known in advance how the system worked and how much you were required to pay? Abuse of power/ How would you rate the integrity/honesty of the following public BEEPS, GAC BEEPS: 1 round; 28 Dishonesty institutions in your country (list of country-specific government Diagnostics economies institutions)? (Households, GAC: ad hoc, 17 1= “Extremely dishonest” businesses and economies 5= “Extremely honest” … 9= “DK/NS” public officials) Reporting Do you know what process to follow in reporting a corrupt act by GAC Diagnostics Households; ad hoc, 17 economies corruption/ a public official? Business managers accountability and firm owners; and Public officials Reporting During the past twelve months have you or anyone in your GAC Diagnostics Households; ad hoc, 17 economies corruption/ firm/household/office observed a corrupt act by a public Business managers accountability official/colleague? If so, have you or anyone in your firm/ and firm owners; household/office reported it? and Public officials Reporting Would you say that the process of reporting corruption is: GAC Diagnostics Households; ad hoc, 17 economies corruption/ 1. Extremely difficult; 2. Difficult; 3. Fairly simple; 4. Simple 5. Business managers accountability Extremely simple;9.DK/NS and firm owners; and Public officials Reporting How much would you say the person reporting a potential GAC Diagnostics Households; ad hoc, 17 economies corruption/ corruption case was protected from potential harassment? Business managers accountability 1. Extremely unprotected 2. Unprotected 3. Fairly protected 4. and firm owners; Protected 5. Extremely protected 9.DK/NS and Public officials Reporting If you or someone in your household / firm/ office decided not to GAC Diagnostics Households; ad hoc, 17 economies corruption/ report corrupt act by public officials, how important were the Business managers accountability following reasons behind his/ her decision and firm owners; Didn’t know where to report and Public officials The case could not be proven The process is too complex and long Corruption is a common practice Corruption can be justified under the current economic situation Investigation would not be made about the report There would be no enforcement even if the decision is made I was concerned about potential harassment and reprisal 84 Governance Indicator Data source Respondent Field application variable profile Behavior of In law, state civil servants convicted of corruption are prohibited State Integrity Journalist- 2 rounds; 50 US states corrupted officials from future state government employment. Reports researchers (evidence-based) Behavior of In practice, state civil servants convicted of corruption are State Integrity Journalist- 2 rounds; 50 US states corrupted officials prohibited from future state government employment. Reports researchers (evidence-based) Budget In practice, significant government expenditures (projects costing State Integrity Journalist- 2 rounds; 50 US states embezzlement more than 1% of the total state budget) require legislative Report researchers approval. (evidence-based) 85 Table 2A.23. Additional variables on respondent’s characteristics Governance Indicator What it measures Data source variable Revolving doors Have you ever been elected democratically for a Potential existing connections between GAC Diagnostics political position or in the public administration? private sector and government (firms) Revolving doors Have you ever worked for the public sector? Potential existing connections between GAC Diagnostics private sector and government (firms) State Capture From the total amount of your business, what Potential existing links between private GAC Diagnostics percentage is done with the state? sector and government (firms) State Capture What percentage of your high-level managers have Potential existing connections between GAC Diagnostics been politicians? private sector and government (firms) State Capture Please evaluate the tendency of the public officials Potential existing connections between GAC Diagnostics in their institution to change jobs and rotate from private sector and government (Public Officials) the private sector to the public sector, how frequent are the rotations of the following public officials? (Use the scale from 1 to 7, where 1 indicates that it never happens, 4 means that sometimes and 7 means that it always happens). • Directors • Executives • Technical staff • Support staff 86 Table 2A.24 Corruption Indicators: Sources and Methodologies Dataset Organization Methodology Frequency Country coverage Business Environment and World Bank and European Bank for Face-to-face survey Irregular implementation. 28 economies from Enterprise Performance Reconstruction and Development Every 3-4 years Eastern Europe and Surveys (BEEPS) Former Soviet Union Quality of Government University of Gothenburg Face-to-face survey 2010, 2013 28 EU member states plus Serbia and Turkey at the NUTS 1 and NUTS 2 level Institutional Profile French Development Agency Mail-in surveys to local 2001, 2006, 2009, 2012 143 countries Database experts working in key ministries and government agencies. The indicators can take value from 0 to 4. TI Global Corruption Transparency International Face-to-face survey Unknown (2015/16 most Global coverage Barometer recent) but sample varies depending on the year Governance and Anti- The World Bank Face-to-face survey of Irregular 17 economies Corruption Diagnostics firms, public officials and private citizens World Justice Project Rule World Justice Project Expert assessment plus Yearly 113 countries of Law Index household survey Public Administration UNDP Face-to-face survey of Every year; representative Vietnam Performance Index (PAPI) households at the provincial level Latin America Public Vanderbilt University Face-to-face survey of Every two years 28 countries of the Opinion Project (LAPOP) households (excluding Americas Canada and US where they are web-based surveys) Regional Barometers Varies Face-to-face interviews Every three years (Afro); Varies by region (Afro, Euro, Arab, Latino) twice a year (Euro); three or phone interviews (flash rounds (Arab); irregular Eurobarometer) (Latino) 87 Global Integrity Local Global Integrity In-depth interviews based Irregular implementation, All US states Integrity Initiative21 on standardized 2012; 2015 questionnaire ERCAS Indicators of European Center for Anti-corruption Composite index consisting First round 2016 105 economies Trust and Integrity in and State Building of six components: judicial Europe independence, administrative burden, trade openness, budget transparency, e-citizenship and freedom of the press. Currently calculated only at the national level Global Competitiveness World Economic Forum Expert interviews Every two years global Index 21 Global Integrity also produces country-level indicators on anticorruption. These indicators have been put on hold since 2011 and the team is revising the methodology. 88