63327 TP – 34 TRANSPORT PAPERS FEBRUARY 2011 Road Asset Governance Filter: Case Study Of Kazakhstan And Armenia Cesar Queiroz, Alejandro Lopez Martinez, Satoshi Ishihara and Kirsten Hommann 2 ROAD ASSET GOVERNANCE FILTER: CASE STUDY OF KAZAKHSTAN AND ARMENIA Cesar Queiroz, Alejandro Lopez Martinez, Satoshi Ishihara and Kirsten Hommann 3 2011 The International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone 202-473-1000 Internet: www.worldbank.org This volume is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. To order additional copies of this publication, please send an e-mail to the Transport Help Desk transport@worldbank.org Transport publications are available on-line at http://www.worldbank.org/transport/ Cover photo: Winter maintenance on the Astana-Schuchinsk expressway, Kazakhstan. By Cesar Queiroz. 2010. 4 CONTENTS Acknowledgements ................................................................................................................................................................ 7 Executive Summary ................................................................................................................................................................ 9 1. Introduction and Objectives of the Study .......................................................................................................... 13 2. Assessing Governance................................................................................................................................................ 17 2.1. Governance Indicators at the National Level .............................................................................................. 17 2.2. Governance in the Road Sector ......................................................................................................................... 18 3. Improving Governance: Initiatives in the Road Sector ................................................................................ 23 4. Data and Methodology ............................................................................................................................................... 29 4.1. Survey Design and Implementation ................................................................................................................ 30 4.2. Aggregating Indicators ......................................................................................................................................... 32 4.3. Potential Data Constraints .................................................................................................................................. 34 5. Results .............................................................................................................................................................................. 37 5.1. Armenia ....................................................................................................................................................................... 37 5.2. Kazakhstan................................................................................................................................................................. 41 5.3. Comparison of Results for Armenia and Kazakhstan............................................................................... 46 5.4. Feedback from Survey Respondents............................................................................................................... 49 6. Recommendations and Conclusions .................................................................................................................... 53 6.1. Recommendations for Armenia ........................................................................................................................ 53 6.2. Recommendations for Kazakhstan .................................................................................................................. 53 6.3. Summary and Conclusions .................................................................................................................................. 54 ANNEX 1: SURVEY INSTRUMENT TO ASSESS ROAD SECTOR GOVERNANCE ............................................. 47 ANNEX 2: DETAILED CALCULATIONS OF THE ROAD AGENCY GOVERNANCE INDEX (RAGI) SCORES FOR ARMENIA......................................................................................................................................................................... 69 ANNEX 3: DETAILED CALCULATIONS OF THE ROAD AGENCY GOVERNANCE INDEX (RAGI) SCORES FOR KAZAKHSTAN................................................................................................................................................................ 71 ANNEX 4: UNEDITED COMMENTS FROM INTERNAL AND EXTERNAL STAKEHOLDERS FROM ARMENIA .................................................................................................................................................................................. 73 ANNEX 5: UNEDITED COMMENTS FROM INTERNAL AND EXTERNAL STAKEHOLDERS FROM KAZAKHSTAN ......................................................................................................................................................................... 77 ANNEX 6: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA AND KAZAKHSTAN ......................................................................................................................................................................... 81 ANNEX 7: RESULTS FROM ARMENIA USING AN ALTERNATIVE SCORING METHOD .............................. 89 ANNEX 8: RESULTS FROM KAZAKHSTAN USING AN ALTERNATIVE SCORING METHOD ..................... 91 5 List of Figures Figure 1: Governance structure and road sector performance under budget constraints ..................... 20 Figure 2: Armenia-Comparison of internal and external stakeholders under four dimensions .......... 40 Figure 3: Kazakhstan –Comparison of internal and external stakeholders under four dimensions .. 45 Figure 4: Scores from internal and external stakeholders for Armenia dimension 1............................... 81 Figure 5: Scores from internal and external stakeholders for Armenia dimension 2............................... 82 Figure 6: Scores from internal and external stakeholders for Armenia dimension 3............................... 82 Figure 7: Scores from internal and external stakeholders for Armenia dimension 4............................... 83 Figure 8: Scores from internal and external stakeholders for Armenia ......................................................... 84 Figure 9: Scores from internal and external stakeholders for Kazakhstan dimension 1 ........................ 85 Figure 10: Scores from internal and external stakeholders for Kazakhstan dimension 2 ...................... 85 Figure 11: Scores from internal and external stakeholders for Kazakhstan dimension 3 ...................... 86 Figure 12: Scores from internal and external stakeholders for Kazakhstan dimension 4 ………….......86 Figure 13: Scores from internal and external stakeholders for Kazakhstan…………...................................87 List of Tables Table 1: Category of Interviewees in Armenia and Kazakhstan ........................................................................ 30 Table 2: Correlation Coefficients between Selected Questions .......................................................................... 32 Table 3: Rank Of Scores For Each Stakeholder, Using Simple Averaging And Factor Analysis............. 34 Table 4: Summary results of RAGI calculation for Armenia ................................................................................ 37 Table 5: Armenia - Assessment of Governance according to four different dimensions......................... 38 Table 6: Armenia - Standard deviation for Internal and External Stakeholders ......................................... 39 Table 7: Armenia – Summary of main issues ............................................................................................................. 41 Table 8: Summary results of RAGI calculation for Kazakhstan .......................................................................... 42 Table 9: Kazakhstan–Assessment of governance according to four different dimensions .................... 43 Table 10: Kazakhstan – Standard deviationfor internal and external stakeholders ................................. 44 Table 11: Kazakhstan- Summary of main issues ...................................................................................................... 46 Table 12: Comparison of RAGI results for Armenia and Kazakhstan .............................................................. 46 Table 13: Testing for differences between RAGI for Armenia and Kazakhstan .......................................... 47 Table 14: Alternative RAGI calculation dropping budget related questions ................................................ 48 Table 15: Main issues raised by stakeholders in Armenia ................................................................................... 49 Table 16: Main issues raised by stakeholders in Kazakhstan ............................................................................. 50 6 ACKNOWLEDGEMENTS This paper was prepared by a team comprised of Cesar Queiroz, Alejandro Lopez Martinez, Kirsten Hommann and Satoshi Ishihara with the essential support of a number of World Bank colleagues. Anca Dumitrescu, Andreas D. Kopp, and Jacques Bure were the peer reviewers. Special thanks are extended to the World Bank country teams of Armenia and Kazakhstan, in particular Olivier P. Le Ber, Jacques Bure, Aliya Bultrikova, Ben Gericke, Aliya Mukay, Aliya Egubayeva, Aksulu Kushanova, Arpine Kostanyan, and Irina Tevosyan. An invaluable contribution was provided by the 38 public and private sector interviewees in Armenia and Kazakhstan, whose names are not mentioned here for confidentiality reasons. The guidance and support of Catherine Revels and Gouthami Padam were essential for the completion of this paper. The team is also grateful to Alexander Bakhtamyan, Director, Armenia Transport Project Implementation Unit for his support to the study. The team is also grateful for the helpful comments provided by the participants in a brown-bag lunch meeting, chaired by Marc Juhel, Sector Manager, Transport which was held at the World Bank on November 16, 2010. 7 8 EXECUTIVE SUMMARY Building upon the Transport Governance Filter developed by the Europe and Central Asia (ECA) transport team, which identified several thematic principles and actionable indicators on the governance of the transport sector at large, this paper seeks to assess the overall governance performance of the road sector as well as the concrete issues that road administrations should address in order to improve sector governance. A pilot survey was conducted in Armenia and Kazakhstan, in which road sector stakeholders were asked to evaluate more than seventy questions structured along four governance dimensions: (i) transparency, disclosure and accountability of the road agency; (ii) transparent and accessible procurement processes; (iii) financial management system; and (iv) administrative procedures and anticorruption effort. Road sector stakeholders were broadly divided into the “internal” stakeholders, who are generally involved in the policy making and financing of the road sector, and the “external” stakeholders who are involved in the road sector but are not directly involved in road sector financing or the design of road sector governance. The scores provided by the internal and external stakeholders were compared to identify specific areas which both external and internal stakeholders agree are highly performing or in need of intervention, and issues where opinions diverge between them. The overall scores obtained were aggregated to develop the Road Agency Governance Index (RAGI), which would indicate the overall governance performance of the respective countries’ road sector. The study showed that the results for Armenia and Kazakhstan are relatively similar, with an overall RAGI of 76 percent for Armenia and a slightly lower RAGI of 74 percent for Kazakhstan. A statistical test indicated that there is no significant difference between the overall RAGI obtained for Armenia and for Kazakhstan. For both Armenia and Kazakhstan, variations in the individual scores are, on average, higher for external stakeholders, suggesting lesser agreement or perhaps lesser precise knowledge among the external stakeholders compared to the internal ones, which in turn implies a need for stronger public outreach and information dissemination should road administrations aspire more public support for their activities. In all four dimensions considered in this study, the average differences between the scores of internal and external stakeholders are higher in Kazakhstan than in Armenia. This may suggest that in Armenia there has been better communication between the public and the private sectors. Nevertheless, the road administrations in both countries should enhance their outreach to road users and other stakeholders, which will help improve the perception of the road sector held by the general public. In particular, the following actions may strengthen the governance, 9 perceived or real, of the Armenian road sector: (i) a review of the access to information policy; (ii) publication and better dissemination of information such as road condition, road works, performance indicators, and application of resources in the road sector; (iii) review of investment decision tools and procedures, including stakeholder participation; (iv) enhancement of training opportunities for sector staff; and (v) improved supervision to increase the quality of road works (through, for instance, technical audits). Regarding Kazakhstan, measures that are likely to enhance its road sector governance include: (i) improvement of anticorruption efforts in the road sector; (ii) an increase in road sector funding through, for example, road user fees or public-private-partnerships (PPP); (iii) dissemination of the results of financial audits; (iv) review of investment decision tools and procedures, including stakeholder participation; (v) development and publication of information such as performance indicators; and (vi) an increase in the efficiency of public spending, through, for example, the introduction of new technologies and standards, and use of performance based contracts (PBC). The result of the pilot study shows that the methodology of comparing differences in perception on specific governance issues between “internal” and “external” stakeholders at the issue, dimension and aggregate levels can generate “actionable indicators” to measure governance in the road sector and identify priority governance issues. On the one hand, the study highlighted potential “blind spots”, or where governments should strengthen the existing governance programs or conduct stronger public outreach to better inform the public. It also identified issues that are considered as needy of intervention across the board, or issues of high ownership that can be addressed with fewer political hurdles. On the other hand, the aggregation of indicators using simple averaging, which is confirmed as acceptable under the study based on robustness checks, allows an easy measurement of the current governance performance of the road sector in given countries. It also allows measurements across countries and over time, subject to confirmation that broad macro conditions are comparable. The following includes some of the lessons learnt that should be taken into account in better interpreting study results and designing a similar study in future: (i) since the data used are comparable only if stakeholders, especially external stakeholders, feel free to express their opinions, various “macro” level issues, especially those regarding the freedom of speech and the independence of businesses from state, must also be assessed; (ii) allocation of road sector budget, including administrative budget, should be carefully assessed as it will determine not just the current road sector performance but also how much road administrations can improve sector governance over time; (iii) caution should be exercised when conducting a cross country comparison since only a small number of stakeholders can be interviewed and macro conditions inevitably differ across countries. (Under this study, Armenia and Kazakhstan are found to exhibit comparable governance performances at the macro level. For future work, countries may first be grouped in terms of the freedom of speech and the independence of businesses from state, and cross country comparison may be limited only between comparable countries); (iv) survey instruments will need further improvement in ways that will better balance methodological simplicity and rigor. 10 11 12 1 INTRODUCTION AND OBJECTIVES OF THE STUDY While governance is defined in different ways by different parties, the World Bank defines governance as “the manner in which public officials and institutions acquire and exercise the authority to shape public policy and provide public goods and services.” 1 This includes the process by which governments are selected, monitored and replaced; the capacity of the government to effectively formulate and implement sound policies; and the respect of citizens and the state for the institutions that govern economic and social interactions among them. Governance therefore refers not only to how governments plan and execute policies but in large part depends on the traditions and institutions by which authority in a country is exercised. 2 The quality of governance affects many economic sectors and political functions, ranging from access to information about the use of public funds to the provision of clean water and basic health services. 3 Corruption is one of the greatest obstacles to economic and social development. 4 Because it involves the abuse of public office for private or political gain, 5 corruption undermines development by distorting the rule of law and weakening the institutional foundation on which economic growth depends. The harmful effects of corruption are especially severe on the poor, who are hardest hit by economic decline, are most reliant on the provision of public services, and are least capable of paying the extra costs associated with bribery, fraud, and the misappropriation of economic privileges. 6 The World Bank guidelines for selection of consultants provide useful guidance with regard to the definition of various concepts in the sphere of fraud and corruption. 7 1World Bank. 2007. “Strengthening World Bank Group Engagement on Governance and Anticorruption.” Joint Ministerial Committee of the Boards of Governors of the Bank and the Fund on the Transfer of Real Resources to Developing Countries, Washington, D.C. http://www.worldbank.org/html/extdr/comments/governancefeedback/gacpaper.pdf 2 Governance Matters 2009: Worldwide Governance Indicators, World Bank. http://info.worldbank.org/governance/wgi/index.asp 3 World Bank Institute. “Governance.” http://wbi.worldbank.org/wbi/about/topics/governance 4 Overview of Anticorruption, World Bank: http://go.worldbank.org/K6AEEPROC0 5 Perceived Corruption and Low Public Respect, World Bank: http://go.worldbank.org/CIGBDAVG40 6The World Bank report "Anticorruption in Transition: Confronting the Challenge of State Capture" distinguishes between state capture and administrative corruption. "State capture" refers to the actions of individuals, groups or firms both in the public and private sectors to influence the formation of laws, regulations, decrees and other government policies to their own advantage as a result of the illegal transfer or concentration of private benefits to public officials." "Administrative corruption" refers to the intentional imposition of distortions in the prescribed implementation of existing laws, rules and regulations to provide advantages to either state or non-state actors as a result of the illegal transfer or concentration of private gains to public officials. http://go.worldbank.org/CIGBDAVG40 7Guidelines for the Selection and Employment of Consultants by World Bank Borrowers (May 2004, revised October 2006 and May 2010): http://go.worldbank.org/U9IPSLUDC0 13 Over the past several years, the World Bank’s infrastructure units have sought to address Governance and Anticorruption (GAC) concerns in a deliberate and measured way. Various streams of work have raised awareness of GAC issues in infrastructure operations among appropriate sector practitioners. However, translating that awareness into systematic treatment of GAC across infrastructure sectors requires expanding the coverage of GAC works (across sectors, countries, and types of engagement), improving the quality of GAC related works (developing and applying appropriate tools in specific country contexts), and building capacity within the infrastructure practice to engage partner countries, industry players and other donors. Also, action-oriented indicators must be developed that would both inform appropriate actions tailored to address critical sector issues and help measure improvements over time. The Governance and Anticorruption in Infrastructure Advisory Program (GAC Squad) has therefore been established to provide direct assistance to Task Team Leaders (TTLs) to build or improve governance measures under their projects across the energy, transport, water, and urban sectors through a combination of hands-on learning-by-doing and knowledge dissemination. This study aims to develop and pilot test a set of “actionable indicators” 8 to measure the level of governance in the road transport sector and identify actions that need to be taken to strengthen the aid and development effectiveness of road transport investments in respective countries. Building upon the Transport Governance Filter 9 developed by the Europe and Central Asia (ECA) transport team, which identified five thematic principles and actionable indicators on the governance of the transport sector, this study sought to develop the list of actionable indicators specifically tailored to the governance challenges of the road sector. The Road Agency Governance Index (RAGI) was thus developed and pilot-tested to assess the relevance of RAGI to address governance issues in the road sector, fine-tune specific indicators and identify issues that need further improvements. Two pilot projects were selected in consultation with the ECA transport team and based on their respective project portfolios, namely, Kazakhstan South-West Roads: Western Europe-Western China International Transit Corridor Project10, and the Armenia Lifeline Roads Improvement Project 11. This report starts with an overview of the existing approaches to governance and corruption with a particular focus on the road sector. It then outlines the methodological framework developed under the study to assess the governance challenges facing the road sector, and report the governance challenges in Armenia and Kazakhstan using the methodology developed. The report will conclude with an 8The “Governance and Anti-Corruption Strategy” (World Bank, 2007: 34-35) defines the need of actionable governance indicators which are commonly defined as “narrowly defined and disaggregated indicators that focus on relatively specific aspects of governance and could provide guidance on the design of reforms and monitoring of impacts.” For details, see: https://www.agidata.info/main/AboutAGI.ashx 9Khan, Maha and Satoshi Ishihara. 2008. “Transport Governance Filter” (October 2008 Draft). World Bank. 10Project details are available at: http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSitePK=40941&menuPK =228424&Projectid=P099270 11Project details are available at: http://web.worldbank.org/external/projects/main?pagePK=64283627&piPK=73230&theSitePK=40941&menuPK =228424&Projectid=P115486 14 assessment of the strengths and weaknesses of the methodological approach used and concrete suggestions to strengthen governance in the road sector of Armenia and Kazakhstan. 15 16 2 ASSESSING GOVERNANCE 2.1 GOVERNANCE INDICATORS AT THE NATIONAL LEVEL Recent years have seen a plethora of indicators that were developed to measure governance, transparency and other measures of government effectiveness. These are typically produced at the country level, which allows for broad comparisons between countries. One example of a frequently used national level governance indicator is the Corruption Perceptions Index (CPI), developed annually by Transparency International (TI), which defines corruption as the abuse of entrusted power for private gain. 12 The CPI ranks 180 countries by their perceived levels of corruption, as determined by expert assessments and opinion surveys. CPI scores for countries vary between 0 (“corrupt”) and 10 (“clean”). Different indicators have different strengths and weaknesses, and vary with respect to the extent of subjectivity embedded in their measurement, their reliability and their coverage of multiple countries and multiple years. Some indicators are based on the opinion of selected ‘experts’ within the country or outside, some indicators are created from enterprise opinion surveys, and others infer governance from observable data such as the backlog of court cases or the time between an application for opening a business until its approval. Often coverage and reliability are negatively associated because it is expensive to conduct expertly prepared governance surveys for many countries over multiple periods. Kaufmann et al (1999) 13 sought to increase coverage by creating composite indicators for six different governance dimensions 14 using indicators already measured by other institutions as input. They grouped these ‘input’ indicators under the respective governance dimensions to be assessed, aggregated them using principal component analysis, and estimated for each country and over multiple years (i.e., from 1996 until 2008 for every second year) the score and the underlying standard error of the estimate. This way, they were able to generate governance estimates for a wide range of countries and years, allowing for careful comparisons of countries’ relative performance on governance and their change in ranking over 12See Transparency International website: http://www.transparency.org/about_us 13Kaufmann, Kraay, Mastruzzi and Zoido-Lobaton published in various team formations four publications that initially created and subsequently updated the six governance indicators. The first paper was published in 1999 explaning the methodology. Kaufmann, d., a. Kray, et al. (1999). "Aggregating Governance Indicators." policy research working paper 2195, the World Bank. 14They include (i) Voice & Accountability, (ii) Political Stability and Lack of Violence/Terrorism, (iii) Government Effectiveness, (iv) Regulatory Quality, (v) Rule of Law, and (vi) Control of Corruption. WGI country reports are available at: http://info.worldbank.org/governance/wgi/pdf_country.asp 17 time. 15 The large coverage, together with the estimation of the standard error for each composite indicator which helps gauge its reliability, made the use of these Governance Indicators (WBIGI)16 quite popular in applied research. Such national level governance indicators provide useful entry points for the assessment of sector level governance as they provide a macro perspective on broader governance issues that may bear on the operational environment in which road sector stakeholders operate. Indeed, the Transport Governance Filter suggests that GAC actions be best developed against the backdrop of the national governance context. By assessing the national governance context relevant to the transport sector prior to the design of any operation, project teams can not only identify cost- effective approaches to reducing corruption risks but can also assess the suitability of existing country systems. For instance, when national procurement capacity is very weak, project teams may want to consider “ring-fencing” projects by hiring procurement management consultants as a short term solution to weak procurement capacity, even if efforts for long-term national capacity development should continue. Also, as mentioned below, governance performance at the macro level in areas such as freedom of expression and civil society development provides an important background to interpret data on sector governance. 2.2 GOVERNANCE IN THE ROAD SECTOR An estimate of the extent of total annual worldwide transactions that are tainted by corruption puts it close to US$1 trillion. 17 In particular, the road transport sector is widely considered to suffer from poor governance. Transparency International 18 rates the infrastructure sector, under which road transport falls, as the most corrupt sector worldwide – ahead of defense, oil and gas. Investments in road infrastructure typically comprise between 0.5 percent and 5 percent of a country’s budget, making it a significant expenditure category. Corruption in roads and other transport projects can account for as much as 5 to 20 percent of project costs. 19 Corruption is often compounded by poor financial accounting practices and a lack of proper auditing. It is estimated that strengthening governance in the transport sector could potentially save 10 to 40 percent of overall spending in the sector. 20 Given the size of the public money used, improving governance in the transport sector could yield substantial returns. 15The underlying data as well as methodological issues are described in "Governance Matters VIII: Governance Indicators for 1996-2008," published in June 2009. 16WBIGI country reports are available at http://info.worldbank.org/governance/wgi/pdf_country.asp 17Kaufmann, Daniel. Myths and Realities of Governance and Corruption: The World Bank (2005). http://siteresources.worldbank.org/INTEAPREGTOPRURDEV/Resources/573691- 1175901454225/seminar1_myths.pdf 18 Transparency International homepage: http://www.transparency.org 19 The World Bank, South Asia: Governance and Transport: Developing Frameworks, available at: http://go.worldbank.org/S1ARP5PNI0 20 ibid. 18 Transparency International observes that corruption in roads and other construction projects is detrimental to society for the following three reasons: 21 It damages developed and developing countries by allowing projects to be implemented that are sometimes unnecessary and often overpriced. Shortcuts on safeguard standards can also lead to loss of life and exacerbate poverty and underdevelopment. It damages companies, from tendering uncertainty, wasted tender expenses, increased project costs, reduced project opportunities to extortion and blackmail, criminal prosecutions, fines, blacklisting 22, and reputational risk. It damages individuals, resulting in reduced morale, criminal prosecution, fines and imprisonment. Poor governance impacts everyone but is often felt the most by the poor, who are not able to alleviate its effects. Failure to deliver adequate road infrastructure or transport services means that project outcomes for target beneficiaries may not be realized. 23 The poorest usually do not have representation to voice their preferences and grievances, nor do they have access to complaint mediation forums. Improving governance will help make transport services more oriented toward the needs of users, reduce the cost of travelling, improve access to basic services and reduce the price of goods and services. Improved governance will help strengthen road sector performance through enhancing the efficiency at which a limited budget is put to use. As illustrated in Figure 1, underlying governance structures will determine the extent to which available resources are used effectively for the improvement of the road sector, and thus determine road sector performance under budget constraints. In other words, while allocation of sufficient budget remains critical for the provision of adequate road transport services, road administrations will be able to improve service levels even at the same budget level by strengthening sector governance. Conversely, if underlying governance structures remain poor, improvements in service levels will be limited even if more funding is made available. 21 Transparency International, Preventing Corruption in Construction Projects: http://www.transparency.org/tools/contracting/construction_projects 22For example, the list of firms and individuals ineligible to be awarded a World Bank-financed contract is available at: http://web.worldbank.org/external/default/main?contentMDK=64069844&menuPK=116730&pagePK=64148989 &piPK=64148984&querycontentMDK=64069700&theSitePK=84266 23 ibid. 19 FIGURE 1: GOVERNANCE STRUCTURE AND ROAD SECTOR PERFORMANCE UNDER BUDGET CONSTRAINTS Transparent and Accountable Road Sector Management Safer and better maintained road network Open and Fair Public Procurement State budget Roads are built Sound Financial Management according to the needs Donor Efficient Administrative funding Procedures Social and environmental concerns are better observed Source: Adapted from Khan and Ishihara 2008 20 21 22 3 IMPROVING GOVERNANCE: INITIATIVES IN THE ROADSECTOR Based on an increasing recognition that governance needs to be improved in the transport sector in general, and the road sector in particular, several initiatives have been conducted, as described below, that seek to identify specific areas susceptible to corruption and to propose actions that counter corruption. The Construction Sector Transparency Initiative 24 (CoST) has been established under the support of the UK Department for International Development (DFID). CoST aims at introducing transparency and accountability in the construction sector with a particular focus on public disclosure of information. CoST thus seeks to strengthen the collection and disclosure of information related to: (i) the basis on which the project was commissioned, (ii) project evaluation, and (iii) any significant changes from the original tender document and reasons for the changes. CoST also prepares a series of working papers aiming to support its initiative in pilot countries by providing information and tools that will: (i) illustrate the benefits of improved sector governance (including transparency and oversight) in terms of sector development outcomes; (ii) outline in greater detail transparency and oversight mechanisms that can be introduced in the sector, including guidance on requirements, costs, and benefits of varied levels of transparency and oversight; and (iii) provide particular guidance on costs and benefits, good practices, requirements and methods for procurement and contract publications, third party oversight, and community oversight mechanisms. 25 While its immediate objective is to enhance the accountability of procurement institutions and construction companies, CoST aims to help societies improve the governance in public construction projects in the long run by creating awareness and allowing the public to draw comparisons between what was planned and what was delivered. The global Transport Knowledge Partnership (gTKP) 26has identified 45 ways in which corruption may occur in road projects. These include corruption 24 CoST homepage: http://www.constructiontransparency.org 25 See, for example, “How Expert Third Party Oversight of Public Construction Projects Can Increase Value for Money and Fight Corruption.” Working Paper No. 3, Draft, August 2009. 26 global Transport Knowledge Partnership (gTKP): http://www.gtkp.com 23 opportunities under the following project stages:27 (i) selection of the project; (ii) planning permission for the project; (iii) choice of design and method of construction for the project; (iv) selection of project participants; (v) collusion and bribery at tender stage; (vi) execution of the project works; (vii) contract claims; and (viii) completion of the project. gTKP provides a detailed examination of corruption opportunities under each one of the above stages. For example, regarding selection of project participants, a proper tender process may be omitted in order to allow for the corrupt nomination of a particular contractor. Even where there is a tender process, the process may still be corrupted in a number of ways: (a) the tender criteria may be distorted in favor of a particular tenderer; (b) information may be leaked for the benefit of a tenderer; (c) there may be collusion between tenderers; (d) there may be bribery by tenderers to the project owner representative and/or the engineer; and (e) there may be extortion of tenderers by the project owner representative and/or the engineer. Paterson and Chaudhuri 28 developed an analytical framework to identify areas prone to corruption in the transport sector. They analyzed different corrupt behaviors prevalent in the transport sector, corrupt practices systematically employed by state officials in road administrations, and corruption at the project and transaction levels. Additionally, they proposed a twin-pronged strategy for improving governance in the transport sector which includes: (i) a focus on short- term controls, investigations, fraud detection, sanctions, and their enforcement; and (ii) a longer-term preventive approach that aims to build internal controls and capacity in ways appropriate to local conditions and the prevailing modes of corruption. Ehrhardt, Oliver and Kenny 29 assessed and identified four focus areas where governance can be improved in the road sector: (i) Planning and project selection - Good planning would promote efficiency and accountability by: giving stakeholders a chance to be involved in developing plans, making the intended direction of the sector (and government) expenditure clear, and providing a justification for this, 27Road Projects: How does corruption occur?: http://www.gtkp.com/theme.php?themepgid=265 28Paterson, W.D.O. and P. Chaudhuri, “Making Inroads on Corruption in the Transport Sector through Control and Prevention”, in The Many Faces of Corruption: Tracking Vulnerabilities at the Sector Level, ed. J Edgardo Campos and Sanjay Pradhan. (Washington DC, World Bank: 2007). http://www.igac.net/pdf/publications_adb_manyfacesofcorruption.pdf 29Ehrhardt, David, Chloë Oliver and Charles Kenny. 2009. “Deterring Corruption and Improving Governance in Road Construction and Maintenance.” World Bank Transport Paper Series TP-27, September 2009 (Washington DC). http://siteresources.worldbank.org/INTTRANSPORT/Resources/336291-1227561426235/5611053- 1229359963828/TP-27_Governance_Sourcebook.pdf 24 allowing stakeholders to judge whether construction and development activities are conducted as planned, and enabling good project selection in line with planning, and providing clear justification for why these projects are being implemented, again helping with accountability and performance. (ii) Capital works - Corruption in capital works happens in many ways including bribes, kickbacks, collusion, bid rigging, and fraud. These activities can take place in the following areas: selection and award of contract (for instance, restrictive or deceptive bid specifications), design and construction specifications (for instance, padding of design and cost estimates), project supervision (for instance, false certification of compliance), and contract claims (for instance, falsifying need or quantities in change orders). (iii) Government engineering and construction units - When a government agency is employing large numbers of laborers, buying lots of materials for a roads project, and maintaining equipment and property there are a number of additional corruption risks involved with respect to human resources, suppliers, and department property. (iv) Public-private partnerships – While PPPs can reduce corruption in some areas, they can also generate new governance problems, including those related to contract selection and award, and post-procurement governance issues, including renegotiations. Queiroz and Kerali assessed the governance risks in PPP projects 30 and found that, because PPP projects in transport infrastructure tend to have monopolistic features, good governance in managing them is essential to ensure that the private sector’s involvement yields the maximum benefit for the public. Good governance in this case requires: (i) competitive selection of the private investor, (ii) proper disclosure of relevant information to the public, and (iii) having a regulatory entity oversee the contractual agreements over the life of the concession. Some Bank-financed roads projects adopt anticorruption plans to mitigate corruption risks and improve sector governance. Examples include the Orissa State Roads Project in India, the Kazakhstan South West Roads Project, the Western Indonesia National Roads Improvement Project, and the Ukraine Road Safety and Improvement Project. Some of the initiatives include a disclosure policy, website development and information management system, complaints handling system, citizen oversight/civil society involvement, and actions for mitigating collusion in 30Queiroz, Cesar, and Henry Kerali. 2010. “A Review of Institutional Arrangements for Road Asset Management: Lessons for the Developing World.” World Bank Transport Papers Series TP-32. Washington DC. http://siteresources.worldbank.org/INTTRANSPORT/Resources/336291-1227561426235/5611053- 1229359963828/TP-32-Road_Asset_Mgmt.pdf 25 procurement and payment fraud. 31 For instance, the South Asia Transport Unit 32 has developed a Governance and Accountability Action Plan (GAAP) Guidance Note for its India projects. It identifies areas critical to implement quality road development programs and develops indicators to measure the performance of the construction industry. The Western Indonesia National Roads Improvement Project established six pillars to tackle corruption: enhanced disclosure, civil society oversight, mitigation of collusion risks, mitigation of forgery and fraud risks, complaint handling system, and sanctions and remedies. 33 As can be seen, most existing initiatives to improve governance and minimize corruption in the road sector primarily aim at identifying specific governance challenges in the road sector and proposing measures to address them. While they provide useful guidance to strengthen governance in the road sector, the heterogeneous approaches adopted do not allow for an easy comparison of initial conditions and improvements over years. This study seeks to complement these existing efforts, by creating a systematic tool that allows the assessment of road sector governance and its improvement over time through interventions, and the systematic identification of the key problems areas that warrant interventions. 31Project Appraisal Documents for Bank-financed roads projects: http://www.wds.worldbank.org/external/default/ main?pagePK=64187835&piPK=64620093&theSitePK=523679&menuPK=64187283&siteName=WDS&colTitle=& dAtts=DOCDT,DOCNA,REPNB,LANG,DOCTY,VOLNB,REPNME&sortDesc=DOCDT&query=&sType=2&report=&trustf und=&author=&pageSize=20&projid=&docTY=540656&docType=3&docTitle=®ion=119225&loan=&credit= 32The South Asia Transport website provides several examples of how governance and corruption issues are integrated into transport projects in the region: http://go.worldbank.org/S1ARP5PNI0 33Project Appraisal Document for Western Indonesia National Roads Improvement Project, World Bank 2007. 26 27 28 4 DATA AND METHODOLOGY The Road Agency Governance Index (RAGI) was developed taking into consideration the key areas identified in the Transport Governance Filter, prepared in 2008 by Khan and Ishihara. 34 The five governance dimensions developed originally under the Transport Governance Filter were consolidated into four main dimensions in order to avoid overlap and improve comparability. A questionnaire used under the study was developed along these governance dimensions, which are summarized as follows: Dimension 1: Transparency, Disclosure and Accountability of the Road Agency (questions Q2-Q19). This dimension addresses issues such as disclosure of annual budgets and expenditures, disclosure of audit reports, road user satisfaction surveys, and consideration of the results of public hearings. Dimension 2: Transparent and Accessible Procurement Processes (Q20-Q34). This dimension addresses issues such as tender processes, selection of consultants, and bid evaluation procedures. Dimension 3: Financial Management System (Q35-Q49). This dimension addresses issues such as external audit reports, cash flow planning, budget execution reports, and fiduciary functions and procedures. Dimension 4: Administrative Procedures and Anticorruption Effort (Q50-Q70). This dimension addresses issues such as unit price of works, road users’ feedback mechanism, corruption, and construction quality. The interview with stakeholders in Armenia and Kazakhstan took place between October 2009 and March 2010. A total of 18 interviews were conducted in Armenia and 20 in Kazakhstan. The list of those consulted, grouped by external or internal, is detailed in Table 1. 34Khan, Maha and Satoshi Ishihara. 2008. “Transport Governance Filter” (October 2008 Draft). World Bank. 29 TABLE 1: CATEGORY OF INTERVIEWEES IN ARMENIA AND KAZAKHSTAN Category of Interviewees Subgroup Armenia Kazakhstan Symbol Internal External Internal External Ministry of Transport (MOT) and Road A 3 4 Directorate (Rd. Dir.) Ministry of Finance (MOF) and Ministry of B 3 4 Economy (MOE) Private Sector, e.g., freight forwarders, C 3 3 insurance companies Road Contractors RC 2 3 Academia, e.g., dean of engineering, professors D 2 1 Consultants E 3 1 Non Government Organizations (NGO) and F 2 4 International Financing Institutions (IFI) Total 8 10 9 11 4.1 SURVEY DESIGN AND IMPLEMENTATION Each of the dimensions outlined above consists of 15 to 21 questions. The survey instrument consists of 72 questions (see Annex 1), of which: 68 questions are of a ‘yes,’ ‘no’ or ‘uncertain’ nature; four questions (Q66-69, with Q69 having six sub-questions) require assessment on a scale between 1 to 5, with 5 assessing the issue raised as very critical; two questions (Q1 and Q70) are of qualitative nature that cannot be quantified; and two further questions (Q71 and Q72) require a percentage response with the higher the better. During the interview, it was noted that respondents were sometimes not inclined to commit to either response option offered under the binary questions (i.e., ”yes” or ”no”), so an option for ”uncertain” was added at the beginning of the survey implementation. This may cause some problems with respect to the reliability of the answers, and the next round of survey implementation, after this pilot phase, will consider a revised answer code. In particular, for future works it seems appropriate to add “don’t know” as a fourth option to the answers. The interviews were conducted with stakeholders within (internal) and outside (external) the public sector; yet all stakeholders are considered knowledgeable about the road sector in their respective countries. Internal stakeholders are 30 involved in some way in the decision-making process in the sector as far as planning, funding or the provision of infrastructure and services are concerned. As such, they might have different perceptions and opinions about sector governance issues compared to external stakeholders, which include construction/ consulting industries, transport/ freight service providers and civil society watchdogs. The study explicitly sought to explore the difference in perception as indications of potential entry points. The comparison of differing perceptions between internal and external stakeholders will not only help reveal gaps with regards to the perceived levels of governance in the road sector, but it will also help prioritize and formulate specific actions not only to strengthen road sector governance but also to gain public trust in road sector administration. Even under the same dimension (e.g., transparency), gaps in perception and the level of knowledge between external and internal stakeholders are larger for some specific concerns than others, signaling “blind spots” where road government agencies could do more to improve public perception with respect to their administration. Academia was considered “internal” under this survey because interviewees are professors of state universities which are as in many former Soviet Union countries, strongly affiliated with state agencies and considered to be involved in sector planning in varying degrees. Questions Q71 (percentage of paved roads in good condition) and Q72 (percentage of unpaved roads in good condition), unlike other questions, address governance issues only indirectly. While conditions of roads will likely be poor when governance is weak, it may be due to other factors including overall shortage of road sector funding or weak technical capacity of the road sector at large. The level of governance is only one of many factors that determine the condition of roads. Nonetheless, it was decided to include these questions as part of the assessment of RAGI since they reflect sector governance to a certain degree, and because they represent only two of 72 questions posed to construct the RAGI. The two indicators account for only 2.7% of the overall RAGI score. Moreover, regular monitoring and public reporting of road conditions is a key aspect of accountability and sound road asset management. These questions provided interesting information about the different perceptions on the condition of the road network between “internal” and “external” stakeholders. For future work, consideration should be given to the pros and cons of keeping these questions in the questionnaire. Typically one would expect similar questions posed under one dimension to experience some correlation in the answers provided. However, even though the questions can be grouped under the same dimension, one could perceive opposite answers in many. Table 2 below shows the percentage correlation 35 between questions which one would expect similarity in the answers. As one can see, correlations in the responses were very high for Kazakhstan (all around 90 percent), but correlations for the first and last pair of questions for Armenia are low with 12 35Non-parametric Spearman’s correlation coefficients were computed, since the variables are not continuous. See for example: Siegel, Sidney. “Nonparametric Statistic for the Behavioral Sciences.” McGraw Hill, 1956. 31 and 14 percent, respectively. Since the questions posed are different, lower correlations do not necessarily suggest inconsistency in the responses; however, such low correlation may mean that some respondents have more detailed or lesser knowledge on a specific topic. TABLE 2: CORRELATION COEFFICIENTS BETWEEN SELECTED QUESTIONS Questions Armenia Kazakhstan Q6 Are the road works contracts and names of winning bidders disclosed to the public? 14% 89% Q15 Are progress and costs of road works (particularly large contracts) reported to the public? Q11 Are public hearings conducted for policy changes or large investment projects? 64% 94% Q12 Are the results of the public hearings (see Q.11) taken into account in the decision making process? Q17 Is the progress in performance indicators published in annual reports that are available to the public? Q18 In the case of poor performance, does the annual (or other) 66% 88% report discuss the corrective measures taken (e.g., to address poor road condition)? Q38 Does the Road Agency implement all recommendations from internal controls within one year? 89% 90% Q39 Does the Road Agency implement all recommendations from external audit reports within one year? Q48 Does the Road Agency allocate adequate administrative budget for capacity development of fiduciary staff? 12% 90% Q49 Does the Road Agency give fiduciary staff sufficient authority on day-to-day execution of fiduciary functions and procedures? Variation in the responses to each question between internal and external respondents was also examined and their standard deviation computed. For both Armenia and Kazakhstan, the standard deviation is, on average, higher for external stakeholders by 0.09 (Armenia) and 0.13 (Kazakhstan) points, suggesting lesser agreement or perhaps lesser precise knowledge among the external stakeholders compared to the internal ones. 4.2 AGGREGATING INDICATORS All quantifiable questions were arranged such that the higher the score, the better the perceived situation in the road sector. For example, answers ranging from 1 to 5 (with 5 rating the governance situation worst) were recoded to generate 0 points for 5, 0.25 points for 4, 0.5 points for 3, 0.75 for 2, and 1.0 for 1. Other questions were already scaled suitable for aggregation, and respondents that were ‘uncertain’ were assigned 0.5 points. One of the key questions when aggregating indicators to form an index is the weight one assumes for each of the indicators. 32 Some studies that have aggregated different governance indicators 36 used principal component analysis (PCA). PCA does not assign weights for each indicator but identifies patterns in the data on which basis weights are computed. Other studies have explicitly assigned weights by making an informed judgment on the importance of certain indicators or questions. This embeds a certain amount of subjectivity, even if the weighting is informed by expert knowledge. It should be noted that simple averaging makes a similar judgment, as it assumes that all indicators or questions have equal importance. In this review of road sector governance, we simply averaged the answers received under each dimension, but verified – as a robustness check – whether PCA would generate the same results in terms of ranks. PCA was applied by aggregating questions within one dimension for each respondent. Each respondent, therefore, received a score based on the responses to the questions (in Kaufmann's analysis the score was assessed for a country) according to which one can rank respondents in terms of how they judge the governance in the road sector for the respective dimension. The ranking that emerged was compared based on factor analysis (one sub-form of PCA), to the ranking obtained when simple averaging was conducted. As can be discerned from Table 3, differences in the ranks are relatively small, that is, in many of the cases, the same respondents had identical ranks whatever methodology was applied and if the rank differs it does only by a few positions. 36For instance, Kaufmann et al, mentioned earlier, and also the Fraser Institute http://www.freetheworld.com/book.html 33 TABLE 3: RANK OF SCORES FOR EACH STAKEHOLDER, USING SIMPLE AVERAGING AND FACTOR ANALYSIS Armenia Kazakhstan Simple Averages Factor Analysis Simple Averages Factor Analysis Best 1st 16 16 16 16 2nd 5 5 3 3 3rd 11 7 2 2 4th 3 6 1 11 5th 13 13 15 15 6th 6 3 11 8 7th 7 11 19 1 8th 14 14 8 7 9th 8 18 13 19 10th 18 15 7 18 11th 1 1 18 13 12th 12 8 10 10 13th 15 12 9 9 14th 17 17 20 20 15th 10 10 12 12 16th 4 4 17 6 17th 9 9 6 17 18th 2 2 4 4 19th 5 5 Worst 20th 14 14 Based on these brief comparisons, it was decided to keep simple averaging as the preferred aggregation method, since it is more easily understandable and replicable. Moreover, the scores in a PCA can only be used for ranking (i.e., determining a relative position among the respondents to each other, whilst the distance from one score to another is meaningless). This reduces the usefulness of PCA, since the difference in ratings between respondents could provide vital information. The average scores presented in the results section represent percentage points, whereby higher percentage points represent higher ratings on governance. A total of 75 answers were given equal weights and grouped into one of the four dimensions. The average percentage points were obtained for each dimension, which were then averaged out again to obtain the overall score. If all answers had the maximum score, the governance rating would be 100 percent. 4.3 POTENTIAL DATA CONSTRAINTS The study used as the primary source of data the perceptions and knowledge expressed by internal and external stakeholders. It needs to be remembered that the extent to which they felt free to express their true perceptions and knowledge determines the reliability of data. The study did not endeavor to internalize such external factors in interpreting data and constructing RAGI, but assessed the macro 34 governance indicators with regard to freedom of speech in the pilot countries. According to the “Freedom of Expression” indicator developed by the Freedom House 37, both Armenia and Kazakhstan are “partially free” for civil liberty including the freedom of expression, where Armenia scored eight and Kazakhstan scored seven in 2009 on a scale between 0 and 16, where higher scores indicate greater freedom. While Armenia and Kazakhstan are therefore considered to be comparable in terms of freedom of expression, allowing a meaningful comparison of data collected in both countries, the same cannot be guaranteed for all countries. For future work, macro level data such as freedom of speech, civil society development, and independence of businesses from state should first be assessed and cross country comparison should only be allowed for those countries with comparable traits. As mentioned above, questions of ‘yes,’ ‘no’ or ‘uncertain’ answer coding were used under the survey. However, some of the interviewees preferred to choose a medium response, thereby declaring that they are between ‘yes’ and ‘no’, while others did not know the answer to the question. As both are lumped under the category of ‘uncertain’ with a 0.5 point attributed to the question, it has reduced the accuracy of the information provided, since ‘not knowing the answer’ should have amounted to a missing value. While the coding of responses would be reviewed in the future, as this initiative is being scaled up, for the purpose of this assessment we checked whether the omission of ‘uncertain’ responses would dramatically alter the results. A brief visual inspection as well as a comparison of the averages (see Annexes 7 and 8) showed that the difference is minor, and omitting ‘uncertain’ responses yields an overall improved governance rating. A second and related problem emerges from some of the questions themselves. Some questions require intimate understanding of specific aspects of sector governance, which not all of the selected stakeholders have. For example, civil society watch dogs and academia are not as familiar with procurement and financial management issues as they are with transparency related issues. Some questions relate to information more internally available (i.e., to staff in the public institutions that make the decisions in that sector). Consequently, some external stakeholders were unable to answer some of the questions. This may be mirrored by the higher variation in responses by external stakeholders compared to those given by internal ones. Again, the survey instrument could be revisited, as this road governance assessment is being scaled-up to other countries. 37Freedom House is dedicated to promoting free institutions worldwide. It publishes surveys detailing state of civil liberties, political rights, economic freedom: www.freedomhouse.org 35 36 5 RESULTS 5.1 ARMENIA The overall Road Agency Governance Index for Armenia, as indicated in Table 4, is 76%, with a range between 36% and 92% and a standard deviation of 0.155, or coefficient of variation of 20.3%. The RAGI obtained from internal stakeholders, as expected, is higher than the one from external stakeholders, respectively 82% and 62%. Both the range and the standard deviation for the external stakeholders are higher than for the internal stakeholders, reflecting a broader variety of perceptions among the former. TABLE 4: SUMMARY RESULTS OF RAGI CALCULATION FOR Average and Range Road Agency Governance Index (RAGI) by internal and external stakeholders: Average Min Max STD Dev Internal 82% 63% 91% 9.1% Stakeholders External 72% 36% 92% 18.5% Stakeholders Overall Average and Range Road Agency Governance Index (RAGI): STD Coefficient of Average Min Max Dev variation 76% 36% 92% 15.5% 20.3% Table 5 provides a summary of the scores obtained under each one of the four dimensions for both internal and external stakeholders. As can be seen, data can be divided into three broad categories:(i) issues highly rated by both internal and external stakeholders, indicating areas where stakeholders generally agree that the level of governance is high; (ii) issues highly rated by internal stakeholders but less so by external stakeholders, indicating areas where road agencies need to do more to gain public trust; and (iii) issues poorly rated by both internal and external stakeholders, indicating areas where stakeholders generally agree that more needs to be done to strengthen governance. 37 TABLE 5: ARMENIA - ASSESSMENT OF GOVERNANCE ACCORDING TO FOUR DIFFERENT DIMENSIONS Scores Range Dimension Internal External Average Difference between Stakeholders Stakeholders Internal & External Min Max Stakeholders (%) 1. Strengthen Transparency, Disclosure of information and 0.76 0.73 0.74 0.25 1.00 4.41% Accountability of the Road Agency (Q2-Q19) 2. Promote Transparent and Accessible Procurement 0.90 0.82 0.86 0.75 1.00 9.82% Processes (Q20-Q34) 3. Establish Sound Financial Management System (Q35- 0.89 0.78 0.84 0.63 1.00 14.32% Q49) 4. Strengthen Administrative Procedures and Anticorruption 0.73 0.56 0.64 0.29 1.00 30.84% Effort (Q50-Q70) Examples of issues highly rated by both internal and external stakeholders include: conducting public hearings for large investment projects; invitation extended to all bidders to attend bid opening sessions; records kept by the Road Agency for all contracts; provision of sufficient information by the Road Agency to bidders; and an effective payroll control system maintained by the Road Agency. As one can see, issues highly rated by both internal and external stakeholders are mostly related to transparency and disclosure, and concerns knowledge in whether certain events or actions are factually conducted, rather than perception as to whether certain events or actions are conducted properly. It thus indicates that the Armenian Road Agency communicates relatively clearly to the public certain events or activities they conduct. Issues that are rated highly by internal stakeholders but poorly by external stakeholders include the following: whether performance indicators are developed for the road sector; whether road condition data are periodically collected; whether the Road Agency management is organized on a merit basis; whether technical audits are conducted for road works; whether the Road Agency uses objective investment decision tools; and whether the Road Agency has a functional management information system. As one can see, issues where opinions differ between internal and external stakeholders are mostly related to administrative procedures of the Road Agency. Interestingly, external stakeholders consider that certain activities or events are not conducted, while internal stakeholders provide the opposite answers. This indicates that not all activities or events the Armenian Road Agency conduct are clearly communicated to the public. Some issues are poorly rated by both internal and external stakeholders. They agree across the board that more needs to be done in those areas. They include: inadequate current public sector policies, including the salary scale of public servants; difficulty in hiring or retaining qualified staff by the Road Agency; and inadequate road sector budget. As one can see, they mostly refer to administrative procedures, especially in terms of human resources issues. 38 Interestingly, there are issues more highly rated by external stakeholders than by internal stakeholders. Such issues relate to whether the annual budget and expenditures are disclosed to the public; whether the Annual Reports of the road sector are published; and whether the Road Agency has a homepage to disclose road sector information. A closer look at the data, however, revealed that this is because such issues are poorly rated by internal stakeholders who have more accurate knowledge in these areas, such as the Road Agency and Ministry of Transport. A comparison between the standard deviation for internal and external stakeholders is shown in Table 6. The results illustrate that the standard deviation is higher for external stakeholder in each one of the four dimensions. TABLE 6: ARMENIA - STANDARD DEVIATION FOR INTERNAL AND EXTERNAL STAKEHOLDERS Standard Deviation Dimension Internal External Stakeholders Stakeholders 1. Strengthen Transparency, Disclosure of information and Accountability of the Road 0.15 0.24 Agency (Q2-Q19) 2. Promote Transparent and Accessible 0.11 0.25 Procurement Processes (Q20-Q34) 3. Establish Sound Financial Management System 0.12 0.19 (Q35-Q49) 4. Strengthen Administrative Procedures and 0.07 0.18 Anticorruption Effort (Q50-Q70) Overall, in Armenia, perception diverges most significantly between internal and external stakeholders on administrative procedures. Interestingly, opinions mostly converged on procurement and financial management issues – both internal and external stakeholders generally hold a positive view on procurement and financial management issues. This may be because few problems are considered to exist in Armenia on procurement and financial management, or because few stakeholders find the current practice problematic. While available data do not allow making any conclusive judgment on this matter, it is interesting to note that perceptions of road contractors are very close to those of internal stakeholders and diverge from other external stakeholders – the average RAGI of road contractors stands at 83% with a range between 81% and 85%, which is very close to the corresponding rating by internal stakeholders, especially the Road Agency and the Ministry of Transport. Figure 2 presents a graphical representation of the scores from internal and external stakeholders for Armenia. Annex 6 presents such scores separately for the four dimensions previously defined. 39 For Armenia, as indicated in Figure 2, there are several aspects showing highly positive perceptions, or areas of strength, that include procurement, general management, and financial management. Conversely, there are several aspects of low perception, or areas deserving more attention, which include communications between the public and private sectors, road condition, public sector policies, road financing, staff qualification, high cost of road works, and anticorruption effort. The highest differences in perceptions by the public and private sectors are in communications and staff training. In particular, the cluster of lowest scores, by both internal and external stakeholders, refers to questions Q66 to Q72 - dimension 4, (see Annex 6). The main issues in this area are in public sector policies, road financing, staff skills, corruption, and road condition. Also remarkably low is the perception on how the road agency seeks feedback from road users, as well as the dissemination of audit and annual reports. FIGURE 2: ARMENIA-COMPARISON OF INTERNAL AND EXTERNAL STAKEHOLDERS UNDER FOUR DIMENSIONS Average four dimensions 1. Transparency, Disclosure of information and Accountability of the Road Agency (Q2-Q19) 1.00 0.75 0.50 2. Transparent 4. Administrative 0.25 and Accessible Internal Stakeholders Procedures and 0.00 Procurement Anticorruption External Stakeholders Processes (Q20- Effort (Q50-Q70) Q34) 3. Sound Financial Management System (Q35-Q49) 40 Table 7 summarizes the main issues rated high-high, high-low and low-low by internal and external stakeholders respectively. TABLE 7: ARMENIA – SUMMARY OF MAIN ISSUES Issues highly rated by Invitation to all bidders to attend bid opening sessions both internal and Conducting public hearings for large investment projects external stakeholders Effective payroll control system maintained by the Road Agency Whether the Road Agency uses objective investment decision Issues highly rated by tools internal and lowly by Whether performance indicators are developed for the road external stakeholders sector Whether road condition data are periodically collected Issues lowly rated by Inadequate current public sector policies both internal and Difficulty in hiring or retaining qualified staff by the Road Agency external stakeholders Inadequate road sector budget 5.2 KAZAKHSTAN The overall Road Agency Governance Index for Kazakhstan, as indicated in Table 8, is 74%, with a range between 40% and 96% and a standard deviation of 0.162, or coefficient of variation of 22.0%. The RAGI obtained from internal stakeholders, as expected, is higher than the one from external stakeholders, respectively at 86% and 64%. Both the range and the standard deviation for the external stakeholders are higher than for the internal stakeholders. 41 TABLE 8: SUMMARY RESULTS OF RAGI CALCULATION FOR KAZAKHSTAN Average and Range Road Agency Governance Index (RAGI) by internal and external stakeholders: Average Min Max STD Dev Internal 86% 73% 96% 7.0% Stakeholders External 64% 40% 82% 14.7% Stakeholders Overall Average and Range Road Agency Governance Index (RAGI): STD Coefficient of Average Min Max Dev variation 74% 40% 96% 16.2% 22.0% Table 9 provides a summary of the scores obtained under each of the four dimensions considered in the questionnaire for both internal and external stakeholders in Kazakhstan. Similar to the Armenia study, data can be divided into three broad categories: (i) issues highly rated by both internal and external stakeholders, indicating areas where stakeholders generally agree that the level of governance is high; (ii) issues highly rated by internal stakeholders but less so by external stakeholders, indicating areas where road agencies need to do more to gain public trust; and (iii) issues poorly rated by both internal and external stakeholders, indicating areas where stakeholders generally agree that more needs to be done to strengthen governance. Examples of issues highly rated by both internal and external stakeholders include: disclosure of the road programs and procurement plans to the public; availability of independent venues (internet, hotlines) for citizens and firms to raise concerns related to the road sector; public advertisement of all tenders by the Road Agency; provision by the Road Agency of sufficient information to all bidders; and technical audits for large works contracts by the Road Agency. As one can see, issues highly rated by both internal and external stakeholders are mostly related to communication and disclosure, and the provision of information to bidders. This indicates that the Kazakhstan Road Agency communicates well with the public in certain events or activities they conduct. 42 TABLE 9: KAZAKHSTAN - ASSESSMENT OF GOVERNANCE ACCORDING TO FOUR DIFFERENT DIMENSIONS Scores Range Difference Dimension Internal External between Internal Average Stakeholders Stakeholders & External Min Max Stakeholders (%) 1. Strengthen Transparency, Disclosure of information 0.89 0.62 0.75 0.23 1.00 43.67% and Accountability of the Road Agency (Q2-Q19) 2. Promote Transparent and Accessible Procurement 0.90 0.80 0.85 0.55 1.00 13.39% Processes (Q20-Q34) 3. Establish Sound Financial Management System (Q35- 0.95 0.66 0.81 0.45 1.00 42.87% Q49) 4. Strengthen Administrative Procedures and 0.70 0.47 0.58 0.16 1.00 48.05% Anticorruption Effort (Q50- Q70) Issues that are rated highly by internal stakeholders but poorly by external stakeholders include the following: whether public hearings are conducted for policy changes or large investment projects; whether performance indicators are established and published; whether the Road Agency has established an adequate internal control framework; whether external audits are conducted and followed up; and whether the Road Agency allocates adequate budget for capacity development of fiduciary staff. As one can see, issues where opinion diverges between internal and external stakeholders are mostly related to performance indicators and audit reports. Some issues are poorly rated by both internal and external stakeholders. They agree across the board that more needs to be done in those areas. They include: insufficient publication of annual reports; unreasonable number of tenders rejected; and inadequate balance of expenditures on road maintenance and construction. In Kazakhstan, the perceptions of road contractors showed an average RAGI of 75% with a range between 69% and 81%. Therefore the perceptions by road contractors are close to the average perceptions of all stakeholders. For Kazakhstan, the highest difference between internal and external stakeholders’ views can be found for the fourth dimension, thus signaling that the highest diversion between the public and private perceptions of road sector governance refers to administrative procedures and anticorruption effort. As the public perception in this dimension is substantially more favorable than the private perception, one might identify a “blind spot” in this area. This may be the result of actual deficiencies in the road sector, or the result of insufficient communication to the private sector regarding these issues. Regarding procurement procedures 43 (dimension 2), the internal and external stakeholders seem to share most of their views, as there is relatively little difference between their scores. Table 10 shows the standard deviation for internal and external stakeholders in Kazakhstan. Similar to the Armenia case, the standard deviation is higher for external stakeholders in all the four dimensions. TABLE 10: KAZAKHSTAN - STANDARD DEVIATION FOR INTERNAL AND EXTERNAL STAKEHOLDERS Standard Deviation Dimension Internal External Stakeholders Stakeholders 1. Strengthen Transparency, Disclosure of information and 0.12 0.25 Accountability of the Road Agency (Q2-Q19) 2. Promote Transparent and Accessible Procurement 0.07 0.14 Processes (Q20-Q34) 3. Establish Sound Financial Management System (Q35- 0.07 0.16 Q49) 4. Strengthen Administrative Procedures and 0.12 0.17 Anticorruption Effort (Q50-Q70) Figure 3 presents a graphical representation of the scores from internal and external stakeholders for Kazakhstan; and Annex 6 provides the details by dimensions. 44 FIGURE 3: KAZAKHSTAN - COMPARISON OF INTERNAL AND EXTERNAL STAKEHOLDERS UNDER FOUR DIMENSIONS Average four dimensions 1. Transparency, Disclosure of information and Accountability of the Road Agency (Q2-Q19) 1.00 0.75 0.50 2. Transparent 4. Administrative 0.25 and Accessible Internal Stakeholders Procedures and 0.00 Procurement Anticorruption External Stakeholders Processes (Q20- Effort (Q50-Q70) Q34) 3. Sound Financial Management System (Q35- Q49) The results show several aspects having highly positive perceptions, or areas of strength, that include procurement, dissemination of information through the internet, and financial management. Conversely, there are several areas where perceptions are low, or areas deserving more attention, which include the anticorruption effort, dissemination of financial audits, construction quality, high cost of roads works, and road condition. The highest differences in perceptions by the public and private sectors can be found for public consultation and annual report, where the private perception is substantially lower than that of the public sector. In particular, results show that the cluster of lowest scores, by both internal and external stakeholders, refers to questions Q66 to Q72 - dimension 4, (see Annex 6). The main issues in this area regard public sector policies, road financing, staff skills, corruption, and road condition. Also remarkably low is the perception of an imbalance between expenditures on maintenance and new construction (apparently the latter being too high), corruption, and high cost of civil works. Table 11 summarizes the main issues rated high-high, high-low and low-low by internal and external stakeholders respectively. 45 TABLE 11: KAZAKHSTAN – SUMMARY OF MAIN ISSUES Public advertisement of all tender by the Road Agency Issues highly rated Carrying out technical audits for large works contract by by both internal and the Road Agency external stakeholders Disclosure of the road programs and procurement plans to the public Whether performance indicators are established and Issues highly rated published by internal and lowly Whether external audits are conducted and followed up by external stakeholders Whether the Road Agency has established an adequate internal control framework Issues lowly rated by Insufficient publication of annual reports both internal and Unreasonable number of tenders rejected external Inadequate balance of expenditures on road maintenance stakeholders and construction 5.3 COMPARISON OF RESULTS FOR ARMENIA ANDKAZAKHSTAN As summarized in Table 12, the results for Armenia and Kazakhstan are relatively similar, with an overall RAGI of 76% for Armenia and a slightly lower 74% for Kazakhstan. As indicated in Table 13, we cannot reject the null hypothesis (p- value>0.01), that is, there is no significant difference between the overall RAGI obtained for Armenia and Kazakhstan. This statistical test shows that the groups of stakeholders surveyed in Armenia are comparable to those surveyed in Kazakhstan. TABLE 12: COMPARISON OF RAGI RESULTS FOR ARMENIA AND KAZAKHSTAN Country Overall RAGI Internal External RAGI Range Stakeholders Stakeholders RAGI RAGI Average STD Armenia 76% 15.5% 82% 72% 36%-92% Kazakhstan 74% 16.2% 86% 64% 40%-96% 46 TABLE 13: TESTING FOR DIFFERENCES BETWEEN RAGI FOR ARMENIA AND KAZAKHSTAN t-Test: Two-Sample Assuming Unequal Variances (alpha 0.01) Armenia Kazakhstan RAGI RAGI Mean 0.764789095 0.737069444 Variance 0.024142698 0.026203687 Observations 18 20 Hypothesized Mean Difference 0 df 36 t Stat 0.5383273 P(T<=t) one-tail 0.296832113 t Critical one-tail 2.43449404 P(T<=t) two-tail 0.593664227 t Critical two-tail 2.719484627 As an interesting coincidence, the Transparency International CPI is also the same for Armenia and Kazakhstan, which are both ranked 120, with a CPI of 2.7 each. 38 However, under the Worldwide Governance Indicators sixth dimension (i.e., Control of Corruption), 39 the percentile ranks for Armenia 40 and Kazakhstan 41 are different, at 35 and 16, respectively. This indicates that 35 percent of the countries surveyed rate below Armenia, and that only 16 percent of them rate below Kazakhstan. In all four dimensions (see Tables 5 and 9), the differences between the scores from internal and external stakeholders are higher in Kazakhstan than in Armenia. This tends to suggest that in Armenia there has been better communication between internal (the public sector) and external (the private sector) stakeholders. Figures 2 and 3 present a graphical representation of the differences in scores from internal and external stakeholders, respectively for Armenia and Kazakhstan. Annex 6 presents such differences separately for the four dimensions previously defined. As observed in Figures 2 and 3, external stakeholders have generally a lower perception than internal stakeholders regarding sector governance in both Armenia and Kazakhstan. The differences tend to be higher in Kazakhstan than in Armenia. Some exceptions to this pattern are observed under dimension 1 for Armenia and for a few specific questions for Kazakhstan. For both countries, there is a cluster of 38The 2009 CPIs are available at: http://www.transparency.org/policy_research/surveys_indices/cpi/2009/cpi_2009_table 39Kaufmann, Daniel, Kraay, Aart and Mastruzzi, Massimo, Governance Matters VIII: Aggregate and Individual Governance Indicators, 1996-2008 (June 29, 2009). World Bank Policy Research Working Paper No. 4978. Available at: http://go.worldbank.org/4MWW40KWH0 or http://econ.worldbank.org/external/default/main?pagePK=64165259&theSitePK=469372&piPK=64165421&men uPK=64166093&entityID=000158349_20090629095443&cid=decresearch 40 Country report for Armenia: http://info.worldbank.org/governance/wgi/pdf/c10.pdf 41 Country report for Kazakhstan: http://info.worldbank.org/governance/wgi/pdf/c115.pdf 47 lowest scores obtained by both internal and external stakeholders over questions Q66 to Q72 (dimension 4). We assigned a score of 0.5 to the questions where the answers were “uncertain.” It can be argued that some of these answers in fact refer to situations where the interviewee does not know the answer. Consequently, we decided to redo the calculations by disregarding the “uncertain” answers. In other words, we considered these answers as missing data. The results (see Annexes 7 and 8) showed a slightly higher (less than 10 percent) overall RAGI for Armenia and Kazakhstan. However, as mentioned above in the methodology section, differences obtained are minor, and no change in the relative indicators was found for either country. For example, using the “0.5” score, the RAGI for Armenia was 3 percentage point higher than the one for Kazakhstan; using the “no data” score, the RAGI for Armenia was 2 percentage point higher than that of Kazakhstan. The majority of questions were constructed so that the answers indicate the level of governance independent of the availability of budget. Answers to six of the questions in the questionnaire, however, are determined as much by the absolute amount of budget made available to road administrations as by the efficiency at which the budget is put to use. As such, answers to these questions indicate the level of governance only partially. While the study assessed the current government performance mostly independent from budget availability, addressing the existing governance gaps may require additional budgets. Therefore, when improvements in governance performance over time are assessed, additional budgets may well need to be available. It can be argued that the bundling of information on resources provision, political administrative processes and outcomes may lead to ambiguities in the interpretation of RAGI. Bad funding and poor road quality may occur, for example, in poor countries with good governance. A low RAGI value would wrongly rate such countries. Therefore, an alternative RAGI calculation was carried out by dropping several questions which are partially influenced by the availability of budget (i.e., Q48, Q51, Q52, Q66, Q67, and Q68). The results, shown in Table 14, indicate that this would cause very minor changes in RAGI. Consequently it was decided to keep the answers to these questions in the definition of RAGI. TABLE 14: ALTERNATIVE RAGI CALCULATION DROPPING BUDGET RELATED QUESTIONS Country Original RAGI Alternative RAGI calculation Armenia 76% 77% Kazakhstan 74% 74% 48 5.4 FEEDBACK FROM SURVEY RESPONDENTS An unedited list of the main comments provided by the stakeholders interviewed in Armenia and Kazakhstan is provided in the Annexes 4 and 5, respectively. Comments from Armenia: The main issues raised by stakeholders in Armenia are indicated in Table 15. For example, insufficient financing for the road sector was raised by 62.5 percent of the internal stakeholders and 40 percent of the external stakeholders, while 50 percent of both internal and external stakeholders raised the issue of poor road condition or deficient maintenance. Only 10 percent of the external stakeholders, and none of the internal stakeholders, considered lack of transparency or corruption as a main issue. Comments from Kazakhstan. The main issues raised by stakeholders in Kazakhstan are indicated in Table 16. For example, insufficient financing for the road sector was raised by 44.4 percent of the internal stakeholders and 27.3 percent of the external stakeholders, while 22.2 percent of the internal stakeholders and 45.5 percent of the external stakeholders raised the issue of lack of transparency or corruption. In view of the country’s difficult climatic and territorial conditions, 11.1 percent of the internal stakeholders and 9.1 percent of the external stakeholders considered this as a main issue. TABLE 15: MAIN ISSUES RAISED BY STAKEHOLDERS IN ARMENIA Stakeholders ARMENIA (percentage of interviewees) Internal External Overall Insufficient financing 62.5 40.0 50.0 Poor road condition or deficient maintenance 50.0 50.0 50.0 Poor road management 50.0 20.0 33.3 Deficient or outdated equipment, standards or specifications 25.0 30.0 27.8 Poor quality of works 12.5 20.0 16.7 Poor road safety 0.0 20.0 11.1 Lack of motorways 0.0 20.0 11.1 Insufficient number of skilled staff 12.5 0.0 5.6 Lack of transparency or corruption 0.0 10.0 5.6 49 TABLE 16: MAIN ISSUES RAISED BY STAKEHOLDERS IN KAZAKHSTAN Stakeholders KAZAKHSTAN (percentage of interviewees) Internal External Overall Insufficient financing 44.4 27.3 35.0 Lack of transparency or corruption 22.2 45.5 35.0 Poor road condition or deficient maintenance 22.2 36.4 30.0 Poor road management 22.2 27.3 25.0 Deficient or outdated equipment, standards or specifications 22.2 18.2 20.0 Insufficient number of skilled staff 22.2 18.2 20.0 Poor quality of works 11.1 9.1 10.0 Poor road safety 11.1 9.1 10.0 Difficult climate and territory 11.1 9.1 10.0 Comparison of Perceived Issues in Armenia and Kazakhstan. The issues most frequently raised by stakeholders in Armenia were insufficient financing, and poor road condition or deficient maintenance. In Kazakhstan, the most frequently issues raised by stakeholders were insufficient financing, and corruption or the lack of transparency. Corruption was not raised as a main issue by any of the internal stakeholders in Armenia. 50 51 52 6 RECOMMENDATIONS AND CONCLUSIONS This section proposes several recommendations to improve the governance of the road sector in countries analyzed in the study. 6.1 RECOMMENDATIONS FOR ARMENIA In view of the survey results, the Armenian Roads Directorate and the Ministry of Transport and Communication may consider several measures that are likely to enhance the level of governance for the country’s road sector (including a higher RAGI): i. Review the access to information policy; ii. Publish and better disseminate annual and other reports, which include information such as road condition, road works, and application of resources in the road sector; iii. Enhance training opportunities for sector staff; iv. Seek additional funding for roads, for example through public-private- partnerships (PPP); v. Increase the efficiency of utilization of public resources, for example through new technology and standards, and improved use of performance based contracts (PBC); vi. Improve supervision and other measures (e.g., technical audits) to increase the quality of road works; vii. Revisit investment decision tools and procedures, including stakeholder participation; and viii. Establish and publish performance indicators for the road sector. 6.2 RECOMMENDATIONS FOR KAZAKHSTAN In view of the survey results, the Committee for Roads and Ministry of Transport and Communication of Kazakhstan may consider several measures that are likely to 53 enhance the level of governance for the country’s road sector (including a higher RAGI): i. Improve the anticorruption effort in the road sector; ii. Seek additional funding for roads, for example through road user fees and public-private-partnerships (PPP); iii. Publish and better disseminate annual and other reports, which include information such as road condition, road works, and application of resources in the road sector; iv. Disseminate the results of financial audits; v. Increase competition to reduce the cost of road works; vi. Improve public consultation, for example through more frequent public hearings; vii. Increase the efficiency of utilization of public resources, for example through new technology and standards, and use of performance based contracts (PBC); viii. Improve supervision and other measures to increase the quality of road works; ix. Revisit investment decision tools and procedures, including stakeholder participation; and x. Establish and publish performance indicators for the road sector. 6.3 SUMMARY AND CONCLUSIONS The study reported in this paper consisted of the following main phases: (a) Internal discussions in the Bank of a draft survey instrument (questionnaire) proposed to assess the degree of governance in the road sector; (b) Incorporation of comments and preparation of a revised questionnaire to be used in the pilot countries; (c) Application of the revised questionnaire to internal and external stakeholders in Armenia and Kazakhstan; (d) Analysis of the data obtained and discussions of governance challenges in the roads subsector in Kazakhstan and Armenia; (e) Assessment of a Road Agency Governance Index (RAGI) for Armenia and Kazakhstan and development of a set of recommendations to improve road sector governance for each of the pilot countries; and (f) Review of methodological strengths of the approach used under the study and issues that remain. 54 In terms of the Road Agency Governance Index (RAGI), the study showed that the results for Armenia and Kazakhstan are relatively similar, with an overall RAGI of 76 percent for Armenia and a slightly lower 74 percent for Kazakhstan. A statistical test indicated that there is no significant difference between the overall RAGI obtained for Armenia and Kazakhstan. For both Armenia and Kazakhstan, the standard deviation is, on average, higher for external stakeholders, suggesting lesser agreement or perhaps lesser precise knowledge among the external stakeholders compared to the internal ones. In all four dimensions considered in this study, the average differences between the scores from internal (public sector) and external (private sector) stakeholders are higher in Kazakhstan than in Armenia. This may suggest that in Armenia there has been better communication between the public and the private sectors. Nevertheless, the public sector in both countries should enhance their outreach to road users and other stakeholders. This will help improve the credibility of the road sector in the view of the general public. The study findings clearly indicate that the approach used under the study is both useful in identifying specific issues facing the road sector of respective countries and establishing a benchmark comparable across countries and over time. In other words, by: (i) establishing four dimensions relevant to road sector governance; (ii) comparing differing perspectives between “internal” and “external” stakeholders both at the aggregate level as well as within each dimension; and (iii) aggregating data as the Road Agency Governance Index (RAGI) which allows comparison across countries and over time for the same country, the methodology used under the study helps overcome some of the methodological issues facing the approach to governance issues in transport reviewed initially in the report. In particular, comparison of perception on specific GAC issues between internal and external stakeholders helps highlight where stakeholders across the board agree as needy of intervention, areas where ownership already exists, and where significant gaps exist between internal and external stakeholders, areas of “blind spots” where road government agencies could do more to improve the perceived level of governance and gain public trust. It will help policy makers prioritize and formulate actions for which demands already exist, or amend and disseminate existing initiatives which will increase the credibility of road administration. Armenia and Kazakhstan are considered “partially free” in terms of the freedom of speech, as shown by the Freedom House indicators. The results of this study are thus considered to be comparable for both countries. For future work and development of the study in different countries, it is recommended to group countries into those with high, partial, or low freedom of expression, and assess the governance in the road sector within the same group in order to ensure comparability. Not all stakeholders selected have the same range and depth of knowledge on every issue of road sector governance. Indeed, the pilot study found that, although all 55 stakeholders interviewed are involved in the road sector, they tend to be knowledgeable only about certain issues of their interest, except for those directly involved in road administration. This is an inevitable methodological drawback of relying on perception and knowledge of diverse stakeholders, an issue that should be carefully assessed for future work using this methodology. Also, it may be useful to include as interviewees more representatives from International Financing Institutions to provide a broader perspective. The survey instrument used in the study has been proven to be a useful tool to assess governance in the road sector. Nevertheless, some questions may lead to multiple interpretations (e.g., Q55), while other questions are related with the availability of budget and they may create a small distortion in the RAGI if there are budget constraints (e.g., Q48, Q51). The availability of budget influences how much road administrations can invest to improve governance over time, and thereby partially determines the RAGI score in future. It can also taint the perception of respondents in assessing even current governance issues that are independent of budget allocation. For future work, these questions should be revised. This might consist in dropping some questions involving internal knowledge and adding others involving more general perception of governance. The study used a very simple methodology of simply adding and averaging numerous responses obtained from binary questions for ease of implementation and duplication and as a way to minimize potential bias stakeholders may have. Robustness checks were conducted which found that results were reliable. For future work and further research, consideration should be given to the pros and cons of keeping the “yes/no/uncertain” form of questions, or adopting a scale of four or five options, which would require more time for the interviews but could provide more realistic assessment of stakeholders’ perceptions. In particular, it seems appropriate to add “don’t know” as a fourth option to the answers. In view of the experience gained with the governance survey in the two pilot countries, it can be concluded that the Road Agency Governance Index (RAGI) appears to be a reasonable indicator to be used to develop regional benchmarks. It is recommended that the procedure applied to the pilot countries be extended to other countries in the region and elsewhere. Furthermore, RAGI seems to be a useful tool for engaging civil society into governance dialogue. 56 ANNEX 1: SURVEY INSTRUMENT TO ASSESS ROAD SECTOR GOVERNANCE SURVEY INSTRUMENT 42 ROAD ASSET GOVERNANCE FILTER: KAZAKHSTAN AND ARMENIA GAC SQUAD OPERATIONAL ASSIGNMENT: 2009-5 Name of Interviewer: ............................................................................................................................... Date: ................................................................ Time: ............................................................. INTERVIEWER INSTRUCTIONS AND INTRODUCTION Do not discuss the purpose of the survey in advance. If the respondent asks any questions about the purpose of the survey, reply with the words: “I would prefer not to answer your questions now, because they might influence your answers. But I will to the best of my ability be happy to answer any questions that you have at the end.” The questions to be asked are in bold letters. Start by reading the words in bold to the interviewee: “The World Bank is carrying out some research on the views that different stakeholders have about the governance and anticorruption effort in the road sector in transitional and developing economies. The contents of the surveys will be regarded as confidential and used only for the purposes of compiling data for preparation of the final report.” Name of Interviewee: ............................................................................................................. Position/Title: ......................................................................................................................... Time in office/position: ............................................................................................ (in years) Organization: ......................................................................................................................... Interview Location: ................................................................................................................. 42 A Russian translation of this questionnaire is available on files. 57 SECTION A – General Views Q.1 Please identify the two or three main problems facing the road sector in your country today: a.) ....................................................................................................................................................................... b.) ....................................................................................................................................................................... c.) ....................................................................................................................................................................... SECTION B – Governance and Anticorruption Effort in the Road B1. Strengthen Transparency, Disclosure and Accountability of the Road Agency Rationale: Transparency in policy formulation and implementation (including planning and execution of investment projects) is a key factor in strengthening governance and preventing fraud and corruption. Laws and regulations should define a line of actions needed to make transport policy planners and implementers accountable to the public. Q.2 Are the annual budget and expenditures disclosed to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.3 Are actions taken by the road agency to address concerns raised by the public made public? (Please circle the answer) Yes No Uncertain ........................................................................................................................................ Q.4 Is the audit report of the Road Agency made available to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.5 Are the road work programs and procurement plans disclosed to the public? (Please circle the answer) 58 Yes No Uncertain ......................................................................................................................................... Q.6 Are the road works contracts and names of winning bidders disclosed to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.7 Are the policy objectives of the Road Agency disclosed to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.8 Are the Annual Reports of the Road Agency published? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.9 Does the Road Agency conduct regular press releases? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.10 Does the Road Agency have a homepage or information center to disclose road sector information (e.g., updated road condition)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.11 Are public hearings conducted for policy changes or large investment projects? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.12 Are the results of the public hearings (see Q.11) taken into account in the decision making process? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.13 Are results of road user satisfaction surveys disclosed to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... 59 Q.14 Are independent venues (internet, hotline) available for citizens and firms to raise concerns related to the road sector (e.g., large road contracts, quality of works, road condition)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.15 Are progress and costs of road works (particularly large contracts) reported to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.16 Have performance indicators been developed for the road sector (e.g., defining road condition and safety)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.17 Is the progress in performance indicators published in annual reports that are available to the public? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.18 In the case of poor performance, does the annual (or other) report discuss the corrective measures taken (e.g., to address poor road condition)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.19 Can the legal and contractual relationships between the Road Agency and bidders be considered fair? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... B2. Promote Transparent and Accessible Procurement Processes Rationale: Transparent and accessible procurement processes are key to prevent fraud and corruption. Q.20 Does the Road Agency publicly advertise all tenders? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... 60 Q.21 Are there any members of the tender commission(s) with potential conflict of interest? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.22 Are all bidders invited to bid openings? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.23 For the selection of consultants, are all shortlisted firms qualified for the proposed job? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.24 Are third parties allowed to attend bid opening sessions? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.25 Can the number of days elapsed from tender advertisement to contract award be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.26 Can the number of complaints for the procurement of goods, works and services be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.27 Can the number of tenders rejected in each bidding be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.28 Does the Road Agency keep records (including completion reports and communications with bidders) for all contracts? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.29 Does the Road Agency timely prepare procurement plans that adequately consider technical, financial, managerial and implementation constraints? (Please circle the answer) Yes 61 No Uncertain ......................................................................................................................................... Q.30 Does the Road Agency adopt clearly defined criteria to evaluate bids? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.31 Can the difference between contract values and engineer’s estimates be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.32 Are international firms generally allowed to participate in biddings for road works? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.33 Does the Road Agency provide bidders with sufficient information, clarification and time to prepare bids? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.34 Does the Road Agency appoint members of tender commissions who are competent individuals? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... B3. Establish Sound Financial Management System Rationale: Strong control/ monitoring mechanisms should be established to ensure value for money in transport investments and operation. Q.35 Are external audits of the Road Agency conducted on an annual basis? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.36 Has the Road Agency established an adequate internal control framework, including financial audit? (Please circle the answer) Yes No 62 Uncertain ......................................................................................................................................... Q.37 Can the scope, nature and follow-up of external audit reports be considered adequate? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.38 Does the Road Agency implement all recommendations from internal controls within one year? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.39 Does the Road Agency implement all recommendations from external audit reports within one year? (please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.40 Does the Road Agency monitor/clear expenditure payment arrears on a regular basis, so there are no arrears over 30 days? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.41 Does the Road Agency carry out effective cash flow planning, management and monitoring? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.42 Is the Road Agency’s external audit report reviewed by the parliament (or other appropriate body)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.43 Does the Road Agency have an effective payroll control system? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.44 Does the Road Agency have a system to properly monitor the use of resources allocated to each of its departments/divisions? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... 63 Q.45 Does the Road Agency prepare budget execution reports in a timely manner? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.46 Does the Road Agency keep aggregate budget overruns (if any) within a reasonable range? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.47 Does the Parent Ministry (or MoF) have functional reporting systems to follow up on the flow of budgetary resources transferred to the Road Agency? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.48 Does the Road Agency allocate adequate administrative budget for capacity development of fiduciary staff? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.49 Does the Road Agency give fiduciary staff sufficient authority on day-to-day execution of fiduciary functions and procedures? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... B4. Strengthen Administrative Procedures Rationale: Effective and transparent processes and procedures for business administration will improve governance. Q.50 Is adequate qualification a main factor in the selection of members of the management team (including board members) of the Road Agency? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.51 Does the Road Agency periodically collect data on the level of service/condition of its road network? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... 64 Q.52 Does the Road Agency keep its key positions properly filled with qualified individuals (or, in the case of vacancies, they are filled in less than a year)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.53 Can the unit prices of works (e.g., the cost of a cubic meter of asphalt concrete) carried out by the Road Agency be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.54 Does the Road Agency conduct technical audits for large works contracts (or a sample of them)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.55 Can the proportion of expenditures on road maintenance and expenditures on road construction be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.56 Does the Road Agency have well established road users’ feedback mechanisms, including satisfaction surveys? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.57 Do the Road Agency’s management and staff regularly discuss skills development needs and service improvement? (Please circle the answer) Yes No Uncertain ................................................................................................................................................ Q.58 Does the Road Agency adopt objective investment decision tools (such as HDM)? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.59 Can the Road Agency’s share of administration cost, as a percentage of the total budget, be considered reasonable? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... 65 Q.60 Does the Road Agency periodically update unit prices used for cost estimates based on the latest contracts or other information available? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.61 Has the Road Agency established a code of ethics? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.62 Is the Road Agency’s overall Human Resources policy linked with and reviewed against policy objectives? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.63 Is the Road Agency’s staff performance evaluated regularly against outcomes and professional behaviors? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.64 Is the Road Agency’s management performance periodically evaluated against policy objectives? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.65 Does the Road Agency maintain a Management Information System well established and functional? (Please circle the answer) Yes No Uncertain ......................................................................................................................................... Q.66 On a rating scale from 1 to 5, with 1 being not at all critical and 5 very critical, how significant a problem to the operation of the Road Agency is the current public sector policies (e.g., public servant salaries) in the country? (Please circle the number for the answer) Not at all Very Critical Critical 1 2 3 4 5 Q.67 What percentage of the requested budget does the Road Agency actually receive on average on an annual basis? (Please circle the number for the answer) 66 100 percent 1 Over 75 percent 2 Between 50 and 75 percent 3 Between 25 and 50 percent 4 Less than 25 percent 5 Q.68 On a rating scale from 1 to 5, with 1 being not at all critical and 5 very critical, how significant a problem to the operation of the Road Agency is attracting and retaining qualified employees? (Please circle the number for the answer) Not at all Very Critical Critical 1 2 3 4 5 Q.69 On a rating scale from 1 to 5, with 1 being not at all critical and 5 very critical, how would you rate the following issues in your country: (circle the number for the answer) Not at all Very Issues Critical Critical a. Corruption in general 1 2 3 4 5 b. Corruption in the road sector 1 2 3 4 5 c. High cost of civil works in 1 2 3 4 5 general d. High cost of road works 1 2 3 4 5 e. Poor construction quality in 1 2 3 4 5 general f. Poor road construction quality 1 2 3 4 5 Q.70 In your opinion, please describe how competition for road works can be increased. Do you have any suggestions on how the Road Agency could improve its performance? ………………………………………………………………………………………………………………………………………… …………………………………………………… Q.71 Regarding the paved road network managed by the Road Agency, what is the percentage of roads in good condition? ………………% in good condition. Q.72 Regarding the unpaved road network managed by the Road Agency, what is the percentage of roads in good condition? ………………% in good condition. 67 68 ANNEX 2: DETAILED CALCULATIONS OF THE ROAD AGENCY GOVERNANCE INDEX (RAGI) SCORES FOR ARMENIA GAC Squad Operational Assignment: 2009-5 The Road Organization as viewed by internal and external stakeholders Internal Stakeholders Public Sector including Academia (A, B, D) - External Stakeholders Private Sector including NGOs, IFIs (C, RC, E, F) Internal Stakeholders External Stakeholders Difference Average Ref. No. 5 6 7 1 10 3 11 15 Average 4 8 9 2 12 13 14 16 17 18 Average between Int. score Dimensions Group A A A B D D B B score C RC C E F E RC C F E score and Ext. views Q2 1 1 1 1 0 1 1 0 0.75 1 1 1 1 1 1 1 1 1 1 1.00 -0.25 0.89 Q3 1 1 0 1 0 0.5 1 1 0.69 1 1 0 0 0.5 0.5 1 1 0.5 1 0.65 0.04 0.67 Q4 1 0 0 0.5 0 0.5 0.5 0 0.31 0 1 0.5 0 1 1 1 0 0.5 0.5 0.55 -0.24 0.44 Q5 1 1 1 1 1 1 1 1 1.00 1 1 1 0 1 1 1 1 1 1 0.90 0.10 0.94 Q6 1 1 1 1 1 1 1 1 1.00 1 1 1 0 0.5 1 1 1 1 1 0.85 0.15 0.92 Q7 1 1 0 1 1 1 1 1 0.88 0 1 1 0 1 1 1 1 1 1 0.80 0.08 0.83 Q8 0 0 0 0.5 0 0.5 1 0 0.25 1 1 0.5 0 1 1 1 0 1 0.5 0.70 -0.45 0.50 Q9 1 1 1 0.5 1 1 1 1 0.94 1 1 1 0 0.5 0 1 1 1 0 0.65 0.29 0.78 Q10 0 0 1 1 1 1 0.5 1 0.69 1 1 1 0 1 1 1 1 1 0.5 0.85 -0.16 0.78 1 Q11 1 1 1 1 1 1 1 0 0.88 1 1 1 0 0.5 1 1 1 1 1 0.85 0.03 0.86 Q12 1 1 1 0.5 0.5 1 1 0 0.75 0 1 1 0 0.5 1 1 1 0.5 1 0.70 0.05 0.72 Q13 1 0 1 0.5 0.5 0.5 0.5 0.5 0.56 0 1 0.5 0 0.5 0.5 1 1 0 0.5 0.50 0.06 0.53 Q14 1 1 1 1 1 1 1 0 0.88 0 1 1 0 0 1 1 1 1 0.5 0.65 0.23 0.75 Q15 1 1 0 1 1 1 1 1 0.88 1 1 0 1 0.5 1 1 1 1 0 0.75 0.13 0.81 Q16 1 1 1 1 1 1 1 1 1.00 1 1 0.5 0 0.5 1 1 1 0 1 0.70 0.30 0.83 Q17 1 0 1 1 0 0.5 1 0 0.56 1 1 0.5 0 0.5 0.5 1 1 0 0 0.55 0.01 0.56 Q18 1 1 1 1 0 1 1 0 0.75 1 1 0.5 0.5 0.5 1 1 1 0 1 0.75 0.00 0.75 Q19 1 1 1 1 0.5 1 1 0.5 0.88 0.5 1 0 0.5 0.5 1 1 1 1 0 0.65 0.23 0.75 Average (1) 0.89 0.72 0.72 0.86 0.58 0.86 0.92 0.50 0.69 1.00 0.67 0.17 0.64 0.86 1.00 0.89 0.69 0.64 Q20 1 1 1 1 1 1 1 1 1.00 1 1 0 0 1 1 1 1 1 1 0.80 0.20 0.89 Q21 1 1 0 1 0.5 1 1 1 0.81 0 1 0.5 0 1 1 1 1 1 1 0.75 0.06 0.78 Q22 1 1 1 1 0.5 1 1 1 0.94 1 1 0.5 1 0.5 1 1 1 1 1 0.90 0.04 0.92 Q23 1 1 1 1 0.5 1 0.5 1 0.88 0 1 0.5 0.5 1 1 1 1 1 1 0.80 0.08 0.83 Q24 1 1 1 1 1 1 0.5 0 0.81 1 1 0.5 0.5 1 1 1 1 1 1 0.90 -0.09 0.86 Q25 1 1 1 1 1 1 0.5 1 0.94 0.5 1 0.5 0.5 1 1 1 1 1 1 0.85 0.09 0.89 Q26 1 1 1 0.5 0.5 1 1 1 0.88 0 1 0.5 0.5 1 1 0.5 1 1 1 0.75 0.13 0.81 2 Q27 Q28 1 1 1 1 1 1 0.5 1 0.5 0.5 1 1 0.5 1 0.5 1 0.75 0.94 0 1 1 1 0 0.5 0.5 0.5 1 1 1 1 1 1 1 1 1 1 1 1 0.75 0.90 0.00 0.04 0.75 0.92 Q29 1 1 1 1 1 0.5 1 1 0.94 0.5 0.5 0.5 0.5 1 1 1 1 1 0.5 0.75 0.19 0.83 Q30 1 1 1 1 0 1 1 1 0.88 0 1 0 0.5 1 1 1 1 1 1 0.75 0.13 0.81 Q31 1 1 1 1 0.5 1 0.5 1 0.88 1 1 0.5 0.5 0.5 0.5 1 1 1 1 0.80 0.08 0.83 Q32 1 1 1 1 1 1 1 1 1.00 1 1 0 0.5 1 1 1 1 1 1 0.85 0.15 0.92 Q33 1 1 1 1 0.5 1 1 1 0.94 1 1 0.5 1 1 1 1 1 1 1 0.95 -0.01 0.94 Q34 1 1 1 1 1 1 1 1 1.00 1 1 0.5 0 1 1 1 1 1 1 0.85 0.15 0.92 Average (2) 1.00 1.00 0.93 0.93 0.67 0.97 0.83 0.90 0.60 0.97 0.37 0.47 0.93 0.97 0.97 1.00 1.00 0.97 Q35 1 1 1 1 1 1 1 1 1.00 1 0.5 0.5 0.5 1 1 0.5 1 0.5 1 0.75 0.25 0.86 Q36 1 1 1 1 1 1 1 1 1.00 1 0.5 0.5 0.5 1 1 1 1 1 1 0.85 0.15 0.92 Q37 1 1 1 1 0.5 1 1 1 0.94 1 0.5 0.5 0.5 0.5 1 1 1 1 1 0.80 0.14 0.86 Q38 1 1 1 0.5 0.5 0.5 1 1 0.81 0.5 0.5 0.5 0.5 1 0.5 1 1 0.5 1 0.70 0.11 0.75 Q39 1 1 1 0.5 0.5 1 1 1 0.88 0.5 0.5 0.5 0.5 1 0.5 1 1 0.5 1 0.70 0.18 0.78 Q40 1 1 1 0.5 0.5 1 0.5 0.5 0.75 0.5 1 0 0.5 1 1 1 1 0.5 1 0.75 0.00 0.75 Q41 1 1 1 0.5 1 0.5 1 0.5 0.81 0 1 0.5 0 1 1 1 1 1 1 0.75 0.06 0.78 3 Q42 Q43 1 1 1 1 1 1 1 0.5 0.5 1 1 1 1 1 1 1 0.94 0.94 1 0.5 0.5 1 0.5 0.5 0.5 1 1 1 1 1 1 1 1 1 0.5 1 1 1 0.80 0.90 0.14 0.04 0.86 0.92 Q44 1 1 1 0.5 1 1 1 1 0.94 0.5 0.5 0.5 0.5 1 1 1 1 1 1 0.80 0.14 0.86 Q45 1 1 1 1 0.5 1 1 1 0.94 0.5 0.5 1 0.5 1 1 1 1 1 1 0.85 0.09 0.89 Q46 1 1 1 1 0.5 1 1 1 0.94 1 1 0.5 0.5 0.5 1 1 1 0.5 1 0.80 0.14 0.86 Q47 1 1 1 1 1 1 1 1 1.00 1 0.5 1 0.5 1 1 1 1 1 1 0.90 0.10 0.94 Q48 1 1 0 0.5 0 0.5 1 1 0.63 1 0.5 0.5 0.5 1 0.5 0 1 0.5 1 0.65 -0.03 0.64 Q49 1 1 1 0.5 1 1 1 0.5 0.88 0.5 0.5 0.5 0.5 1 0.5 1 1 0.5 1 0.70 0.18 0.78 Average (3) 1.00 1.00 0.93 0.73 0.70 0.90 0.97 0.90 0.70 0.63 0.53 0.50 0.93 0.87 0.90 1.00 0.73 1.00 Q50 1 1 1 1 0.5 1 1 1 0.94 1 1 0.5 0 1 0.5 1 1 0.5 1 0.75 0.19 0.83 Q51 1 1 1 1 1 1 1 1 1.00 1 1 0.5 0 0.5 1 1 1 0.5 1 0.75 0.25 0.86 Q52 1 1 1 0.5 0.5 1 1 0.5 0.81 0 1 0.5 0 1 1 1 1 1 0.5 0.70 0.11 0.75 Q53 1 1 1 1 0.5 1 1 1 0.94 0 1 0.5 1 0.5 1 1 1 1 1 0.80 0.14 0.86 Q54 1 1 1 1 1 1 1 1 1.00 0.5 1 1 0 1 0 1 1 0.5 1 0.70 0.30 0.83 Q55 1 0 0 1 1 1 1 1 0.75 1 1 0.5 1 1 1 0 1 0 0.5 0.70 0.05 0.72 Q56 0 0 1 0.5 0 1 1 1 0.56 0 0.5 0 0 0 1 0.5 1 0 0 0.30 0.26 0.42 Q57 1 1 1 0.5 0 1 1 1 0.81 0 0.5 0 0 1 0 0.5 1 0.5 1 0.45 0.36 0.61 Q58 1 1 1 1 1 0.5 1 1 0.94 0 0.5 0.5 0 0.5 1 0 1 0.5 1 0.50 0.44 0.69 Q59 1 1 0.5 1 0.5 0.5 1 0.5 0.75 1 0.5 0.5 0.5 0.5 1 1 1 0.5 0.5 0.70 0.05 0.72 Q60 1 1 1 1 1 0.5 1 1 0.94 0.5 1 0.5 0 1 1 1 1 0.5 1 0.75 0.19 0.83 Q61 0 0 1 0.5 1 1 1 0 0.56 0 1 0.5 0 1 0.5 0.5 1 0.5 0.5 0.55 0.01 0.56 Q62 1 1 1 1 0 1 1 1 0.88 0 1 0.5 0.5 1 1 0.5 1 0.5 1 0.70 0.18 0.78 4 Q63 Q64 0 1 0 1 1 1 0.5 0.5 1 1 1 1 1 1 1 1 0.69 0.94 0 0 0.5 0.5 0.5 0.5 0.5 0.5 1 1 1 1 0.5 0.5 1 1 1 0.5 0.5 0.5 0.65 0.60 0.04 0.34 0.67 0.75 Q65 1 1 1 0.5 1 1 1 1 0.94 0 0.5 0.5 0 0.5 1 1 1 0.5 1 0.60 0.34 0.75 Q66 0.75 0.75 0.25 0.25 0.5 0 0.5 0.5 0.44 0 0.5 0 0.5 0.75 0.5 0.25 0.5 0.25 0.75 0.40 0.04 0.42 Q67 0.5 0.5 0 0.75 0.5 0.75 0.5 0.75 0.53 0.5 0.5 0.5 0.75 0.75 0.25 0 0.25 0.5 0.5 0.45 0.08 0.49 Q68 0.5 0.75 0 0.25 0.25 0 0.5 0.5 0.34 0 1 0.5 0 0.25 0.75 0 0.25 0.5 0.75 0.40 -0.06 0.38 Q69 a 1 0.75 1 0.5 0.25 0.5 0.75 0.25 0.63 0 0.25 0 0 0.25 1 0.5 0.75 0.25 0.5 0.35 0.28 0.47 Q69 b 1 0.75 1 0.5 0.25 0.5 0.75 0.5 0.66 0 0.75 0 0 0.25 1 0.5 1 0.5 0.75 0.48 0.18 0.56 Q69 c 0.75 0.75 0.5 0.5 0.5 1 0.25 0.75 0.63 0.5 0.25 0.5 0.5 0.25 1 0.5 0.5 0.75 0.5 0.53 0.10 0.57 Q69 d 0.75 0.75 0.5 0.5 0.25 1 0.25 0.5 0.56 0.5 0.5 0.5 0.5 0.25 1 0.5 1 0.75 0.75 0.63 -0.06 0.60 Q69 e 0.75 0.75 1 0.75 0.5 0.75 0.75 0.75 0.75 0.25 0.5 0 0.5 0.25 0.75 0.5 0.5 0.5 0.5 0.43 0.33 0.57 Q69 f 0.75 0.75 1 0.5 0.25 0.75 0.5 0.75 0.66 0 0.75 0 0.25 0.25 0.75 0.5 0.5 0.5 0.5 0.40 0.26 0.51 Q71 0.45 0.8 0.55 0.75 0.75 0.95 0.55 0.65 0.68 0.65 0.2 0.25 0.65 0.7 0.45 0 0.5 0.5 0.55 0.45 0.24 0.55 Q72 0.5 0.5 0.1 0.25 0.5 0 0.5 0.05 0.30 0.5 0.1 0.5 0.5 0.5 0.1 0 0.05 0.2 0.45 0.29 0.01 0.29 Average (4) 0.77 0.73 0.76 0.67 0.57 0.77 0.81 0.74 0.29 0.66 0.38 0.30 0.63 0.76 0.53 0.81 0.51 0.69 Road Agency 91% 86% 84% 80% 63% 87% 88% 76% 57% 81% 49% 36% 78% 86% 85% 92% 73% 82% Governance Index 69 70 ANNEX 3: DETAILED CALCULATIONS OF THE ROAD AGENCY GOVERNANCE INDEX (RAGI) SCORES FOR KAZAKHSTAN GAC Squad Operational Assignment: 2009-5 The Road Organization as viewed by internal and external stakeholders Internal Stakeholders Public Sector including Academia (A, B, D) - External Stakeholders Private Sector including NGOs, IFIs (C, RC, E, F) Internal Stakeholders External Stakeholders Difference Average Ref. No. 1 2 3 7 8 9 11 15 16 Average 4 5 6 10 12 13 14 17 18 19 20 Average between Int. score Dimensions Group B A D B B B A A A score F F E RC C C F F RC C RC score and Ext. views Q2 1 1 1 1 1 1 0 1 1 0.89 1 0.5 0 1 0 1 0 1 1 1 1 0.68 0.21 0.78 Q3 1 1 1 0.5 1 1 1 1 1 0.94 0.5 1 0 1 0 1 1 0 1 1 1 0.68 0.26 0.80 Q4 1 1 1 1 0 0 0 0.5 1 0.61 0.5 0 0 1 0 1 0 0 1 1 0 0.41 0.20 0.50 Q5 1 1 1 1 0 1 1 1 1 0.89 1 1 1 1 1 1 1 0.5 1 1 1 0.95 -0.07 0.93 Q6 1 1 1 1 0 1 1 1 1 0.89 1 0.5 0 1 1 1 0 0.5 1 1 1 0.73 0.16 0.80 Q7 1 1 1 1 1 1 1 0.5 1 0.94 1 0 0 1 1 1 1 1 1 1 1 0.82 0.13 0.88 Q8 1 0.5 1 0 0 0 1 1 1 0.61 1 0.5 0 1 1 1 1 0 1 1 0 0.68 -0.07 0.65 Q9 1 1 1 0.5 0 0 1 1 1 0.72 1 0.5 0 1 0 1 1 1 1 1 1 0.77 -0.05 0.75 Q10 1 1 1 1 1 1 1 1 1 1.00 1 1 0 1 0 1 1 1 1 1 1 0.82 0.18 0.90 1 Q11 1 1 1 1 1 1 1 1 1 1.00 0 0 0 0 0 1 1 0 0.5 0.5 0 0.27 0.73 0.60 Q12 1 1 1 1 1 1 1 0.5 1 0.94 0 0 0 0 0 1 0.5 0 0.5 0.5 0 0.23 0.72 0.55 Q13 1 1 1 1 0.5 0 1 0 1 0.72 0 0 0 0 0 1 0 0 0.5 1 1 0.32 0.40 0.50 Q14 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 0 1 1 1 1 0.91 0.09 0.95 Q15 1 1 1 1 0.5 1 1 1 1 0.94 1 0 0 1 1 1 0 1 1 1 1 0.73 0.22 0.83 Q16 1 1 1 1 1 1 1 1 1 1.00 0.5 0 1 1 1 0.5 0 1 1 1 1 0.73 0.27 0.85 Q17 1 1 1 1 1 1 1 1 1 1.00 0.5 0 0 0.5 0 0.5 0 0.5 1 1 1 0.45 0.55 0.70 Q18 1 1 1 0.5 1 1 1 1 1 0.94 0.5 0 0 0.5 0 1 0 0.5 1 1 1 0.50 0.44 0.70 Q19 1 1 1 1 0.5 1 1 1 1 0.94 0 0.5 0.5 1 0 1 0 1 0 1 0 0.45 0.49 0.68 Average (1) 1.00 0.97 1.00 0.86 0.64 0.78 0.89 0.86 1.00 0.64 0.36 0.19 0.78 0.39 0.94 0.42 0.56 0.86 0.94 0.72 Q20 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q21 1 1 0.5 1 0.5 0.5 1 1 1 0.83 0.5 0.5 0.5 1 0 0 0.5 0 1 1 1 0.55 0.29 0.68 Q22 1 1 1 1 1 1 1 1 1 1.00 1 0.5 1 1 1 1 0.5 1 1 1 1 0.91 0.09 0.95 Q23 0.5 1 1 1 0.5 0.5 0.5 0.5 1 0.72 1 0.5 1 1 0 1 0.5 0.5 0.5 1 0.5 0.68 0.04 0.70 Q24 1 1 1 1 0.5 0.5 0.5 1 1 0.83 1 1 1 1 0.5 0.5 0 0.5 1 0.5 1 0.73 0.11 0.78 Q25 1 1 1 1 1 1 1 1 1 1.00 0 0.5 1 1 1 1 0.5 0.5 1 1 1 0.77 0.23 0.88 Q26 0.5 0.5 1 0.5 0.5 0.5 1 0.5 1 0.67 0.5 0.5 0.5 1 1 1 0.5 1 1 1 1 0.82 -0.15 0.75 2 Q27 Q28 0.5 1 1 1 1 1 0.5 1 0.5 1 0.5 1 1 1 0.5 1 1 1 0.72 1.00 0.5 0.5 0.5 1 0.5 1 1 1 1 1 0.5 1 0.5 0.5 0.5 1 0 1 1 1 1 1 0.64 0.91 0.09 0.09 0.68 0.95 Q29 1 1 1 1 1 1 1 1 1 1.00 0.5 1 1 1 1 1 0.5 1 1 0.5 1 0.86 0.14 0.93 Q30 1 1 1 1 1 1 1 1 1 1.00 0 0.5 0.5 1 1 1 0.5 0.5 1 1 1 0.73 0.27 0.85 Q31 1 1 1 1 0.5 0.5 0 1 1 0.78 0.5 1 1 1 1 1 1 0.5 0 0.5 0.5 0.73 0.05 0.75 Q32 1 1 1 1 1 1 1 1 1 1.00 1 1 0.5 1 1 1 0.5 1 1 1 1 0.91 0.09 0.95 Q33 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 0.5 1 1 1 1 0.95 0.05 0.98 Q34 1 1 1 1 1 1 1 1 1 1.00 0.5 0.5 1 1 1 1 0.5 0.5 1 1 0.5 0.77 0.23 0.88 Average (2) 0.90 0.97 0.97 0.93 0.80 0.80 0.87 0.90 1.00 0.63 0.73 0.83 1.00 0.83 0.87 0.53 0.70 0.83 0.90 0.90 Q35 1 0.5 1 1 1 1 1 1 1 0.94 0.5 0.5 0.5 1 0.5 0.5 0 1 0.5 1 0.5 0.59 0.35 0.75 Q36 1 0.5 1 1 1 1 1 1 1 0.94 1 0 0.5 1 0.5 1 1 1 0.5 1 0.5 0.73 0.22 0.83 Q37 1 0.5 1 1 1 0 1 1 1 0.83 0.5 0.5 0.5 1 0.5 0.5 0 0 0.5 1 0.5 0.50 0.33 0.65 Q38 1 0.5 1 1 1 1 1 1 1 0.94 0.5 0.5 0.5 1 0.5 1 0.5 0 0.5 0.5 0.5 0.55 0.40 0.73 Q39 1 0.5 1 1 1 1 1 1 1 0.94 0.5 0.5 0.5 1 0.5 0.5 0 0 0.5 0.5 0.5 0.45 0.49 0.68 Q40 0.5 1 1 1 1 0 1 1 1 0.83 0 0.5 0.5 1 0 1 0.5 0 1 0.5 1 0.55 0.29 0.68 Q41 1 1 1 1 1 1 1 1 1 1.00 0.5 0.5 0.5 1 1 1 0 1 0.5 1 0.5 0.68 0.32 0.83 3 Q42 Q43 1 1 1 0.5 1 1 1 0.5 1 1 1 1 1 1 1 1 1 1 1.00 0.89 0.5 0.5 0 1 1 0.5 1 1 1 0.5 1 0.5 0 1 1 1 1 0.5 1 1 0.5 0.5 0.73 0.73 0.27 0.16 0.85 0.80 Q44 1 1 1 1 1 1 1 1 1 1.00 0 0.5 0.5 1 0 1 1 1 0.5 1 0.5 0.64 0.36 0.80 Q45 1 1 1 1 1 1 1 1 1 1.00 0.5 0.5 1 1 1 1 1 1 0.5 1 1 0.86 0.14 0.93 Q46 1 1 1 1 1 1 1 1 1 1.00 1 1 0.5 1 1 0.5 0.5 1 1 1 1 0.86 0.14 0.93 Q47 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 0.5 1 0.5 1 1 1 0.5 0.86 0.14 0.93 Q48 1 1 1 1 1 1 0.5 1 1 0.94 0.5 0.5 1 1 1 0.5 0.5 0 0 0.5 0.5 0.55 0.40 0.73 Q49 1 1 1 1 1 1 0.5 1 1 0.94 0.5 0.5 1 1 1 1 0.5 0.5 0.5 0.5 0.5 0.68 0.26 0.80 Average (3) 0.97 0.80 1.00 0.97 1.00 0.87 0.93 1.00 1.00 0.53 0.53 0.67 1.00 0.63 0.80 0.47 0.63 0.60 0.83 0.60 Q50 1 1 1 1 1 1 1 1 1 1.00 0.5 0 1 1 0.5 1 0 0 1 1 0.5 0.59 0.41 0.78 Q51 1 1 1 1 1 0 1 1 1 0.89 0.5 1 1 1 1 1 1 0 1 0.5 1 0.82 0.07 0.85 Q52 1 1 1 1 1 0 1 1 1 0.89 0 0.5 0.5 1 1 1 0 1 1 1 0 0.64 0.25 0.75 Q53 1 1 1 1 1 0 1 1 1 0.89 0 1 1 1 1 1 0.5 0.5 1 0.5 1 0.77 0.12 0.83 Q54 1 1 1 0 1 1 0.5 1 1 0.83 1 1 1 1 1 1 0 1 1 1 1 0.91 -0.08 0.88 Q55 1 0 1 0 1 0 0 0 0 0.33 0 0 0 0 0 1 0 1 0 0 0 0.18 0.15 0.25 Q56 1 1 1 0 0.5 0 0.5 0.5 1 0.61 0 0 0 0 0 0.5 0 0 1 1 0 0.23 0.38 0.40 Q57 1 1 1 0.5 0.5 1 1 1 1 0.89 0.5 0.5 0 0 1 1 0 0 1 0.5 0.5 0.45 0.43 0.65 Q58 1 1 1 1 0.5 1 1 1 1 0.94 0.5 0.5 0.5 0 0 0.5 0 0.5 0.5 0.5 0.5 0.36 0.58 0.63 Q59 1 0 1 1 1 1 0 1 1 0.78 0.5 0.5 1 1 0 0.5 0 1 1 1 1 0.68 0.10 0.73 Q60 1 1 1 0 1 1 1 1 1 0.89 0.5 0.5 1 0 1 1 0.5 1 1 0.5 0 0.64 0.25 0.75 Q61 0.5 1 1 0 1 1 1 1 1 0.83 1 1 1 0 1 1 1 0.5 0 1 0.5 0.73 0.11 0.78 Q62 1 1 1 1 1 1 1 1 1 1.00 0.5 0.5 0.5 1 1 1 0 0 1 1 0.5 0.64 0.36 0.80 4 Q63 Q64 1 1 1 1 1 1 0.5 1 0.5 0.5 1 1 0.5 0.5 1 1 1 1 0.83 0.89 0.5 0.5 0.5 0 0.5 0.5 0 1 0.5 1 1 1 0.5 0.5 0 0 1 1 1 1 0.5 0 0.55 0.59 0.29 0.30 0.68 0.73 Q65 1 1 1 0 1 0 1 1 1 0.78 0.5 0 0.5 0 0 1 0 0 1 1 0 0.36 0.41 0.55 Q66 0 0.25 0.5 0.5 0.75 0.25 0.5 0.5 1 0.47 0.25 0.25 0.25 0.5 0.5 0.5 0 0.25 0.75 0.5 0.25 0.36 0.11 0.41 Q67 0.75 0.5 0.5 0.75 0.5 0.25 0.75 0.75 0.5 0.58 0.5 0.5 0.25 0.5 1 0.5 1 0.75 0.75 0.5 0.75 0.64 -0.05 0.61 Q68 0 0.5 0.5 0 0.75 0 0.75 0 1 0.39 0 0 0 0 0 0 0 0 0.75 0.5 0.5 0.16 0.23 0.26 Q69 a 0 0.5 0.75 0.5 0.75 0.5 0.5 0.25 0.5 0.47 0 0 0.25 0.5 0.25 0.25 0 0 0.25 0.5 0.75 0.25 0.22 0.35 Q69 b 0.25 1 0.75 0.5 0.75 0.25 0.75 1 1 0.69 0 0 0.25 0.5 0.25 0.25 0 0 0.5 0.5 0.75 0.27 0.42 0.46 Q69 c 0 0.75 0.25 0.25 0.75 0.25 0.5 0.5 0.5 0.42 0 0.25 0.25 0.5 0.5 0 0 0.25 0.75 0.25 1 0.34 0.08 0.38 Q69 d 0 0.75 0.25 0.25 0.75 0.25 1 0.5 0.5 0.47 0 0.25 0.25 0.5 0.5 0.25 0 0.25 0.75 0.25 1 0.36 0.11 0.41 Q69 e 0 0.5 0.5 0.5 0.75 0 0.75 0.25 1 0.47 0.25 0.25 0.25 0.5 0 0.25 0 0 0.75 0.5 0.75 0.32 0.15 0.39 Q69 f 0.5 0.75 0.5 0.5 0.75 0 1 0.5 1 0.61 0.25 0.75 0.25 0.5 0 0.25 0 0 0.75 0.25 0.75 0.34 0.27 0.46 Q71 0.5 0.6 0.35 0.35 0.67 0.5 0.6 0.3 0.3 0.46 0.2 0.8 0.3 0.2 0.08 0.35 0.2 0.5 0.3 0.27 0.5 0.34 0.13 0.39 Q72 0.5 0.4 0.25 0.5 0.43 0.5 0.5 0.3 0.7 0.45 0.1 0.15 0.3 0 0 0.3 0 0.5 0.1 0.27 0.1 0.17 0.29 0.30 Average (4) 0.67 0.76 0.78 0.50 0.78 0.47 0.73 0.72 0.85 0.32 0.40 0.47 0.45 0.48 0.64 0.19 0.33 0.74 0.62 0.52 Road Agency 88% 87% 94% 82% 81% 73% 85% 87% 96% 53% 51% 54% 81% 59% 81% 40% 56% 76% 82% 69% Governance Index 71 72 ANNEX 4: UNEDITED COMMENTS FROM INTERNAL AND EXTERNAL STAKEHOLDERS FROM ARMENIA Q1 MAIN PROBLEMS OF THE ROAD SECTOR From Internal Stakeholders Overall repair of main section of roads Conservation and maintenance of roads Access to villages, rural roads Need better inventory of the road network New modern technology for road works Need more training of specialist, engineers… Quality control of roads works Limited financing Very bad condition of part the road network, particularly local roads Rehabilitation and maintenance of old standard roads Low budget Old equipment Improvement of roads – capital repair Operation in monitoring of roads Fragmentation of road management Shortage of funds Many km of secondary roads in poor condition Lack of financing Extremely low budget for roads Delays in providing the budget to the road sector Payment delays to employees by road contractors From External Stakeholders Absence of monitoring systems Management of road sector Very low quality of roads Lack of motorways in the main corridors Rural villages access roads in poor condition Too little funds for road maintenance Lack of motorways in the main corridors Quality of road works – deterioration in short time Safety Poor quality of roads Deficient public transportation High pollution by vehicles Poor road safety features Deficient maintenance of roads 73 Outdated standards and specifications for roads Poor general approach to roads, including outdated standards Insufficient funds Poor road condition Very low budget for roads Lack of road rehabilitation Poor safety design of roads Funding for roads Poor public awareness Need to establish safety audit Old technology – Old equipment Q70 SUGGESTIONS TO IMPROVE THE PERFORMANCE OF THE ROAD AGENCY From Internal Stakeholders To increase transparency, more advertising, fair tender process No need to increase competition Reasonable competition. Money fluctuation is a problem. For big contracts, only a few contractors are qualified to bid. So if we divide bigger contracts into smaller contracts (typically less than 2 or 3 US$ M.) we can attract more bidders including newcomers Now it is open competition Performance is good Difficult because only a few contractors for big contracts Make sure that appropriate stakeholders are represented in bid committees Increase budget Increase the number of qualified employees Improve the road construction industry, and be more strict of the contractors Improve the quality of works Provide more training opportunities From External Stakeholders Make the process more transparent and objective Too narrow roads built Fight corruption in general Open the market to new contractors More quality control of works Making everything more public Better transparency Advertisement Increase the participation of the public including the involvement of NGOs More transparency Use independent organizations such as NGOs for road assessment Facilitate contact between MOT and the public Give more opportunities to companies with good equipment 74 Provide more specialized training to the road sector Not enough opportunities for each contractor (too much competition) Competition is already high 6-10 bidders per contract Increase the road budget Already good competition More interaction with the public Increase public awareness Improve design to improve safety More open information on contracts available ADDITIONAL COMMENTS From Internal Stakeholders The lower the salary, the higher the corruption After the tender, special notices are published, the MoT website is being built Performance indicators: Travel time, roughness (IRI), traffic volume. A special group collects data from road police on road crashes. HDM4 is used to select sections for rehabilitation All contractors are now private companies Armenia uses a Central Procurement Agency The external audit is reviewed by the Control Chamber MoT has a department responsible for the budget. The Roads Directorate has an annual contract with MoT There is no Road Board The Roads Directorate has Pavement Management System (PMS) and Bridge Management System (BMS) under operation by its own staff The Ministry of Civil Works publishes a monthly report with market prices (e.g., materials)that are used in cost estimates Under WB contracts, a high level individual engineering consultant (international) carries out technical audits HDM4 data collection and analysis is done in-house MoT is responsible for policy formulation Roads Directorate salaries are higher than in MoT. MoT has state employees. Road Directorate employees are hired on the basis annual contracts. Under ADB financed contracts an independent consultant carries out technical audits Bitumen price varies too much There is a need to allocate more fund for maintenance From External Stakeholders Formally road work programs and procurement plans are disclosed but this process is not transparent There is a need to control of trucks quality. Annual checks of truck The quality of the works is not good enough Some members of the tender commission have some potential relation with contractors Tenders are not transparent 75 76 ANNEX 5: UNEDITED COMMENTS FROM INTERNAL AND EXTERNAL STAKEHOLDERS FROM KAZAKHSTAN Q1 MAIN PROBLEMS OF THE ROAD SECTOR From Internal Stakeholders Poor road condition Poor inspection and construction quality of highways Climate and large territorial area in KZ Slow attraction and implementation of new technologies Road Safety Poor condition of roads. Quality and maintenance deficient. Lack of appropriate training Lack of financing Insufficient management capability Corruption Insufficient financing Lack of accountability by Roads Directorate and contractors Corruption during bidding Shortage of qualified technical staff Financing Lack of domestic manufacturing of road equipment Lack of financing From External Stakeholders High level of corruption Low quality of the works Breaching of legislation and human rights during project implementation Traffic congestion in Almaty and Astana Poor quality of roads Poor infrastructure, quality of roads in central and west KZ Poor quality of road Poor road safety Old soviet construction methodology still used Need of rehabilitation and reconstruction of roads Maintenance – not enough financing Poor organization of the road sector Lack of specialists Consolidation of knowledge Lack of transparency Financing resources Lack of domestic manufacturers of road equipment 77 Qualified staff, only one university in KZ for road engineers Lack of accountability of firms, government and banks Lack of openness, information on roads No public hearings Corruption Huge size of the country Climate Excessive and not always fair competition Quality of the roads Lack of roads to some places Financing Q70 SUGGESTIONS TO IMPROVE THE PERFORMANCE OF THE ROAD AGENCY From Internal Stakeholders Timely preparation of bidding documents Improve dissemination of information Competition is already very high Invite international companies Education and train staff Improve supervision of road works Insure that budget are used for intended purposes Strengthening of control and quality in the use of funds Improve the level of management by private organizations More transparency in procurement Streamline the relationship between the Road Directorate and the contractors Roads committee should have more specialists More funds for roads Training of specialized workers Increase salaries of committee and the number of staff Qualified staff More stable cost of the equipment From External Stakeholders Eradicate corruption Adopt new technologies, including materials Carry out user surveys – what people need Ask the people’s opinion on MOTC performance Engage qualified persons Tender committees should comprise independent individuals Roads committee should invite international customers officers 78 Should increase competition – improve quality of design Carry out road user satisfaction surveys Many issues Increase transparency Provide equal condition to all contractors Create system for free access and dissemination of information Respect human rights Strengthen the local currency Minimize conflict of interest through more transparent procurement procedures Competition is more than reasonable, more than sufficient Assign more power to the roads committee Facilitate the entry of new companies Acquire and apply new technologies taking into consideration local conditions Introduce FIDIC for locally-financed contracts Create Road Fund ADDITIONAL COMMENTS From Internal Stakeholders HDM-4 available and used There is a program approved by the President to control corruption Too much expenditures on construction and too little on maintenance Not enough funding for maintenance From External Stakeholders Decisions are made without listening to the public Too much spent on administration Level of corruption much higher in locally funded contracts; if IFI financed, almost no opportunity for corruption 79 80 ANNEX 6: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA AND KAZAKHSTAN Armenia FIGURE 4: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA DIMENSION 1 Dimension 1 Transparency, Disclosure of information and Accountability of the Road Agency (Q2- Q19) Q2 Q19 1.00 Q3 Q18 Q4 0.75 Q17 0.50 Q5 Q16 0.25 Q6 Internal Stakeholders 0.00 External Stakeholders Q15 Q7 Q14 Q8 Q13 Q9 Q12 Q10 Q11 81 FIGURE 5: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA DIMENSION 2 Dimension 2 Transparent and Accessible Procurement Processes (Q20-Q34) Q20 Q34 1.00 Q21 Q33 0.75 Q22 0.50 Q32 Q23 0.25 Internal Stakeholders 0.00 Q31 Q24 External Stakeholders Q30 Q25 Q29 Q26 Q28 Q27 FIGURE 6: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA DIMENSION 3 Dimension 3 Sound Financial Management System (Q35-Q49) Q35 Q49 1.00 Q36 Q48 0.75 Q37 0.50 Q47 Q38 0.25 Internal Stakeholders 0.00 Q46 Q39 External Stakeholders Q45 Q40 Q44 Q41 Q43 Q42 82 FIGURE 7: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA DIMENSION 4 Dimension 4 Administrative Procedures and Anticorruption Effort (Q50-Q70) Q50 Q721.00 Q51 Q71 Q52 Q69 f Q53 0.75 Q69 e Q54 0.50 Q69 d Q55 0.25 Q69 c Q56 Internal Stakeholders 0.00 Q69 b Q57 External Stakeholders Q69 a Q58 Q68 Q59 Q67 Q60 Q66 Q61 Q65 Q62 Q64 Q63 83 FIGURE 8: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR ARMENIA Q2 Q72 Q3 Q4 Q69 f Q71 1.00 Q5 Q69 e Q6 Q69 d Q7 Q69 c Q8 Q69 b Q9 Q69 a Q10 Q68 0.75 Q11 Q67 Q12 Q66 Q13 Q65 Q14 Q64 0.50 Q15 Q63 Q16 Q62 Q17 Q61 0.25 Q18 Q60 Q19 Q59 Q20 Q58 0.00 Q21 Q57 Q22 Q56 Q23 Q55 Q24 Q54 Q25 Q53 Q26 Q52 Q27 Q51 Q28 Q50 Q29 Q49 Q30 Q48 Q31 Q47 Q32 Q46 Q33 Q45 Q34 Q44 Q35 Q43 Q36 Dimension Questions Q42 Q41 Q37 Q40 Q39 Q38 1. Transparency, Disclosure of information and Accountability of Q2-Q19 the Road Agency 2. Transparent and Accessible Q20-Q34 Procurement Processes Internal Stakeholders 3. Sound Financial Management Q35-Q49 System External Stakeholders 4. Administrative Procedures and Q50-Q70 Anticorruption Effort Note: The numbers following the letter Q refer to the questions given in Annex 1 84 Kazakhstan FIGURE 9: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR KAZAKHSTAN DIMENSION 1 Dimension 1 Transparency, Disclosure of information and Accountability of the Road Agency (Q2- Q19 Q2 Q19 1.00 Q3 Q18 Q4 0.75 Q17 0.50 Q5 0.25 Q16 Q6 Internal Stakeholders 0.00 External Stakeholders Q15 Q7 Q14 Q8 Q13 Q9 Q12 Q10 Q11 FIGURE 10: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR KAZAKHSTAN DIMENSION 2 Dimension 2 Transparent and Accessible Procurement Processes (Q20-Q34) Q20 Q34 1.00 Q21 Q33 0.75 Q22 0.50 Q32 Q23 0.25 Internal Stakeholders 0.00 Q31 Q24 External Stakeholders Q30 Q25 Q29 Q26 Q28 Q27 85 FIGURE 11: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR KAZAKHSTAN DIMENSION 3 Dimension 3 Sound Financial Management System (Q35-Q49) Q35 Q49 1.00 Q36 Q48 0.75 Q37 0.50 Q47 Q38 0.25 Internal Stakeholders 0.00 Q46 Q39 External Stakeholders Q45 Q40 Q44 Q41 Q43 Q42 FIGURE 12: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR KAZAKHSTAN DIMENSION 4 Dimension 4 Administrative Procedures and Anticorruption Effort (Q50-Q70) Q50 Q721.00 Q51 Q71 Q52 Q69 f Q53 0.75 Q69 e Q54 0.50 Q69 d Q55 0.25 Q69 c Q56 Internal Stakeholders 0.00 Q69 b Q57 External Stakeholders Q69 a Q58 Q68 Q59 Q67 Q60 Q66 Q61 Q65 Q62 Q64 Q63 86 FIGURE 13: SCORES FROM INTERNAL AND EXTERNAL STAKEHOLDERS FOR KAZAKHSTAN Q2 Q72 Q3 Q4 Q69 f Q71 1.00 Q5 Q69 e Q6 Q69 d Q7 Q69 c Q8 Q69 b Q9 Q69 a Q10 Q68 0.75 Q11 Q67 Q12 Q66 Q13 Q65 Q14 Q64 0.50 Q15 Q63 Q16 Q62 Q17 Q61 0.25 Q18 Q60 Q19 Q59 Q20 Q58 0.00 Q21 Q57 Q22 Q56 Q23 Q55 Q24 Q54 Q25 Q53 Q26 Q52 Q27 Q51 Q28 Q50 Q29 Q49 Q30 Q48 Q31 Q47 Q32 Q46 Q33 Q45 Q34 Q44 Q35 Dimension Questions Q43 Q36 Q42 Q41 Q37 Q40 Q39 Q38 1. Transparency, Disclosure of information and Accountability of Q2-Q19 the Road Agency 2. Transparent and Accessible Q20-Q34 Procurement Processes Internal Stakeholders 3. Sound Financial Management Q35-Q49 System External Stakeholders 4. Administrative Procedures and Q50-Q70 Anticorruption Effort Note: The numbers following the letter Q refer to the questions given in Annex 1 87 88 ANNEX 7: RESULTS FROM ARMENIA USING AN ALTERNATIVE SCORING METHOD GAC Squad Operational Assignment: 2009-5 The Road Organization as viewed by internal and external stakeholders Internal Stakeholders Public Sector including Academia (A, B, D) - External Stakeholders Private Sector including NGOs, IFIs (C, RC, E, F) Internal Stakeholders External Stakeholders Difference Average Ref. No. 5 6 7 1 10 3 11 15 Average 4 8 9 2 12 13 14 16 17 18 Average between Int. score Dimensions Group A A A B D D B B score C RC C E F E RC C F E score and Ext. views Q2 1 1 1 1 0 1 1 0 0.75 1 1 1 1 1 1 1 1 1 1 1.00 -0.25 0.89 Q3 1 1 0 1 0 1 1 0.71 1 1 0 0 1 1 1 0.71 0.00 0.71 Q4 1 0 0 0 0 0.20 0 1 0 1 1 1 0 0.57 -0.37 0.42 Q5 1 1 1 1 1 1 1 1 1.00 1 1 1 0 1 1 1 1 1 1 0.90 0.10 0.94 Q6 1 1 1 1 1 1 1 1 1.00 1 1 1 0 1 1 1 1 1 0.89 0.11 0.94 Q7 1 1 0 1 1 1 1 1 0.88 0 1 1 0 1 1 1 1 1 1 0.80 0.08 0.83 Q8 0 0 0 0 1 0 0.17 1 1 0 1 1 1 0 1 0.75 -0.58 0.50 Q9 1 1 1 1 1 1 1 1.00 1 1 1 0 0 1 1 1 0 0.67 0.33 0.81 Q10 0 0 1 1 1 1 1 0.71 1 1 1 0 1 1 1 1 1 0.89 -0.17 0.81 1 Q11 1 1 1 1 1 1 1 0 0.88 1 1 1 0 1 1 1 1 1 0.89 -0.01 0.88 Q12 1 1 1 1 1 0 0.83 0 1 1 0 1 1 1 1 0.75 0.08 0.79 Q13 1 0 1 0.67 0 1 0 1 1 0 0.50 0.17 0.56 Q14 1 1 1 1 1 1 1 0 0.88 0 1 1 0 0 1 1 1 1 0.67 0.21 0.76 Q15 1 1 0 1 1 1 1 1 0.88 1 1 0 1 1 1 1 1 0 0.78 0.10 0.82 Q16 1 1 1 1 1 1 1 1 1.00 1 1 0 1 1 1 0 1 0.75 0.25 0.88 Q17 1 0 1 1 0 1 0 0.57 1 1 0 1 1 0 0 0.57 0.00 0.57 Q18 1 1 1 1 0 1 1 0 0.75 1 1 1 1 1 0 1 0.86 -0.11 0.80 Q19 1 1 1 1 1 1 1.00 1 0 1 1 1 1 0 0.71 0.29 0.85 Average (1) 0.89 0.72 0.72 1.00 0.60 1.00 1.00 0.50 0.71 1.00 0.75 0.13 0.86 0.93 1.00 0.89 0.73 0.69 Q20 1 1 1 1 1 1 1 1 1.00 1 1 0 0 1 1 1 1 1 1 0.80 0.20 0.89 Q21 1 1 0 1 1 1 1 0.86 0 1 0 1 1 1 1 1 1 0.78 0.08 0.81 Q22 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q23 1 1 1 1 1 1 1.00 0 1 1 1 1 1 1 1 0.88 0.13 0.93 Q24 1 1 1 1 1 1 0 0.86 1 1 1 1 1 1 1 1 1.00 -0.14 0.93 Q25 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1.00 0.00 1.00 Q26 1 1 1 1 1 1 1.00 0 1 1 1 1 1 1 0.86 0.14 0.92 2 Q27 Q28 1 1 1 1 1 1 1 1 1 1 1 1.00 1.00 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0.78 1.00 0.22 0.00 0.85 1.00 Q29 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1.00 0.00 1.00 Q30 1 1 1 1 0 1 1 1 0.88 0 1 0 1 1 1 1 1 1 0.78 0.10 0.82 Q31 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1.00 0.00 1.00 Q32 1 1 1 1 1 1 1 1 1.00 1 1 0 1 1 1 1 1 1 0.89 0.11 0.94 Q33 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q34 1 1 1 1 1 1 1 1 1.00 1 1 0 1 1 1 1 1 1 0.89 0.11 0.94 Average (2) 1.00 1.00 0.93 1.00 0.86 1.00 1.00 0.93 0.62 1.00 0.00 0.40 1.00 1.00 1.00 1.00 1.00 1.00 Q35 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1.00 0.00 1.00 Q36 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1.00 0.00 1.00 Q37 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1.00 0.00 1.00 Q38 1 1 1 1 1 1.00 1 1 1 1 1.00 0.00 1.00 Q39 1 1 1 1 1 1 1.00 1 1 1 1 1.00 0.00 1.00 Q40 1 1 1 1 1.00 1 0 1 1 1 1 1 0.86 0.14 0.91 Q41 1 1 1 1 1 1.00 0 1 0 1 1 1 1 1 1 0.78 0.22 0.86 3 Q42 Q43 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1.00 1.00 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1.00 1.00 0.00 0.00 1.00 1.00 Q44 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1.00 0.00 1.00 Q45 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1.00 0.00 1.00 Q46 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1.00 0.00 1.00 Q47 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q48 1 1 0 0 1 1 0.67 1 1 0 1 1 0.80 -0.13 0.73 Q49 1 1 1 1 1 1 1.00 1 1 1 1 1.00 0.00 1.00 Average (3) 1.00 1.00 0.93 1.00 0.88 1.00 1.00 1.00 0.88 1.00 0.67 0.50 1.00 1.00 0.93 1.00 1.00 1.00 Q50 1 1 1 1 1 1 1 1.00 1 1 0 1 1 1 1 0.86 0.14 0.93 Q51 1 1 1 1 1 1 1 1 1.00 1 1 0 1 1 1 1 0.86 0.14 0.93 Q52 1 1 1 1 1 1.00 0 1 0 1 1 1 1 1 0.75 0.25 0.85 Q53 1 1 1 1 1 1 1 1.00 0 1 1 1 1 1 1 1 0.88 0.13 0.93 Q54 1 1 1 1 1 1 1 1 1.00 1 1 0 1 0 1 1 1 0.75 0.25 0.88 Q55 1 0 0 1 1 1 1 1 0.75 1 1 1 1 1 0 1 0 0.75 0.00 0.75 Q56 0 0 1 0 1 1 1 0.57 0 0 0 0 1 1 0 0 0.25 0.32 0.40 Q57 1 1 1 0 1 1 1 0.86 0 0 0 1 0 1 1 0.43 0.43 0.64 Q58 1 1 1 1 1 1 1 1.00 0 0 1 0 1 1 0.50 0.50 0.77 Q59 1 1 1 1 1.00 1 1 1 1 1.00 0.00 1.00 Q60 1 1 1 1 1 1 1 1.00 1 0 1 1 1 1 1 0.86 0.14 0.93 Q61 0 0 1 1 1 1 0 0.57 0 1 0 1 1 0.60 -0.03 0.58 Q62 1 1 1 1 0 1 1 1 0.88 0 1 1 1 1 1 0.83 0.04 0.86 4 Q63 Q64 0 1 0 1 1 1 1 1 1 1 1 1 1 1 0.71 1.00 0 0 1 1 1 1 1 1 1 0.80 0.75 -0.09 0.25 0.75 0.91 Q65 1 1 1 1 1 1 1 1.00 0 0 1 1 1 1 0.67 0.33 0.85 Q66 0.75 0.75 0.25 0.25 0.5 0 0.5 0.5 0.44 0 0.5 0 0.5 0.75 0.5 0.25 0.5 0.25 0.75 0.40 0.04 0.42 Q67 0.5 0.5 0 0.75 0.5 0.75 0.5 0.75 0.53 0.5 0.5 0.5 0.75 0.75 0.25 0 0.25 0.5 0.5 0.45 0.08 0.49 Q68 0.5 0.75 0 0.25 0.25 0 0.5 0.5 0.34 0 1 0.5 0 0.25 0.75 0 0.25 0.5 0.75 0.40 -0.06 0.38 Q69 a 1 0.75 1 0.5 0.25 0.5 0.75 0.25 0.63 0 0.25 0 0 0.25 1 0.5 0.75 0.25 0.5 0.35 0.28 0.47 Q69 b 1 0.75 1 0.5 0.25 0.5 0.75 0.5 0.66 0 0.75 0 0 0.25 1 0.5 1 0.5 0.75 0.48 0.18 0.56 Q69 c 0.75 0.75 0.5 0.5 0.5 1 0.25 0.75 0.63 0.5 0.25 0.5 0.5 0.25 1 0.5 0.5 0.75 0.5 0.53 0.10 0.57 Q69 d 0.75 0.75 0.5 0.5 0.25 1 0.25 0.5 0.56 0.5 0.5 0.5 0.5 0.25 1 0.5 1 0.75 0.75 0.63 -0.06 0.60 Q69 e 0.75 0.75 1 0.75 0.5 0.75 0.75 0.75 0.75 0.25 0.5 0 0.5 0.25 0.75 0.5 0.5 0.5 0.5 0.43 0.33 0.57 Q69 f 0.75 0.75 1 0.5 0.25 0.75 0.5 0.75 0.66 0 0.75 0 0.25 0.25 0.75 0.5 0.5 0.5 0.5 0.40 0.26 0.51 Q71 0.45 0.8 0.55 0.75 0.75 0.95 0.55 0.65 0.68 0.65 0.2 0.25 0.65 0.7 0.45 0 0.5 0.5 0.55 0.45 0.24 0.55 Q72 0.5 0.5 0.1 0.25 0.5 0 0.5 0.05 0.30 0.5 0.1 0.5 0.5 0.5 0.1 0 0.05 0.2 0.45 0.29 0.01 0.29 Average (4) 0.77 0.73 0.77 0.73 0.59 0.80 0.81 0.76 0.28 0.72 0.27 0.27 0.66 0.78 0.54 0.81 0.51 0.74 Road Agency 91% 86% 84% 93% 73% 95% 95% 80% 62% 93% 42% 32% 88% 93% 87% 92% 81% 86% Governance Index 89 90 ANNEX 8: RESULTS FROM KAZAKHSTAN USING AN ALTERNATIVE SCORING METHOD GAC Squad Operational Assignment: 2009-5 The Road Organization as viewed by internal and external stakeholders Internal Stakeholders Public Sector including Academia (A, B, D) - External Stakeholders Private Sector including NGOs, IFIs (C, RC, E, F) Internal Stakeholders External Stakeholders Difference Average Ref. No. 1 2 3 7 8 9 11 15 16 Average 4 5 6 10 12 13 14 17 18 19 20 Average between Int. score Dimensions Group B A D B B B A A A score F F E RC C C F F RC C RC score and Ext. views Q2 1 1 1 1 1 1 0 1 1 0.89 1 0 1 0 1 0 1 1 1 1 0.70 0.19 0.79 Q3 1 1 1 1 1 1 1 1 1.00 1 0 1 0 1 1 0 1 1 1 0.70 0.30 0.83 Q4 1 1 1 1 0 0 0 1 0.63 0 0 1 0 1 0 0 1 1 0 0.40 0.23 0.50 Q5 1 1 1 1 0 1 1 1 1 0.89 1 1 1 1 1 1 1 1 1 1 1.00 -0.11 0.95 Q6 1 1 1 1 0 1 1 1 1 0.89 1 0 1 1 1 0 1 1 1 0.78 0.11 0.83 Q7 1 1 1 1 1 1 1 1 1.00 1 0 0 1 1 1 1 1 1 1 1 0.82 0.18 0.89 Q8 1 1 0 0 0 1 1 1 0.63 1 0 1 1 1 1 0 1 1 0 0.70 -0.08 0.67 Q9 1 1 1 0 0 1 1 1 0.75 1 0 1 0 1 1 1 1 1 1 0.80 -0.05 0.78 Q10 1 1 1 1 1 1 1 1 1 1.00 1 1 0 1 0 1 1 1 1 1 1 0.82 0.18 0.90 1 Q11 1 1 1 1 1 1 1 1 1 1.00 0 0 0 0 0 1 1 0 0 0.22 0.78 0.61 Q12 1 1 1 1 1 1 1 1 1.00 0 0 0 0 0 1 0 0 0.13 0.88 0.56 Q13 1 1 1 1 0 1 0 1 0.75 0 0 0 0 0 1 0 0 1 1 0.30 0.45 0.50 Q14 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 0 1 1 1 1 0.91 0.09 0.95 Q15 1 1 1 1 1 1 1 1 1.00 1 0 0 1 1 1 0 1 1 1 1 0.73 0.27 0.84 Q16 1 1 1 1 1 1 1 1 1 1.00 0 1 1 1 0 1 1 1 1 0.78 0.22 0.89 Q17 1 1 1 1 1 1 1 1 1 1.00 0 0 0 0 1 1 1 0.43 0.57 0.75 Q18 1 1 1 1 1 1 1 1 1.00 0 0 0 1 0 1 1 1 0.50 0.50 0.75 Q19 1 1 1 1 1 1 1 1 1.00 0 1 0 1 0 1 0 1 0 0.44 0.56 0.71 Average (1) 1.00 1.00 1.00 0.93 0.67 0.78 0.89 0.93 1.00 0.69 0.31 0.18 0.81 0.39 1.00 0.41 0.57 0.93 1.00 0.72 Q20 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q21 1 1 1 1 1 1 1.00 1 0 0 0 1 1 1 0.57 0.43 0.77 Q22 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q23 1 1 1 1 1.00 1 1 1 0 1 1 0.83 0.17 0.90 Q24 1 1 1 1 1 1 1.00 1 1 1 1 0 1 1 0.86 0.14 0.92 Q25 1 1 1 1 1 1 1 1 1 1.00 0 1 1 1 1 1 1 1 0.88 0.13 0.94 Q26 1 1 1 1.00 1 1 1 1 1 1 1 1.00 0.00 1.00 2 Q27 Q28 1 1 1 1 1 1 1 1 1 1 1 1 1 1.00 1.00 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0.80 1.00 0.20 0.00 0.89 1.00 Q29 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q30 1 1 1 1 1 1 1 1 1 1.00 0 1 1 1 1 1 1 0.86 0.14 0.94 Q31 1 1 1 1 0 1 1 0.86 1 1 1 1 1 1 0 0.86 0.00 0.86 Q32 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q33 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q34 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1.00 0.00 1.00 Average (2) 1.00 1.00 1.00 1.00 1.00 1.00 0.92 1.00 1.00 0.75 1.00 1.00 1.00 0.86 0.92 0.67 0.88 0.86 1.00 1.00 Q35 1 1 1 1 1 1 1 1 1.00 1 0 1 1 0.75 0.25 0.92 Q36 1 1 1 1 1 1 1 1 1.00 1 0 1 1 1 1 1 0.86 0.14 0.93 Q37 1 1 1 1 0 1 1 1 0.88 1 0 0 1 0.50 0.38 0.75 Q38 1 1 1 1 1 1 1 1 1.00 1 1 0 0.67 0.33 0.91 Q39 1 1 1 1 1 1 1 1 1.00 1 0 0 0.33 0.67 0.82 Q40 1 1 1 1 0 1 1 1 0.88 0 1 0 1 0 1 1 0.57 0.30 0.73 Q41 1 1 1 1 1 1 1 1 1 1.00 1 1 1 0 1 1 0.83 0.17 0.93 3 Q42 Q43 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1.00 1.00 0 1 1 1 1 1 1 0 1 1 1 1 1 1 0.78 1.00 0.22 0.00 0.89 1.00 Q44 1 1 1 1 1 1 1 1 1 1.00 0 1 0 1 1 1 1 0.71 0.29 0.88 Q45 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q46 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q47 1 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1 1.00 0.00 1.00 Q48 1 1 1 1 1 1 1 1 1.00 1 1 1 0 0 0.60 0.40 0.85 Q49 1 1 1 1 1 1 1 1 1.00 1 1 1 1 1.00 0.00 1.00 Average (3) 1.00 1.00 1.00 1.00 1.00 0.87 1.00 1.00 1.00 0.60 0.60 1.00 1.00 0.75 1.00 0.44 0.64 0.80 1.00 1.00 Q50 1 1 1 1 1 1 1 1 1 1.00 0 1 1 1 0 0 1 1 0.63 0.38 0.82 Q51 1 1 1 1 1 0 1 1 1 0.89 1 1 1 1 1 1 0 1 1 0.89 0.00 0.89 Q52 1 1 1 1 1 0 1 1 1 0.89 0 1 1 1 0 1 1 1 0 0.67 0.22 0.78 Q53 1 1 1 1 1 0 1 1 1 0.89 0 1 1 1 1 1 1 1 0.88 0.01 0.88 Q54 1 1 1 0 1 1 1 1 0.88 1 1 1 1 1 1 0 1 1 1 1 0.91 -0.03 0.89 Q55 1 0 1 0 1 0 0 0 0 0.33 0 0 0 0 0 1 0 1 0 0 0 0.18 0.15 0.25 Q56 1 1 1 0 0 1 0.67 0 0 0 0 0 0 0 1 1 0 0.20 0.47 0.38 Q57 1 1 1 1 1 1 1 1.00 0 0 1 1 0 0 1 0.43 0.57 0.71 Q58 1 1 1 1 1 1 1 1 1.00 0 0 0 0.00 1.00 0.73 Q59 1 0 1 1 1 1 0 1 1 0.78 1 1 0 0 1 1 1 1 0.75 0.03 0.76 Q60 1 1 1 0 1 1 1 1 1 0.89 1 0 1 1 1 1 0 0.71 0.17 0.81 Q61 1 1 0 1 1 1 1 1 0.88 1 1 1 0 1 1 1 0 1 0.78 0.10 0.82 Q62 1 1 1 1 1 1 1 1 1 1.00 1 1 1 0 0 1 1 0.71 0.29 0.88 4 Q63 Q64 1 1 1 1 1 1 1 1 1 1 1 1 1 1.00 1.00 0 0 1 1 1 1 0 0 1 1 1 1 0 0.60 0.63 0.40 0.38 0.82 0.80 Q65 1 1 1 0 1 0 1 1 1 0.78 0 0 0 1 0 0 1 1 0 0.33 0.44 0.56 Q66 0 0.25 0.5 0.5 0.75 0.25 0.5 0.5 1 0.47 0.25 0.25 0.25 0.5 0.5 0.5 0 0.25 0.75 0.5 0.25 0.36 0.11 0.41 Q67 0.75 0.5 0.5 0.75 0.5 0.25 0.75 0.75 0.5 0.58 0.5 0.5 0.25 0.5 1 0.5 1 0.75 0.75 0.5 0.75 0.64 -0.05 0.61 Q68 0 0.5 0.5 0 0.75 0 0.75 0 1 0.39 0 0 0 0 0 0 0 0 0.75 0.5 0.5 0.16 0.23 0.26 Q69 a 0 0.5 0.75 0.5 0.75 0.5 0.5 0.25 0.5 0.47 0 0 0.25 0.5 0.25 0.25 0 0 0.25 0.5 0.75 0.25 0.22 0.35 Q69 b 0.25 1 0.75 0.5 0.75 0.25 0.75 1 1 0.69 0 0 0.25 0.5 0.25 0.25 0 0 0.5 0.5 0.75 0.27 0.42 0.46 Q69 c 0 0.75 0.25 0.25 0.75 0.25 0.5 0.5 0.5 0.42 0 0.25 0.25 0.5 0.5 0 0 0.25 0.75 0.25 1 0.34 0.08 0.38 Q69 d 0 0.75 0.25 0.25 0.75 0.25 1 0.5 0.5 0.47 0 0.25 0.25 0.5 0.5 0.25 0 0.25 0.75 0.25 1 0.36 0.11 0.41 Q69 e 0 0.5 0.5 0.5 0.75 0 0.75 0.25 1 0.47 0.25 0.25 0.25 0.5 0 0.25 0 0 0.75 0.5 0.75 0.32 0.15 0.39 Q69 f 0.5 0.75 0.5 0.5 0.75 0 1 0.5 1 0.61 0.25 0.75 0.25 0.5 0 0.25 0 0 0.75 0.25 0.75 0.34 0.27 0.46 Q71 0.5 0.6 0.35 0.35 0.67 0.5 0.6 0.3 0.3 0.46 0.2 0.8 0.3 0.2 0.08 0.35 0.2 0.5 0.3 0.27 0.5 0.34 0.13 0.39 Q72 0.5 0.4 0.25 0.5 0.43 0.5 0.5 0.3 0.7 0.45 0.1 0.15 0.3 0 0 0.3 0 0.5 0.1 0.27 0.1 0.17 0.29 0.30 Average (4) 0.67 0.76 0.78 0.50 0.85 0.47 0.77 0.73 0.85 0.21 0.36 0.46 0.45 0.48 0.66 0.14 0.31 0.75 0.65 0.53 Road Agency 92% 94% 95% 86% 88% 78% 89% 91% 96% 56% 57% 66% 82% 62% 90% 42% 60% 83% 91% 81% Governance Index 91