33184 E-Strategies MON I T O R I N G A N D EV A L U A T I O N TO O L K I T January 2005 The World Bank GLO B A LINF O R M AT I O N A N DCOM M U N I C ATIONTEC H N O L O G I E SDEPA R T M E N T T H E W O R L D B A N K ­ I N F / G I C T V 6 . 1 B ( 3 J A N 0 5 ) E-STRATEGIES MONITORING AND EVALUATION TOOLKIT E-STRATEGIES MONITORING AND EVALUATION TOOLKIT ABOUT THIS TOOLKIT PURPOSE The focus of this toolkit is premised on the fact that effective Monitoring and Evaluation (M&E) is integral to the design and implementation of effective e-strategies. Developing M&E components of e-strategies is a means by which to ensure that the strategies are explicit and realistic with regard to what they aim to achieve, and that their implementation is regularly assessed and realigned to ensure the efficient use of scarce resources. In many respects, the credibility and efficiency of e-strategies depends on their having a strong M&E spine. Based on a review of some 50 e-strategies conducted by the authors, this toolkit advances a framework by which to integrate M&E into e-strategies. The M&E framework expands on indicators that relate to core elements of the strategy formulation process, namely the development of policy goals, strategic priorities, and key initiatives and actions that are embodied by e-strategies. TARGET AUDIENCE This toolkit it targeted at anyone who is involved in, or even just has an interest in, the development of national e-strategies. This includes decision-makers and staff from government agencies, development practitioners from international and non-governmental organizations (NGOs), members from the private sector involved in ICT development, and the interested general public. This toolkit should be regarded as a `living document' which will evolve with time. In future editions, the M&E framework applied here in detail to several of the key thematic areas on which national e-strategies commonly focus (namely infrastructure, ICT sector and e-government) will be deepened in other similarly important domains such as e-health, e-education, and e-business. This initial version of the toolkit simply gives an overview of these areas. As the toolkit benefits from users' feedback, and as additional evidence becomes available about various e-strategy players' experience with M&E, the online version of the toolkit will attempt to capture best practices and offer additional tools for monitoring and evaluating progress in the implementation of e-strategies. This online version will be linked from http://www.worldbank.org/ict/ III ACKNOWLEDGEMENTS This toolkit was written by Aref Adamali and Bruno Lanvin, under the general direction of Robert Schware. Additional research and writing support was provided by John Coffey and Charles Watt. Mark Wahl developed the web site and edited the document. The above team could not have completed the project without the support, ideas and feedback from a number of people. They include, Pierre Guislain, Christine Qiang, Charles Kenny, and Lorenzo Pupillo, as well as other colleagues from the Policy Division of the World Bank's Global Information and Communication Technologies Department. Before reaching its present form, this toolkit went though various intermediate versions. Comments and advice have been received from within and from outside the World Bank Group, and are gratefully acknowledged here. The authors would like to thank in particular Susan Teltscher from UNCTAD and colleagues in the e-Development Services Thematic Group. DISCLAIMER The views expressed are those of the authors and do not necessarily reflect those of the World Bank, its executive directors, or the countries they represent. Any errors remain the authors' own. IV TABLE OF CONTENTS About this Toolkit........................................................................................................................III Purpose...................................................................................................................................III Target audience.......................................................................................................................III The team.................................................................................................................................IV Acknowledgement...................................................................................................................IV Table of Contents...........................................................................................................................V Preface........................................................................................................................................VII Acronyms and Keywords ...........................................................................................................VIII MODULE 1 -Background and Purpose.........................................................................................1 I. Structure and methodology of the toolkit.....................................................................................2 II. A simple framework to monitor and assess e-strategies...............................................................4 III. Where does M&E fit ? .............................................................................................................5 MODULE 2 - A Strategic Approach to Monitoring and Evaluation...............................................7 I. Methodological context..............................................................................................................8 II. Institutional and strategic context............................................................................................14 MODULE 3 -Building an E-strategy on a Sound Foundation......................................................18 I. E-strategies VS D-strategies......................................................................................................18 II. E-strategies VS information societies........................................................................................19 I. E-readiness VS access AND usage..............................................................................................21 MODULE 4 - ICT Infrastructure and ICT sector.........................................................................25 I. Rationale: Why focus on ICT infrastructure?............................................................................25 II. M&E frameworks...................................................................................................................27 III. Assumptions and risks............................................................................................................34 MODULE 5 - ICT Sector..............................................................................................................37 I. Rationale: Why focus on the ICT sector?..................................................................................37 II. M&E framework(s).................................................................................................................39 III. Assumptions and risks............................................................................................................48 V MODULE 6 - E-government.........................................................................................................50 I. Rationale: Why focus on E-government?..................................................................................50 II. Definition: What is E-government?.........................................................................................50 III. Overview of E-government strategies......................................................................................51 IV. M&E frameworks..................................................................................................................54 MODULE 7 - Other Strategic Sectors...........................................................................................63 I. E-business...............................................................................................................................63 II. E-education and e-learning.....................................................................................................64 III. E-health................................................................................................................................65 ANNEXES Annex 1. The basics on indicators and evaluation .........................................................................67 Annex 2. Data and indices..........................................................................................................71 Annex 3. List of E-strategies .......................................................................................................79 VI PREFACE Technology and infrastructure are never ends in themselves. However, when it comes to fighting poverty, they play a pivotal role. Telecommunications and information technologies in particular are offering new ways to address old and stubborn obstacles to development. Yet the presence of telecommunications lines, computers, or even Internet connectivity is not a sufficient condition in itself to turn such technologies into instruments of development. In the absence of relevant applications in health, education, governance, or employment, many high-tech white elephants are likely to emerge. The ultimate reason to devote resources to information technologies rests in their development impact, their socio-economic value, their ability to empower, and their contribution to job creation and competitiveness. Over the past ten years or so, e-strategies have received increasing attention in this respect, because they have been seen by many as a way to transcend rivalries and divergent views among ministries, because they were perceived as a way to involve all stakeholders (government, business, and civil society) in a common project, and because they could allow countries to mobilize their energies around a few key development objectives, rather than pure ICT goals. Yet, in many cases, efforts to design and implement e-strategies have remained disappointing. Among the reasons for failed approaches in this area, the following have been common: (a) lack of ability or political will to link e-strategy objectives to broader macro-economic objectives; (b) excessive focus on some visible indicators, and perceived `best practices' from countries with different contexts and constraints; and (c) absence of a reliable set of `impact indicators' to assess the present and future value of efforts to design and implement such e-strategies. It is on this third roadblock that this toolkit focuses. Based on the evaluation of some fifty e-strategies, it attempts to provide an array of emerging best practices in the area of monitoring and evaluating e-strategies and their components. As the international community pursues its effort to make the best possible use of information technologies to fight poverty, the need for reliable and comparable indicators has never been more acute. In the context of upcoming high-level discussions--whether for the `Millennium Summit + 5' meetings (New-York, September 2005), or for the second part of the World Summit on Information Society (WSIS, Tunis, November 2005)--it is likely that such issues will receive renewed attention. We believe that, if a significant number of countries refer to the tools offered here and use them to make their e-strategies more efficient, successful, and measurable, such discussions could gain in relevance. It is our hope that this modest and down-to-earth contribution will contribute to enhance the ability of those who build information societies to make the best of existing resources and energies. Mohsen Khalil, Director Global ICT Department The World Bank Group VII ACRONYMS AND KEYWORDS B2B Business-to-Business KAM Knowledge Assessment Methodology B2C Business-to-Consumer KEI Knowledge Economy Index B2G Business-to-Government KI Knowledge Index (see KAM) CAS Country Assistance Strategy M&E Monitoring and Evaluation CDT Center for Democracy and MDG Millennium Development Goal(s) Technology NGO Non-government organization CIO Chief Information Officer NRI Network Readiness Index DAC Development Assistance Committee NSO National Statistics Office (OECD) OBA Output Based Aid DAI Digital Access Index OECDOrganization for Economic DOT Digital Opportunity Taskforce Cooperation and Development EU European Union OUP Oxford University Press FAQ Frequently Asked Question PC Personal Computer FDI Foreign Direct Investment PDA Personal Digital Assistant G2B Government-to-Business PIAP Public Internet Access Point G2C Government-to-Consumer PRSP Poverty Reduction Strategy Paper G2G Government-to-Government QQT Quantity, Quality, Time GDP Gross Domestic Product R&D Research and Development GITR Global Information Technology ROI Return on Investment Report (INSEAD-WEF-infoDev) Rs Rupees HR Human Resources RTC Record of Rights, Tenancy and Crops ICT Information and Communication SME Small and Medium Size Enterprise Technology TOR Terms of Reference ICT4DInformation and Communication UNESCO United Nations Educational, Technology for Development Scientific and Cultural Organization IFC International Finance Corporation UQAM University of Montreal, Quebec infoDev Information for Development USPTO United States Patent and Trademark Program (hosted by the World Bank) Office IP Intellectual Property VC Venture Capital IPR Intellectual Property Regime/Right WB World Bank IS Information Society WBI World Bank Institute ISDN Integrated Services Digital Network WDI World Development Indicators ISP Internet Service Provider WEF World Economic Forum IT Information Technology WSIS World Summit on the Information ITU International Telecommunications Society Union (United Nations) WTO World Trade Organization VIII Module 1 BACKGROUND AND PURPOSE At the World Summit on the Information Society (WSIS) in December 2003, leaders of the world made a commitment to develop national e-strategies by the time the world convenes for the second phase of WSIS in November 2005.1 This represents a major challenge for individual countries, as well as a significant risk for many of them. A first risk is that individual e-strategies be launched in the absence of a common reference framework, which will make it difficult to evaluate their impact, compare their achievements, and consolidate them at sub-regional or regional levels. A second risk is that errors of the past might be repeated: over the past decade, many countries have spent significant time, energy, and resources to design e-strategies which often remained blue prints, or white elephants because no systematic set of indicators had been agreed upon and established to monitor and evaluate their implementation. The purpose of the present toolkit is to address such risks by offering a simple guide to the promoters, supporters, designers, and implementers of national e-strategies, with a special focus on developing countries. It is based on the combination of two ingredients, namely: § A detailed review of some fifty e-strategies (identifying common trends in ICT policy in general and M&E in particular); § The use of a simple logical framework to identify, track, monitor, and evaluate the various policy goals, strategic priorities, key initiatives, and actions that make an e-strategy. A fundamental premise of this toolkit is that monitoring and evaluation ( `M&E') is not an `ex-post facto' component of an e-strategy, but a vital part of its design and implementation, and a condition of its effectiveness. Developing M&E components of e-strategies is a means to ensure that the strategies are explicit and realistic with regard to what they aim to achieve, and that their implementation is regularly assessed and realigned to ensure the efficient use of scarce resources, particularly in terms of the opportunity costs of those resources when they might alternatively be used for poverty reduction, healthcare, or non-ICT infrastructure. Removing M&E from its `ex post facto' status, however, immediately raises the issue of sequencing: when and how should the M&E component be introduced in an e-strategy? The present toolkit will explore such sequencing in two fashions: (1) within specific activities (referred to as `modules' here), and (2) for the strategy as a whole. In both cases, it will be argued that the sooner M&E concerns are articulated, the better. 1Article 8 of the WSIS Plan of Action states that the "Development of national e-strategies (...) should be encouraged by all countries by 2005." See http://www.itu.int/wsis/docs/geneva/official/poa.html 1 I. STRUCTURE AND METHODOLOGY OF THE TOOLKIT This toolkit has been designed and produced as a `hands-on' modular tool. This means that its reader may chose to use any of the toolkit's components (modules), and adapt it to his or her own objectives and decision sequence. Apart from modules One, Two and Seven (which concern elements of methodology and consistency, and apply `across the board', the rest of the toolkit can be seen as a collection of stand-alone sub-kits (called thematic modules), aimed at providing practical recipes and indicators to monitor and evaluate particular components of an e-strategy. Outline of the toolkit Modules 1 and 2 of the toolkit will provide a strategic description of the approach and methodology used to make M&E an integral part of e-strategies. Modules 3 to 7 will offer specific M&E tools for (a) the emergence of the adequate social, economical and institutional environment for a successful e-strategy implementation, and (b) thematic areas of e-strategies. By reviewing a significant number of national e-strategies,1 the authors have been able to select a small number of thematic areas that reflect the essential interests of the promoters of such strategies. They basically fall under two major categories: · ICT development (infrastructure, equipment and services) · Sectoral applications development and usage (e-government, e-business, e-health, e- education, in particular).2 In order to be as relevant as possible to the needs of those who will have the responsibility to promote, design, and implement e-strategies, the structure of the toolkit reflects those priorities. Finally, Annexes provide complementary elements regarding key ICT-related indices, as well as the data, methodologies, and references on which the Toolkit is based.. 1 See Annex 3 for a list of the e-strategies considered. 2 E-government is the subject of Module 5, while e-business, e-health and e-education are addressed in a succinct fashion in Module 6. Later editions of this toolkit are expected to provide more detail on M&E in those areas. Methodology and structure of the thematic modules Each thematic module is based on a common format, and includes the following sections: Rationale. Why the theme under consideration was selected as a module of the toolkit. Definition. What is understood by the specific theme being addressed, and how it will be covered in the module. 2 Overview. How e-strategies have addressed this theme to date. M&E tables. The M&E framework is applied to a selected number of the strategic interventions that are commonly undertaken under the theme considered. The matrices thus developed provide examples for users to develop similar matrices for areas that the module does not cover. The tables form the heart of each module.2 Assumptions and risks. Some key assumptions and risks that are commonly related to the theme are addressed, in particular regarding how they may affect e-strategies targets and ways to reach them. Toolkit Structure Module 1 Background and rationale Module 2 Methodology Sectoral Modules Module 3 Modules 4 & 5 · Rationale · M&E Tables (6 & 7) Framework Infrastructure E-government · Definition · Assumptions · Overview and risks and ICT sector · Rationale · Rationale · M&E Tables · Definition · Assumptions · E-readiness E-business · Overview and risks · Definition · Security · Overview · Rationale · M&E Tables · Digital Divide E-learning · Definition · Assumptions · M&E Tables · Overview and risks · Assumptions · Rationale and risks E-health · M&E Tables · Definition · Assumptions · Overview and risks Annexes Indicators and quantification This internal structure mirrors (a) the overall socio-economic and institutional context in which any e-strategy can be designed and implemented, and (b) the logical framework which is used by a majority of institutions involved in development projects and programs. Both are briefly described below. 2The tables focus on strategic priorities that relate to each thematic area. The choice of focus areas is driven by: (a) what is found in similar components of over 50 e-strategies reviewed by the authors (i.e. common strategic priorities); (b) the complexity of the M&E challenge (i.e. where possible, tables are developed for areas that are more challenging to monitor and evaluate than others). No formal value judgment is attached to the selection process. However, there is an implicit acceptance that such priorities are at least useful, and no tables are developed for initiatives that the authors think potentially ineffective. 3 II. A SIMPLE FRAMEWORK TO MONITOR AND ASSESS E-STRATEGIES An `e-strategy pyramid' The Logical Framework Pyramid Policy goals Strategic priorities Key initiatives Actions Resources Assumptions and risks Based on the `Logframe Handbook' developed in the World Bank, the M&E approach offered here refers to a rather simple way of considering the inter-relationship between policy, strategy, and implementation. The overall policy of a specific country will determine how and why themes such as `building an information society', or `implementing a national e-strategy' are priority objectives.3 Notwithstanding the reasons why a particular country may select such objectives, this toolkit considers that in any particular sector or area, policies, strategies, and implementation are the respective responses to three main questions: why ? what ? and how ? · Policy ­ Why · Strategy ­ What · Implementation plan ­ How 3Policy can be singular if it refers to the overall objectives of the country (e.g. `development policy'), or plural if it includes some fundamental subsets of objectives (e.g. trade policy, education policy, ). 4 The implementation level will itself be divided into two operational levels, namely `key initiatives' (how certain objectives will be implemented), and `actions' (which will be more specific to one area of responsibility ­ e.g. institutional or geographic). Moreover, the inputs and resources required to implement the e-strategy need to be addressed. These can be institutional structures, staff, or financial resources. A clear understanding of resource requirements is an important link between the strategy and its implementation, and forms the base upon which all elements of the strategy depend. Finally, a strategy may need to consider elements that are outside the focus of the strategy that will affect its implementation. Identifying the assumptions and risks on which the strategy is based is critical to setting parameters around which to measure its success or failure. It is also the only means by which to begin developing risk-mitigation measures and, where possible, incorporating these into the strategy itself. Those elements can be summarized graphically in the `Logical Framework Pyramid' presented at the beginning of this module.4 III. WHERE DOES M&E FIT ? M&E applies to all `layers' of the strategy pyramid introduced above. However, various levels of the pyramid will clearly require different types of indicators. For example, policy objectives, which will typically be longer-term and society-wide have traditionally been assessed in terms of `impact', i.e. in rather broad and largely unquantified ways. One of the ambitions of this toolkit is precisely to offer simple ways to attach indicators to such objectives.5 Strategic priorities have proved more amenable to quantification. However, such quantification has often remained limited to broad aggregates (e.g. percentage of the national population that has reached a certain level of ICT education); one will hence consider `outcomes', which will typically be society-wide indicators. By contrast, once one reaches the implementation layers of key initiatives and specific actions, indicators will be easier to design and use, referring respectively to outputs (e.g. number of computers installed in classrooms) and deliverables (e.g. so many computers installed and connected in so many schools in a certain region). 4See previous page. 5If a country happens to adopt a policy objective such as `to become a knowledge society within twenty years', or `to stimulate the growth of the national ICT sector', various strategic goals will need to be articulated to assess progress in achieving them. Such goals could include, respectively `provide primary education to 80 percent of a class age by a certain date', or `generate a certain percentage of national income through the ICT sector by a certain date', for example. 5 From LogFrame to M&E Indicators Policy goals Impact Strategic priorities Outcomes Key initiatives Outputs Actions Deliverables Resources Assumptions and risks Whatever the level of the e-strategy pyramid one may wish to consider, each and every one of the indicators selected is potentially a basis for an M&E component. However, for reasons of practicality and in order to account for local specificities, an efficient M&E approach will often have to be designed and implemented in a customized fashion. The thematic modules of this toolkit will provide practical ways to exercise such selectivity. However, it is important at this stage to examine in greater detail what each of the indicators mentioned in the diagram above (namely: impact, outcomes, outputs, and deliverables) might mean for the institutions and individuals involved in each of the respective levels of the e-strategy pyramid. This is the purpose of Module 2. 6 Module 2 A STRATEGIC APPROACH TO MONITORING AND EVALUATION As mentioned earlier, the way in which an M&E model and its indicators are incorporated in an e- strategy will influence its feasibility and hence its credibility and wider application. It is therefore important that the M&E system be designed in such a way as to make it comprehensible and usable by domestic participants (government, ministries, enterprises, and civil society) as well as external stakeholders (investors, donors, partners). If understood in that manner, M&E ceases to be a mere component of e-strategy. It becomes a powerful instrument to make such strategy more meaningful and convincing for those who will have the task to implement it and to support it. Designing the M&E component of an e-strategy hence requires that priority attention be given from the start to two of its main aspects, namely: · Methodological aspects, and · Institutional and strategic aspects The methodological context of M&E refers mainly to the ways in which it will offer relevant tools to monitor and evaluate progress made vis-à-vis the various levels of decision-making mentioned earlier (policy goals, strategic priorities, key initiatives) and to their expected results (impact, outcome, output). On the other hand, the institutional and strategic context of M&E will include the ways and means by which the M&E model can best be adapted to local constraints, and maximize ownership and `buy-in' from the various players/stakeholders involved. 7 I. METHODOLOGICAL CONTEXT I.1 - Policy Goals/Impact indicators At the top of the strategy pyramid is the policy or vision The indicators by which the policy that the strategy ultimately attempts to fulfill. The goals will be measured are generally indicators by which such policy goals will be measured are development-focused indicators and generally development-focused indicators, pertaining to not ICT-focused the country's economy and society as a whole, and not necessarily ICTs in particular. For example, a policy that seeks to grow the country's ICT sector to make it a leading growth factor in the economy may choose to measure this by GDP growth, total employment growth, or total productivity growth. These indicators are considered to be impact indicators. Impact indicators are often the most difficult to assess. This is due to two key factors: Time horizons. Impact indicators often only show change after a considerable time lag. This may be years after the e-strategy has been undertaken. Positive changes to these indicators may be of limited concern to the original formulators or implementers of the strategy, as they may have moved on to other duties by the time the impact of their efforts begins to show on the key indicators. Due to the time delay, such indicators are seldom monitored on a regular basis. Establishing causality. Changes in impact indicators are likely to be influenced by interventions undertaken as a part of the e-strategy, but almost certainly not as a result of these interventions alone. Therefore establishing causality between the e-strategy's interventions and the changes in an impact indicator is difficult - many other factors come into play, making it hard to establish whether or to what degree an ICT strategy or intervention is responsible for indicator change. For example, GDP growth is clearly driven by a vast array of factors, of which the ICT-producing sector may be just one. I.2 - Strategic Priorities/Outcome indicators Converting a policy or vision into tangible change on the ground requires choosing what initiatives to undertake and establishing goals for how far to advance in each particular initiative. Choosing what to do also implies undertaking the difficult task of choosing what not to do. Equally important, is the process of establishing how far to go in pursuit of any given objective. This requires establishing indicators to track achievement against core objectives that the strategy prioritizes. This requires clarifying what will be the tangible outcomes of the strategy. The decision of what to do (and not do), and how much of it to do lies at the heart of e-strategy formulation. 8 For example, if a country has selected the growth of its ICT sector as a policy goal, it will have to make choices among a number of possibly viable strategic priorities, such as: · Develop ICT infrastructure · Develop high-bandwidth technology parks · Encourage high-tech foreign direct investment (FDI) · Increase the stock of locally trained ICT professionals. For example, should the country choose to increase the stock of ICT professionals among its key strategic priorities, it will require a number of outcome indicators to assess its progress towards this objective. This may include the number of people graduating from tertiary and professional education institutions or the number of people employed by the high-tech sector. The last indicator could be segmented by domestic or foreign firms to provide meaningful data for FDI- related strategic objectives. Since the time required for change to be reflected in outcome indicators is likely to be shorter than for impact indicators, causality will be somewhat easier to determine. Though the stock of locally trained ICT professionals may only show an increase some time after the initiation of the strategy, such time should not be expected to be more than a few years. Therefore, outcome indicators are easier to monitor than impact indicators, as they are likely to show the results of the e-strategy intervention over a shorter time horizon. However, there is still a key role for evaluation, in assessing to what degree the interventions are responsible for a certain outcome, and in assessing what would have been the outcome had the intervention not occurred. I.3 - Key Initiatives/Output indicators To meet the strategic objectives outlined above, a number of Outputs or products that should be distinct initiatives can be undertaken. For each initiative, the qualified in terms of their number and key deliverable, or output indicator, should be specified in the level of quality strategic plan. For example, increasing the number of qualified ICT workers will require a variety of initiatives, all of which will generate outputs or products that should be qualified in terms of their number and level of quality. Qualification is achieved through the selection of key indicators that measure both the quantity and the quality of the deliverables or outputs. For example, increasing the stock of ICT professionals will require a number of interventions, each of which will create different outputs. These may include: · Improvement of capacity of ICT-focused learning institutions · Increase in demand for ICT education/training · Improvement of quality of ICT education at tertiary/vocational level. · Assessing success or failure of an initiative may require establishing measures for the capacity, demand, and quality of ICT-focused education6. 6Appropriate indicators will be both quantitative and qualitative. 9 The table below provides some examples: Output Quantitative indicator Qualitative indicator § Teachers of general and § Rating of graduates' capabilities Capacity of ICT-focused learning vocational schools trained in by private sector increases by X institutions improved, e.g.: basic ICT skills and use ICT in points - a measurable improvement teaching increases by X% § Rating of institutions by of teachers' qualification in § In-service training of managers in standards agency increases by X such institutions, the use of ICT in educational points - quantified support to settings increases by X% teachers responsible for § Training programs and materials introducing computers in for in-service training staff classes and curricula, designed and applied increases by - similar output for the X % business sector (on-the-job § Funding provisions to institutions training) increases by X% § Number of professional teaching staff increases by X% § Number of students graduating increases by X% § %ofgraduatesthatarewomen § Number of students applying to § Secondary curricula places greater Demand for ICT education/training technical institutions increases by emphasis on ICT-focused increased X% subjects § X number of partnerships formed § Tertiary/vocational curricula Quality of ICT education improved at with private sector includes market-leading tertiary/vocational level § X number of partnerships formed techniques and knowledge with foreign institutions § An established information § Distance education services, environment that provides a extend access to X number of range of support systems through students to a full curriculum use of ICT. § Xnumberofstudents graduating § Hot-line services established to with recognized certification from support teachers and advisors in accredited ICT-training their use of hardware and institutions software. Aside from measuring the quantity and quality of outputs, initiatives should also be assessed for how effectively they have been undertaken, both immediately after implementation is complete as well as during implementation. This will entail conducting periodic assessments of distinct initiatives. This will allow the implementation team to understand areas of comparative strength to build on further, and to incorporate them into other elements of the strategy. It will also allow the team to address areas of relative weakness, to make necessary adjustments, or even bring them to an early close. Mid-stream evaluation plays a key role in ensuring that the strategy is implemented well and resources are spent efficiently. It will ultimately help to ensure that the strategy meets its intended goals. 10 I.4 - Actions/interim deliverables indicators Details of each initiative in terms of its main stages, dependencies, and resources required for each stage of implementation should be included in the implementation plan. The strategy should present an overview of the actions involved in each initiative, as well as key milestones by which to gage their progress.7 At this layer, the indicators are the interim deliverables or sub-products that are generated by each key task of the initiative. They are closely tied to how the initiative is designed and the specific approaches selected. They serve as milestones against which to track the progress of the project through its various stages, with shorter completion timeframes than impact, outcome, or output indicators. For example, building the capacity of ICT learning institutions requires a number of interrelated activities. They may all comprise part of a single initiative, or a number of separate initiatives that coalesce to meet a larger capacity building objective. Depending on how the project is structured, some action or activity indicators could be: : · Assessment of capacity needs of higher educational and technical institutions. Assessments completed of X percent of institutions by month A · Program to provide grant funding to institutions established by month B, of amount of $X · X percent of grant facility funds disbursed to eligible institutions by month C · Recruitment criteria for staff completed. Staffing needs for X percent of institutions completed by month D Many of the initiatives undertaken as part of national e- strategies are related to creating institutions or building the Many of the initiatives undertaken as part of national e-strategies are related capacity of existing ones. For example, an ICT to creating institutions or building the infrastructure component of a strategy may focus on capacity of existing ones. establishing a regulatory agency to ensure an open and competitive telecommunications market. Monitoring and evaluating the success of institution building will focus on some of the key elements that starting and running a well-functioning organization requires. This can range from the development of a physical location for the institution, to whether it has been staffed, to the sustainability of its financing. I.5 - Resources/Input indicators The resources required to undertake these projects, and ultimately meet the strategic and policy objectives, should be specified in the strategy. These make up the project's inputs, or input indicators. Inputs take a variety of forms. They can be institutional structures, including the mechanisms required to implement initiatives or supervise the over-all strategy. They will include staff, oftentimes highly skilled professionals with expertise in ICTs as well as in the area of thematic focus (such as e-education or e-health). Financial resources are undoubtedly a key input. 7This is particularly important in the case of ICTs as many of the initiatives being proposed will be unfamiliar to policy-makers and reviewers of the e-strategy. Details of the actions required to implement the initiatives will make them more tangible and therefore comprehensible. 11 A clear understanding of required financing and, importantly, its source, is the link between the strategy and its implementation. It therefore forms a base on which all elements of the strategy depend. Assessing the outputs, outcomes, and ultimately the impact of a strategy must be conducted relative to the Financial resource requirements serve as a common language to provide level of resources that have been dedicated to the strategy. context for what may otherwise be a Clearly some countries have more resources to dedicate to number of unfamiliar activities their e-strategies than others. Therefore, assessing the outputs of an e-strategy, and therefore its success, cannot be done in absolute terms alone, but requires integrating the resources that have been dedicated to the strategy into the indicators themselves as a common denominator. For example, the performance measures of an ICT sector incubator may include the number of firms launched that are financially sustainable after a certain number of years. However, they should also assess how many financially sustainable businesses were launched for a given amount of money invested in the incubator. Definition of required resources also plays a role in facilitating communication with regard to the e-strategy. Many of the initiatives contained in the strategy will have little precedent to go by. Therefore, understanding the scale of the activities will not come naturally to a number of stakeholders. Financial resource requirements are the most basic means by which a variety of stakeholders will be able to understand the scale of the e-strategy and the activities it embodies. It can serve as a common language to provide the necessary context for what may otherwise be a number of unfamiliar activities. I.6 - Assumptions and Risks The development and implementation of an e-strategy is necessary to bring about effective ICT development across a range of sectors. But it is not sufficient in itself. There are a number of other factors on which ICT development is dependent, but over which both the formulators and implementers of the e-strategy have little control. Many of these factors relate to the political, economic, and social environment in which the strategy exists, that when combined with the outputs of the strategy lead to the outcomes and impact that the strategy intends. These environmental factors are often pre-requisites or assumptions that strategy-makers take for granted in developing their targets and goals. At the most general level, strategy-makers may assume that a country remains politically stable. Without political stability, the strategy is unlikely to attain its overall outcomes or impact, no matter to what degree it delivers on its outputs. A change in the assumptions on which the strategy is based A change in the assumptions on which necessitates re-evaluating the goals that the strategy sets. the strategy is based necessitates a This need not be negative. A strategy that focuses on ICT reevaluation of the goals that the sector development for export purposes may assume an strategy sets export market of a certain size, of which the country intends to generate revenues worth $Y million. However, should the market suddenly boom, the country may revise its revenue targets to say $Y million plus 20 percent. 12 While many assumptions on which a strategy is based are outside the control of the strategy, this is not always the case, particularly at the component level. For example, ICT sector development initiatives may be dependent on the advancement of e-government programs, based on the assumption that the government will be a major source of demand for locally developed ICT products and services. Reductions or delays in e-government initiatives will therefore adversely impact the development of the ICT sector. Similarly, the ICT sector is also dependent to a large extent on the establishment and enforcement of an intellectual property rights (IPR) regime to safeguard investments in knowledge intensive products. Creation of an IPR regime may be covered as part of a component of legislative reform. However, strategy-makers may choose to make the development and enforcement of an IPR regime an element of an ICT sector component, thereby wielding better control over the outcome of related initiatives and `internalizing' the risks associated with them. Incorporating activities on which the success of the strategy is dependent into the strategy itself is one way to mitigate risk. However, the ability to do this is usually limited. It is also for the most part inadvisable, as the strategy will become excessively fragmented as it strives to control the bearing of a wide array of factors, many of which have little to do with ICTs. Other risk mitigation measures, such as monitoring progress or change in certain key areas on which the success of the strategy is dependent may be all that can reasonably be done. PROTECTING INTELLECTUAL PROPERTY RIGHTS Many products developed by the ICT sector are `virtual.' A software program may have no more tangible representation than a collection of CD-ROMs on which it is stored. The substantial investments to develop such products must then be recouped through sales. However, thes e products are often easily copied at a fraction of the costs that the original developer incurs. Unless such products are protected from low-cost duplication, entrepreneurs will have little incentive to invest time and money in developing new products. The knowledge and effort that has gone into new product development must be protected from unauthorized duplication. Creation of an intellectual property rights (IPR) regime ­ a legal and regulatory framework to protect original developers' intellectual property ­ is therefore often a point of focus of national e-strategies that look to develop a local ICT sector. An IPR regime generally has two facets: legal and operational. Establishing the laws and regulation for IPR protection is comparatively less complex, as there is international precedent on which to base legislation. However, creating the machinery to ensure compliance with regulation is somewhat more challenging. Private investors (foreign and local) will require confidence that IPRs are being enforced before they engage in substantial ICT investment. Operationalizing IPR legislation will require assigning responsibility to an institution to investigate and prosecute cases of abuse. This will require knowledgeable staff as well as a dedicated budget. As with regulatory structures, it is vital that the legal framework and compliance institutions should be seen to be transparent, fair, and predictable in the eyes of international and domestic ICT sectors. Note: ICTs can also play an important role in facilitating the enforcement of IPR (and other laws) through the deployment of systems to manage court cases and related workflow. The implementation of a court filing system in the Slovak republic reduced the time between filing and first hearing from 73 to 27 days, and the number of procedures required from 23 to 5. Source : World Bank. 2004. Doing Business in 2004. World Bank. 13 II. INSTITUTIONAL AND STRATEGIC CONTEXT II.1 - M&E mechanisms and institutions All major initiatives pertaining to an e-strategy's key objectives require clear definition in the strategy. The Unambiguously stating implementation responsibility and resource requirements strategy should also specify which agencies will take lead in a strategy is an important means by responsibility for each project, and estimate the resources which to ensure that the projects actually required to complete the projects. Unambiguously stating get done ... The same applies to M&E implementation responsibility and resource requirements activities in a strategy is an important means by which to ensure that the projects actually get done. A lack of clarity on responsibility and budget reduces the chances of the strategy moving forward to the implementation phase. The same applies to M&E activities. An e-strategy should clearly define the roles, responsibilities and financing options for M&E. The choice of which institutions should take primary responsibility for the M&E effort will depend on (a) which `layer' of the strategy is being addressed, and (b) existing national M&E capacity. In general, as one moves down the strategy pyramid from the apex to the base, the location of the M&E capabilities should move closer to the agencies responsible for project implementation. In some cases there may be an existing agency that can take primary responsibility for M&E-related activities, while in others a team may have to be established for this purpose. Selecting which agency should take lead responsibility, or where to locate a new team, should be determined by striking a balance between ownership, access, and capacity. Ownership - M&E activities are conducted to inform and guide e-strategy decision-making and implementation processes. They also serve to encourage accountability and transparency of the processes of public office. Agencies responsible for making decisions and undertaking implementation should see the M&E information gathering and analysis as an integral component of what they do, and develop a sense of ownership for that component. Should M&E be conducted by an external agency, there is a risk that the agency will be seen as an external auditor. It may face resistance in so far as its ability to gather data and information and, worse still, parties responsible for implementation may not act on the M&E findings. The benefit of being able to make adjustments and improvements mid-stream through implementation will therefore be lost. Data access - The ability to conduct good M&E is dependent on access to data. Some data is available at a national level, and so gathering and analyzing it can be effectively conducted by a national organization. This would apply more commonly to M&E relating to policy and strategic objectives (impact and outcome data). Some of this data may reside with a National Statistical Office (NSO), or with a line ministry. For example, the NSO may have data on the growth of the ICT sector (a policy objective), while the ministry of education may have information on the number of locally trained ICT professionals (strategic objective). Lower down the pyramid, the relevant M&E data is more likely to reside with the project team that is responsible for implementation. 14 Responsibility for Pyramid Layer Objective Indicator gathering and analyzing M&E data Policy goals Grow the country's ICT § Total sector industry revenues § % contribution to NSO or Ministry of GDP growth Trade & Industry Strategic Increase stock of locally § Number of people priorities trained ICT graduating with ICT- professionals related qualifications § Number of people Ministry of Education employed in ICT sector Key initiatives Improve capacity of § Funding provisions ICT-focused learning to institutions institutions increased by X% § Number of profes- sional teaching staff Ministry of Education or increased by X% Project Team § Number of students graduating increased by X% Actions § Conduct capacity § Assessments needs assessment completed in X% of § Create grant institutions by month program A. § Establish staff § Established by recruitment criteria month B. X% of grant facility funds disbursed to eligible Project Team institutions by month C. § Staffing needs for X% of institutions completed by month D. Capacity leverage - An efficient means by which to conduct e-strategy M&E may be to use established M&E agencies and institutions such as National Statistical Offices (NSOs), leveraging their data gathering and analytical capacity. However, ICT is a comparatively new field, and thus there may be little existing data and limited applicable capacities8. 8Building the capacity of National Statistical Offices is clearly a priority in this area, and remains complementary to efforts made by certain organizations (such as ITU and the World Bank) to maintain worldwide databases on connectivity or ICTs for example. Such efforts will require significant financing; one way to optimize the use of the resources and knowledge available Is to enhance coordination and cooperation among the various agencies involved. This is precisely the purpose of the 'Partnership on Measuring ICT for Development', launched during UNCTAD XI (Sao Paulo, June 2004) by ITU, OECD, UNCTAD, UNESCO's Institute for Statistics, the UN Regional Commissions (UNECLAC, UNESCWA, UNESCAP, UNECA), the UN ICT Task Force and the World Bank. See http://measuring-ict.unctad.org. 15 II.2 - Where should the `M&E team' be located ? It is clear that the institutional location of the team that will eventually be responsible for formulating and discharging M&E responsibilities may have a significant impact on its ability to do so. On one hand, such a team should not be seen by the operational entities involved in the e- strategy as `a remote judge and censor'. On the other hand, if the team is too close to implementation tasks, it runs a distinct risk of becoming `judge and party', and losing credibility in the process. To perform its work efficiently, the `M&E team' will need to receive its legitimacy from the highest levels of government, i.e. above the level of specific ministers involved in the strategy.9 It will also need to exercise its responsibilities with the necessary levels of visibility and transparency. Whether this is performed through the establishment of a `special' centrally-located government unit, or through a more flexible network of individuals involved in various aspects of the formulation and implementation of the strategy will heavily depend on the pre-existing local institutional framework and work habits of government, business, and civil society. In any case, the efficiency and credibility of the M&E team will require that it base its work on the highest technical and methodological standards. Key Activities of an e-strategy M&E Agency Management · Develop a formal plan and business processes, including a budget and goals/targets for staff · Develop human resource management systems, assessing training needs of other agencies · Conduct regular training on ICT-related M&E · Develop internal communications and team building. Indicator development · Assess existing data sources and their relevance to the strategy and implementation plan · Recommend improvements in specific data series in terms of timeliness, coverage, or level of disaggregation · Develop and publish new data series · Ensure compliance with international standards for specific data items · Create new data products, for instance, presenting existing data in new ways, or including new types of analysis and discussion · Improve response rates for specific surveys. Outreach · Establish regular consultations between users and providers of statistical data · Establish processes to receive regular feedback from customers · Update statistical legislation · Establish links with the media. Source : Adapted from Achikbache, B., Belkindas, M., Eele, G., Swanson, E. "Strengthening Statistical Systems." PRSP Source Book. World Bank. 9An increasing number of countries has chosen to pursue a `CIO' approach, whereby a personality (often issued from the private sector) is given high visibility (and sometimes significant powers) to promote national e-strategies. 16 References Achikbache, B., Belkindas, M., Eele, G., Swanson, E. "Strengthening Statistical Systems." PRSP Source Book. World Bank. http://www.worldbank.org/poverty/strategies/sourcons.htm Association for Progressive Communications. 2003. "ICT Policy: A Beginner's Handbook." APC bridges.org. 2001. "Comparison of E-Readiness Assessment Models." bridges.org. http://www.bridges.org/ereadiness/ bridges.org. 2002. "E-readiness Assessment: Who is Doing What and Where." bridges.org. http://www.bridges.org/ereadiness/ Grace, J., Kenny, C. and Qiang, C. "Information and Communication Technologies and Broad-Based Development: A Partial Review of the Evidence." World Bank Working Paper No. 12. World Bank. http://poverty.worldbank.org/files/10214_ict.pdf Hanna, N. 2003. "Why National Strategies are Needed for ICT-Enabled Development." ISG Staff Working Papers. World Bank. Kenny, C. and Qiang, C. 2003. "ICT and Broad-Based Development." ICT and Development: Enabling the Information Society. World Bank. http://lnweb18.worldbank.org/ict/projects.nsf/WSISPublications#ictDev Lanvin, B. and Qiang, C., 2003. "Poverty `e-Readication': Using ICT to the Meet MDG." The Global Information Technology Report 2003-2004. Oxford. http://www.weforum.org/site/homepublic.nsf/Content/Global+Competitiveness+Programme%5CGloba l+Information+Technology+Report NORAD. 1999. The Logical Framework Approach. http://www.baltichealth.org/customers/baltic/lfa/LFA%20handbook.htm Nordic-Dutch Trade Union Centers. 2003. "Handbook of Participatory Project Planning." http://www.fnv.nl/download.do/id/100001160/ Prennushi, G., Rubio, G., Subbarao, K., "Monitoring and Evaluation." PRSP Source Book. World Bank. http://www.worldbank.org/poverty/strategies/sourcons.htm Qiang, C., Pitt, A., with Ayers, S. 2003. "Contribution of Information and Communication Technologies to Growth." World Bank Working Paper No. 24. World Bank. http://lnweb18.worldbank.org/ict/projects.nsf/WSISPublications Schware, R. 2003. "Information and Communications Technology (ICT) Agencies: Functions, Structures, and Best Operational Practices." Emerald. http://hermia.emeraldinsight.com/vl=12696689/cl=14/nw=1/fm=docpdf/rpsv/cw/mcb/14636697/v5n 3/s1/p3 World Bank. The LogFrame Handbook. World Bank. World Bank. 2002. ICT Sector Strategy Paper. World Bank. http://info.worldbank.org/ict/ICT_ssp.html 17 Module 3 BUILDING AN E-STRATEGY ON A SOUND FOUNDATION I. E-STRATEGIES VS D-STRATEGIES As with any complex socio-economic endeavor, the success of e-strategies will rely heavily on the sequence of actions to be undertaken, and activities to be organized. The previous two modules pointed to the fact that M&E concerns should be part of the early stages of an e-strategy, and that the relevant methodological and institutional choices must be at the core of such concerns. It is equally important to keep in mind that an e-strategy, however far-reaching and broad-ranging, cannot be a substitute for a development strategy (d-strategy). From an `M&E' point of view, this has important practical consequences, including the following: · Formulation Some indicators (especially regarding `Impact') will have to be formulated at a level of decision making which is higher than that of e-strategy (namely that of the country's overall development strategy ­ d-strategy) or even that of its socio-economic policy; the designers and promoters of e-strategies should hence refrain from `re-inventing the wheel' when such M&E indicators already exist, and focus on making them a fully integrated component of their own efforts. · Linkage The M&E indicators should be related to each other in a way that reflects the sequencing of objectives at the various levels of decision making For instance, if an e-strategy includes initiatives regarding distance education, it will be important that such activities (and their outputs) be connected not only to broader e-strategy objectives (such as promoting e-literacy or enhancing the use of ICT in education), but also to `d-strategy' objectives (e.g. developing ICT usage in general), and more generic policy objectives (which could be in this case a diversification of the economy from traditional sectors into newer ones). 18 This is illustrated in the following diagram: Policy Decision level -Fight analphabetism D- Strategy -Diversify economy -Eradicate malaria -Reconcile -Develop tourism E- Strategy communities -Develop fisheries -... -Develop ICTs -Promote regional -Attract FDI to ICT customs union sector (infrastructure) Initiatives -... -Promote e-literacy -Stimulate local entrepreneurship in -PCs in primary Actions ICT (SMEs) schools -Provide adequate -Adapt curricula regulatory -Train teachers -Identification of environment - Establish exams one `pilot school' in -... and standard tests district 29 - ... -5 PCs delivered to school 29-3 -1 diesel power supply installed in school 29-3 - ... Contribution # of ICT products & # of certified graduates Ratio PC/pupils # of public schools of ICT to GDP services companies in e-literacy courses in public schools connected & running Typical M&E indicators/ time horizon II. E-STRATEGIES VS INFORMATION SOCIETIES Over the last few years, an increasing amount of international effort has been devoted to the building of information societies.10 Still, evidence shows that there remains a limited awareness about the potential role of ICT in the fight against poverty.11 Even in the significant and cross-sectoral intellectual effort mobilized behind the objectives of the Millennium Declaration, ICTs appear largely as a second thought, and a relatively minor tool to reach the Millennium Development Goals (MDGs). In spite of the myriad of findings regarding ICT projects in the field, such evidence has not yet been aggregated or scaled up in a way that would easily convince decision makers at the policy level. To a large extent, the case for ICT for development (ICT4D) still needs to be made12. M&E indicators and processes have a crucial role to play in this respect. This role however will not be fully realized unless the following objectives are clearly recognized as priorities, both at the national level and international level: 10Chief among those are the European Union's 'e-Europe Initiative', the G-8 DOT Force (Digital Opportunity Task Force), the United Nations' ICT Task Force (UNICTTF), and the whole process of the World Summit on Information Society (WSIS). 11 Recently, a survey carried out by OECD/DAC underlined the remarkably small proportion of PRSPs (Poverty Reduction Strategy Papers) mentioning ICTs. See `Role of Infrastructure in Economic Growth and Poverty Reduction ­ Lessons learned from PRSPs of 33 countries', www.oecd.org/dataoecd/57/60/33919674.pdf. 12See for example K. McNamara "ICT for Development: What Works & What Does Not" (infoDev, 2003). 19 M&E Integration. When selecting e-strategy M&E tools and indicators, the designers and promoters of e-strategies should make them as compatible as possible with existing objectives and targets regarding development in general, and the building of information societies. This may include a search for homogeneity in terminology, or more importantly, possibilities to establish causal linkages between objectives and indicators; this will enhance their ability to receive international support for their efforts, and to benefit from existing or emerging best practices in the field.13 Indicator Quality. Every effort to enhance the quality, coverage and detail of ICT and e-economy indicators should be pursued. Major gaps currently exist for data regarding applications and usage, i.e. about all indicators beyond physical measurement of tele-density, connectivity, equipment or information traffic flows. In most cases, the first step will consist of strengthening local statistical and data collection capacities.14 M&E Compatibility. As underlined earlier, M&E instruments attached to e-strategies should me made as compatible as possible with those existing for `traditional areas of d-strategies'. This will be necessary to (a) achieve consistency in pursuing overall national policy objectives, and (b) obtain the support and `buy in' or those parts of government and civil society who might otherwise see e- strategies as a fad or a distraction from other, more fundamental development objectives. WSIS MDGs Renewed `IS' context (Information Society approach) (Declaration and (Goals and specific Plan of Action) targets under Goal 8) New `E' context Is ICT Evidence from important YES ICT4D experience for the success E-STRATEGY DOT Force, etc .. of the country's D-Strategy ? Changes (PRSP, CAS, ..) Monitoring NO and Evaluation M&E Traditional `D' context Development Strategy OUTCOME D-STRATEGY (e-readiness, connectivity, (PRSP, CAS, ..) usage .. etc..) Monitoring and Evaluation M&E E-strategies and Development : OUTCOME from D to E to IS (growth, competitiveness, poverty, health, education, etc ..) 13Not to mention the significant impact that it would have on countries' abilities to benchmark their efforts vis-a-vis those of other economies. 14As underlined in the previous module, capacity building at the local/national level is a priority; considering the expected cost of such efforts, coordination at the international level (such as the one advocated by members of the 'Partnership on Measuring ICT for Development') is likely to receive Increasing attention. 20 I. E-READINESS VS ACCESS AND USAGE The M&E component of e-strategy should reflect the fact that ICT is only a tool and not an end for development. In other words, the number of telephone lines, personal computers, or even Internet hosts available in one country are not the ultimate indicators to be used to assess whether or not an e-strategy has been successful. On the other hand, while the economic and social value that people will derive from a greater use of ICTs is clearly a much better indicator of such success, it is also much more complex to measure, monitor, and evaluate. If it is clear that usage is a better indicator than access, it remains equally obvious that there will be no usage if there is no access. Moreover, both access and usage will depend heavily on the legal, regulatory, and other economic and social frameworks within which information and information technology can be accessed and used, and on whether government, business, schools, and individuals are interested and able to access and use them. Such elements are generally understood as being part of `e-readiness..' E-Readiness Most business strategies begin with a review or assessment of the current state of business. They focus on key elements of the business ­ such as its customer base, its operations, and its product line ­ and describe where the business stands with regard to each of these areas, what it has achieved in the recent past, and highlights areas of relative strength, weakness, and opportunity. The assessment of how well (or badly) things are working now for a business drives the degree of change that will be required in the future. A similar approach is required for the development of national e-strategies. Understanding where the country stands with regard to key elements of its ICT development agenda must form the base from which a national e-strategy is developed. Fortunately, this has already occurred in a number of countries. E-readiness assessments have been conducted in over 137 countries.15 E-readiness assessments are central to the ability to formulate e-strategies, in two key ways: § What to do. E-readiness assessments provide the basic information from which to determine the themes or sectors on which to focus the country's e-strategy. They provide information on where a country has made good progress and help to identify areas of continuing weakness. Oftentimes this is done through comparison to other similar countries, so as to provide context in which to understand the country's current position. § How much of it to do. E-readiness assessments also facilitate the process by which a country develops targets on how far to go in pursuit of each key objective (i.e. once it has been decided what to do, how much of it to do). They provide data regarding the current level of ICT development for a specified country, baseline data against which the progress of the e-strategy can be measured. 15One can even consider such efforts as excessive (or at least redundant) in a certain number of cases: 55 countries have been assessed for e-readiness at least 5 times, and 10 countries at least 8 times. See http://www.bridges.org/ereadiness/where.html 21 Country comparisons play an important role in selecting strategic priorities and establishing growth targets. If a comparator country is considerably `more ready' in an area of importance, for example ICT infrastructure, then strategists may choose to put more emphasis on this area. It is also on the basis of such a comparison that reasonable estimates of growth targets can be established. If a comparator country previously grew its infrastructure, measured by tele-density for example, at a rate of X percent per year, strategists may chose to establish the same (or a slightly more ambitious) target for their own ICT infrastructure development. Depending on the specific context of a particular country, e-readiness may also give different emphasis to issues regarding, for instance, security and privacy, consumer protection, or `digital divide' issues.16 Access and Usage Access (both physical and economic) is only one dimension of possible `digital divides.' It is whether or not the connectivity and equipment provided (to businesses, local governments, schools, hospitals, community access points, or individuals) will be actually used in a productive and sustainable fashion that will determine how ICTs actually contribute to local and international development objectives. Beyond access and usage, it will also be important to assess the economic and social value that are derived from e-strategies. Because it has much more to do with `outcomes' than with `outputs,' and because it requires reference to pre-existing values in a society, this is an exercise that goes far beyond the scope of this toolkit. 16`Digital divide issues' refer here to disparities among various groups of the national population, between urban and rural areas for example. 22 Summary Readiness, access, and usage constitute the three layers against which the chances of success of an e-strategy can be rated. It will hence be important, from an M&E point of view, to link indicators (whether they concern impact, outcome, or specific deliverables) to at least one component of the following three sets of elements: · Readiness o Legal, regulatory and overall institutional framework (rule of law, IPR regimes, trade and investment openness, regulatory framework, competition framework, etc ..) o Society's support (at all levels) for innovation, reform, and ICT o Human resources (education in general, e-literacy in particular, ..) o Perceptions about security/privacy17 o Digital divide issues (e.g. rural/urban) · Availability/Access o Infrastructure (e.g. telecom) and network penetration o Equipment (computers in business, administrations, schools, homes, ..) · Usage o Applications (e.g. e-government, e-business, e-education, e-health) o Specific usage modalities (e.g. community access points) o Specific sectoral or policy objectives (e.g. export competitiveness) Moreover, the more the indicators chosen under each of those three sets are compatible with internationally agreed objectives and targets (such as those attached to the MDGs), the easier it will be to generate international support for particular e-strategies, benchmark national efforts vis-à-vis those of other countries, and encourage foreign direct investment. 17Security concerns have received increasing attention in the recent past. They are no longer restricted to digital signature, encryption, consumer protection, or intellectual property issues. Topics such as cyber-crime, identity theft, phishing or spam are progressively finding their way into e-strategies. E-security is hence expected to be addressed as a separate item in future editions of this toolkit. 23 e-Strategies and MDGs In order to reach the targets set by the MDGs, countries can either increase the resources they allocate to specific objectives, or increase the efficiency with which they use their available resources. At the core of the discussion about ICT and MDG is the question of whether ICT can contribute to improving efficiency in delivering the MDGs. This way of representing ICT investment as a source of increased efficiency in pursuing MDGs 2 to 7 (and hence MDG 1) opens a number of policy and strategic avenues. Additional efforts will be required from statisticians and econometricians to help quantify the relevant elasticities and dynamic linkages between multiple variables, which may vary from one country to another. International efforts are hence required to collect relevant data at the local, regional, and global levels. Excerpt from Lanvin, B. and Qiang, C. 2003. `Poverty "e-Readication" - Using ICT to Meet MDG:', in `Global Information Technology Report, 2003-2004.' INSEAD-WEF-infoDev, 2003. 24 Module 4 ICT INFRASTRUCTURE AND ICT SECTOR I. RATIONALE: WHY FOCUS ON ICT INFRASTRUCTURE? Widely accessible, affordable, and reliable ICT infrastructure must form the foundation of any ICT development strategy. Without ubiquitous infrastructure, Without ubiquitous infrastructure, the benefits of ICTs will the benefits of ICTs will accrue to the few people who have access accrue to only the few people who have access to to communications networks, most communications networks, most often the better off. often the better off Therefore, the majority of national e-strategies entails elements that focus on developing a country's ICT infrastructure, in particular on broadening access. `Access' is conceived of as not only increasing geographic coverage, but also addressing issues of affordability. Developing countries vary considerably with regard to the state of their regulatory environment and the level of competition in their ICT infrastructure market. Some developing countries have yet to privatize their incumbent operators and open their mobile markets to competition. Others have begun this process but have yet to complete it, oftentimes introducing competition in mobile telephony while leaving the fixed line market state-owned. This module addresses monitoring and evaluation across the spectrum of infrastructure development interventions, beginning with increasing competition in ICT infrastructure delivery, licensing private operators and service providers, and the privatization of state operators. It then addresses the establishment of a regulatory framework and mechanisms with which to ensure compliance. It ends by looking at means to promote universal access. Definition: What is ICT Infrastructure?18 Information and communication infrastructure refers to the telecommunication and information networks through which information is transmitted, stored, and delivered.19 Networks may be independent, or interconnected and interoperable. They are "public" or "open" if they can transmit information from any source, or "private" or "closed" if they are restricted to members of a closed user group. Providers of ICT infrastructure services include the operators of 18This definition is taken from the World Bank's ICT Sector Strategy Paper. http://info.worldbank.org/ict/ICT_ssp.html 19Types of networks include cellular, data, broadband, backbone, satellite, broadcasting, the Internet, and other networks; they may be wireline, wireless, or a combination of both. Network components may include terrestrial wires, undersea cables, radio waves, satellites, towers, base stations, equipment (transmitters, repeaters, switches, routers), and related hardware and software. 25 the various networks, as well as specialized network services such as Internet service providers (ISPs), web hosting companies, and data centers. This module focuses on public or open networks, and primarily on telecommunications and Internet service provision. Overview of ICT Infrastructure Development Strategies The vast majority of e-strategies focus on increasing ICT infrastructure access, primarily through increased competition and private participation.20 This is based on the assumption that ICT infrastructure development is more effectively undertaken by the private sector. Driven by incentives to increase profits, competing private firms prove to be effective engines of ICT network expansion, continuously striving to reach new customers and broaden the range of affordable services offered to existing customers. This requires creating laws to allow for private investment (often including foreign investment) and issuing licenses for the entry of multiple network operators and service providers. ICT infrastructure components of e-strategies often also focus on increasing private participation in existing state-owned enterprises through privatization. Attracting private investment into the ICT infrastructure market requires not only opening the sector, but also creating a regulatory environment that ensures that all service providers are treated equally. This includes making certain that advantages enjoyed by incumbent operators are made available to new entrants into the market. One of the most important levers to this effect is the creation of an interconnection regime that allows new operators to connect to existing operators' networks and share inter-network revenue. Private participation alone will not ensure that all parts of a country are provided access to ICT infrastructure. Some Poor areas or hard to reach regions are generally difficult to serve parts, such as poor areas or hard to reach regions, are profitably and so tend to be left generally difficult to serve profitably and tend to be left unserved by the private sector unserved by the private sector. State telecommunications operators, as public entities, would usually subsidize service provision to non-profitable segments using revenues from more profitable segments or services. To ensure that this continues once telecommunications operators are privatized, e-strategies often advocate establishing universal service obligations together with public funds to support such a commitment to universal service provision. 20ICT infrastructure development is a major focal point in almost 70 percent of over 20 e-strategies reviewed for this toolkit. 26 II. M&E FRAMEWORKS Increase private sector-led competition - Increased private sector-led competition in the ICT infrastructure sector lies at the heart of most e-strategies' plans for infrastructure. This usually entails encouraging the entry of new service providers to compete against incumbents (primarily through the issue of licenses) as well as increasing private ownership of existing incumbents through privatization. Policy goals Develop access to ICT infrastructure Strategic priorities Increase private-sector Establish Undertake universal Key initiatives -led competition regulatory framework Access schemes Actions License private Privatize providers incumbent Resources Assumptions and risks License private providers - Increased competition in the ICT infrastructure sector often requires unbundling services that may have previously been provided by a single operator (usually the state- owned incumbent). This allows private firms to serve limited segments of the larger ICT infrastructure market, such as international long-distance traffic or the retailing of last-mile services to the end consumer. However, unbundling requires an interconnection regime to ensure that different operators can reasonably connect to one another's networks. Approaches to monitoring and evaluation during this process focus mainly on (a) whether an environment is in place that allows for multiple operators and service providers to serve the market, and (b) whether multiple providers are actually doing so. In terms of the e-strategy pyramid, licensing is considered a strategic priority for which key initiatives and actions can be undertaken; sample indicators and data sources are included in the following diagram. Privatize state-owned assets - Reforms to introduce new operators and service providers in the ICT infrastructure market are common in many countries. Mobile licenses have been issued to private firms other than the incumbent operator in many developing and emerging markets. However, privatization of the incumbent is lagging ­ only 15 percent of low-income countries have fully privatized their fixed local loop markets. 27 Therefore, privatization of the incumbent telecommunications operator is becoming a common component of e-strategies. Aside from bringing improvements in telephone coverage, reduced costs of calls, and more reliable services, it is also expected that privatization will lead to the development of a more efficient telecommunications operator. This will be reflected in lower call prices as cost savings are passed on to consumers, as well as a higher lines-to-worker ratio. A more profitable and growing fixed line operator will also yield higher taxation revenues for the government. Pyramid Layer Objective Indicator Data source Policy goals Develop access to ICT · Total teledensity Ministry of infrastructure · Wait time for line Information and · Cost of a local call Communications · Cost of Internet service Strategic Increase private sector-led · % of equity owned by Ministry of priorities competition private investors a Information and · Average lines per worker a Communications · Taxation revenue from sector ($ and % of GDP) a · Assessment of sector openness (from ITU) Key initiatives Create licensing · License/registration Ministry of environment available Information and · Number of licenses issued a Communications · Number of enterprises or Project Team operational a · License revenues ($ - initial and ongoing)a Actions · Develop policy for · Assessment conducted by Project Team license issuance month A · Establish · Policy adopted by month B interconnection · Interconnection regime regime created by month C · Issue licenses · Tender documents complete by month D · First licenses issued by month E (a) Disaggregated by type of service, such as international long-distance, local calling, Internet service, mobile, ISDN, etc. Categories will differ based on sector policy and how licenses are bundled/sold. 28 Still, privatization is a complex process in which achievements and success occur incrementally and usually slowly. Indicators that monitor project progress at the level of key actions are therefore critical to ensure that (or at least assess whether) the process is moving forward. The table below presents some possible means by which to monitor and evaluate this challenge. Pyramid `Layer' Objective Indicator Data source Policy goals Develop access to ICT · Total teledensity Ministry of infrastructure · Wait time for line Information and · Cost of a local call Communications · Cost of Internet service Strategic priorities Increase private sector-led · % of equity owned by Ministry of competition private investors b Information and · Average lines per worker b Communications · Taxation revenue from sector ($ and % of GDP) b · Assessment of sector openness (from ITU) Key initiatives · Privatization of fixed · % of equity government- Ministry of line operator owned Information and · $ raised through sale Communications or · Public participation in sale Project Team (% of equity that is publicly traded) · Taxation revenue of privatized enterprise(s) ($, years 1-5) · Sale complete by month F Actions § Conduct valuation of § Contract awarded for Project Team incumbent privatization assistance by § Prepare for month A (financial and privatization, operational) including organization § Restructuring analysis reengineering complete by month B § Conduct sale § Organizational change transaction initiatives complete by § Undertake awareness- month C raising campaign § Tender documents complete by month D § Public awareness-raising campaign begun by month E § Sale complete bymonth F (b) Disaggregated by type of service, such as international long-distance, national long-distance, local calling, Internet service, mobile. Categories will differ based on how licenses are bundled/sold. 29 Establish a regulatory framework - Creating an environment to attract private investment in ICT infrastructure requires (a) the establishment of laws to sanction and govern private investment in infrastructure, and (b) the establishment of a non-political and independent regulatory authority to ensure the laws and regulation are enforced. Policy goals Strategic priorities Develop access to ICT infrastructure Key initiatives Actions Increase private-sector Establish Undertake universal Resources -led competition regulatory framework Access schemes Assumptions and risks Develop Establish legal framework regulatory authority There is considerable precedent on which to base a country's legal code that governs the private telecommunications sector. Most countries have telecommunications acts or laws, and there are also guidelines established by the World Trade Organization (WTO) that specify provisions that member states should incorporate into national legislation. However, equally important as a comprehensive legal code is the means by which to ensure its compliance. This requires establishing a technically competent regulatory authority, one that is independent of both operators and the political process.21 Monitoring and evaluating the strength and independence of the regulator will involve analysis of its degree of independence (in terms of its relationship with the regulated firms, consumers, and political authorities), financing sources, and accountability (in terms of the transparency of the decision-making processes). It may also include assessment of dispute resolution channels and the role of the regulator in resolution. 21It is worth noting here that the regulator is also responsible for managing radio spectrum, which is of increasing importance with the explosive growth of mobile technologies. 30 Pyramid Layer Objective Indicator Data source Policy goals Develop access to ICT § Total teledensity Ministry of infrastructure § Wait time for line Information and § Cost of a local call Communications § Cost of Internet service Strategic Establish regulatory regime§ Assessment of sector Ministry of priorities openness (from ITU) Information and § Number of independent Communications operators Key initiatives Create an independent § Independent source of Ministry of regulatory authority financing (Y/N) Information and § Main sources of financing Communications (segmented by % of total) or Project Team § Responsibility for (Y/N): § licensing otariff setting ospectrum allocation odispute resolution ouniversal service Actions § Staff regulatory § TORs established for all Project Team authority staff § Purchase spectrum § % of staff positions filled management by month A equipment § Business plan completed § Launch regulatory for authority by month B authority § Spectrum management § Draft licensing equipment operational by procedures month C § Draft tariff setting § All draft procedures procedures complete by month D § Draft inter-connection § All approved procedures procedures (including by month E asymmetric interconnection) § Draft spectrum allocation procedures § Draft dispute resolution procedures § Draft universal service obligation procedures Develop universal access - Increased private participation in the telecommunications sector, combined with technological There will always be some parts of a country that will not be advances, has broadened the reach and lowered the cost of served if left to the market alone telephone and Internet access in virtually all parts of the world. However, there will always be some parts of a country that will not be served if left to the market alone. These may be out of the way and sparsely populated areas that are costly to reach and offer low volume traffic, or less well-to-do communities that cannot afford to pay for telephone or Internet service. Often the two go hand-in-hand, with rural communities being both costly to 31 reach yet with limited capacity to pay. E-strategies therefore often focus elements of their ICT infrastructure components on service provision to these segments of society. To do so, some countries have opted to align the pricing of telecommunications services with the true cost of provision. Users are charged more for calling a high-cost rural area than a lower-cost urban area, with a higher portion of the call revenue going to the provider with the higher-costs (referred to as asymmetric interconnection).22 This spreads the cost of service provision to high- cost areas more equitably among service providers. Asymmetric interconnection schemes may also be combined with Asymmetric interconnection the establishment of a fund to subsidize the cost of universal schemes may also be combined access, financed either directly from the government budget or with the establishment of a fund from mandatory contributions by telecommunications operators to subsidize the cost of universal access (often between 1 and 2 percent of sector revenue).23 Chile pioneered an innovative approach in the early 1990s, requiring telecommunications firms that were interested in providing rural access to competitively bid for the government subsidy. The firm requesting the lowest subsidy to meet the tender requirements (such as the provision of public pay phones within a specified distance from all households in a certain area) won the bid. Policy goals Develop access to ICT infrastructure Strategic priorities Key initiatives Increase private-sector Establish Undertake universal -led competition regulatory framework Access schemes Actions Resources Establish and Develop OBA Finance Fund schemes Assumptions and risks There are two key elements of providing universal service through such schemes (referred to as reverse subsidy or output-based aid (OBA) schemes). The first entails establishing a fund; the second a mechanism through which to disburse it. 22None of the reviewed e-strategies emphasized asymmetric interconnection. Those that address universal access mainly focus on the need to establish a fund. 23How such funds are used for rural access varies (they often cover part of network development cost, but not operation). 32 As mentioned above, universal access funds are usually capitalized through universal service obligation levies on telecommunications operators, with the government often providing the initial start-up or seed capital. How the funds are disbursed is often more complex. The table below focuses on OBA-based approaches. It is based on an assumption that the scheme addresses both public telephone and Internet access. Pyramid Layer Objective Indicator Data source Policy goals Develop access to ICT · Total teledensity Ministry of infrastructure · Wait time for line Information and · Cost of a local call Communications · Cost of Internet service Strategic Develop universal access · % population with no Ministry of priorities mechanism telephone access c Information and · % population with no Communications internet access c and Project Team · % population not under mobile signal · Households with telephone · Households with internet · Public subsidy-to-private investment ratio Key initiatives Develop OBA scheme · Public subsidy-to-private Ministry of investment ratio Information and · Number of households Communications reached per $ of subsidy or Project Team (telephones) · Number of households reached per $ of subsidy (internet) · Cost per public payphone · Cost per public Internet access point Actions · Choose locality · Assessment report on Project Team · Issue call for target locality complete by proposals month A · Draft asymmetric · Official call for proposals interconnection by month B procedures · Assessment complete by · Award concession month C · Deliver service · Award by month D · Beginning of service provision by month E · Service to complete locality completed by month F · Service level assessment yearly (years 1-5) (c) Based on a pre-established definition of `access' (for example, public access within 5 kilometers of every household). 33 III. ASSUMPTIONS AND RISKS Private investment in ICT networks came to a virtual standstill after the dotcom crash. Though it initially drew less attention than the rise and fall of the dotcoms, the telecommunications sector underwent a more dramatic change in fortunes. When the telecom bubble eventually burst in 2000, investors lost approximately $1 trillion worldwide ­ about three times more than what was lost in dotcom investments. Following this, interest in telecom investments has been low, and has only recently started to show some recovery. Some countries, despite the best of intentions, have been unable to attract sufficient private competition and investment into their ICT infrastructure sectors. This demonstrates the degree to which the success of strategies to increase private investment in ICT infrastructure is dependent on investor appetite. Components of e-strategies that deal with ICT infrastructure sector development are therefore subject to substantial risk; however, they are also a considerable source of risk. Virtually every element of an e-strategy is based on the assumption that reasonable access to ICT infrastructure exists, at a suitable level of quality. Where access levels are low, other components are often developed based on assumptions that ICT access will grow substantially over the period that these components are being implemented. This therefore places considerable importance on `getting right' the components of e-strategies that focus on ICT infrastructure. Failure to meet access targets will lead to lower-than-expected achievement across most other components of the e- strategy. Summary The networks that carry voice and data traffic are the foundations of e-strategies. Therefore, progress on components that focus on increased access ­ defined both in terms of reach and affordability ­ must be carefully monitored and evaluated, as it is upon such components that all other elements of an e-strategy depend. Interventions to increase ICT access focus on increasing the role of the private sector in building ICT infrastructure and delivering related services. This requires increasing competition and private investment through issuing licenses and privatizing incumbents. It requires establishing a pro- competitive regulatory framework and an independent authority to ensure the compliance of operators. It also entails establishing means with which to extend access to areas of the country that would otherwise not be served if left to market forces alone. 34 Case: Planning for Infrastructure Development in Mozambique Mozambique's e-strategy focuses on seven key areas: human resource development, content and applications, governance, policy and regulation, enterprise development, provincial development, and ICT infrastructure. The infrastructure component comprises over 50 percent of the total planned expenditure of $280 million. The ICT infrastructure component of the strategy lays out in detail the key initiatives that will be undertaken to develop the country's infrastructure. The main focus of these initiatives is on modernizing, and increasing the coverage, of the national telecommunications network, and promoting universal access to telecommunication and Internet services. For each initiative it specifies: § Prioritization relative to other initiatives in terms of long, medium, or short term § Main outputs and deliverables § Implementation timeframe § Implementing partners § Estimated budget § Whether finances for the initiative have been secured, and if not, the level of the financing gap. For example, an initiative to set-up Digital Agencies, where the public can access the Internet and other ICT-related services, establishes a target of 36 agencies covering 23 districts. The strategy details a three-phase rollout, covering incrementally more regions over an 18-month period. The strategy estimates a budget of $1 million for the initiative (which is fully funded) and specifies the national telecommunications service provider, Telecommunications Mozambique, as responsible for implementation. Source : Mozambique Commission for Information and Communication Technology Policy. "Draft Policy for Information and Communication Technologies; Toward the Global Information Society." 2000. http://www.uneca.org/aisi/nici/Mozambique/mozambique.htm 35 References Cannock, G. 2001. "Telecom Subsidies." Public Policy for the Private Sector, Note No. 234. World Bank. http://rru.worldbank.org/PublicPolicyJournal/Summary.aspx?id=234 Dymond, A. "Telecommunications Challenges in Developing Countries: Asymmetric Interconnection Charges for Rural Areas." World Bank Working Paper No. 27 2003. World Bank. The Economist. October 2003. "A Survey of Telecoms." The Economist. http://www.economist.com/displaystory.cfm?story_id=2098913 Fink, C., Mattoo, A. and Rathindran, R. 2002. "An Assessment of Telecommunications Reform in Developing Countires." World Bank Policy Research Paper No. 2909. World Bank. http://econ.worldbank.org/view.php?type=5&id=20745 Intven, H., Oliver, J., Sepulveda, E. 2000. Telecommunications Regulation Handbook. infoDev. http://www.infodev.org/projects/314regulationhandbook/ ITU. 2003. Trends in Telecommunications Reform 2003: Promoting Universal Access to ICTs. ITU. http://www.itu.int/publications/docs/trends2003.html ITU. 2003. World Telecommunication Development Report: Access Indicators for the Information Society. ITU. Web: http://www.itu.int/ITU-D/ict/publications/wtdr_03/ Wellenius, B. 2000. "Extending Telecommunications Beyond the Market." Public Policy for the Private Sector, Note No. 206. World Bank http://rru.worldbank.org/PublicPolicyJournal/Summary.aspx?id=206 World Bank. 2003. ICT and Development: Enabling the Information Society. World Bank. http://lnweb18.worldbank.org/ict/projects.nsf/WSISPublications World Bank. Draft paper. 2004. "Recent Economic Developments in Infrastructure." 36 Module 5 ICT SECTOR I. RATIONALE: WHY FOCUS ON THE ICT SECTOR? The second most common focus area in ICT strategies after e-government is development of a domestic ICT sector.24 Recently, interest has been driven by substantial opportunities offered by the outsourcing industry (often offshore), that has been growing by some estimates at 30 to 40 percent per annum.25 This industry caters mainly to large corporations from developed economies, and includes the development and delivery of ICT services, such as software development or IT applications management, as well as ICT-enabled services, such as call centers or back-office accounting functions.26 However, opportunities offered by outsourcing-related services are not the only driver of interest in ICT sector development. The promotion of local ICT sector growth is recognized as a means to develop locally appropriate ICT products and services; this stimulates ICT demand and usage, in both the public and private sectors. A dynamic and growing local ICT sector is also a source of employment, providing opportunities for highly skilled people (and thus stemming the `brain drain' as local talent finds rewarding opportunities within the domestic market). This module looks at means by which to monitor and evaluate ICT Sector strategy focus: interventions to stimulate the development of a local ICT sector, § Opportunitiesforout- focusing on small and medium-sized enterprises (SMEs). Three sourcing and exports main areas are covered: streamlining business regulation, improving § Meetingdomestic access to capital, and business incubation. demand for ICTs 24ICT sector development is a major focal point in the majority of over 20 e-strategies reviewed for this toolkit. 25The McKinsey Quarterly. "On Outsourcing and Offshoring." January 2004. 26Outsourcing is certainly not the only reason why foreign investors would be interested in funding local ICT activities. It should be noted however that activities undertaken to create an environment conducive to private investment do not apply solely to the ICT sector, and form an element of a number of different national e-strategy themes, in particular those that focus on infrastructure development and e-business application (both of which are driven primarily by the private sector). 37 Definition: What is the ICT Sector ? The ICT sector is made up of private enterprises that produce ICT goods and provide ICT-based services. For the purposes of this module the focus is on small and medium-sized enterprises (SMEs).27 ICT goods. These consist of hardware, software, and network equipment that are required to generate, process, store, transmit, and present electronic information. This can cover a range of hardware products from desktop computers (and components) to digital cameras. It also includes software, such as back-office accounting packages or telemedicine applications. ICT consulting services. These consist of service companies that predominantly focus on the ICT sector and the application of technology products. They offer ICT-focused management consulting services to public and private enterprises, which include a variety of services such as ICT strategy formulation and systems implementation. ICT-enabled services. These consist of information-intensive services that are conducted by third party service providers on behalf of their clients, such as customer call centers and data processing services. This service sector is uniquely enabled as a result of ICTs, as it provides back-end business services remotely to clients who were previously obliged to fulfill these needs on their own. These services can be (and are increasingly being) provided from a wholly different geographic region. The ability to `offshore' such services has been driven by declining voice and data communication costs, coupled with improved quality and reliability. Overview of ICT Sector Development Strategies ICT sector development is a main priority in the majority of the e-strategies that were reviewed for this toolkit.28 Common themes are addressed in many of the strategies (see diagram below). The top three ICT sector components focused on are: § establishingcyber-parksandbusinessincubationactivities § encouragingventurecapitalfinancing(eitherbyprovidingincentivestotheprivatesectoror through government funding § streamliningregulationforenterprisestart-upandoperations(fromdutiesandtaxationto zoning and inspection exemptions). Other areas include interventions focused on ICT infrastructure development, human capital development, fostering professional networks, and stimulating demand of ICT goods and services. In pursuit of these goals, most strategies focus on stimulating the development of small and medium ICT enterprises, to serve both domestic and international markets. 27This rather broad definition (in contrast with statistical definitions used by OECD or WTO for instance) stems from operational concerns: when designing and implementing an e-strategy, the enterprises involved (in developing and transition economies, a majority of these are SMEs) will present many differences in concerns and focus. It can be argued that some of the issues addressed in this section would better fit under an e-business module for example. It is likely that, in future editions of this toolkit, some sections of the present module will migrate towards the sectoral modules still remaining to be developed. 28This figure is partly skewed by the fact that a number of sub-national strategies from India were included in the sample, all of which focus predominantly on the development of state-level ICT sectors. 38 Commonly addressed sub-components of ICT Sector Strategies Incubation Financial Support Business Regulation Infrastructure Human Capital Networks Demand 0 1 2 3 4 5 6 7 8 9 10 Number of strategies II. M&E FRAMEWORK(S) Business Regulation The private sector is the main driver of ICT sector development. Therefore, all elements of national e-strategies that deal with this theme prioritize the creation of a regulatory and legal environment that is conducive to private investment. This is a substantial and challenging agenda, that is necessarily broad in scope. It may include: § Businessregistration § Employmentregulation § Contractenforcement § Creditmarketregulation § Procedures for closing abusiness.29 An e-strategy that undertakes change across these areas will be far-reaching, perhaps excessively so. It will also have more to do with general private sector reform than the ICT sector per se. However, there are elements of such a reform agenda that are more pertinent to the ICT sector than others, and upon which e-strategies generally focus: § Business registration and related costs of business start-up are often prioritized, as many ICT businesses in developing countries have yet to be created. Streamlining business incorporation processes encourages entrepreneurs to start businesses, testing and hopefully profiting from their new business ideas. § Contract enforcement is primarily addressed from the perspective of enforcing intellectual property rights (IPR). A strong IPR regime helps to protect the products that the sector 29These are the main areas of focus of the World Bank's Doing Business in 2004: Understanding Regulation. Much of the information in this section draws from this report. Future reports will focus on: 2005 - registering property, dealing with government licenses and regulation, and protecting investors; 2006 ­ paying taxes, trading across borders, and improving law and order. 39 generates and ensure that entrepreneurs and investors benefit from the time and money invested into their development. § Access to credit is vitally important for the ICT sector as it facilitates the commercialization of innovations and expansion of commercially proven business concepts. On the whole, e- strategies have addressed this area through encouraging venture capital development. Other useful initiatives may also include developing central data repositories of credit history. Policy goals Grow the country's Strategic priorities ICT sector Key initiatives Improve SMEs' ImproveSMEs' Create business Actions regulatory environment access to capital incubation services Resources Assumptions and risks Business Contract Contract registration enforcement enforcement This section focuses on the first point, business start-up, and the means by which it can be streamlined. The ease with which an entrepreneur is able to register a business and comply with all business incorporation requirements varies considerably from country to country.30 Reducing barriers to business start-up may require a variety of significant institutional changes, such as reducing the number of required procedures, improving coordination amongst different government agencies, and reducing (or eliminating) capital requirements. ICTs themselves can play a role in this process. At the very least, information can be posted on a web site to ensure easy access.31 This will facilitate the basic processes of finding regulatory information, understanding it, and complying with it. However, it is possible to go further than this. A recent report by the World Bank, Doing Business in 2004, notes that :'in business entry, reforms that are easy to implement include the adoption of better information and intra-government communications technology ­ to inform prospective entrepreneurs and to serve as a virtual one-stop shop for business registration.' 32 A number of e-strategies therefore advocate establishing a one-stop online portal to ease SME registration. As with most ICT initiatives, the application of ICTs needs to be accompanied by fundamental organizational change. Web-enabling outdated and cumbersome business registration processes will lead to little tangible benefit. Processes must be overhauled and stream-lined upfront before online versions are developed.33 30In Australia, for example, this requires two procedures, two days and 1 percent of per capita income. In contrast, it costs $5,531 to register a business in Angola, which amounts to 883 percent of per capita income. 31An example from Venezuela is http://economia.eluniversal.com/guiadinero/micro3.shtml# 32World Bank. 2004. Doing Business in 2004: Understanding Regulation (World Bank). World Bank, IFC, OUP. p. xviii 33Online registration has a number of advantages: (a) it de-personalizes the registration process, thereby reducing the opportunity for (and so the cost to the entrepreneur of) rent-seeking, and (b) reduces the costs of interacting with numerous government agencies or bodies, by bringing the necessary transactions of such agencies to a single place. 40 There are a number of indicators that can be used to monitor and evaluate strategy components that streamline business registration. Some are presented in the table below. Basic measures that are commonly used to compare different countries or changes over time within a single country may include: § Numberofproceduresrequired § Timerequiredtocompleteprocedures § Cost 34 Pyramid Layer Objective Indicator Source of data Policy goals Grow the domestic ICT § Total sector revenues NSO or sector § % Contribution to GDP growth Ministry of § Number of patents filed Trade & § Total number of people Industry employed by sector Strategic Improve SME regulatory § Number of hi-tech SMEs Ministry of priorities environment launched per year Trade & § Totalstart-up costs Industry § Ongoing regulation compliance costs § Perception of laws relating to ICTs § Perception of overall administrative burden Key initiatives Streamline business § Number of procedures required Ministry of incorporation procedures a § Time required to complete Trade & procedures Industry or § Cost Project Team § Number and % of new high- tech SMEs that register online § % growth in online registration each year Actions § Assess key registration § Assessments completed of X% Project Team procedures of institutions by month A § Reengineer/streamline § X number of procedures processes integrating streamlined by month B procedures from all § % of registration procedures relevant agencies integrated into single one-stop- § Launch shop facility communications and § Portal online/live by month C outreach campaign § Number of online registrations § Create online of hi-tech SMEs registration portal a § % growth in on-line registration each year (a) This is based on an assumption that an online portal is developed as part of the business registration reform process. 34Data for these measures are available online for a number of countries as part of Doing Business in 2004 at http://rru.worldbank.org/DoingBusiness/. Doing Business in 2004 also includes minimum capital requirements. This is less relevant for initiatives that focus mainly on establishing online registration portals. 41 Access to capital Access to venture capital (VC) has been a key explanatory factor behind the growth of the ICT sector in the US, particularly in the late 1990s and early 2000s. While this form of financing is not unique to the US, it is not as pervasive or accessible in other parts of the world, particularly in developing economies. Instead, SMEs are often dependent on traditional bank loans or even family savings for their start-up capital. Policy goals Grow the country's Strategic priorities ICT sector Key initiatives Actions Improve SMEs' Improve SMEs' Create business regulatory environment access to capital incubation services Resources Assumptions and risks Stimulate VC Reform credit funding market E-strategies commonly address capital access issues through encouraging venture capital development.35 The government, often in partnership with the private sector, often establishes VC funds to play a demonstrative role in stimulating private VC investment in the ICT sector. On the whole, strategies have avoided government administration of VC funds. It is believed that governments lack experience and expertise in assessing business proposals that may one day spawn successful enterprises. Therefore, while public monies are used to capitalize such funds (often matching a certain percentage of private investment), the disbursement and management of the fund is often delegated to private agencies. However, e-strategies are often silent on the initial size of the funds and the financing sources. While the specifics may not be available at the time of strategy formulation, some estimate in this regard is important ­ otherwise, it will not be possible to establish reasonable indicators and related targets with which to guide the strategy implementation. 35Capital market reform would also be particularly useful, however, this may be considered beyond the scope of an e- strategy. A useful initiative, however, may include developing central data repositories of credit history (public or private). 42 Pyramid Objective Indicator Source of data Layer Policy goals Grow the domestic ICT · Total sector revenues NSO or Ministry sector · % Contribution to GDP of Trade & growth Industry · Number of patents filed · Total number of people employed by sector Strategic Provide access to start-up · Number of hi-tech Ministry of Trade priorities financing SMEs launched per year & Industry and · Number of SMEs financed Ministry of per year by each channel Finance (VC, banks, etc.) · Perception of VC availability · Perception of subsidies for firm-level R&D Key initiatives Launch and operate a VC § Fund value ($) at launch Project Team fund § Public-private financing ratio § Number of new hi-tech SMEs supported by fund (by year) § % operational after each year (years 1-5) § Average capital infusion per company § Fund ROI Actions § Create VC fund § Fund value ($) at creation Project Team § Establish partnerships § Public-private financing with private funders ratio and business § Number of partnerships incubation service established providers § Selection guidelines § Develop guidelines developed for fund eligibility § Number of new hi-tech § Fund proposals SMEs supported by fund § Promote fund (by year) § # of funding proposals submitted Monitoring and evaluating the effectiveness of a VC fund will depend on a number of factors outside of the control of a fund.. Macro-economic and environmental conditions that are not conducive to private investment, such as high inflation or a lack of legal and judicial transparency, will limit private entrepreneurs' willingness to take on debt and launch new business ventures. Furthermore, regardless of capital availability, the capacity of local entrepreneurs to create an effective and profitable enterprise is of crucial importance. Operationalizing an innovative idea requires considerable management experience. Therefore, the creation of VC funds is often tied to business incubation activities, as the success of a fund will ultimately be measured by the number of sustainable enterprises that it launches, which has as much to do with good management as financial backing. 43 Business Incubation While it is widely believed that establishing the correct regulatory and incentive structures is a necessary step to ensuring the development of an ICT sector, this step in itself is insufficient. A number of national e-strategies also advocate a strong leadership role for the government, both to establish VC funds, as previously mentioned, and to provide ICT business incubation services. Policy goals Grow the country's Strategic priorities ICT sector Key initiatives ImproveSMEs' ImproveSMEs' Createbusiness incubation services Actions regulatory environment access to capital Resources Assumptions and risks Incubate Create business and launch incubator SMEs The bottom line in assessing the effectiveness of a business incubation strategy is the number of SMEs an incubator has launched, combined with some estimation of their success (such as years in operation, cumulative profits, or number of patents registered). However, the incubator itself as an enterprise should also be assessed, in terms of its own operational effectiveness and long-term sustainability.36 Establishing a business incubator Evaluating an incubator's operational effectiveness requires clarifying its value proposition. At the most basic level, incubators provide value through: · physical infrastructure · business and management expertise · access to networks of other start-ups, established businesses, investors, and research and development (R&D) institutions. Indicators to measure the development of a business incubator will require establishing parameters regarding the size and scope of the initiative. The e-strategy should therefore outline the: · type and scale of infrastructure that will be provided · types of business support services to be provided · means by which networks will be developed, and which organizations to include. 36Indicators proposed in this section are drawn from a number of proposals submitted to the infoDev Incubator Initiative. Web: http://www.infodev.org/html/programs_incubators.html. See also infoDev Incubator Support Center web site: http://www.idisc.net/ 44 The final factor ­ access to businesses, investor and R&D networks ­ is arguably one of the most valuable and yet difficult-to-measure products that an incubator can provide to budding entrepreneurs.37 The table below provides suggestions on measuring the successful establishment of an incubator: Pyramid Layer Objective Indicator Source of data Policy goals Grow the domestic ICT § Total sector revenues NSO or Ministry sector § % Contribution to GDP of Trade & growth Industry § Number of patents filed § Total number of people employed by sector Strategic Create a business incubator § Number of new hi-tech Ministry of Trade priorities SMEs supported by & Industry and incubator (by year) Project team § % of SMEs operational after each year (years 1-5) § Average profits generated per company (by year) Key initiatives § Build physical § Funding of $X established Project Team infrastructure § Public-private financing § Staff the incubator ratio § Launch the incubator § Total incubator ROI after year X § Functional office space of X m2 created § Connectivity of X Mbts/sec § % space rented after month A § Staff-to-client ratio Actions § Develop business plan § Business plan and budget Project Team § Build physical complete infrastructure § TORs established for all § Recruit management staff by month A team § X% of staff recruited by § Create partnerships month B § Begin operations § Incubator launched by month C 37The final measure of success of an incubator as an enterprise is the whether it is privately sustainable. The level of private participation in the early stages of the initiative is a good initial indicator of this. Ultimately, the final measure would be whether the incubator is successfully privatized. 45 Incubating and launching ICT SMEs A functional, well-connected and well-managed business incubator provides the necessary foundations for launching hi-tech SMEs. However, ensuring that these SMEs grow to become successful firms depends on the specific interventions that the incubator's management team will undertake. This requires: · careful selection of firms to incubate (those that offer a promising product backed by a dedicated and well-qualified team) · actively engaging firms, and providing them with business coaching in areas such as accounting, marketing, HR, business plan creation, finance and operations · helping launch into stable and profitable markets. Establishing indicators to measure this undertaking will begin with the bottom line ­ as stated before, this might include the number of SMEs an incubator has launched, together with some measure of their success (such as years in operation, cumulative profits, or number of patents registered). However, indicators should then focus on the activities the incubator conducts as part of its regular operations, and how effectively it undertakes them. These may focus primarily on the provision of value-added services (such as management coaching) and the use of networks with other businesses, investors, and R&D institutions. The indicators should begin, however, with measures of how the incubator chooses its future winners ­ its selection criteria. 46 Pyramid Layer Objective Indicator Source of data Policy goals Grow the country's ICT § Total sector revenues NSO or Ministry sector § % Contribution to GDP of Trade & growth Industry § Number of patents filed § Total number of people employed by sector Strategic Create a business § Number of new hi-tech Ministry of Trade priorities incubator SMEs supported by & Industry and incubator (by year) Project team § % of SMEs operational after each year (years 1-5) § Average profits generated per company (by year) Key initiatives § Provide business and § Number of firms by Project Team management month A expertise § Average number of firms § Ensure access to (by year) business and R&D § Number of trainings networks conducted (by topic) § % of incubator staff participating in training (by topic) § Number of joint projects with other businesses § Number of joint projects with R&D institutions § Satisfaction survey results Actions § Select firms § Firm selection criteria Project Team § Assess learning established needs and develop § Knowledge/skills gap plan analysis complete by § Provide training month B § Create partnerships § Learning plan complete by § Operationalize month C partnerships § Training content for X topics completed by month D § Number of partnerships with other businesses § Number of partnerships with investors § Number of partnerships with R&D institutions 47 III. ASSUMPTIONS AND RISKS Access to reliable high bandwidth infrastructure, and the availability of skilled workers that will both start and staff ICT enterprises, are key ingredients to the development of a domestic ICT sector. Components of e-strategies that deal with ICT sector development usually have little direct bearing on these two elements. While access and human resources are often referenced in ICT sector components, they are covered in more detail as stand-alone components of e-strategies. Responsibility for implementation of these components is likely to reside with different parts of the government. Therefore, tracking progress against key measures of connectivity and human capacity will be important in assessing whether the ICT sector development components will meet its own targets and goals. At the most basic level, access and human resource-related indicators may include tele- density or the number of students graduating with technical qualifications. Aside from tracking such measures ­ which reflect changes at only a very high level ­ mid-term evaluations may be required to assess whether or not infrastructure and human resource growth are sufficient to meet the ICT sector's growth in demand, and what interventions are required to address any (controllable) impediments or constraints to the success of the ICT sector. Finally, the state of the market that the budding ICT sector seeks to serve and profit from is of crucial importance. Most ICT sector development strategies focus on both the domestic and international markets, tending more towards the latter. Aligning targets for growth of the domestic ICT sector needs to be done in light of reasonable projections of domestic and export market growth and the ability of the sector to meet changes in demand. This requires careful tracking of both the market as a whole as well and key segments within it including demand, supply, and types of ICT products and services. Summary ICT sector development strategies focus on a broad variety of issues that often form part of the larger private sector reform agenda. However, there are some elements that are particularly relevant to developing and launching a successful local ICT sector. These include incubation support, financial support, and improved business regulation. Within these areas there are a variety of feasible interventions, each of which require a set of key indicators to monitor and evaluate progress. However, regardless of how effectively such interventions are undertaken, the ultimate success of ICT sector development in national economic development ­ the growth of the ICT sector and its ability to contribute to GDP growth and employment ­ is heavily dependent on exogenous factors. They include the quality of ICT infrastructure access and the depth of the human capital base from which the sector can draw its most valuable asset ­ its people. It is also dependent on the health of the ICT market, both local and international, and the level of competition within it. Tracking changes in this market, and adjusting the targets established in the strategy as necessary, is therefore of vital importance to ensuring that the strategy is sustainable and its goals remain realistic and meaningful. 48 Case: Measuring (and Adjusting) the Growth of the ICT Sector in Jordan The primary focus of Jordan's e-strategy is on developing the ICT sector. Launched in 2000, it is rich in baseline data that covers the state of the ICT sector in Jordan. The strategy also draws on data from other countries ­ mainly Egypt, Israel, India and Ireland ­ that includes: § NumberofsoftwareandITservicesfirms § Numberofpeopleemployedinsector § Sectorrevenues § Annualgrowthrates § Employeesperfirm § Revenue per employee § Totalvalueofsectorexports § Value of exports per employee § Majorproducts § Sourceofforeigndirectinvestment(FDI) § Leadingprivateinvestors/partners. Based on this and other data, the strategy reviews Jordan's comparative strengths and weaknesses. It also establishes three high-level goals and targets, towards which all of the activities recommended in the strategy focus: § Create30,000IT-relatedjobsin2004 § Generate$550millioninannualexportsby2004 § Ensure$150millionincumulativeFDIby2004. The strategy then lays out specific actions and deliverables for individual initiatives, grouped in six key focus areas, to achieve these overarching targets. An updated strategy, developed four years later in 2004, reviews progress made against both the core three high-level targets, as well as the deliverables for the individual initiatives (that are assessed in terms of being complete, partially complete, or not addressed). In light of the rate of growth achieved in the preceding years, the revised strategy updates the initial targets substantially. The date by which to achieve the targets is extended from 2004 to 2006, the annual export target is reduced from $550 million to $100 million, but a new target of $550 million in domestic revenues is added. Finally, the FDI target is increased by an additional $20 million. Sources Information and documents relating to Jordan's e-strategy are available at: http://www.reach.jo/ 49 Module 6 E-GOVERNMENT I. RATIONALE: WHY FOCUS ON E-GOVERNMENT? For many government officials concerned with extending the benefits of Information and Communication Technologies (ICTs) to their country, e-government is a natural point of entry, and is a core element of most e-strategies designed so far.38 E-government has the potential to greatly and immediately improve how government operates internally and how it serves its customers. At the very least e-government can bring cost savings to businesses (e.g. faster business registration), and time savings to citizens (e.g. online tax returns). The best e-government implementations however address the ways in which internal government processes are executed, as well as how government transacts with society as a whole. The potential of ICT in government should therefore be understood as a paradigm shift that improves how government operates and how society interacts with and views government. More than many other components of an e-strategy, e-government efforts must be measurable (and generally visible) in order to attract support from civil society at large.39 Hence, any failure to meet deadlines or reach milestones will be more damaging in e-government than in most other areas. Offering transparent ways to measure progress (through benchmarking, and more generally through M&E) will therefore be vital to for success in e-government. Since the toolkit focuses on monitoring and evaluation (M&E), it will base its approach on the range of `best practices' that emerges from the relatively broad array of experiences available in this area.40 II. DEFINITION: WHAT IS E-GOVERNMENT? E-government consists of a set of activities and instruments through which ICTs are fully or partially integrated in some of the core functions of governments, administrations, and public service entities. The purpose of such integration is generally a combination of the following: 38E-government development is the most frequently cited (over 60%) focus area of the strategies surveyed. 39Some internal (G2G) transactions may remain invisible (if not hidden) from the end user; however, even in such cases, civil society may require transparency (as in the case if the French Legislation `Informatique et Libertes', which limits some linkages between various governmental databases ) 40 This best-practice approach was already the starting point of the `E-government Handbook for Developing Countries', published by infoDev and the Center for Democracy and Technology, in November 2002 (see www.infodev.org). Many of the items described in this module refer to the classification used in this milestone publication. 50 Efficiency - Greater efficiency in delivering government services to citizens and businesses as well as improved intra-government services Provision - Development and delivery of new services to the population, or provision of services to populations previously underserved, especially in rural or less densely populated areas. Responsiveness - Increased responsiveness of governments to the needs of their citizens, including new possibilities for citizens and governments to interact with each other. Accountability - Greater transparency and accountability of governments and administrations, including in the area of public procurement. Participation - Higher levels of citizen participation in public decisions and management, hence strengthening democracy. Understood as such, e-government is much more than a tool E-government is not primarily about for improving cost-quality ratios in public services. It is also automation of existing procedures but an instrument of reform and a tool to transform about changing the way in which government conducts business and government. Thus, e-government is not primarily about delivers citizen services automation of existing procedures (which may or may not be effective), but about changing the way in which government conducts business and delivers services. III. OVERVIEW OF E-GOVERNMENT STRATEGIES E-government strategies vary considerably in terms of their focus and the degree of change they aspire to undertake. However, most e-government strategies include the following elements: § A precise identification of the prerequisites for success (and possible indications on how to meet these prerequisites if necessary) § A clear definition of the objectives being pursued (output and outcome), ways of measuring success, and a time horizon within which such success is expected § A set of M&E indicators, linked to specific objectives, levels of responsibility, and milestones embedded in the e-government strategy. Prerequisites The achievement of e-government goals requires that ICTs in a country are developed to a sufficient level to allow measurable changes to take place in the way government functions and citizens are served. The ability of citizens to access ICTs will depend on the existence of infrastructure (e.g. Internet access), affordability of access (which will itself critically depend on the existence of a competitive regulatory environment), and of the availability of basic knowledge (e- literacy) across society. Moreover, since e-government is not just about saving money by computerizing procedures, but also about reforming and improving government, other pre-requisites include (a) the political will of government authorities to reform and improve government processes, and (b) the support and engagement that such efforts will gather from civil society as a whole. Although many of the pre-requisites relevant to e-government also happen to be necessary conditions for the development and successful implementation of e-strategies as a whole, one 51 cannot over-emphasize the importance of adequate legal, regulatory, and institutional environments as necessary conditions for success in e-government. Whether such frameworks are designed and enforced at central or local (i.e. sub-national) levels, they have proven to be the single most important pre-condition for the successful and society-broad use of information technologies, mainly because they have allowed cost and prices to diminish. In areas in which public entities (i.e. governments, administrative departments, or state-owned enterprises) remain major players (as in e-government), the creation of such environments becomes particularly critical (and visible) because government is on both on the supply side (e.g. for on-line services to citizens) and on the demand side (e.g. for e-procurement) of the equation. Such pre-requisites can be summarized by the `ABCDE' of e-government: Prerequisite Concern Activity (typical indicators) Access Infrastructure, costs, competition/ § Equipment (PCs, kiosks, community centers) regulation (hence includes proper § Teledensity regulatory and competition § Rule of law frameworks) § Pro-competitive ICT regulation (tariff and non-tariff barriers, competition in the ICT sector) § Cost (fixed line calls and Internet access) § Access for disadvantaged or excluded Basic Skills Basic education, vocational training, § Literacy (alphabetization rates) ICT awareness § E-literacy ratios per age/group/sex/region § Vocational training Content Value to government and citizens § Questionnaires on value to users/citizens and government § Content in local languages Desire Political leadership and will to § Public statements/decisions reform § Laws & regulations (perceptions of quality of legal system) Engagement Commitment of all components of § Broad involvement of civil society civil society (questionnaire/survey) § Local awareness of ICT potential for development (questionnaire/survey) Expected Outcomes and Sequencing Depending on their own development goals and available resources (financial, technological, and human), governments may have different levels of ambition regarding e-government. It is generally recognized that three types (or layers) of e-government may be considered as part of a national e- strategy, namely: § Publishing information (one-way communication: (G2B, G2C) § Interacting with the citizen (two-way communication: (G2B, G2C) § Contracting with citizens and government on-line: (G2B, G2C, G2G) 52 Although those three layers undoubtedly correspond to increasing levels of institutional, legal, and technological sophistication, they should not be seen as stages in a required strategy `sequence'. Various layers may coexist within e-government, some services being more advanced than others at any point in time. Publish Interact Contract Rationale Bring information quickly Engage civil society in reformOffer cost-effective government and more directly to citizens process and generate support services anywhere, anytime Focus Area Rules, regulations, Two-way communications Services such as ID cards, documents, forms, (e.g. email), feedback forms certificates, land ownership institutional structures, and online discussion fora titles, registrations processes and procedures (automobiles, change of address, public procurement tenders), tax and fine collection Good Practices § Strategy to get § Show citizens that their § Enlist the support of information online, with engagement matters, by those who will be using appropriate milestones informing them of the the site § Post information of outcomes of their online § Integrate e-government value to people in their comments with process reform, daily lives, and § Break down complex streamlining and emphasize local language policy issues into easy- consolidating processes content to-understand before putting them § A mandate that all components online agencies publish a § Be proactive about § Address the concerns of specified range of soliciting participation - government workers information online use traditional media to whose role will change § Design sites so they are publicize online as a result of the easy to maintain, and consultations innovation sustain funding to ensure § Recognize that initial that information is investments in updated regularly transaction systems can § Focus on content that pay off over time in supports other goals, e.g. terms of cost savings economic development, and increased revenue anti-corruption, § Create a portal for attracting FDI transaction services An important point needs to be made regarding the level of government at which an e-government strategy is being considered. Since social consensus is often less difficult to generate at the local (e.g. state or municipal) level than at the national (e.g. federal) level, e-government strategies are in general most productively designed and implemented by local governments (state or cities). E- government strategies, whether national or local, will involve virtually all parts of government. As 53 the e-strategy is formulated and customized, this should be reflected in its objectives and relevant M&E instruments. The e-government components of an e-strategy should consider various best practices, depending on which level of governance they focus upon. The following table provides an initial basis for their identification in a possible `e-government roadmap'.41 IV. M&E FRAMEWORKS It is important to note that e-government initiatives have commonly been incorrectly conceived, haphazardly applied and rarely measured in terms of their relative success.42 The most common mistake is the use of IT to automate existing processes when what is required is a comprehensive business case to define how technology can positively impact government and society in the short and longer terms, and a corresponding M&E system to reflect progress and return on investment. More than many other components of an e-strategy, e-government efforts are highly visible. To attract support from civil society at large, they are required to establish clear objectives and be transparent with regard to their implementation. Failure to meet deadlines or reach milestones may be Failure to meet deadlines or reach more damaging in e-government than in many other milestones may be more damaging in e- areas. The specification and design of indicators and government than in many other areas methodology must therefore be rigorous and precise, while also leaving a reasonable margin for adjustments during implementation. They also require a high level of commitment and endorsement from relevant authorities. For each of the three types of e-government approaches, an appropriate set of M&E indicators needs to be implemented. The following tables offer proposals in this regard. It is up to each decision maker to select and adapt those indicators best fitted to their needs and constraints. It should be noted that many of the initiatives and actions that are covered in the tables relate to putting information and services on-line, therefore focusing to a large extent on the `front-end' of e-government. However, substantial `back-end' or organizational changes are required to undertake many of these initiatives. Information must be shared across government departments, requiring (at the very least) standardization in collection and processing. Processes and procedures have to be adjusted ­ even completely overhauled ­ to respond to the different requirements of online service provision. Staff skill requirements are also likely to change, necessitating adjustments to staffing and hiring practices; such organizational change is taken into account in the M&E frameworks that follow. Indicators to track progress and measure success of this change will be required as well. 41Based on elements from in the E-government Handbook for Developing Countries. infoDev and CDT. 2003. Web: http://www.cdt.org/egov/handbook/2002-11-14egovhandbook.pdf 42http://www.e-devexchange.org/eGov/topic1.htm 54 E-government Level 1: Publish It can be particularly challenging to develop indicators to measure progress made in putting government information online and measuring the degree to which people are accessing it. At the initiative (output) level it is possible to simply count web sites, which provides a good measure of progress made. However, assessing the strategic value of this outcome is considerably more difficult. Counting website visits is of limited help, as this cannot be tied back to the number of users (or percent of the population). Conducting surveys of the population would be an effective way to gauge usage, but this can be costly.43 Create efficient, Policy goals responsive, transparent government Strategic priorities Publish: Interact: Contract: Provide valuable Engage society to Offer cost-effective Key initiatives on-line information improvegovernment online services Actions Resources Assumptions and risks E-government and efficiency : examples from Singapore and India Advances in information technology, including the Internet, are paving the way for investment climate improvements that reduce demands on public administration, enhance transparency, and ease compliance burdens on firms. Approaches to business regulation in Singapore and land titling in India's Karnataka state illustrate the potential. The e-government initiative launched by Singapore in 2000 included business registration and licensing procedures. It provides an online application system for business registration and licensing and a one-stop online application system for certain special licenses (for example, building and construction permits) that previously required separate submissions to as many as 12 regulatory authorities. The integrated approach reduced the cost of incorporating a new company from anywhere between S$1,200 and S$35,000 (around $700 to $20,000) (depending on the capital of the company) to a flat fee of S$300 ($175). What used to require two days now requires less than two hours. Streamlining the submission process for construction permits saves applicants more than S$450 ($260). India's Karnataka state introduced an electronic land-titling system, Bhoomi, in the late 1990s.The online system is delivered through kiosks installed in all land offices of Karnataka. These kiosks provide copies of a Record of Rights, Tenancy, and Crops (RTC). Obtaining an RTC once required up to 30 days, and typically a bribe of as much as Rs. 2,000 (about $43). Land records could be deliberately "blurred" for fees of Rs. 10,000 ($220).These records were not open to the public, and it sometimes took two years for the records to be updated under the manual accounting system maintained by 9,000 "village" accountants - state employees responsible for three to four villages each. Today an RTC can be obtained for a fixed fee of Rs. 15 ($0.32) in 5 to 30 minutes. The records are open for public scrutiny. Citizens can now request that land titles be updated quickly through the kiosks, a process that has increased the number of annual applications for updates by 50 percent. Source: Tan (2004); Bhatnagar and Chawla (2004); and Lobo and Balakrishnan (2002). Quoted in `A Better Investment Climate for Everyone' World Development Report, 2005, The World Bank. (Box 2.16, `E-government and the investment climate') 43 One way to minimize costs would be to include relevant questions in national household and business surveys, carried out by NSOs. Efforts in this area should ideally be developed with a view to collect data in a way that would ensure international comparability. 55 Pyramid Layer Objective Indicator Data source Policy goals Create an efficient, responsive and Perception of overall Office of transparent government administrative burden Government and Perception of government Ministry of Local effectiveness Government Strategic Bring valuable information online to § Perception of E-government CIO's priorities the public, anytime anywhere a government online Office presence § % pop. using govt. sites § Usage growth rate Key initiatives § Roll-out of online information § No. of agencies E-government CIO's services b with web sites Office § Raise public awareness through § % of agencies with online and offline channels web sites § % of information services rolled-out on time § Actions § Establish an independent central e- § Central e- Project team or government group and M&E unit c government team Central M&E unit Establish selection guidelines for and M&E unit information/content to be posted, established by including nature and volume month A Assess technology and organizational needs/requirements § Guidelines for Develop online information platforms M&E established by Offer information services on line, month B including local language content § Relevant Develop publicity campaign to promote information sources new e-government initiatives identified by month Solicit feedback on usability and usefulness C of online government services § System functional requirements completed by month D § Mid-term implementation review conducted by month E § Public awareness survey results (a) There should be a clear linkage between the type of content that is brought on-line and larger development goals, such as economic development, anti-corruption, and attracting FDI. (b) Criteria used for information publication should be related to cost and time savings and envisaged productivity gains. (c) Central e-government group responsible for interoperability and inter-agency consistency of e-government services, security, consolidation of records, and M&E. E-government Level 2: Interact Measuring levels of interaction is similarly challenging to publishing-focused initiatives. Indicators can focus on the types and number of interactive channels available (such as e-mail or discussion fora) and the government's responsiveness. As the sophistication of the online services increases, the indicators should be disaggregated to reflect progress made by individual government agencies. 56 Policy goals Create efficient, Strategic priorities responsive, transparent government Key initiatives Publish: Interact: Contract: Provide valuable Engage society to Offer cost-effective Actions on-lineinformation improve government online services Resources Assumptions and risks Pyramid Layer Objective Indicator Data source Policy goals Create an efficient, responsive and Perception of overall Office of transparent government administrative burden Government and Perception of government Ministry of Local effectiveness Government Strategic Engage society to improve government, Perception of government E-government priorities and enhance efficiency of delivery of online presence CIO's Office government services No. of improvements to government services resulting from public suggestions Key initiatives Deploy online interactive tools % govt. agencies with E-government Raise public awareness through online interactive sites CIO's Office and offline channels No. of online interaction Examples: e-mail Q/A, threaded channels (by agency) discussions, feedback forms, `Ask the % of possible reforms/policy issues that have interactive policy maker' tools (by agency) Actions § Establish selection guidelines for § Guidelines for Project team or opportunities for interaction selection established Central M&E unit § Assess technology and by month A organizational needs/requirements § Relevant opportunities § Develop online interactive for interaction platforms (`interact' stage), identified by month B integrating with information § System functional platforms (`publish' stage) requirements § Publish government contact completed by month information C § Develop publicity campaign to § Mid-term promote new e-government implementation review § Solicit online public consultation conducted by month through other media D § Publish deliberations of § Public awareness questions/queries from the public survey results in FAQ, to show that engagement is taken seriously § Solicit feedback on usability and usefulness of online government services § Link to other online government services 57 E-government Level 3: Contract At this level of e-government, quantitative M&E indicators become easier to use. These may include the number of agencies and functions online, average time for processing citizen requests or applications, number of complaints about the level and quality of government services, or depending on the e-government initiative, reduction in government costs for service delivery (procurement), increased revenue (tax collection) and better governance (voter turn-out). Indicators at this level will not only be agency-specific, but also focus on individual online services.44 Create efficient, responsive, transparent Policy goals government Strategic priorities Publish: Interact: Contract: Provide valuable Engage society to Offercost-effective on-line information improve government online services Key initiatives Actions Resources Assumptions and risks 44See also M&E table in module 4 on development of an online business registration portal. 58 PyramidLayer Objective Indicator Data source Policy goals § Createanefficient,responsive § Perception of overall Office of and transparent government administrative burden Government and § Perceptionofgovnt Ministry of Local effectiveness Government Strategic § Offer cost-effectiveonline § Perceptionofgovnt E-government priorities government transactional online services CIO's Office services anywhere anytime § %govt.agencieswith transactional sites § %ofpossibleservices online (by agency) Key initiatives § Createonlineversions of § No.ofonlineservices E-government offline services (to cut costs § %ofpossibleservices CIO's Office and redeploy resources more that are online (by efficiently) agency) § Raise public awareness § % of total customers through online and offline transacting online (per channels service)45 § Usagegrowthrate Examples: ID cards, certifications (per service) (death, birth, marriage, divorce), land § Time to complete ownership titles, registrations transaction (per (automobiles, change of ownership), service) public procurement (tenders), tax and fine collection Actions § Establishguidelinesfor § Guidelines for Project team or selecting online services selection established Central M&E unit § Establish mechanism for by month A interagency coordination and § Relevant services system integration identified by month B § Assess technology and § Interagency organizational needs mechanisms and § Developonlinetransactional procedures established platforms, integrating with by month C interactive and informational § Systemfunctional platforms requirements § Addresstheconcernsof completed by month government workers whose D roles will change § Mid-term § Providenecessaryfeedback implementation and possible training review conducted by § Buildconfidenceinsecurity month E and privacy § Staffandusertraining § Develop publicity campaignto complete by month F promote new e-government § Public satisfaction § Solicitfeedbackonusability survey results and usefulness of online government services § Benchmark processing times for individual services and transactions 45Available from `Weblogs' generated by the computers that `host' the e-government website or portal. 59 Summary The incentive to undertake e-government development is based not in technology itself, but rather in the real benefits that can be realized over the short, medium, and long term, in particular where e-government goals are aligned with larger governance objectives. The drivers for the success and sustainability of e-government are true government commitment to improving administrative processes, increasing productivity and generating savings, and building public confidence in the government's ability to execute the necessary reforms and transformations. Developing effective indicators to monitor such change and evaluate the means by which e- government initiatives are delivering on their promise is particularly important, as e-government is often the most visible (and closely watched) component of an e-strategy. 60 Case: Benchmarking E-government in Europe The E-Europe Plan of Action for 2005 establishes 7 targets which are specific to e-government: § Interactive public services accessible to all § Electronic procurement for a significant part of public procurement § Public Internet Access Points (PIAP's) for all citizens, preferably with broadband connections § Broadband connections for all public administrations § Interoperability for e-government services § E-services to promote culture and tourism § Secure communications between public services for the exchange of classified government information. With regard to the first target, 20 recommended services are listed (12 to citizens and 8 to businesses): Public Services for Citizens Public Services for Businesses Income taxes Social contribution for employees Job search Corporation tax Social security benefits Value Added Tax Personal documents Registration of a new company Car registration Submission of data to statistical offices Application for building permission Customs declarations Declaration to the police Environment-related permits Public libraries Public procurement Certificates (birth, marriage) Enrollment in higher education Announcement of moving Health-related services Monitoring and evaluating progress achieved in establishing these online services is based on a five stage framework that assesses the level of maturity or sophistication of each e-government service. While the definition of each stage of maturity varies depending on the service being assessed, in general terms they translate as: Stage 0 ­ No online presence Stage 1 ­ Information (on-line information about public services) Stage 2 ­ Interaction (downloading of forms) Stage 3 ­ Two way interaction (processing forms, including authentication) Stage 4 ­ Transaction (case handling, decision and delivery) Based on this classification, member states are scored for the maturity level of each of the 20 e-government services. This allows them to compare their levels of maturity relative to one another, as well as progress made in different groupings of services, such as services for citizens versus services for businesses. Because the framework has been in existence since 2001, it also allows member states to compare changes over time. Though this framework is very useful, it has limitations. First, it only applies to Internet-based electronic services, and not those delivered through other channels such as mobile devices or call centers. Second, it focuses only on the front-office of e-government, and does not account for important back-offices changes that e-government entails, such as process redesign and system integration. Sources E-Europe. 2001. "E-government Indicators for Benchmarking E-Europe." http://europa.eu.int/information_society/eeurope/2002/action_plan/pdf/egovindicators.pdf E-Europe. 2002. "E-Europe 2005: An Information Society for All." http://europa.eu.int/information_society/eeurope/2005/all_about/action_plan/index_en.htm E-Europe and Cap Gemini Ernst & Young. 2003. "Web Based Survey on Electronic Public Services: Report of the Fourth Measurement, October 2003." http://europa.eu.int/information_society/eeurope/2005/all_about/benchmarking/index_en.htm 61 References Commonwealth Business Council. 2001. E-government ­ Modernizing Commonwealth Governments. Digital Opportunity Task Force. 2001. Digital Opportunities for All. http://www.g7.utoronto.ca/summit/2001genoa/dotforce1.html E-Europe. 2001. "E-government Indicators for Benchmarking E-Europe." http://europa.eu.int/information_society/eeurope/2002/action_plan/pdf/egovindicators.pdf E-Europe. 2002. "E-Europe 2005: An Information Society for All." http://europa.eu.int/information_society/eeurope/2005/all_about/action_plan/index_en.htm E-Europe and Cap Gemini Ernst & Young. 2003. "Web Based Survey on Electronic Public Services: Report of the Fourth Measurement, October 2003." http://europa.eu.int/information_society/eeurope/2005/all_about/benchmarking/index_en.htm ESIS Report. 2000. "Public Strategies for the Information societies in the Member States of the European Union." infoDev and CDT. 2003. The E-government Handbook for Developing Countries. http://www.cdt.org/egov/handbook/2002-11-14egovhandbook.pdf G-8 Kananaskis Summit, 2002. "The Italian Initiative on E-government for Development." National Audit Office. 2002. "Government on the Web II." http://www.nao.org.uk/pn/01- 02/0102764.htm OECD. 2003. The E-government Imperative. OECD. http://www1.worldbank.org/publicsector/egov/E- GovernmentImperative.pdf Schware, R. and Deane, A. 2003. "Deploying E-government Programs: The Strategic Importance of `I before E'." World Bank. http://www1.worldbank.org/publicsector/egov/ReinventingGovWorkshop/Deployingegovt.pdf UNPAN. Benchmarking E-government: A Global Perspective - Assessing the Progress of the UN Member States. http://www.unpan.org/egovernment2.asp World Bank e-government web site: http://www1.worldbank.org/publicsector/egov/ OECD. (2003) The E-Government Imperative. Paris: Organization for Economic Cooperation and Development. Tan, Eng Pheng. 2004. "The Singapore E-Government Experience." Paper presented at Foro Internacional: Hacia una Sociedad Digital. Panamá City, Panamá. April 6. Luc De Wulf Luc De Wulf and Jose B. Sokol, eds., "Customs Modernization Handbook" (Washington, DC: The World Bank, 2004). UNCTAD "E-commerce and Development Report (Geneva, 2004) http://www.unctad.org/Templates/WebFlyer.asp?intItemID=3084&lang=1 62 Module 7 OTHER STRATEGIC SECTORS I. E-BUSINESS Rationale E-business has been a key driver of past and current demand for e-strategies. E-business has benefited global and national economies by lowering the transaction costs of doing business and creating opportunities for greater integration with global markets. For both government and business, it is seen as a way of transforming the manner in which they operate, bringing about new and more effective means of developing products and servicing customers. Overview E-business covers three major areas of activity, depending on the partners involved: § B2B ­ businesses transacting with businesses, e.g. electronic procurement of inputs and components for car manufacturing § B2G ­ businesses transacting with government, e.g. electronic procurement, online company registration and tax returns § B2C­businessessellingtoconsumers,e.g.vendorssellingbooksonline. E-business components of e-strategies encompass a range of activities needed to improve, and in the case of some countries, establish the ICT infrastructure and legal and regulatory systems that enable e-business. Often these initiatives focus on small and medium-sized enterprises (SMEs), promoting technology application among SMEs from a variety of sectors, while also encouraging the establishment of ICT-focused SMEs. Typical e-business initiatives include: Awareness creation. Awareness of the benefits of ICT application among government, business and consumers can help to promote its growth. In this regard, B2G initiatives (which may sometimes be considered as `e-government' rather than `e-business' initiatives) have a significant demonstration role to play. (See module 6, `E-government'). ICT infrastructure development. E-business will not grow without accessible, good quality and affordable infrastructure. This requires increasing competition in the ICT infrastructure market, combined with ensuring that those parts of the market that are less attractive to the private sector are reached (such as poor rural areas). (See module 4). 63 Human resource development. People's ability to effectively use ICTs is the foundation on which the success of e-business strategies is based. Legal infrastructure. A legal framework is required to build public and business trust in online transacting. This includes initiatives to manage digital rights, codify e-contracts, ensure privacy and data protection, and combat cyber crime and protect intellectual property. Macro-economic and financial infrastructure. Instruments need to be put in place to ensure that e-business transactions can be completed and contracts honored. Such instruments will be found in areas as diverse as exchange rate policies, on-line payment regulation, or credit card infrastructure. Logistical and trade infrastructure. The existence of reliable domestic transport and delivery systems, as well as ­ from an international point of view ­ trade facilitation, customs efficiency, and port management services, will be key to the success of e-business strategies. Promoting e-business among SMEs. Interventions may include ICT skills development, business incubation, and programs to increase access to financing. (See Module 4). II. E-EDUCATION AND E-LEARNING Rationale Education and learning are fundamental to the development of a dynamic society that can participate in today's global knowledge-based economy. Knowledge and skills are becoming the key differentiating factors of production in modern economies. E-education strategies focus on developing such skills and harnessing ICTs to ensure that this is done as effectively as possible. Overview An e-education strategy encompasses (a) the development of knowledge and skills required for a knowledge-based economy, as well as (b) the application of ICTs in education administration, delivery, and teacher training. E-literacy: Knowledge and Skills The modern economy is characterized by constant and rapid change. This requires a national base of `knowledge workers' who can quickly adapt old skills and gain new ones throughout their careers. Today's workers need to be competent in the use of ICTs that are increasingly being integrated in all work processes. This requires embedding e-literacy into all levels of a country's education system, integrating ICT usage into curricula and course content. 64 ICTs in Education: efficiency and effectiveness Administration. ICTs can make the administration of education more effective, cost-efficient, and productive through the use of education support technologies (such as student and course management software). Delivery. Networked technologies have the potential to improve access to, and the choice of, educational materials. These channels can also be used to bring education to those who have historically been excluded, including populations in rural areas (through distance learning), women facing social barriers that limit their access to education, and students with disabilities or with specific vocational training needs. Teacher Training. Teachers are both information consumers, using the Internet to access resources, as well as information providers, acting as information filters and dissemination channels for students. ICTs can be used to increase the reach and quality of teacher training in the same way that it benefits students in the classroom. The M&E component of an e-education strategy will follow the distinction between the productivity benefits deriving from (a) the economic benefits of ICT education, and (b) the actual use of ICT in education. For example, indicators that reflect change in ICT knowledge and skills could be measured by the degree to which university education meets the needs of a competitive economy. The productivity gains from ICT use in the education sector may be measured, for example, by the cost per student graduating from primary, secondary or tertiary institutions. III. E-HEALTH Rationale A healthy workforce is a key factor of economic growth and stability, and also provides a basis for a healthy society in general. The healthcare industry is characterized by a constant need to innovate, devise cost-efficient ways to treat patients, manage healthcare organizations, and educate the public. Because many national health care industries are publicly owned, the operational efficiency of this sector can have a strong impact on the national budget. Overview The use of ICTs in the healthcare sector can (a) improve delivery of clinical services, (b) streamline the administration of healthcare organizations, and (c) increase the reach of public health education. Clinical Services Delivery. ICTs are being used to more effectively diagnose and provide subsequent patient treatment in hospital and primary care settings. Healthcare organizations use telemedecine (i.e. real-time communications and transmission of images) for doctor-patient and doctor-specialist consultations. 65 Administration. Healthcare administration departments use ICTs to improve operations. Hospital information systems are used to consolidate patient data from disparate units, allowing for the sharing of individual medical records between different care providers. Medical practitioners use computers for better patient management (such as maintaining patient records), while Personal Digital Assistants (PDAs) are used to reduce medication error rates and costs by issuing prescriptions electronically. Healthcare organizations can also take advantage of the ICT marketplace to outsource non-core business processes (such as IT system management and applications development) in order to focus on the core business of delivering clinical services. Health Education. ICTs facilitate improvements in health education by providing the means to publish and communicate medical and training information to healthcare practitioners and the public. Governments can use ICTs to increase health literacy by publishing diet, exercise, wellness, and medical information on websites. This can serve to reduce unnecessary demands on medical facilities and services. In healthcare organizations, ICTs are also improving the skills of staff and specialists by providing training through simulations and providing online access to medical best practices. 66 A N N E X E S ANNEX 1. THE BASICS ON INDICATORS AND EVALUATION Indicators Indicators are used at every level of the e-strategy development and implementation process to define and describe the deliverables, outputs, outcomes, and impact of the strategy components. Well-designed indicators describe results by highlighting the success or failure of objectives, and thus act as `triggers' to managerial decision-making during implementation. Likewise, such indicators also provide a means for evaluation of the success of the strategy once implementation is complete. While not a substitute for comprehensive evaluation, indicators can complement in- depth analyses by providing preliminary assessment or "quick-tracking" at a lower cost.46 Indicator Design. Indicators can be viewed as belonging to one of three types: 1. Soft: Reflect qualitative values, such as perceptions of the existence or quality of a number of ICT services. For example, "Do you think better training has improved productivity? Answer on a scale of 1 ­ 10." 2. Hard: Reflect concrete and quantitative measurements. For example GDP/capita, teledensity, adult literacy. 3. Binary: Reflect the simple positive/negative state of an objective. For example "Does an independent telecommunications regulator exist? Answer: Yes/No". 46Gannon, C. and Shalizi, Z. 1995. "The Use of Sectoral and Project Performance Indicators In Bank-Financed Transport Operations, Environmentally Sustainable Development." World Bank. 67 Each indicator should be measurable in terms of quantity, quality, and time (QQT). A four-step process illustrates this design guideline:47 Example 1 Example 2 1. Basic Indicator More and better computer engineering More software and technology s ervice graduates. startups 2. Add Quantity The quantity of graduates increased The quantity of startups increased (how much) from 5,000 to 14,000. from 25 to 75. 3. Add Quality The number of graduates passing The number of technology startups (what kind of change) standard exams (40% female / 60 % (50% software development, 50% male) from lower income families in technology services) from non northwest districts increased from government-subsidized programs 5,000 to 14,000. increased from 25 to 75. 4. Add Time The number of graduates (40% / The number of technology startups (by when) 60%) from lower income families in (50% software development, 50% northwest districts increased from software services) from non- 5,000 to 14,000 per annum starting in government subsidized programs year 3 of project. increased from 25 to 75 in year 4 of the investment credit program. Leading and Process Indicators. Some strategies show impact only after completion of the implementation. For long-term strategies, it may not be possible to wait long enough to validate the outcomes or impact of the strategy. Traditionally, impact evaluation has been a separate process from monitoring project progress. However, since most e-strategies demand a high degree of responsiveness and flexibility in order to improve and direct the implementation, feedback is needed in the nearer term about the potential effectiveness of the project. This may require a special kind of indicator, called a Leading Indicator, that may provide some confidence (or cause for alarm) early in the life of the project. Leading indicators do not measure the impact we desire, but they do signal whether or not a process is in place that is moving toward the desired impact. For example, the ICT sector may not fully develop until year 7 after the start of a government investment incentive program. It might be acceptable to consider the size and survival rate of younger companies at interim stages of development as a leading indicator of long-term impact. Proxy Indicators. Under certain circumstances it may not be feasible to use certain indicators, they being either too difficult or too expensive to gather and use. For example, measuring actual rural Internet usage may be difficult or costly. Instead, proxy indicators ­ such as the number of computers, or the number of regional ISP accounts ­ may be a more practical indicator of the general trend. 47This information comes from the World Bank Logframe Methodology Handbook. 68 Evaluation At its core, evaluation of e-strategies entails an assessment of: § What an e-strategy (or component) has achieved § How well it has gone about achieving it Evaluating what an e-strategy has achieved can often only be done some years after the start of the strategy. This is particularly true when it comes to the outcomes and larger development impact of the strategy. Furthermore, while it is likely (or at least hoped) that the strategy will lead to positive developmental outcomes, establishing clear causality is often complex. However, doing so is particularly important to justify future investment in e-strategy initiatives and promote ICT for development. The evaluation of how a strategy has been undertaken can take place once the implementation is complete. Lessons learned from this process can then be incorporated into the design and execution of future initiatives. However, some evaluation should also be undertaken during implementation. This entails conducting periodic assessments of distinct initiatives, to understand areas of comparative strength (to build on them further and introduce them to other elements of the strategy) and weakness (in order to make adjustments or bring them to an early close if necessary). Mid-stream evaluation and related realignment of the implementation plays a key role in ensuring that the strategy is implemented well and resources are spent efficiently. It will ultimately help to ensure that the strategy meets its intended goals and that a positive evaluation is ultimately concluded upon. Targets and Benchmarks As this toolkit makes clear, e-strategies require clear targets and goals on which to focus their activities, and indicators with which to measure the achievement of those targets and goals. The toolkit presents suggestions for appropriate indicators to give meaning to, and make measurable, some of the goals set by e-strategies. However, this toolkit does not give an indication of how much change should be undertaken in each area. For example, the toolkit highlights teledensity as one of many measures of ICT infrastructure development, but does not address at what rate teledensity should grow. Setting such goals, however, is a key element of both the formulation as well as the monitoring and evaluation of e-strategies. Establishing meaningful guidelines for setting targets is particularly challenging. Because every country begins at a different stage of development in the various areas addressed by e-strategies, goals and rates of advancement are likely to vary considerably from country to country. The level of resources available for undertaking any given e-initiative will also vary by country, which will influence targets and growth rates as well. However, there are some useful starting points. Chief among them is comparison to other countries. If a comparator country is more advanced in an area of importance, for example ICT infrastructure, then strategy-makers may chose to put a considerable amount of strategic emphasis on this area and direct comparatively more resources to it. It is also on the basis of such a 69 comparison that reasonable estimates of growth targets can be established. If a comparator country was able to grow its infrastructure, measured by teledensity for example, at a rate of X percent per year, strategy-makers may choose to establish the same (or a slightly more ambitious) target for their own ICT infrastructure development.48 This places a premium on the careful selection of appropriate comparator countries. A number of criteria may be used, such as countries from within the same region or income bracket. Countries that have undertaken similar development policies may also be selected, or countries with a similar size economy. Whatever combination is used, it is important that selection is done well. If comparator countries are too far advanced, unrealistic targets may be set, making the strategy unachievable. On the other hand, if the comparator countries are not sufficiently advanced, the strategy's targets may not be ambitious enough, and the country may lose opportunities to make far-reaching advances in ICT development. 48Information on other countries' performance is not always readily available. E-readiness assessments provide a good potential source, as do ICT indexes such as the Networked Readiness Index (see Annex 2 on Indices and Data). 70 ANNEX 2. DATA AND INDICES A number of agencies (private or public) offer an array of indices, that countries can chose to use in order to assess their advancement in ICT development and compare their progress to that of other countries. These indices rely on a range of data that are publicly available ­ though not always for free ­ and cover a wide variety of countries49. This module looks at four public indices that are focused on (or are particularly relevant to) ICT development. They include: § The World Bank Institute's (WBI) Knowledge Assessment Methodology (KAM) and the Knowledge Economy Index (KEI)50 § The Global Information Technology Report's (GITR) Networked Readiness Index (NRI) § Orbicom's Index of countries' "Infostates" § International Telecommunications Union's (ITU) Digital Access Index (DAI). This module begins with an explanation of what an index is, and summarizes each of the indices in turn, looking at their focus and structure. It then considers similarities between them and the data on which they are based. I. WHAT IS AN INDEX? An index aggregates a variety of indicators into a single over-all value that allows a country to An index aggregates a variety of indicators compare itself to other countries, as well as to into a single over-all value that allows a country to compare itself to other countries, compare its current position to that of its past. as well as to compare its current position to Some indices are an aggregation of a number of sub- where it was in previous years indices that focus on particular sectors or themes of a country's ICT development (such as ICT infrastructure or education). Such comparisons can provide a quick and effective means to identify areas of relative strength or weakness and focus on key issues. Comparisons over time can also help policy-makers form an understanding of the success of their ICT interventions.51 The choice of the number of indicators used in an index entails a trade-off between the number of indicators and the number of countries covered. Some indicators, for example, are not available for certain countries in existing datasets. Therefore, on the whole, the greater the number of indicators, the lower the country coverage (i.e. the DAI, with only 8 indicators, covers 176 49 Much of the same data is also often used in conducting country e-readiness assessments though, depending on the focus of the e-readiness assessment and the methodology used, additional country- or sector-specific data is likely to be included in the assessments. 50WBI's KAM allows users to work with a full set of data of 76 indicators, or with a basic set of 14 indicators. The basic set are used to compile the KEI (based on 12 indicators) and the KI (based on 9 indicators). 51See Annex 2 for basic introduction to Indicators. 71 economies). Indices with larger data sets either make do with gaps in the data or make reasonable estimations based on a country's level of development or comparisons to other similar indicators. Once the set of indicators and countries are chosen, the data in each index is then standardized to allow for easier aggregation and comparison.52 II. REVIEW OF KEY INDICES There are a number of ICT-related indices available. They include the four that are covered in this module as well as others developed by private firms such as International Data Corporation's Information Society Index, or the Economist Intelligence Unit's e-readiness rankings. This module focuses on publicly available indices that are relatively easy and cost-effective (or even free) to access. Organization ­ Index Countries Indicators Type of indicators Main source WBI - KAM/KEI 121 76 Hard and soft WDI, WEF, IMD GITR - NRI 102 48 Hard and soft WDI, WEF, ITU Orbicom - Infostate 139 19 Hard ITU ITU - DAI 178 8 Hard ITU WBI's Knowledge Assessment Methodology (KAM) and the Knowledge Economy Index (KEI) Web: http://info.worldbank.org/etools/kam2004/index.htm. The World Bank Institute has developed a comprehensive set of data relating to the knowledge economy that is accessible through an online analysis tool - the Knowledge Assessment Methodology (KAM). It comprises 76 indicators and covers 121 countries. Focus WBI's knowledge assessment methodology (KAM) "helps to benchmark how an economy compares with its neighbors, competitors, or others it wishes to emulate. The KAM is designed to help countries understand their strengths and weaknesses in making the transition to the knowledge economy. It is thus useful in identifying the challenges and opportunities that a country faces, and where it may need to focus policy attention or future investments." 52 Usually this entails dividing the data by the total population, or GDP per capita. It also requires "normalizing" the data, which entails ranking countries with regard to individual or groups of indicators (on a scale of say 0-10). This is done for all indicators, and allows for easier aggregation and comparison. 72 Structure The KAM is focused on four `pillars' of the knowledge economy around which WBI's methodology focuses, namely: § economicincentiveandinstitutionalregimes § education § innovation § ICTinfrastructure. KAM provides online access to 76 indicators that relate to each of these four key areas, as well as over-all country performance and gender data. Through various online analysis and graphing tools, it is possible to compare countries using any of the 76 variables. The main focus of the KAM is on the "basic scorecard," a selection of 14 indicators from the list of 76 (three from each of the four pillars and two country performance measures). The basic scorecard forms the basis for the knowledge economy index (KEI), an index based on the average of a country's performance across each of the four pillars of the knowledge economy. An unweighted version of the scorecard is also available, in which data for the innovation pillar is presented in absolute terms and not per capita terms.53 The KAM provides a variety of analytical tools to allow users to self-select data and conduct comparisons against individual countries, groups of countries (grouped by region, income, or human development index categories), or over time (comparing most current data to data from 1995). The data is presented in an array of graphical formats, including cobweb diagrams, bar graphs, and color-coded maps. Furthermore, all data is accessible in both original format and as a normalized score. 53Innovation variables can be presented in terms of absolute values because "in innovation, absolute size of resources matters, because there are strong economies of scale in the production of knowledge and because knowledge is not consumed in its use. Populous countries, such as India and China, have a critical mass of innovative capacity which is not reflected when scaled by population." 73 GITR's Networked Readiness Index (NRI) Web: http://www.weforum.org/site/homepublic.nsf/Content/Global+Competitiveness+Programme% 5CPurchasing+and+Contact+Information The Networked Readiness Index (NRI) is part of the Global Information Technology Report (GITR), developed by INSEAD, WEF and the World Bank's infoDev program. The most recent version of the NRI covers 102 countries, comprising a mix of 48 quantitative and qualitative indicators. The NRI has been compiled for the past three years, covering an increasing number of countries (77 in 2001, 88 in 2002, 102 in 2003). For countries for which data is missing, reasonable estimates are developed based on a variety of techniques including regression analysis (using other variables that are highly correlated to the missing variable) or clustering techniques (estimating data based on country groups of a similar GDP per capita). Focus "The Networked readiness Index (NRI) is defined as the degree of preparation of a nation or community to participate in and benefit from information and communication technologies (ICT) development. By looking at the overall index of a country, one can get an idea of how a country compares to other countries; specifically, to countries facing similar global and ICT challenges."54 Structure The structure of the GITR has changed since its inception. The first year focused on network use and enabling factors, looking at variables of access, policy, society and the economy. The focus has shifted in the last two years' reports to put more emphasis on key stakeholders of ICT development. The table below summarizes the current structure. Index Component Index Sub-indices Market Environment Environment Political and Regulatory Environment Infrastructure Environment Networked Readiness Individual readiness Index Readiness Business readiness Government readiness Individual usage Usage Business usage Government usage The change in structure from the first year means that the comparison of a country's ranking on the index to previous years must be undertaken with caution. However, the index has not changed from 2002 to 2003, which allows for better cross-year comparison. The component and sub-indices allow users to drill down to uncover specific trends and areas of relative strength and weakness. For example: 54Dutta, S., Lanvin, B., Paua, F. (eds). 2003. The Global Information Technology Report, 2003-2004. Oxford. p. 217 74 § India scores low on infrastructure, but comparatively higher on environment (due to large numbers of IT experts) § Korea and Malaysia score high on government readiness in both readiness and usage. § Singapore scores high for government and business usage, but low for individual usage. The GITR presents individual country pages for 102 countries. Rankings are provided for each country as whole, as well as for the three sub-indices and all indicators. Graphs for each of the 48 indicators that compare all countries are also published, presenting specific scores but not the underlying data. Orbicom's `Infostate' Index Web: http://www.orbicom.uqam.ca/projects/ddi2002/2003_dd_pdf_en.pdf The `Infostate' index was developed by Orbicom, a network of UNESCO communications chairs and associated members created by UNESCO and Université du Québec à Montréal (UQAM). The index focuses on developing countries, covering 139 in total, using 19 indicators. It compares countries to each other as well as over time (using baseline data from 1996.) Focus The `Infostate' index is developed as an "instrument [to] quantify the Digital Divide across countries, as well as monitor its evolution."55 This enables policy-makers and the international community to identify needs, allocate investments and monitor performance. Structure "Infostate" comprises two sub-indices: infodensity and info-use. These reflect on the productive capacity (infodensity) and the consumption (info-use) of ICTs in a given country. The main analysis in the report focuses on five income groupings of countries, analyzing trends between the groupings with regard to their over-all infostate, infodensity, and info-use. The report also analyzes changes over time, using baseline data from 1996. The analysis also compares countries against a country "Hypothetica", which is the average of all countries included in the model, and "Planetia", which represents a weighted average of all countries (weighted for population). By using a hypothetical country that represents the global average, the authors introduce the idea of a moving target in terms of achievement of ICT development goals (i.e. while all countries are advancing in ICT development in absolute terms, in relative terms some are doing considerably better than others). 55Sciadas, G. 'Monitoring the Digital Divide ... and beyond'. (Orbicom, 2003) p. 2. 75 The report presents tables with index scores as well as the underlying data for the countries that are covered. It also presents cobweb diagrams for individual countries that compare 1996 and 2001 data, and compare each country against Hypothetica. ITU's Digital Access Index (DAI) Web: http://www.itu.int/ITU-D/ict/publications/wtdr_03/index.html The International Telecommunications Union recently launched the Digital Access Index (DAI) that covers the greatest number of countries of the four indices presented here, totaling 178 economies. The index is limited to eight variables. A key focus of the DAI is to measure change over time. Focus The DAI was developed to "measure the overall ability of individuals in a country to access and use ICTs."56 Its main objectives are to: § measureacountry'scapacityforusingICTs § coverasmanycountriesaspossible § ensuretheindexisastransparentaspossible. Structure The eight indicators used in the DAI are grouped according to five main categories, each of which comprise a sub-index: infrastructure, affordability, knowledge, quality, and usage. They are then combined to form the overall index. Countries are grouped according to their aggregate DAI score into one of four categories: high, upper, middle, low. Areas of Commonality The number of indicators used in the different indices varies, ranging from as many as 48 to as few as eight indicators (NRI and DAI respectively). All four indices share some data in common. The table below highlights indicators that are common to three or more of the indices. (Note: The table represents all of the data that is available for analysis in the KAM. The Knowledge Economy Index (KEI) itself is developed based on only 12 of these 76 indicators). 56ITU. 2003. World Telecommunication Development Report, 2003. ITU. p.103. 76 Organization/Report Indicator Data source WBI GITR Orbicom ITU (76) (48) (19) (8) Adult literacy rate (% age 15 and above) UNESCO x x x x Telephones per 100 people ITU x x x x Tertiary enrollment WDI x x x x (b) Internet users per 100 people ITU BB (a) x x x Secondary Enrollment WDI x x x (b) Mobile phones per 1000 people ITU x x x Computers per 100 people ITU x x x TV Sets per 100 people WDI x x x Internet hosts per 1000 people ITU x x Press freedom Freedom House x x Researchers in R&D UNESCO x x Availability of Venture capital WEF x x Public spending on education as % of GDP WDI x x Extent of Staff Training WEF x x Availability of local management education WEF x x Well educated people do not emigrate abroad IMD x x E-Government WEF x x Cost of internet access ITU x x Int'l Internet bandwidth per 100 people ITU BB (a) x x Broadband subscribers per 100 people ITU BB (a) x x Primary enrollment WDI x x (b) (a) From ITU Internet Reports 2003: Birth of Broadband; http://www.itu.int/osg/spu/publications/sales/birthofbroadband/ (b) Primary, secondary and tertiary enrollment are included as a single indicator in DAI Eight indicators are common to three or more of the indices (inside red above box). In general terms, these indicators are related to ICT infrastructure access, education levels, and usage. The majority of indices have a set of indicators (or a sub-index) that predominantly focus on each of these three aspects of ICT development. Focus Index Sub-Index WBI - KAM/KEI ICT Infrastructure GITR - NRI Environment (Infrastructure) ICT Infrastructure Orbicom - Info-density (Networks) 'Infostate' ITU - DAI Infrastructure WBI - KAM/KEI Education GITR - NRI Readiness (Individual) Education Orbicom - Skills 'Infostate' ITU - DAI Knowledge GITR - NRI Usage (Individual) Usage Orbicom - Info-use (Uptake) 'Infostate' ITU - DAI Usage Three indicators appear across all four indices: adult literacy rates, telephones per 1000 people, and levels of tertiary education. The other indicators that the four indices share in common are a 77 reflection of their respective areas of focus. Both the KAM/KEI and the NRI have a shared focus on businesses' ICT development and the innovation climate (covered under the KAM's Education and Innovation pillars, and the NRI's sub-indices on the Market Environment and Business Readiness). These include indicators such as: perceptions of staff training, access to venture capital financing, access to a local management education, and the number of researchers in R&D. Other areas of commonality include indicators that reflect the quality of ICT access, generally addressed in terms of bandwidth, for the most part highlighted by Orbicom and ITU. Many of the common data sets come from a limited number of sources. The most common sources are ITU data, indicators from the World Bank's World Development Indictors (WDI) data set, and the World Economic Forum's (WEF) perceptions surveys.57 Summary There is a range of public and private indices that provides easy access to key ICT data accompanied by useful analysis. Many use the same indicators, with just over 20 indicators being used by two or more of the four indices that were reviewed in this module. The different indices and related tools or reports each have various strengths. For example, the KAM provides access to a wide range of data with useful online analytical tools. Though focused on the knowledge economy, many of the indicators used are also particularly relevant to ICT development. The NRI provides the ability to drill down from the overall index to component indices, sub-indices and eventually individual indicators, to locate areas of comparative strength or weakness in a country's own ICT performance as well as relative to other countries. Orbicom's `Infostate' index provides original analysis against a hypothetical global country, while the DAI provides broad country coverage ranking 178 economies. The majority of the indicators that appear in the indices are ones that are most relevant to tracking change at higher levels of the e-strategy pyramid, the layers that relate to strategic priorities (outcomes) and policy goals (impact). The initiatives and actions undertaken as part of the e- strategy will in time coalesce to show up as change in these indicators. However, there is likely to be a substantial time lag before such changes appear. For example, privatization of a state-run telecommunications enterprise is likely to lead to higher teledensity levels, but only after a number of years. Therefore, portions of the data that are required to monitor and evaluate e-strategies will not be available on these indices or through other cross-country sources. This is particularly so for indicators that are required to track and reflect progress made at lower levels of the strategy pyramid ­ on individual initiatives and actions. Such indicators are in large part determined by how initiatives chosen as part of the e-strategy are designed, and so will be unique to many countries and their own approaches to ICT development. 57These surveys ask respondents to rate their perceptions of a particular aspect of the country on a scale of 1-7. 78 ANNEX 3. LIST OF E-STRATEGIES The following e-strategies (national or sub-national) were reviewed as part of the background research for this toolkit.58* Country (or region) Web address at which the strategy can be found Albania http://www.undp.org.al/?elib,428 Angola http://www.uneca.org/aisi/nici/Angola/angola.htm Azerbaijan http://www.nicts.az:8101/ Bangladesh www.bccbd.org/html/itpolicy.htm Bhutan http://www.dit.gov.bt/bips/documents/documents.htm Bolivia http://www.aladi.org/nsfaladi/ecomerc.nsf/0/E8147919B55D97A40325 6BEA004D2EDA/$File/lineamientos.pdf?OpenElement Chile http://www.agendadigital.cl/agenda_digital/agendadigital.nsf/vwDocume ntosWebLink/27363116E8E6631704256E5800549FE3?OpenDocument China (Hong Kong) http://www.info.gov.hk/digital21/eng/strategy2004/strategy_main.html Colombia http://www.agenda.gov.co/ Czech Republic http://www.micr.cz/scripts/detail.php?id=1288 Dominican Republic http://www.edominicana.gov.do/interfaz/contenido.asp?Ag=1&Categori aNo=3 Egypt http://www.uneca.org/aisi/nici/Egypt/egypt.htm Finland http:://www.tietoyhteiskuntaohjelma.fi/esittely/en_GB/introduction Ghana http://www.uneca.org/aisi/nici/Ghana/ghana.htm India (National) http://www.gipi.org.in/ITPolicyInIndia.php India (Andhra Pradesh) http://www.gipi.org.in/state_policy/andhra.pdf India (Delhi) http://delhigovt.nic.in/icetpolicy.pdf India (Haryana) http://www.gipi.org.in/state_policy/haryana.pdf India (Orissa) http://www.gipi.org.in/ITPolicyInIndia.php Indonesia http://www.sdnbd.org/sdi/issues/IT-computer/policy/indonesia.pdf 58A more thorough review and analysis of over 40 e-strategies is currently being conducted by the World Bank's Global ICT Department. The findings will be integrated into subsequent versions of this toolkit. * Over time, these links may become outdated. Visit http://www. worldbank.org/ict to link to the online version of this report for the latest links. 79 Ireland http://www.taoiseach.gov.ie/index.asp?locID=181&docID=1773 Jamaica http://unpan1.un.org/intradoc/groups/public/documents/CARICAD/ UNPAN009931.pdf Japan http://www.kantei.go.jp/foreign/policy/it/index_e.html Jordan http://www.reach.jo/ Korea http://www.ipc.go.kr/ipceng/public/public_view.jsp?num=2007&fn=&r eq=&pgno=3 Mauritius http://ncb.intnet.mu/ncb/downloads/Downloads/Reports%20and%20s urveys/Others/finalntp.doc Mozambique http://www.markle.org/downloadable_assets/mz_final_ict_strategy.pdf Namibia http://www.uneca.org/aisi/nici/Documents/ICT%20Policy%20Docume nt%20Ver%208.2.pdf Nigeria http://www.uneca.org/aisi/nici/Documents/IT%20policy%20for%20Ni geria.pdf Norway http://odin.dep.no/nhd/engelsk/publ/rapporter/bn.html Poland http://www.informatyzacja.gov.pl/_d/files/projects/epoland- the_strategy_on_the_development_of_the_information_society.pdf Romania http://unpan1.un.org/intradoc/groups/public/documents/UNTC/UNP AN016044.pdf Russia http://www.e-rus.ru/eng Rwanda http://www.uneca.org/aisi/nici/Documents/rwanpap2.htm Singapore http://www.ida.gov.sg/idaweb/aboutida/infopage.jsp?infopagecategory= &infopageid=I226&versionid=2 Slovenia http://unpan1.un.org/intradoc/groups/public/documents/UNTC/UNP AN015723.pdf South Africa http://www.tsicanada.com/documents/Strategy.pdf Tanzania http://www.tanzania.go.tz/pdf/ictpolicy.pdf Thailand http://www.nectec.or.th/intro/e_nationalpolicy.php Trinidad & Tobago http://www.gov.tt/nict/ Tunisia Hard copy only Ukraine http://www.e-ukraine.com.ua United Kingdom http://e- government.cabinetoffice.gov.uk/assetRoot/04/00/60/69/04006069.pdf Venezuela http://www.mct.gov.ve Viet Nam http://mpt.gov.vn/english/introduction/?thucdon=in 80