84060 8 December 2008 World Bank Marrakech Action Plan for Statistics Report of an Independent Evaluation Christopher Willoughby and Philip Crook 2 This page is blank 3 PREFACE This evaluation was commissioned for the MAPS Advisory Board by the Development Data Group (DECDG), which has been responsible from the beginning for the World Bank contribution to the Marrakech Action Plan for Statistics. In line with standard requirements of the DGF Council, independent consultants were selected competitively on the basis of responses to a public announcement requesting applications from interested individuals.The work has focused very much on the use of the funds made available by DGF, with relatively little attention to the broader objectives of MAPS except to the extent they were reflected in arrangements made for the projects financed. The work began with a Phase 1 focused on the viewpoints of the concerned headquarters bodies, of the World Bank and of the participating intermediary agencies. A few months after completion of the report on Phase 1, a second contract was awarded to the same consultants to conduct discussions in a small number of the intended ultimate beneficiary countries about their experience of MAPS activities, and to integrate the findings into the final report. This document presents the combined results from both phases of the work, providing the first comprehensive review of MAPS since the start of the program in 2005-06. The work has been greatly assisted by many people at the World Bank headquarters, in the other international agencies involved in the Partnership, and in Ethiopia, Malawi, Mali and Niger which generously hosted brief visits to review their experience of the programs. We are most grateful for the patience and care with which our inquiries have been attended to. The authors take full responsibility for errors and omissions remaining. 4 This page is blank 5 List of Abbreviations used in the Report AASDA African Association of Statistical Data Archivists ADP Accelerated Data Programme BCEAO Banque Centrale des Etats de l’Afrique de l’Ouest CALCINDIC Calculate Indicators Tool (UIS) CAR Central African Republic CCSA UN Committee for the Coordination of Statistical Activities CIS Commonwealth of Independent States CSIDB Country Statistical Information Database (WB) CSO Central Statistics Office DCI Data Capture Infrastructure (UIS) DDI Data Documentation Initiative DECDG Development Data Group (of World Bank) DFID UK Department for International Development DGF Development Grant Facility DHS Demographic and Health Survey DIVA Data and Indicators Verification Application (UIS) DOTS Directly Observed Treatment System (for TB) DQAF Data Quality Assessment Framework ENSAE Ecole Nationale de la Statistique et de l’Analyse Economique, Dakar ENSEA Ecole Nationale de Statistique et d’Economie Appliquée, Abidjan EFA Education for All EMIS Education Management Information System ERS Error Reporting System (UIS) EU European Union FAO Food and Agriculture Organization FY Fiscal Year GIS Geographical Information Systems GPP Global Programs and Partnerships GPS Global Positioning System HMN Health Metrics Network ICT Information and Communications Technology IDA International Development Association IHSN International Household Survey Network IIEP International Institute for Educational Planning ILO International Labour Organization IMF International Monetary Fund INSEE Institut National des Statistiques et des Etudes Economiques, Paris IFORD Institut de Formation et de Recherche Démographique, Yaoundé ISCED International Standard Classification of Education ISSEA Institut Sous-Régional de Statistique et d’Economie Appliquée, Yaoundé LIFI Legislative and Institutional Framework Index LRE Light Reporting Exercise LSMS Living Standards Measurement Survey MAPS Marrakech Action Plan for Statistics MDG Millennium Development Goal MICS Multiple Indicator Cluster Survey MSB Monitoring Service Branch (of UN Habitat) NADA National Data Archive 6 NSDS National Strategy for the Development of Statistics NSO National Statistical Office NSS National Statistical System OECD Organisation for Economic Cooperation and Development OPCS Operations Policy and Country Services P&R2 UN Principles and Recommendations for Population and Housing Censuses Revision 2 PARIS21 Partnership in Statistics for development in the 21st century PDO Project Development Objective PER Primary Enrolment Rate PES Project for Education Statistics PREM Poverty Reduction and Economic Management PRSP Poverty Reduction Strategy Paper ROSC Report on the Observance of Standards and Codes (IMF) SCBC Statistical Capacity Building Committee (WB) SDMX Statistical Data and Metadata Exchange SEE South Eastern Europe SNA93 System of National Accounts 1993 revision SQAF Survey Quality Assessment Framework SSA Sub-Saharan Africa STATCAP Statistical Capacity Building Programme (WB) SWAp Sectorwide Approach TA Technical Assistance UIS UNESCO Institute for Statistics UN ECE UN Economic Commission for Europe UNESCAP UN Economic and Social Commission for Asia and the Pacific UNESCO United Nations Educational, Scientific and Cultural Organisation UNESCWA UN Economic and Social Commission for West Asia UNFCCC United Nations Framework Convention on Climate Change UNFPA UN Fund for Population Activities UNHabitat United Nations Human Settlements Programme UNHCR United Nations High Commissioner for Refugees UNICEF United Nations Children’s Fund UNSC United Nations Statistics Commission UNSD United Nations Statistics Division UNWTO United Nations World Tourism Organisation WB World Bank WBI World Bank Institute WHO World Health Organisation 7 Marrakech Action Plan for Statistics Report of an Independent Evaluation Table of Contents Page Preface 3 List of Abbreviations used in the Report 5 Executive Summary 9 I. Introduction 19 Approach to Evaluation 20 Structure of the Report 22 Monitoring and Evaluation 22 Adequacy of Evidence 23 II. The Relevance of MAPS 29 III. Effectiveness of MAPS and Each of its Main Components in Delivering on Intended Purposes 30 IIIA. The MAPS Partnership – A Catalyst for Improved Statistical Capacity Building 25 IIIB. DGF Funding to Institutions – Making Room for Innovation 33 Paris 21 General Operations 33 International Household Survey Network 34 UNESCO Institute for Statistics – IT Operations and Other Services 35 UN Statistics Division – Supporting Census 2010 37 UN Habitat – Urban Indicators and the MDGs 39 IIIC. DGF Funding for Capacity Building via Institutions 40 NSDS Design, Review and Implementation 40 Accelerated Data Program – Discovering Data 43 UIS Statistical Capacity Building 44 UNSD Regional Workshops 46 UNECE Gender Statistics 46 IIID. International Evaluation Data 47 IIIE. Specific Answers 50 IV. Cost-Efficiency of MAPS 53 8 V. Country-Level Perspective 57 Suggestions for the Future 59 VI. Assessment of Governance and Management Structure and Patterns 61 VII. Main Suggestions for Further Action 67 Improving the Performance of the MAPS Partnership 67 Possible Future DGF Support for Programs Started 68 Statistical Annex 73 Table 3.1: MAPS activities in IDA countries in increasing order of CSIDB 2007 Assessment 74 Table 3.2: Main stages in the introduction of strategic planning for statistics (NSDS) showing Progress achieved by IDA member countries of Sub-Saharan Africa 2004-08 81 Table 3.3: CSIDB definitions and scoring 84 Table 3.4: CSIDB scores for IDA countries, 2004-07 86 Table 3.5: CSIDB score for potential indicators of capacity - 2004 to 2006 89 Table 3.6: Availability and source of selected MDG indicators at September 2008 for Ethiopia, Malawi, Mali and Niger 90 Table 3.7: Changes in availability and source of selected MDG indicators in SSA IDA countries, around 2002 and around 2005 98 Table 3.8: Water, sanitation and slum MDG indicator data availability 99 Table 3.9: International reporting of basic CPI data for IDA SSA countries 100 Table 3.10: International Reporting of Basic External Trade data for IDA SSA 101 countries Other Annexes 103 1. List of Persons Interviewed for the Evaluation 103 2. Summary status of recent statistical planning in Sub-Saharan Africa IDA countries, September 2008 110 3. Ethiopia: experience with MAPS 120 4. Malawi: experience with MAPS 125 5. Mali: experience with MAPS 129 6. Niger: experience with MAPS 134 7. Assessment by UIS of Some Main Developments in its Work over the Last Few Years 142 8. Paris 21: NSDS programme 153 9. Paris 21: IHSN and ADP 155 10. UNESCO Institute for Statistics 158 11. UN Statistics Division 161 12. UN Economic Commission for Europe 163 13. UN Habitat 165 14. Terms of Reference for an Independent Evaluation of the Marrakech Action Plan for Statistics 168 9 Marrakech Action Plan for Statistics Report of an Independent Evaluation Executive Summary i. The Marrakech Action Plan for Statistics (MAPS) is one important element of the effort underway to improve the effectiveness of development aid and the management of development more broadly. It was developed and agreed at the 2nd International Roundtable on Managing for Development Results that took place at Marrakech, in Morocco, in February 2004. ii. MAPS aims at increasing consciousness among country leaders and development supporters of the significance of statistical work and activities. It seeks to promote greater cooperation and partnership among different bodies in support of countries’ own efforts to strengthen their statistical services. iii. The plan agreed at Marrakech prioritised six action areas. Two depended mainly on action at the international level, which has moved forward, if less than hoped. Four, more specific areas, required early action at both national and international levels: strategic planning for statistics, the upcoming census round, management of statistical surveys, and MDG monitoring. iv. The novel feature of the plan was to assemble an informal partnership of international agencies with relevant experience who would be able, with the aid of grant finance of limited scale, to develop programs specifically focused to helping poorer developing countries achieve stronger capacities in these four areas. v. A total of eight such programs were developed, including five of relatively large scale, and they have received aggregate financing of nearly $7.5 million each fiscal year since 2006 from the World Bank’s Development Grant Facility (DGF). vi. The principal purpose of the evaluation has been, within the overall MAPS scheme, to examine experience with these programs, to see whether they should be continued and how they might be improved. The evaluators were guided to focus not on program details but on overall contribution to development, and especially to the quality of decisions by governments and others about broader development policies and programs. vii. In view of the scale of the task, the Development Data Group contracted the work in two phases. The first, in April-May 2008, focused on discussions with concerned World Bank staff and the partner agencies, to understand and assess what had actually been done. The second, in August-September 2008, emphasized discussions in a small sample of beneficiary countries to assess what difference the MAPS initiatives had made to development of their capacities to generate and use statistics. viii. Findings from the country visits were very consistent with the conclusions we had drawn from the agency discussions about the individual programs. The four sample countries – Ethiopia, Malawi, Mali and Niger – had been selected deliberately as 10 relatively advanced in both strategic planning and use of MAPS services. Improvement in some fields was slow and difficult, but the overall balance sheet backed the cautious optimism about MAPS that had emerged from the earlier work. ix. An interesting common feature of almost all the programs financed by DGF under the MAPS initiative is that they involve important capacity-building actions for developing countries at agency headquarters as well as in the countries themselves. Headquarters actions produce a wide variety of relevant products, from agreed statistical definitions and standards, or improved techniques for storing and exchanging data, to new international associations. x. The objectives of the DGF-supported programs have never been limited to only one of the MAPS action areas. Most in fact contribute to several. Nonetheless, for the sake of clarity, this summary will present the main findings about each program and its effectiveness together, and under the action area to which it most closely relates. Mainstreaming Systemwide Strategic Planning xi. The central, flagship objective of MAPS for the short-run period that it mainly focused on was that all the poorer countries would have National Statistical Development Strategies (NSDS) under preparation (or further advanced) by 2006, and in execution at latest by 2010. Support to attainment of this objective has been the main purpose of DGF grants of recent years to Paris 21 (about $1.3 million p.a.). The small, regionally focused project run by UNECE also relates more closely to this objective than others. xii. Experience to date suggests that the objective remains very valid and worthwhile, even though a significant proportion of countries are likely to fall short of the 2010 target. Large progress has been made. Paris 21 information as of early 2008 indicated that 35 of the 80 IDA countries were implementing a strategic plan, and another 40 were preparing one. Our review for Sub-Saharan Africa, based on multiple sources of information, indicates that the number of countries implementing a plan there has risen from 13 in 2004 to 19 in 2008, with all but one of the remaining 20 countries preparing one. xiii. Care has to be taken, as Paris 21 often emphasizes, in interpreting figures of this sort. By no means all the plans under implementation are systemwide, let alone comprehensive in coverage of recommended subjects. Plans “under implementation” are often only moving slowly into implementation, while many recorded as being at an earlier stage can hardly be said to be under active preparation. xiv. But the process is moving, due in substantial part to Paris 21. Strong advocacy efforts in all parts of the developing world probably played some part in many country decisions to embark on preparation of a plan. Much more widely influential appear to have been guidance and advice on a participative planning process and comprehensive coverage, including of managerial aspects. These were delivered through conference presentations, pamphlets and documents, website, and personal advice. They appear to have been increasingly followed in the active planning work that has been underway. 11 xv. Paris 21 continues to play a supportive role, in association with regional institutions and development banks, for NSDS work in the more needy sub-regions and countries of Asia and Latin America. But its activity on the subject is mainly concentrated on Sub-Saharan African countries, advising on preparation and implementation, developing materials adapted to different national and donor audiences, and helping to maintain momentum. xvi. The program funded by DGF grants to UNECE has been for an integrated training effort, jointly with the World Bank Institute, to strengthen consideration of gender issues in the statistical offices of seven of the countries that emerged from the break-up of the Soviet Union and some of its neighbors. It appears to have led to useful follow-on initiatives in most of the countries and is likely to have a gradually widening effect on the offices’ work planning. Preparation for 2010 Census Round xvii. The MAPS objective was to ensure a successful ‘2010 round’ (i.e., covering population censuses between 2005 and 2014). It was considered a high priority because of the importance of censuses to much other statistical work and the rather large number of countries that had dropped out of the 2000 round. Prospects for the current round do appear to be better but some 70% of censuses normally occur only in the middle years (i.e., 2009-11 in this round). xviii. The UN Statistics Division in New York takes an active part in promoting and coordinating censuses and generating materials for census managers. DGF has been providing $1 million per year to enable UNSD to do much more than normal for IDA countries, especially in terms of appropriate manuals and technical systems, a website with comprehensive documentation on methodological standards and good practices, a census question bank, and a worldwide program of regional workshops on key topics of interest to developing countries. xix. The evidence appears quite strong that the UNSD enhanced efforts will result in higher average quality of census results in this round than previously, with some carryover in terms of new procedures and techniques for the next round too. Our country visits indicated strong use of UNSD material in the two countries which have already had their census, and similar prospects in the other two. Participant evaluations six months after the UNSD regional workshops suggest continued high appreciation and significant application of learnings. International Household Survey Network (IHSN) xx. MAPS recommended creation of an IHSN, combining main sponsors, financiers and managers of household surveys in developing countries, to introduce more efficient approaches to their conduct, storage and dissemination, and to improve coordination of plans for surveys. The proposal was strongly supported by the international agencies concerned. IHSN was created in September 2004, with a management group including, besides the World Bank and UNSD, also ILO, UNICEF, WHO-HMN, Paris 21 and DFID. 12 xxi. In 2005 IHSN was supplemented by establishment of another program, which had not been foreseen in the original Marrakech plan but would handle most of the country-based work required. It was entitled Accelerated Data Program (ADP), to emphasize the generation of data much needed for policy purposes, whether from improved accessibility of past survey data or implementation of better-designed new surveys. Strong demand caused it to outgrow its envisaged small pilot scale within a few months of its launch. xxii. DGF grants to these very closely connected programs have run about $0.8 million per year for IHSN and $2.0 million for ADP. Not much use has been made of targets in terms of country coverage because the focus was more on developing new and better ways of managing surveys and their results. But country demand has been very strong, and involvement of countries in each program has tended to grow faster than expected. xxiii. The core of IHSN’s work to date has turned out to be on new tools for survey work. It early developed a user-friendly IHSN Toolkit for micro-data management, drawing on U.S. experience. It has made much progress in building up a survey Question Bank, to encourage consistency and facilitate survey design. To enable reliable country storage of survey data, in an internationally consistent format, it inaugurated in 2007 a National Data Archive (NADA), now running in about 10 countries. It is developing a Survey Quality Assessment Framework (SQAF) to improve survey design. xxiv. Responding to very strong demand, ADP has helped spread widely the IHSN Toolkit for survey documentation and storage, and training in its use. It is now being used in about 25 countries, and many past surveys have been retrieved. A further 15 countries are planned to be covered. Several countries have been helped on NADA set- up. A half-dozen or so have been helped with completion of surveys that had run into difficulties, or with design of new surveys. xxv. We have not been able to identify specific cases where the ADP efforts already resulted in the availability of data much needed for policy purposes. Earliest fruits of this sort were expected to be in the form of trend data (from previously under-exploited surveys) that would be of great interest to researchers. Processing or analysis of data from the few new surveys already helped has not yet been completed. Improvements in MDG Monitoring xxvi. Because it was prepared only about 18 months before the UN’s 2005 review of progress on the MDGs, the original MAPS focused entirely on short-term measures. Our discussions at UNSD, which chairs the UN Inter-Agency and Expert Group on the MDGs, indicated that reasonable progress continues to be made. Several of the MAPS programs, including the small UNECE-WBI project, have been contributors. xxvii. A large contributor to this objective has been the UNESCO Institute for Statistics (UIS). It had been receiving DGF support in earlier years and was placed under the MAPS umbrella in FY 2007 at request of the DGF Council, in pursuit of greater cooperation and synergy in statistical capacity building. The annual grant has continued at about $1.8 million, mainly focused on improving, on the one hand, quality, 13 comparability and timeliness of primary and secondary education data gathered from countries and, on the other, data processing and transmission methods. xxviii. Recent DGF funding has played a large part in enabling substantial modernization of methods for handling the worldwide database that UIS is responsible for. Its Data Capture Initiative (DCI) uses a common technology for all surveys, produces them in different languages, and uses Adobe, which requires no special installation. An Error Reporting System (ERS) and other comparable tools have brought economies. UIS is in the process of moving from fixed-date collection campaigns to a system of Rolling Collection and Release, shortening the time between data supply and its reflection in the database. Joint use of SDMX is being developed with Eurostat and OECD. xxix. UIS has also continued to make progress in improving country coverage of the basic statistics on primary and secondary education, their international comparability, quality and timeliness. This has been much helped by periodic meetings on a sub- regional basis with the Education Ministry staff responsible for countries’ own questionnaires and surveys. In a few countries EU aid has enabled UIS to play a much larger role in redeveloping the whole reporting system. xxx. One of the few clear improvements revealed by the extensive review of international databases that we have carried out concerns primary enrolment data for Sub-Saharan African countries: there has been a visible shift from estimated data to country data. Our visits in Africa demonstrated too how all four countries had clearly benefited from UIS advice and exchanges. xxxi. UIS has also been trying to improve data on education expenditure, where response rates from developing countries to its high-quality questionnaire remain only about 60%, and limited to public expenditure. Special effort, focused on eight SSA countries in 2005-07, yielded promising results, but further work is needed, especially to make any progress on private expenditure for education. xxxii. The small program with UN Habitat which was worked up in the months following the Marrakech meeting and supported with annual DGF grants of $200,000 for three years has also helped lay the foundation for better MDG monitoring in regard to slum populations. Habitat developed an index for monitoring security of tenure. As an interim alternative for practical purposes, it successfully steered through the Inter- Agency and Expert Group four alternative characteristics of housing conditions for distinguishing slums. The recommendation has been accepted by the UN agencies. xxxiii. Although coverage and quality of health-related indicators is often considered still one of the weaker aspects of MDG monitoring, it is interesting to find clear improvement over the course of the last decade, in tables regarding source of MDG indicators, for all four of the countries visited. The extent of the improvement, moreover, correlates quite strongly with the relative overall quality of national statistical system planning and management, thus emphasizing the significance of other efforts, including NSDS work, also to MDG monitoring. 14 Governance Arrangements xxxiv. MAPS designers tried to ensure governance arrangements that were light and relied as much as possible on pre-existing structures but nonetheless brought to bear effectively the views of interested stakeholder groups. The DGF Council provides firm guidance on overall program scale, and the internal Statistical Capacity Building Committee (SCBC) on distribution among the constituent programs. Advice on main directions and issues can be obtained from the external MAPS Advisory Board. Day-to- day work is handled by the small MAPS unit and led by the management of the Development Data Group. The structure is efficient and effective. xxxv. The international Advisory Board was intended to provide strategic direction, promote coordination, and monitor implementation of MAPS and especially the DGF- financed components. It was to include very experienced policy makers and statisticians, from a range of different backgrounds. xxxvi. The Board has met once a year, as planned, and provided increasingly valuable advice as participants became more familiar with the programs. The Board assembles an invaluable range and depth of experience (although policy-maker participation has been weak), but it has not so far been given encouragement or time for in-depth discussion of strategic directions. xxxvii. The task assigned to the SCBC was “to promote the coordination of statistical capacity building activities within the World Bank as a key component of the overall results agenda … and report, as required, to the Senior Management Team and the Executive Board.” The Committee has usefully upheld the focus of MAPS on capacity building for generation of statistics to meet national and international needs and yield wider results. But it has met far less frequently than planned, confined its attention to the DGF grant, and not broached the wider within-Bank coordination role requested. xxxviii.One challenging idea for the Advisory Board is that it should seek a mandate from the UN Statistical Commission (UNSC) for promoting cooperation among the statistical community in support of development – just as the Bank itself has a mandate from UNSC for managing the International Comparison Project (ICP). The Board would feel empowered to promote partnerships, and its views would be given more weight by statistical agencies. Stronger support across the statistical community internationally would no doubt be useful, but it is more important that aid agencies collaborate with the statisticians in-country, and with each other. xxxix. It may be more productive, in practice, to concentrate attention on solving the problems in the Board’s operation that were identified above. Now that MAPS is moving beyond its pilot phase and the Statistics for Results Facility is also being created, the Board’s intended role of providing strategic direction needs to be activated. Discussion should move upstream to assessing priorities in improvement of statistical activities to get better overall development, and best ways to advance those priorities. xl. It will also be important to take advantage of forthcoming Board turnover (connected with the normal three-year term for membership) to attract two or three new members with extensive experience in making policy in a developing-country setting and in getting hold of the statistical information and analysis required. If appropriate 15 candidates are reluctant to undertake a full term, they might be offered a Visiting Board Member status (i.e., serving for only a year or two). xli. Careful consideration should be given by the DGF Council and the Operations Policy and Country Services Vice-Presidency to the merits of having the SCBC implement the larger mandate that seems to have been in mind in 2005. The need for a light coordinating overview seems to have grown with the increasing inclusion of statistical and MIS sub-components in IDA operations. DGF grants for statistical operations (e.g., for Health Metrics network) would obtain endorsement from SCBC as well as the concerned Sector Board, as in the case of the UIS program. xlii. The relatively low profile that the Development Data Group have given to MAPS is probably appropriate since it needs simply to be a non-bureaucratic mechanism for encouraging partnership and cooperation among the small number of international agencies involved and to some extent other aid suppliers. Nonetheless, historical and up-to-date information on its activities do need to be made easier to find on the World Bank website than is currently the case – or, if not there, to be provided and regularly updated. Conclusions xliii. The review of effectiveness contained in Chapters III and V of the report and summarized here demonstrates that the various DGF-supported programs are making a difference in the various environments where they operate, and are having substantial impact on the broader objectives which led to their choice. Those objectives are of little changed importance since the time, less than five years ago, that they were prioritised. They are at best half way to attainment. xliv. The various programs are being carried out by partner agencies which are well managed, very dedicated and highly technically competent in their respective fields. The partners have been evolving their activities and offerings in sensible ways in response to demand patterns and emerging opportunities. xlv. Our inquiries have not identified any cases of wasteful or excessive expenditure. But they have highlighted the quite significant savings being gained from technical innovations in the agencies or that will be earned by many developing countries as they apply the new software or procedures that are becoming available from bodies such as IHSN and UIS. The aggregate volume of such productivity gains can be expected to rise in the next few years as innovations continue and spread. xlvi. DGF funding for the present set of programs needs to be maintained at about the current annual level of $7.5 million at least for the next four years through FY 2012. Since commitments for the two small programs (Habitat and UNECE) are ending – due, respectively, to other funds coming forward, and completion of the program – this amount would be available for continuation of the five larger existing programs, with possible room for addition of a small new initiative. xlvii. The main near-term planning work needed is to compare the widely different options that could be envisaged for IHSN and ADP, as to the ranges of statistics and surveys to which they should their apply their efforts, the division of labor between the 16 two bodies, their institutional affiliations, and the extent to which they should depend on core financing as against funding for country work from aid/national budgets. xlviii. It is our understanding that on present prospects DGF would be in a position to consider sustaining the past level of funding for MAPS beyond FY 2010. Today’s uncertainties must nonetheless remind recipients of the contingency that this could prove not possible. Programs which would suffer most severely from sharp reduction in DGF support would be well advised to redouble efforts to attract additional alternative financing. This may apply in particular to ADP and to UIS. Recommendations xlix. Real success stories in statistical development, with the linkages accurately traced from better information to better decisions to more successful development, can yield substantial additional value in terms of building political commitment to statistics in other countries. The MAPS Partnership, in collaboration with national governments and external sources of technical/financial assistance, should explicitly seek to bring about and document some demonstration cases of success. l. Increasing reference to statistical services in the Country Assistance Strategies prepared by the World Bank for IDA countries is to be welcomed. But now that the Bank has its results agenda and the proven instrumentality of MAPS to help support it, coverage of statistical capacities needs to be systematised, to focus on the links between information and development performance, improvements achieved and those most urgently needed. li. Paris 21 and its regional representatives, especially those in Africa, should be asked to prepare, at least for the next 3 or 4 years, six-monthly reports summarizing progress on each of the NSDSs already under implementation, and the problems and obstacles arising. Better still, countries should be encouraged to prepare their own contributions, in line with standards for coverage and length provided by Paris 21. lii. The MAPS Advisory Board should keep these progress reports under close scrutiny. It should review with concerned World Bank staff and other partners what initiatives should be envisaged to help overcome implementation problems. A standing mechanism should be created to enable the MAPS Partnership to offer quickly the best possible advice on specific dilemmas that countries run into with regard to development of statistical services. liii. A useful dimension of the more strategic orientation of future Advisory Board meetings could be consideration of advice from outside bodies that can bring an important perspective, such as the UN Inter-Agency and Expert Group on the MDGs, and deliberation on related new initiatives, such as support for a serious effort with FAO to improve data on the hunger and malnutrition indicator under MDG 1. liv. The Bank’s internal Statistical Capacity Building Committee should now activate the broader mandate it was given in 2005 and fulfil a function of overviewing and fostering coordination, consistency and synergy among Bank actions to strengthen countries’ statistical capacities, whether through IDA-assisted projects, DGF allocations or other channels. Empowering it for this purpose will require the DGF Council to 17 abide by its older, original practice of not approving grants for statistical activities without an endorsement by the Committee. Expansion of Committee membership, to embrace all main sectors and regions, will also be necessary. lv. MAPS should promote design of an intensive, experimental training program, phased over a year, to meet the urgent needs of many younger staff-members, in ministers’ offices and policy divisions, for practice in the interpretation of statistics to diagnose problems, trace their causes, and identify possible chains of steps toward solution. The course would be given in countries’ capital city, and it would also strengthen understanding and demand for statistics from the most senior staff. lvi. As already started by Paris 21, MAPS should work with the broader aid community to find a more satisfactory and lasting solution than seems currently to exist for covering the costs of training statisticians in adequate numbers, especially to degree level and particularly in Africa, where the problem appears sufficiently serious that it could become an effective constraint to NSDS implementation. lvii. MAPS and its component partner agencies should continue to work towards identifying verifiable development objectives for their activities at two levels: (a) ultimate outcome, in terms of the ways that important decisions, or large groups of decisions, are expected to be better as a result of the particular interventions that would have been undertaken, and (b) intermediate objectives, being the values, to be achieved in a year or two or three, of indicators believed to reflect aspects of progress toward the ultimate outcome sought. Both should be discussed for all programs, but it must be recognized that more concreteness and precision on outcome will be possible for some programs (e.g., those of UIS) than others (e.g., IHSN). lviii. Evaluation as well as monitoring of the delivery by NSDSs on their development goals will also be an increasingly important task of the coming years. Particular attention should be given to the higher-level outcome goals, of improvements in decision-making and hence in development, and to how well the resources available for statistical work are distributed among sectors and issues relative to needs and opportunities of these sorts. lix. Partner agencies involved in the development of computer software with a view to its adoption by developing-country statistical offices and units should try to train the local staff up to the level at which they could themselves make small modifications or elaborations to the computer program. In all cases, the international agencies should pre-identify the most cost-effective alternative way of introducing such changes, and take any necessary steps feasible to ensure the effectiveness of such assistance. 18 This page is blank 19 Marrakech Action Plan for Statistics Report of an Independent Evaluation I. Introduction 1.01 This report deals with evaluation of the Marrakech Action Plan for Statistics (MAPS) which was developed and agreed at the 2nd International Roundtable on Managing for Development Results that took place in Morocco in February 2004. Launch of the Action Plan was an important step in pursuit of the results agenda which had been developed following the Monterrey Conference on Financing for Development in 2002. It is now being supplemented by launch of a further new initiative in response to evolving needs, creation of a Statistics for Results Facility to help raise coordinated finance from a variety of sources in support of national programs of statistical develop- ment in interested countries. The Facility would make it easier for countries to implement integrated plans for development of their official statistical services. 1.02 MAPS called for initiatives at three different levels: national, international, and, strengthening both, a more effective partnership between all concerned to help the developing countries meet their statistical capacity needs. Six action areas were highlighted: 1. Mainstream strategic planning of national statistical systems 2. Prepare for the 2010 Census Round 3. Increase financing for statistical capacity building 4. Set up an International Household Survey Network 5. Accelerate improvements in MDG monitoring 6. Improve accountability in the international statistical system While all areas required initiative at national as well as international level, progress on items 3 and 6 was most dependent on formal international or multi-country action. Progress in the other four areas was much more dependent on actions within individual developing countries but it needed to be encouraged and facilitated by provision of better help than had been available in the past from the international community. 1.03 While the World Bank has made important capital commitments to some countries in support of statistical capacity building (especially under the STATCAP program), it is its support for the four areas requiring action at the national level and innovative support from outside which is mainly connected with MAPS. Eight programs were developed with UN and other international agencies for provision of support to developing countries, especially the IDA countries and those of Sub-Saharan Africa, in these four areas. Grants to these agencies from the Bank’s Development Grant Facility were approved in the following amounts: 20 MAPS Direct Partner Agencies: Programs and Grants (in $ millions) Approved FY06 FY07 FY08 FY09 Paris 21 Nat. Stat. Dev. Strategy work 1.50 1.30 1.30 1.00 Intl. Household Survey Network 0.90 0.75 0.75 1.00 “Accelerated Data Program” 2.00 2.00 2.00 2.20 UNESCO Institute for Statistics 1.78 1.80 1.80 1.80 UNSD 2010 Census Round 1.00 1.00 1.00 1.00 UN ECE Gender Statistics - 0.35 0.20 0.15 ILO Gender Statistics (Informal Sector) - (0.10) (0.10) - UNSD Gender Statistics (in lieu of ILO) - - 0.10 0.20 UN HABITAT Urban Statistics 0.20 0.20 0.20 - (MAPS Unit MAPS Advisory Board & Eval’n. 0.12 0.10 0.15 0.15) Total 7.50 7.50 7.50 7.50 Approach to Evaluation 1.04 Given the wide range of the evaluation task, the Development Data Group which manages the overall MAPS scheme decided to break the work on the DGF-funded activities into two parts. The first was to be a rapid fact-finding review, seeking to establish what had been done with the aid of the grants and to elicit the views of the intermediary agencies, and other observers involved in the process, about the results achieved and the management arrangements employed. A long list of quite specific broad questions was helpfully included in the Terms of Reference (Annex 14 to this report). And a specific assignment within the first phase was to reach a judgment on the adequacy of the answers that emerged, and of the foundation for them. This would help to show how necessary or useful it might be to have a second phase of the evaluation and, if recommended, what specific shape it should take. 1.05 Initial conversations at the World Bank headquarters made it clear that the principal interest of the DGF Council (responsible for best use of the resources assigned to the Facility), as of other management levels concerned, was not in detailed follow-up of spending or deliverables under the grants made but rather in assessment of their overall impact, actual and prospective, on countries’ development management. Had improvements in statistical information that were brought about by the programs assisted resulted already in better policies or decisions than countries would otherwise have been able to make, or were there real prospects of this occurring ? Were the overall MAPS management arrangements conducive to efficient channelling of available resources toward such objectives ? For each of the particular programs studied, what measures needed to be taken to move toward phase-out of DGF assistance, or was there a convincing case to be made for further extension ? 1.06 These directions guided the discussions and the analyses that we carried out. Purposes and management of the overall program were discussed quite fully with people we were able to meet at the higher management levels, including a variety of members of the MAPS Advisory Board. But most of the time was devoted to the various programs supported, with the work during the first half of the study, in April- May 2008, typically following four stages: Initial review of the basic documentation on each program’s history since 2005, as provided by the MAPS Unit; Meeting with the World Bank staff member responsible for direct supervision of each grant and 21 preparation of the annual requests for renewal; Fuller analysis of all relevant information available, immediately followed by extensive conversations with managers and staff in the agencies using the grant funds; Execution of small analyses – sometimes by ourselves, sometimes by our interlocutors in the agencies, often jointly – to try better to tie down or to illustrate the broader impact of a program or the future that should be envisaged for it. Some three to eight hours, in several separate sessions, were spent with those principally responsible for each of the programs receiving annual grants of the order of $1 million or more. Similar stages were followed in the work on the smaller programs, but interaction with the agency responsible was handled by e- mail. 1.07 The Report on Phase 1 of the Evaluation (dated May 31 2008) was circulated quite widely by the Development Data Group in June, but only a few requests were received for further clarification or elaboration of particular points. Consequently the Phase 2 evaluation effort was able to concentrate mainly on the visits to a small sample of countries, as proposed in the May 31 report, in order to sound directly their opinion of the MAPS program, to check what we had learned from the partner agencies about country activities, and to clarify some aspects of the situation in regard to National Statistical Development Strategies – particularly the strength of in-country ownership and the actual state of implementation. 1.08 The Phase 1 report had identified seven countries as particularly suitable for such review because they appeared to be relatively advanced in the use of planning for statistics and they had also participated significantly in other programs of the MAPS partner agencies. We finally picked four of these countries as feasible for us to visit in the short time available, and they all kindly accepted and gave good support during the 2-4 day visits undertaken. 1.09 In the course of the visits we tried to learn as much as possible about the progress that had been made in statistical development in recent years, and the experience with planning, in order to be able to assess the nature and significance of the contribution, or value added, from the MAPS initiatives. We sought interviews not only with the producers of statistics (in central statistical agencies and in line ministries) who were normally those who had had the strongest link with the MAPS themes but also with a variety of types of users – policy-makers, PRSP monitors, socio-economic research units – and relevant people from two or three aid agencies active in connection with use and generation of statistics in the country. We looked also for the broader effects of statistical work, on the pace and quality of development that is taking place. 1.10 The focus, albeit only qualitative, was thus very much on evaluation of the MAPS agency action rather than of the country’s own work, and on identifying any ways, large or small, in which the MAPS interventions might have been improved. We emphasized our particular gratitude to the countries that received us, recognizing that they were helping on assessments that were relevant only partly to their future and designed at least as much for the benefit of other countries and the international agencies. 22 Structure of the Report 1.11 Main purposes and content of each of the programs carried out are summarized in brief annexes (Nos. 8 – 13) to the current report, based principally on the interactions we had with their managers and attempting to fulfil much of the fact-finding aspect of the first phase of the evaluation. Those annexes also provide approximate figures for expenditures incurred, mainly to indicate orders of magnitude involved. 1.12 The seven chapters of this report are mainly addressed to answering the broader questions about MAPS that were included in the Terms of Reference, backed by a fuller assessment of the statistical accomplishments of the various component programs in the longer Chapter III. The answers to the questions that were posed are strongly colored by all that we have learned in the last few months about the programs supported, but they make specific reference only to instances in which a particular piece of experience in one or two programs or an analysis carried out of underlying data helps to illustrate a point or better show its implications. 1.13 In the later part of this Introduction, we attempt to answer the questions, included in the Terms of Reference, regarding the adequacy of monitoring and evaluation arrangements in MAPS and to convey the judgment that we reach on the adequacy of the evidence basis available for answering the questions posed to the evaluators. Chapter II deals with the questions about the Relevance of MAPS and its components to the needs of the low-income countries, Chapter III with those about their Effectiveness, and Chapter IV with the little that can be said at the current stage about their Efficiency. The findings from the country studies as well as the agency program reviews are taken into account in those chapters, but Chapter V gives a synthetic overview of the country perspective as it emerges from the four country reports (themselves presented at Annexes 3 – 6). Chapter VI assesses the Governance and Management structures of MAPS and the programs it has supported. 1.14 The seventh and final chapter of the report, Suggestions for Further Action, addresses the questions from the Terms of Reference regarding possible steps to improve effectiveness of MAPS or its components, and puts forward a first tentative set of proposals regarding the scale of future DGF support for the programs reviewed and measures needed to prepare for such future. Monitoring and Evaluation 1.15 The questions raised on this subject in the Terms of Reference for the present study essentially relate to how much the monitoring and evaluation arrangements for the Actions included in MAPS, and especially for the programs supported by DGF, have evolved since 2005 and how adequate they now are. Within the overall restriction that it is extremely difficult, and perhaps impossible, to find objective indicators relating to the crucial objective of all such statistical work – the extent to which statistics generated are effectively used to improve public policies and decisions – the answer seems to be that monitoring and evaluation arrangements have been significantly improved and continue to become more useful. 23 1.16 Original presentations of the agency programs supported tended to elaborate on the broad statement of program objective with an often long and detailed list of deliverables, and frequently considerable attention to the proposed relation between these deliverables and the financial contribution that DGF was expected to make. While this was probably a useful first stage to the work, it seems to have been gradually, and in some cases quite rapidly, abandoned in the communications between the Bank and the agency. This probably reflected fuller recognition that what was really being assisted was execution of a broader program, often financed mostly from other sources, and that the agencies had their own governing bodies (and audit arrangements) giving close attention to program delivery. Hence, the key issue for the Bank was really broader assessment of, and agreement with, the overall direction that the entity was taking. 1.17 An important step over the last year or so has been for the Bank to supplement the broader statements of objectives and deliverables now conveyed in the annual grant agreement letters with statements of improved or more clearly stated ‘Project Development Objective’ (or PDO) indicators, with baseline and target values. Some of the efforts in this direction seem to have generated excessively complex, composite indicators, while others run the opposite danger of being straightforward but less directly related to program impact than needed. But all are receiving attention for further improvement, such as the strengthened version that the Bank’s Development Data Group is now developing of the Statistical Capacity Indicator that it originally introduced in 2002. 1.18 As regards the broader objectives of MAPS (Actions 3 and 6 in the above list), data available on aid flows or investment in support of official statistics in developing countries have been quite insufficient to monitor fulfilment of the projections and targets included in the original MAPS document. For the aid flows aspect this may change as a result of substantial effort over the last three years by Paris 21 task teams, although it will probably remain difficult to capture the many small contributions made within the context of much broader sectoral programs. The accountability action (No. 6) was principally foreseeing completion, and supporting implementation, of the draft “Principles Governing International Statistical Activities,” which refer in part directly to practices in provision of aid for statistical development and are largely in tune with the Paris Declaration on Aid Effectiveness. The Principles were approved and issued by the UN Committee for the Coordination of Statistical Activities (CCSA) in 2005. Adequacy of Evidence 1.19 Recognizing both the obvious importance of the contribution that statistical capacity building has to make to evidence-based management and the detailed assessments of specific country situations which are needed to weigh concretely the extent of such contribution – and thus the limits of the possible – we believe that the evidence presented in this report is generally sufficient to justify the positive conclusion of the evaluation and the recommended continuation of the programs supported. Most of the actions at country level, and especially those aiming at broader capacity building, are at quite early stages, so that the degree of ultimate effectiveness they will have remains a matter of judgment. This is of course even more so in respect of the broader effects of statistical improvements on policy and investment decisions, potential effects that are correctly emphasized by a ‘management for results’ philosophy. But the overall 24 picture that emerges from the country contacts made in the second phase of our work, while it reveals cases of slow or disappointing impact as well as of promising trends, adds credence to the information gained from our interactions with the partner agencies and indicates that cautious optimism remains justified. 25 II. The Relevance of MAPS 2.01 The main idea underlying MAPS was to raise the profile of statistical work and capacities in the eyes of the donors and country leaders, and to attract resources to fill unmet needs, as in the case of the expensive task of carrying out decennial censuses, more efficient use of large-scale surveys, and statistical planning. MAPS has emphasized greater partnership and cooperation between different parties involved in official statistics because those could much facilitate the task. 2.02 MAPS has also benefited, however, from broader world trends that have gradually prompted more sympathetic understanding by the distinguished senior statisticians of the OECD countries for the importance of the needs of the developing countries and the real difficulties that their professional counterparts in those countries face in meeting them. Annual sessions of the central official body of the world’s statisticians, the UN Statistical Commission, now normally have participation from 130 countries, compared with little more than half that number as recently as the early years of the current decade. More explicit attention is also given to subjects of interest to the developing countries. 2.03 The MAPS strategy, with its central focus on the development of a national integrated plan for production of official statistics in service to policy makers and managers of sectoral programs, resonates well with the partner agencies which usually deal mostly with developing countries. They recognize the need for a genuinely owned development strategy that will help generate the variety of quite different statistics that is often required to understand a problem. UN Habitat staff depicted their association with MAPS as the first time they had even heard of specialized assistance being offered in support of program monitoring. The UNESCO Institute for Statistics also greatly appreciated the substance of their relationship with the Bank because of the constant emphasis on what was needed for public policy, and hence on the types of policy issue to which their statistical concepts and methods needed to be adapted. The institute would appreciate further practical help from World Bank country staff, as apparently valuably provided by the Millennium Challenge Corporation, to press developing countries to fill UIS questionnaires more promptly. 2.04 One or two partner agencies, more accustomed to dealing with more developed countries, thought that MAPS’ planning emphasis might sometimes be applied too universally, and one expressed particular reservations about cooperating on limited-term small-scale projects. In another and different case, that of ILO, small grants approved (para 1.03 above) could never in the end be consummated, due to inability to find mutually acceptable legal formulae. But these difficulties or reservations were very much minority affairs, not representative of viewpoints more broadly held among the Bank’s partners in MAPS. 2.05 The comparative advantage of the MAPS partnership lies in the variety of international agencies associated and hence in the possibility of combining different disciplines and viewpoints to create an appropriate framework for dealing with a problem. The intellectual resources assembled, thanks to DGF assistance, for the IHSN, or by UIS and Habitat for developing statistical concepts and methods appropriate to 26 education and to urban issues, are all good examples. The relationship with the Bank as development financier and producer of substantial advisory reports is also important because it can facilitate diffusion of new solutions found. On the other hand the resources assembled are not suitable in scale or in experience to undertake extensive efforts to implant the solutions in individual countries. That is work which normally needs to be left to bilateral donors (or Bank/IDA funding) to finance, usually with the work contracted out to private consultants or research institutes. 2.06 MAPS broadly is focusing on things that remain of highest priority in the development of statistics for management of development in the poorer countries, and indications are that the MAPS partners, with their Bank counterparts, evolve their programs actively in light of new evidence about demand and about potentially viable solutions. UIS, for instance, has been giving increasing attention to gathering of data about financing of education, as strongly urged by the Bank, and IHSN/ADP has substantially revised planned activity mix in light of very strong demand for salvaging past surveys, of which results have almost been lost, and the substantial potential demonstrated by the Question Bank initiative. 2.07 It would no doubt be very valuable from time to time to ask the Advisory Board, with the very wide range of talent and experience that it encompasses, to take a morning or an afternoon for full and open exchange of views about the balance among sectors and between sources of statistics (e.g., surveys as against administrative data) that it would be appropriate for MAPS to achieve in its services in the coming years. Another issue that could become urgent more quickly is what measures MAPS could take to help countries that run into implementation problems with their NSDS, and whether MAPS or the World Bank should take steps to prepare for meeting such needs. 2.08 It is hard to believe that there is any great urgency at the current stage to reformulate the six MAPS objectives that were drawn up four years ago. The emphases on Strategic Planning, Financing for Statistical Capacity Building and the International Household Survey Network (including ADP) (Actions 1, 3 and 4) have seen substantial development but remain of unchanged importance. It is necessary to maintain stress on the 2010 Census Round at this point, and to lay the foundation for rapid processing, analysis and dissemination of results and data after the upcoming peak years of census execution. As regards the MDGs, there are some indications that DGF-supported projects may have contributed less so far than originally expected to short-run improvement in international monitoring, but the UN Inter-Agency and Expert Group seems to have made reasonable progress. It would anyway be undesirable to make any downward displacement of this objective at this time, just before the 2010 check on progress toward the 2015 targets. 2.09 Contributions to MDG measurement that could now be expected from the ongoing MAPS projects in the relatively short term include, in addition to the education figures which improve little by little each year thanks to the efforts pressed by UIS, also some corrections resulting from the results of recent and upcoming censuses, and improvements in methods and frequency of population projections (national and international), and possibly, in a few cases, revisions resulting from deeper analysis of the extensive past survey data now made more accessible thanks to the ADP initiative. 27 2.10 Projections have often to be used to provide estimates of recent years’ population which form the denominator of many of the MDG indicators. These projections – whether from the UN Population Division or from national statistical offices or population commissions – give rise not infrequently to more doubts and controversy than the numerators. It is good to see the increase in the positive dialogue between countries and agencies which such problems promote. The UNSD recognizes the problem and intends to give attention to preparation, jointly with the Population Division, of improved guidelines. Our country visits further underline the frequency with which disagreements on population figures arise between governments’ central and sectoral statistical agencies, particularly on sub-national numbers that are of particular importance for internal planning. The improved guidelines should therefore have high utility within countries as well as at international level. 2.11 Improvement to MAPS’ relevance to country needs which suggest themselves for consideration at this time are matters more of minor adjustment and refinement than any significant change of direction. A keen eye must continue to be kept out for all opportunities of enhancing constructive dialog and influence, rarely sufficient, between policy-making levels of government and the directors of bodies involved in statistical work. Reflecting the strong emphasis on this point and on statistical needs by the countries that fund IDA, World Bank Country Assistance Strategies for IDA countries have begun to give more serious attention to statistical services.1 This practice must be extended, and encouraged to lead on to country-apposite initiatives to get the most out of its statistical services and supply data of types and in ways that will have the largest positive effect on development. 2.12 Another point warranting more systematic attention is the financial effort that countries are making out of their own resources to sustain their statistical services. The increased readiness of the international community to help countries with their statistical needs, and the increased provision of aid generally in the form of budget support, are much to be welcomed. But it remains true that an important dimension of “ownership” of a plan or program is readiness to attach a high priority to it even in face of budgetary shortfalls. Spending levels for statistics should be more fully discussed and better monitored. Lastly, as mentioned above, MAPS should be closely following IDA countries’ experience in implementation of the strategic plans that so many of them have been preparing in recent years. It should be looking now to identify the kinds of obstacles that foreign aid could constructively help resolve and to consider what measures should be taken to accelerate availability of appropriate aid when the need arose. 1 IDA: Operations Policy and Country Services 2007. “Focus on Results: The IDA14 Results Measurement System and Directions for IDA15” pp. 27-28. 28 This page is blank 29 III. Effectiveness of MAPS and Each of its Main Components in Delivering on Intended Purposes An enormous effort has been made behind the scenes to improve the methodological and technical framework for statistics, with the needs of developing countries very much in mind. International advocacy for statistical capacity building has been effective, but more can be done. Capacity building in-country has been programmed in some places but opportunistic in others, and not as well coordinated as it could have been. The potential to improve statistical quality is there, but little is yet visible at the international level. 3.01 MAPS is, in different people’s minds, the Marrakech Action Plan for Statistics itself, the Partnership of statisticians, agencies and donors which is carrying forward the recommendations of the Action Plan, and a group of DGF-funded activities which are presently part of the concrete manifestation of the Plan, with the addition of the Programme for Education Statistics. In this section we are asked to consider the effectiveness of the partnership and the present package of activities. 3.02 It was made clear in our discussions with the World Bank that the focus of our analysis should be on impact: what evidence can be found that MAPS - the partnership and package - has had a measurable impact ultimately on the quality and application of statistics. Immediately we have the issue of attribution. This is largely insoluble but is also largely irrelevant. The MAPS components are just that - components of a worldwide effort to improve statistics. The careful analysis underlying Marrakech pointed to certain lines of action - areas where others were not operating and which needed additional attention - and as long as MAPS is not working against the Plan, it can take a share of the responsibility for the outcomes. 3.03 This is a large topic, and the chapter is divided into five sections. These are: IIIA - a view of the MAPS partnership and the impact of its activities at a global level; IIIB - a view of the work of a global nature of the institutions receiving DGF funds, and its potential or actual impact in-country and internationally. This covers the general operations of Paris 21, IHSN, UN Habitat, and the global work of UIS and UNSD; IIIC - a view of the work these institutions are doing in individual countries or regions, to which DGF resources contribute, and the specific results that have been seen. This includes NSDS, ADP, Gender, and capacity building work by UIS and UNSD; IIID - an analysis of the main sources of data for indications of improvement in statistics and statistical capacity; and IIIE - a summary response to the specific questions posed in the Terms of Reference and not previously treated. 30 IIIA. The MAPS Partnership - a Catalyst for Improved Statistical Capacity Building 3.04 MAPS did not start in isolation. A great deal of work on statistical capacity building was going on already, and had been for some decades with varying degrees of success. The partnership therefore had to work within, and build on, existing structures. This is one of the first reasons why we conclude that MAPS is being effective - for the very point that it is not trying to take over or re-invent statistical capacity building. Many initiatives pre-date MAPS - all the ground work on NSDSs, for example, and the seeds of the IHSN. The setting up of Paris 21 itself can now with hindsight be seen as a necessary precondition for MAPS to operate, and it has been one of the strengths of Paris 21 that here we have an organisation which is seen as an honest broker, and which, while carefully guiding the debate, does so at the right speed. 3.05 But MAPS has brought a new coherence to statistical capacity building efforts. There are different opinions about the practical usefulness of NSDSs, but there is no doubt that they have put decisions about capacity building firmly in-country. The support of the World Bank has been key to this, with their insistence that an NSDS is a prerequisite for a StatCap loan. [The Secretariat of the Trust Fund for Statistical Capacity Building has seen an increase in demand for grants, and has noted that, as a direct result of MAPS, the requests are of higher quality and with greater attention to detail.] At this level of policy, however, the quality of the actual NSDS itself is not so relevant: what matters is that it is a single strategy around which donors and agencies - and the country itself - can focus. The acid test comes with implementation, which will be discussed later. 3.06 Many donors and agencies were already assisting statistical systems when MAPS was created, and so on the one hand MAPS created the umbrella under which everyone could work, and on the other sought to fill specific gaps - in education, gender and slum statistics, in the census 2010 preparations, and in “backroom” intellectual work of general benefit. The MAPS partnership has tried hard to bring together the different actors on the statistical capacity building stage. Chapter 5 goes into the governance issues in more depth, but in terms of impact there are a few things to say. 3.07 We feel that the advocacy aspect of the partnership could be stronger. It is operationalised through the Advisory Board, which has no official mandate and whose membership mixes those representing the World Bank itself and DGF recipients with those who participate as expert advisers. The planned electronic discussion forum never materialised, so this short annual meeting is the only official occasion for people to contribute, question and learn. It is true of course that the statistical development community is relatively small, and people meet elsewhere. 3.08 Where the Advisory Board members can do more is in promoting MAPS – or, to be more precise, country-owned statistical capacity building - within the organisations where they work and especially, in appropriate organisations, down to their country representation. Paris 21 say that explaining MAPS is a never-ending task. Each time an individual changes - be it the World Bank country economist, the EU delegate, the DFID country director, or the head of the visiting IMF team - the process of selling MAPS has to start again. This means that it has not penetrated enough into the various corporate consciousnesses. The situation is ten times worse if the director of statistics 31 changes, which demonstrates the continued fragility of capacity building. But MAPS’ insistence that things proceed at the speed the country wishes, with the team they choose, is another of its great strengths and makes failure less likely if key individuals change. 3.09 Full understanding of, and support for, the objective of building country-owned institutions will be crucial as more and more NSDSs move into implementation. At that stage, any shortcomings in the strategy will become painfully clear. Contributing donors and agencies are most likely to be funding portions of the development programme, rather than following the ideal of providing budgetary support. Expectations will have been raised. There will be immense pressure on the statistical offices to deliver more quickly than is commensurate with carefully building capacity - and if the principal development partners are not fully on board with the MAPS ethos, we risk returning to square one. This country level cooperation and understanding is more important than the exact composition of the Advisory Board. 3.10 We have seen the country assessment framework for the Health Metrics Network, together with the Partnership Review Note which states A partnership with HMN is aligned with the objectives of other Bank programs, including the support to statistical capacity provided in DECDG, and existing GPPs, including the Statistical Capacity Building program. We wonder to what extent alignment of objectives will lead to cooperation in the field and internationally. The assessment framework contains many interesting ideas, and would resonate with the work already done by UIS on technical issues, as well as with the NSDS process nationally. There is potential here for sharing of ideas and experiences. 3.11 We have also seen the 2006 WB Kenya Education Sector Support project documents, where potential synergy between NSDS and UIS looks as though it may have been missed. StatCap is only mentioned as an aid to monitoring good governance, and UIS as a source of EFA indicators, but the project intends to spend $6.9 million on an education management information system which must surely be linked to the NSDS. 3.12 One final word about the Advisory Board - it needs to be flexible enough to engage with or bring in new players. The newly expressed interest of the African Union in capacity building particularly for education statistics needs attention, for example. The Advisory Board might wish to consider whether engagement should be at the Board level, or through partners such as Paris 21, or in specific countries on a case-by-case basis. ~~~~~~ 3.13 Specific questions in the terms of reference which are relevant to this section are as follows. We are asked to examine to what extent the MAPS partnership has sustained its focus on the original action plan; to what extent it has built political commitment for statistical capacity development; to what extent it has influenced the strategies and programmes of developing countries and development partners, and vice versa; to what 32 extent it has contributed to increased coherence of effort; and what evidence there is of outcomes of these. 3.14 The strategic focus on the original action plan has been maintained well. This has both positive and negative connotations. On the one hand, MAPS has followed the mandate given to it in Marrakech (which was indeed fairly broad). But against that is the lack so far of any opportunity for strategic discussions at the Advisory Board meetings. The need for this is becoming more important. It is by no means time to stop any of the present activities, but given the lead-time for new start-ups, it is definitely time to revisit MAPS and review the priorities. A global strategy for assisting NSDS implementation needs discussion. 3.15 True political commitment is extremely hard to judge, and would only be proven when a country chose to follow MAPS and its ethos using its own resources. External assistance must of necessity influence commitment. However, the number of countries which have spent considerable time and effort on statistical legislation is one indicator of genuine commitment to a country-owned process. Future indicators could also include rejecting donor assistance when it is not in line with the country strategy - but so far examples of that have not been drawn to our attention. It may on the other hand be true that it is no longer necessary in most countries to “sell” the benefits of good data, and therefore statistical capacity building, to ministers. 3.16 The influence of the MAPS partnership on the strategies and programmes of countries and their development partners, and vice versa, is an interesting issue. Promoting and, more importantly, supporting NSDSs is the main route, and there is evidence that country experience has fed back into the process. Country experience has led IHSN/ADP to put more resources into the Question Bank. The early change of focus for ADP, from detailed work in a few pilot countries to task 1 work in many is a direct response of MAPS to expressed country needs. This example highlights the flexibility of MAPS, which is another of its strengths, and contributes to its overall effectiveness. 3.17 Another example of influence is the increased attendance at UN Statistics Commission meetings by statisticians from developing countries. Participation has risen from 70 to 130 countries and much of this is a direct result of MAPS, and especially Paris 21, advocacy and funding. Paris 21 also does a lot of work behind the scenes to prepare statisticians who have not attended the UNSC in the past. As a result, the UN Statistics Division is becoming more directly aware of developing country problems and concerns. 3.18 Coherence of effort and building partnerships in-country is a keystone of the MAPS ethos, but evidence or counter-examples would have to be collected country by country to answer properly the question of how much difference MAPS has made to coherence in the round. The test will come, as we said earlier, when NSDSs are being implemented. In Malawi, only two or three donors are active in statistics, and they are working within the plan. One key outcome of the first years of MAPS has been the scaling-up initiative presently being prepared by the World Bank. Donor partnerships are being piloted in 3 countries where there is a lead donor for statistical capacity building – the World Bank in Ghana, the Asian Development Bank in Afghanistan, and Denmark in Mozambique – and the lead donor concept is attracting increasing support elsewhere, as for instance in Mali and Zambia. 33 3.19 Coherence is also tested and put under strain by the large number of regional and sub-regional organisations to which, for example, African countries belong. They all have mandates, many with data requirements, and as yet there is little attempt to harmonise these many demands. Ultimately it is the director of statistics who has to decide on priorities, and has the responsibility to ensure that donor and agency support meets the overall strategic needs. All MAPS can do is help to empower the director to do this. 3.20 The final question about outcomes of these partnerships at the global level cannot be satisfactorily answered except by what has been written above. Later we shall see very good evidence of partnerships linked to the different products which MAPS has developed. IIIB. DGF Funding to Institutions - Making Room for Innovation 3.21 In this section we shall cover what could be described as indirect assistance to countries. Direct assistance will come in section IIIC. Coverage here, on the other hand, includes the general operations of Paris 21, the IHSN, much of the work of UIS and UNSD, and the work of UN Habitat. 3.22 DGF funding has been instrumental in creating a better environment in which statistical capacity building can flourish. We have seen examples of excellent work which would not otherwise have had a very good chance of being funded. In places, funding allowed a closer focus on the needs of developing countries, which was beyond the natural remit of the organisation - UNSD, for example. In UIS and Paris 21, DGF allowed the creation of products which are intellectually sound, PC-based and tested, innovative, well documented, flexible enough to meet country needs, made available without charge, and in demand. 3.23 The overall impression we gained during our conversations with the partner agencies was of the huge amount of background work which has gone on and to which it is impossible to do justice in the space allowed. It has certainly been under-reported in the documents which go to the Advisory Board. But to avoid this report being overlong, we shall restrict ourselves to briefly describing the activity and then trying to focus on its actual or potential impact. Paris 21 General Operations 3.24 Part of this is the reporting and monitoring which Paris 21 does on behalf of the international community, covering mainly progress with NSDSs and monitoring support to statistical capacity building. Regarding the latter, PRESS is being developed to replace the LRE - light reporting exercise - and information is in the process of being collected from donors and agencies. This will when finished be of general use, though it will ultimately be more useful, and in line with MAPS, to have this information compiled by the countries themselves. 3.25 Reporting on NSDSs is also a service to the international community, but Paris 21 staff see a danger if it becomes a purely external exercise, and is very reluctant to sit in judgment on the quality of NSDSs. They argue persuasively that since MAPS is built 34 around country ownership, countries should report themselves on NSDS development and implementation and on quality too. In one or two cases this is beginning to happen. They are developing a central website for NSDSs, but plan for it to be linked to country sites. 3.26 Paris 21 continues to develop and maintain their series of guides to NSDSs. It hardly needs to be stated that these are of great benefit to countries, as is evident from the continued interest, including the demand - which Paris 21 has satisfied - for the documents to be available in many languages. 3.27 The third area where Paris 21 have been active and where good results have been seen is in building partnerships, arranging funding, and general advocacy. It is through building these partnerships that Paris 21 has been able to promote NSDSs in the early years, often making presentations in the margins of other conferences, which is both cost-effective and efficient. Partners have been found in all regions. The country- level advocacy kit has been well received, and Paris 21 continues to to satisfy demands from countries for innovative and informative materials. International Household Survey Network 3.28 The Marrakech Action Plan, under Action 4, recommends three actions: 1. organise a Household Survey Network for the purposes of sharing information and mobilising international support for more efficient approaches to conducting household surveys in developing countries; 2. develop a set of recommendations for household-based economic and social data, taking into account current and planned multinational survey programmes and the needs of developing countries to monitor their own development progress; and 3. work with experienced data archivists and data users to establish a global information centre containing household surveys and metadata; establish good dissemination practices which promote analysis and research while protecting the confidentiality of survey respondents. 3.29 Although the impact indicators for IHSN are not specifically referred to in the progress reports, they can be inferred: • to improve the availability, accessibility and quality of survey data; • to avoid the duplication of survey activities; • to improve cost-effectiveness of surveys; and • to reduce the burden of international survey programmes on national statistical systems. 3.30 IHSN operations are extremely well linked to these aims. The network is now well established with DGF funds contributing to the secretariat, and productive partnerships have been built with internationals such as WHO and FAO which have additional staff in place for this task and lead in their areas of competence. The academic foundations of the work have introduced a much-needed intellectual quality to what is being done. IHSN protocols are now used by MICS, and will probably be used 35 by DHS in the future. DGF funds have paid for the migration of the IHSN from a mainframe to a PC-based system, making it more portable and accessible. 3.31 The immediate impact of improving accessibility to survey data has been all but achieved: and quality is being addressed by other activities. But survey metadata is now in the public domain, linked where possible to microdata on national websites, and includes some surveys done by line ministries. The international interest generated by IHSN will lead quite naturally to quality improvements as dialogue between producers and users develops. The IHSN/ADP work on a Survey Quality Assessment Framework will contribute to this - the ultimate aim is that the SQAF can be used in advance of a survey being taken, to identify potential problems in time to put them right. 3.32 Taking Marrakech a further step, ADP is piloting National Data Archives (NADAs). In line with the MAPS philosophy, and bearing in mind the occasions where survey microdata have been taken away from developing countries, which in a few cases have then even been charged for access to their own data, the aim is to have survey databanks in each country, but with a consistent format. The backroom work on access and confidentiality, and future work on anonymisation, will save countries from reinventing the wheel. This has been seen in action in Ethiopia, where the first national data archive should soon become operational. The CSA have made serious use of the IHSN microdata management toolkit to archive censuses and surveys back to 1995. Over the last three years, the IHSN/ADP team has assisted CSA a great deal. Staff have been trained in the appropriate tools and have attended regional workshops; people from other offices have visited Addis Ababa to share experiences; and the team in Paris has given solid back-up to the CSA when it was required. 3.33 The micro-data toolkit has been a great success with a huge demand from countries, and is now available in 5 languages. Again, the academic roots of the DDI have given a much-needed rigour to the work, which is PC-based. Metadata can be extracted in a number of convenient formats, for a web-page, or into a questionnaire, for example. 3.34 And finally the Question Bank is encouraging a much-needed consistency, not least among the international agencies which often ask for the same information in slightly different forms. At country level, it will allow those designing questionnaires to see what the options are, and to increase the chance that the data thus collected will satisfy international requirements too. 3.35 The activity which has not been successful to date is the information system for proposed and on-going surveys. This has been approached from the agency perspective. The solution perhaps is to start with the country - and to hope that the implementation of the NSDS will allow for proper and planned phasing of surveys. UNESCO Institute for Statistics - IT Operations and Other Services 3.36 The three-part strategy for UIS development with DGF support consists of 1. data development and collection: improving comparability, coverage, reliability and timeliness, with better documentation; 36 2. UIS internal data processing: streamlining and harmonising UIS internal quality assurance procedures; and 3. responding to needs of data users: in terms of timeliness and accessibility, and improved outputs. These have well-documented performance indicators which have been reported against systematically. 3.37 Like the IHSN - more so, in fact - the bulk of the work has been going on in Montreal behind the scenes, and there has been a heavy investment in IT which is beginning to pay off. For detail, see Annex 7, which has been provided by UIS and is briefly summarised below. 3.38 The first area of improvement is in the Data Capture Infrastructure (DCI). This uses a common technology for all surveys, reducing the time required for testing and training. Surveys can be produced in a number of languages without having to check again the questionnaire, and can be viewed as web-based, paper-based or electronic file- based. It uses Adobe, which can run on any platform and requires no special installation, unlike earlier surveys. Users are given instant acknowledgment of receipt of their surveys. 3.39 The immediate impact has been on UIS internal efficiency, with resources freed up to concentrate on data quality rather than data transmission. Since the new survey instruments were only used for the first time at the end of 2006, there is still an element in-country of learning the new system, and so it is too early to see an across-the-board improvement in timeliness. 3.40 The main benefits for countries are a) that they were consulted about the system which they wanted, and this was delivered; b) the survey can be loaded on any operating system, and many computers; c) the survey can be delivered electronically direct to the relevant desk and not get bogged down in bureaucracy; and d) there is some amount of basic validation as data are entered. The almost instant acknowledgement from UIS of receipt is also a very important innovation and morale booster. This has been confirmed by Ethiopia, for example, which has recognised the improvements brought by UIS to the system for international reporting. The Ethiopian EMIS team also noted UIS’s readiness to adapt the software: the original, for example, had no cut and paste facility, but when this was pointed out it was quickly put right. 3.41 The second area covers the three new internal tools: the error reporting system (ERS); the calculate indicators tool (CALCINDIC); and the data and indicators verification application (DIVA). Annex 7 gives the detail, but in essence these three tools provide UIS’s work with an internal consistency and quality assurance, which were lacking before. It does not impact directly on the quality of data coming from countries, but it makes it easier to spot errors. It is now simple for UIS to calculate new indicators at the request of data users, without programming expertise. 3.42 On the side of the data providers, UIS has made some steps to change the way in which country data are handled. We have already seen the IT improvements, but UIS also plan to move towards Rolling Collection and Release. Ultimately, this will mean 37 that data from countries will be validated and placed in the database within a few weeks of being received. This replaces the old system of a paper questionnaire being sent out once a year, and validated when all the replies were in. The database was then treated very much like a paper publication, for updating annually. 3.43 UIS is halfway through this reform. Questionnaires are now sent out electronically in two main batches, corresponding to school years ending in June and December. The difference is that questionnaires are examined when received and feedback/queries made to countries within two weeks. They are surveying countries at the moment on national release dates, to see what further time can be saved. It should be noted though that most education indicators require data from other sources as part of the calculations - population estimates from UNPD, for example, for UIS education indicators - so timeliness is not purely in the hands of the providing countries. 3.44 The final stage will be rolling release, which will come when the systems have been modified to cope with data referring to different time periods, issues of when the database is “up to date”, version control, and so on. 3.45 The SDMX - Statistical Data and Metadata Exchange - initiative is a good example of more efficient working practices. At present, three agencies have a mandate to collect education data - Eurostat, OECD and UIS - and they use three different instruments. Data are then analysed and published in a number of different ways. A joint programme of the three organisations, UIS participation in which is largely funded by DGF, is bringing together the instruments using SDMX, to address such issues as version control, data verification and quality assurance. 3.46 In terms of its service to partners, UIS’s website is targeted mainly at individuals. Institutions and agencies receive their data through tailored files. The World Bank education staff are particularly pleased that they now receive data in advance of their annual Education For All conference. Much work is going on to revise historical series - as, for example, population estimates are revised, or a country classification changes. And of general use to both users and countries is the work on mapping country education systems onto ISCED. This will again reduce the errors and uncertainties in country data as well as making global comparisons more meaningful. 3.47 It should be noted that, in assessing bids for its assistance, the Millennium Challenge Corporation ask UIS to quality-assure education indicators. United Nations Statistics Division - Supporting Census 2010 3.48 The three lines of activity undertaken by UNSD with the support of DGF are: 1. to monitor country progress in conducting population and housing censuses and their financing needs; 2. to assist countries to implement the United Nations Principles and Recommendations for Population and Housing Censuses, Revision 2 (referred to hereafter as P&R2); and 38 3. to build technical systems to help countries conduct their censuses and improve their efficiency. 3.49 UNSD has reported regularly to the Advisory Board against this work programme, using a clear logical framework with performance indicators. They have maintained good records of what has been done. 3.50 The aim, of course, is that all countries of the world will participate in the 2010 Census Round (censuses between 2005 and 2014), following the poor showing, particularly in Africa, for the 2000 Round. For the record, 6 African countries have yet to announce plans. UNSD hasten to point out that the decision whether or not to take a census is nevertheless for the country alone to make. All UNSD can do is ensure that information and assistance are available when required. So a simple count of census promises is not a particularly useful indicator. Of more use is what happens after the census is taken: are data analysed and released in a timely fashion? Are the results accepted? UNSD’s advice and guidance is designed to reduce the risk of failure. 3.51 The UNSD programme follows a logical structure: first, the production and the acceptance of P&R2; then handbooks on topics according to phases of the census developed by the expert groups - covering for example mapping, data processing, dissemination, archiving; then the UNSD census web-site. 3.52 At the global level, UNSD have been very active in enabling a full exchange of knowledge through metadata, a question bank and a general knowledge base. The question bank contains around 8500 questions and answers from previous censuses organised according to the topics in P&R2 and is being intensively used. It differs from the UIS question bank in that the actual way in which questions were asked by countries in previous censuses can be seen and compared. SDMX underpins the work. 3.53 The web-site is being developed as a resource centre, with comprehensive documentation on methodological standards and good practice. It is recording over 300 hits a month on the specialist topics of methods and questionnaires from within the census community of probably around 500 people, which is an excellent result. International standards have been published in clear simple language, agreed by expert groups on which developing countries have been fully represented. DGF funds have been key to ensure this involvement. 3.54 The response from Africa has been positive. In 2007 the African countries created and endorsed a Census Addendum to P&R2 which addresses issues of particular interest to the continent. Countries such as Nigeria, Ethiopia, Sudan, Angola, Liberia and Burundi, are making plans to join the 2010 Round having not conducted a census in the 2000 Round. And a welcome sign of partnerships are the beginnings of South-South cooperation on census issues – between Mozambique and Angola, for instance. 3.55 CensusInfo is an example of a new gobal partnership, involving UNSD (with DGF funding), UNICEF and UNFPA. The aim is for this to be a tool for census dissemination on the same lines of DevInfo, MDGInfo and ChildInfo. 39 UN-Habitat - Urban Indicators and the MDGs 3.56 UN-Habitat has been reporting regularly on progress against a number of targets set out in its agreement with DGF. A recent overall assessment by its Monitoring and Systems Branch (MSB) is summarized in Annex 8. 3.57 There are five areas where UN-Habitat has been working and where positive results have been seen. First, four slum indicators were successfully steered through the Inter-Agency and Expert Group on MDG indicators in March 2007. These define slum- dwellers as suffering one or more of the following: lack of access to an improved water supply; lack of access to improved sanitation; overcrowding; and dwelling made of non- durable material. This is of tremendous importance and long overdue, since MDG Target 11 - By 2020, to have achieved a significant improvement in the lives of at least 100 million slum dwellers - has not been adequately measured since the targets were set. The definitions have been accepted by the UN agencies, and “housing durability” has been included in the UN census recommendations. 3.58 The definitions are efficient, in that they use definitions which are well understood and data which in most cases already exist, thus adding to their value; and they are effective, since they will bring about harmonised methods and consistent analysis. 3.59 The second area is UN-Habitat’s recommendations for disaggregating MDG indicators data into slum and non-slum, and their development of indicators of secure tenure. This has the potential - not yet seen - to produce estimates of the social dimensions of slum dwellers from existing survey tools with little extra cost. MICS have taken the recommendations on board, and DHS are expected to. 3.60 Thirdly, UN-Habitat has developed a new survey tool - the Urban Inequities Survey. This has been used successfully in 12 cities around the world, and has been supported by the Netherlands and Norway. Again, it has potential to give a much greater understanding of the attributes of slum dwellers and to add to the information collected through censuses and surveys such as MICS and DHS. 3.61 The fourth area where DGF funding has aided UN-Habitat is to upgrade its UrbanInfo tool. There has been international interest in it since it was completed towards the end of 2006, and some requests from countries and urban observatories for training, which has been delivered. So far no impact is visible at country level, and UN Habitat continues to make its estimates of the slum population centrally every 5 years - the latest data for 2005 are available. 3.62 And finally, work has been initiated with partners from the EU, Italy and the Netherlands to develop automated mapping methodology from satellite imagery. 3.63 The work then has been, like much of that of UIS and UNSD, of a global and backroom nature. Although the original DGF agreement of 2006 stated that analytical capacity was to be built at national level, and that of 2008 that at least 10 NSOs would be using higher-quality urban indicators, that is so far not evident. The amount of funding allocated for this - totalling $75,000 - was in any case quite inadequate. Also, it 40 is by no means clear that the NSO is the logical home for this sort of analysis, rather than planning ministries or individual city councils. IIIC. DGF Funding for Capacity Building via Institutions 3.64 In this third section, we shall cover the work which has been done in individual countries and regions, to which DGF resources contribute, and describe the outcomes and impact. We shall cover NSDS, ADP, Gender statistics, and capacity building work by UIS and UNSD. 3.65 Due to the limited amount of DGF funds, most of this work has required additional resources and has been done in partnership with other donors and agencies. In some cases, country-level work has been a way of testing global ideas; in others it has been opportunistic; in yet others countries have been chosen for assistance as they are “low hanging fruit”, as we were told several times. Because of the nature of DGF MAPS, which had an institutional focus, a structured and long-term partnership with many countries for capacity building was not a true option. But Table 3.1, in the Statistical Annex, does attempt to give an indication of the extent to which IDA countries have benefited from MAPS/DGF. 3.66 Other work has been done with regional groupings and using the regional networks of some of the partners and their advisers. DGF has enabled this. It also allowed developing countries to participate in regional workshops and receive direct assistance from regional advisers. This has been an efficient way of operating. NSDS Design, Review and Implementation 3.67 National Strategies for the Development of Statistics are at the heart of the MAPS initiative, and despite the excellent guidance which has come out of Paris 21, are still a national responsibility. They pre-date MAPS, but received a new impetus from it, as we described earlier. These few paragraphs will try to say what has been happening in IDA countries using the assistance from Paris 21, both regional advisers and Paris- based staff. 3.68 The main observation to make at the start is to note the large effort that went into the first two years of the programme. A rough count suggests that between December 2004 and October 2006, Paris 21 organised about 40 regional workshops with world- wide coverage; made missions to at least 12 countries and assisted others by video-link; and participated in at least 16 statistics-related events, at which they presented NSDSs. 3.69 The results are set out in Table 3.2. The original aim of MAPS was “to mainstream strategic planning of statistical systems and prepare national statistical development strategies for all low-income countries by 2006”. This date was optimistic, to say the least, and certainly more recent DGF agreements with Paris 21 softened the target. This is sensible - if you want country ownership you must be prepared to go more slowly. But considering that the first regional workshop was only held in December 2004, to set out the roadmap for NSDS development, progress has been excellent. The 2007 grant agreement added to the NSDS target the words “and are able 41 to improve the performance of their statistical systems by 2010” and we come back to this later in the chapter. 3.70 To summarise, of the 39 Sub-Saharan Africa IDA countries: In 2004, 12 countries had no strategic plan: by 2008, • 1 still had no plan; • 10 were developing an NSDS; and • 1 was implementing an NSDS. In 2004, 14 countries were at some stage in the design of a plan: by 2008 • 8 were still at the design stage; • 5 were implementing an NSDS; and • 1 was implementing an NSDS and designing the next one. In 2004, 13 countries were implementing a plan: by 2008 • 1 was still implementing their earlier plan; • 9 were designing a new NSDS while implementing the earlier one; • 2 have designed and started to implement a new NSDS; and • 1 is designing a new plan. 3.71 For the overall IDA list of 80 countries, the situation in February 2008 was (from Paris 21’s own progress report) that 35 countries were currently implementing a strategy, a further 40 were planning an NSDS, and for 5 countries there was no information. 3.72 The extent to which these strategies follow the guidelines is a matter of judgement. It could be argued that the process of producing a strategic plan (at least the first one) is of more import than the final content, and this is to some extent true - one use of the plan lies in the way in which the process, if followed correctly, promotes and builds relations and links within the statistical system which then leads to a better understanding of the role each person and institution plays within it and in time a better strategy. But the choice of whether or not to follow the guidelines lies with the statistics office and their assessment of the current state of affairs and what they think is achievable. 3.73 In Malawi, for instance, the first plan only covered the national statistical office and was very much an internal document but one in which for the first time the statisticians could do some blue-sky thinking. By the time of the third plan, which follows NSDS guidelines, the process had become almost automatic and consultation was widespread and inclusive. In Ethiopia, for their first plan the CSA worked only with the Ministry of Finance and Economic Development, as a representative of all official users. 3.74 Annex 2 contains a rapid attempt to review the strategic plans for Africa which are readily available from the Paris 21 Knowledge Base, supplemented by information from their network of advisers, and national statistical office websites. The judgements contained in it are those of the consultants, and do not necessarily reflect Paris 21 or World Bank opinion. What we were attempting to do was summarise to what extent the 42 strategies produced appeared to follow the NSDS guidelines. One of the main difficulties was to judge the level of consultation which has taken place, since many plans do not describe the process. A second difficulty arose in trying to decide whether the strategy was being implemented, or what stage the design process had reached. Paris 21 advisers were very helpful here, but it is possible that we may have misinterpreted or misjudged the situation and we apologise if that is the case. 3.75 In summary, then, with the above caveats, the current situation seems to be that: • 12 countries are implementing a strategic plan which follows or is very close to NSDS guidelines: Burkina Faso, Cape Verde, Congo-Brazzaville, Djibouti, Kenya, Malawi, Mali, Mauritania, Niger, Nigeria, Rwanda, Uganda; • 13 countries are close to finalising an NSDS which aims to follow the guidelines (and some are currently implementing an earlier strategy): Benin, Comoros, Cote d’Ivoire, Ethiopia, Gambia, Guinea, Lesotho, Madagascar, Mozambique, Senegal, Sierra Leone, Tanzania, Togo; • 4 countries have prepared a road map and are in the process of preparing an NSDS (and may have an earlier plan in operation): Burundi, Cameroon, Ghana, Liberia; • a further 6 countries have prepared a road map for an NSDS but apparently have not moved much further on (though they may have an earlier plan in operation): Angola, CAR, Chad, DRC, Guinea-Bissau, Sao Tome; • and there are 4 countries for which there is not enough information to judge exactly what is happening at the moment: Eritrea, Somalia, Sudan, Zambia. A plan which follows NSDS guidelines but only covers the central statistical office has been counted as an NSDS. 3.76 The fact that by 2008, 19 countries in Sub-Saharan Africa were implementing a strategy, and several were on their second, should, one hopes, mean that statistical standards are being raised. But there is no evidence readily available at international level. The draft framework document Better statistics for better results produced by the World Bank in April 2008 sets out an indicative results framework by which capacity building could perhaps be measured and is a major advance on previous thinking. However, it will require either a careful recording of the current situation in NSDS countries, which is then monitored; or a change in the way international databases record country data - in the same way that the UIS database can now be used to partly judge the quality of education data. In the fourth section of this chapter, we shall examine the main international databases to see what they can tell us about the impact of NSDS. 3.77 Effective implementation of NSDSs is the unknown factor. While we have one or two examples where implementation has started, many have not progressed as yet beyond the necessary changes to the legislative framework, and in some cases creating a new statistical body. Donor funding has been forthcoming, but not always through budgetary support, and due to the structure of statistics, tends to focus on the central 43 statistical office or a single sector and not the whole system. There may be the concept of a national statistical system, but the bottom line is that it does not have a central management structure. The advantage of having a strategic plan is that when an opportunity arises for investment in a statistical unit, be it cross-cutting through the statistics system or sectoral within a line ministry, the strategy is there. 3.78 There do, however, seem to be certain things which aid successful implementation (or whose absence hinders it). These include: • a statistical common service, which links (officially or otherwise) statisticians across the public service; • up-to-date legislation, which correctly recognises the functions of the components of the system; • high-level support from a number of key figures in government, over a number of years and not just at the launch of the plan; • donor willingness to assist with financing implementation of the strategy; • a degree of realism in what is to be attempted - unreality leads quickly to disillusionment; • some stability among the key personnel. There will be many others. Accelerated Data Programme - Discovering Data 3.79 This was envisaged as a pilot programme whose overall objective was to provide policy-makers and other stakeholders with better data and analysis. In a few pilot countries there was to be a coordinated programme of data evaluation, collection, analysis and dissemination. It evolved rather rapidly into a programme which got survey data “out of the drawers” and into the public domain, and from the original four countries has been considerably enlarged. This is principally due to the huge interest shown in ADP by countries and the ease by which the technology can be transferred. The change in emphasis was agreed by the World Bank and the Advisory Board. 3.80 There was no development objective specified in the grant agreement, but the team now implementing the programme suggest the following: to increase the value and foster better use of data by introducing improved practices of data collection, management and dissemination in participating countries. This is very much in line with activities. 3.81 ADP now has three task components which are: 1. to make existing survey micro-data more widely and easily accessible to society; 2. to assess the quality of survey data with a view to improving the relevance and quality of future surveys; and 44 3. to assist countries to design improved mid- and long-term survey programmes and co-finance the implementation of data collection activities where there is a pressing data gap. And to help implement these three tasks, three tools have been developed: 1. the Data Documentation Initiative (DDI), leading to a National Data Archive (NADA) and using the microdata management toolkit; 2. the Question Bank; and 3. the Survey Quality Assessment Framework (SQAF). 3.82 It is hard to disentangle ADP from IHSN in-country, as the two programmes complement each other, but IHSN is a global programme of which Paris 21is a member – a type of research laboratory – while ADP is very much a short-term programme which is run by Paris 21 as the host agency. 3.83 Task 1 is now underway in about 25 countries, with plans for a further 15 or so, and is starting to show results, as demonstrated by the build-up of survey metadata in the IHSN catalogue. The underlying principle has been to find out first what survey data are available in-country before embarking on new ones, preserving and disseminating the existing data. Using the IHSN to reference the surveys, this has filled knowledge gaps at little cost. It is however still very much in a pilot stage with the main impacts yet to come. The micro-data management toolkit is in use in a number of countries following training, and it has proved very easy to use. In Cameroon, for example, the statistics office trained staff of other agencies as well as staff from DRC. The Uganda Bureau of Statistics has trained colleagues from Rwanda and Kenya. 3.84 Task 2 so far has only been implemented in Cameroon, for the education sector; and has begun in Mali (for agriculture), Ethiopia and Uganda. At a regional level, ADP played an active role in an FAO/Paris 21 workshop on agricultural statistics. 3.85 Part of Task 3, financing data collection, is not really feasible with the limited resources available, though help has been given to 5 countries to complete surveys which had run into difficulties. This is not strategic, though it is clearly useful. However, the other aspects of task 3, better design and planning of survey programmes, is showing results. Links are being made between survey planning and an NSDS. 3.86 To summarise, then, the impact at present on statistical quality is potential rather than actual. The focus on quality and standards in the statistics offices is welcome, and should make it less likely that survey data are wasted in future. The SQAF may be a useful tool, though it is as yet only at the development stage. UIS Statistical Capacity Building 3.87 The DGF contribution to statistical capacity building by UIS is minimal compared to what is required to build an administrative system which will generate satisfactory education data, and thereby improve education statistics, even in one country, let alone all those that need help. We have seen earlier how work in Montreal will lead to improvements in the way data are handled once they leave the statistics office; in the way errors will be spotted more quickly and the systems for resolving 45 them made more efficient; and how a variety of other initiatives will leave less room for errors in the first place. 3.88 However, UIS has also received funding from the EU and World Bank for intensive work with a number of countries - 11 for the EU and 19 for the World Education Indicators project (with many of the countries covered being eligible for IDA assistance). This has led to excellent and measurable results. 3.89 UIS highlights the following outcomes: country visits have led to the following improvements in regard to education finance: • Mozambique providing timely data from 2004; • Burkina Faso, Mali, Lesotho and Niger providing timely data from 2005; and • Ethiopia providing timely data since 2006. Kenya and Tanzania received visits in 2004, and now complete all UIS questionnaires. CAR and Sierra Leone have also improved their reporting. Other countries which have been assisted, with similar positive outcomes, are Gambia, Liberia, Eritrea, Madagascar and Rwanda. Overall, for 15 countries data reporting has been moved forward by two years, and for 9 countries it has moved forward by three. 3.90 However, UIS has also introduced a new policy of not publishing estimated data for countries that didn’t submit data for a given year, which is hoped to encourage a more prompt response. In the short term, however, it leads to an increase in apparent non-availability of data. 3.91 Ethiopia is one of the countries that has benefited from a UIS technical assistance project which was set up in 2003 with the aid of European Union financing. The existing education statistics system was surveyed, and recommendations made for improvements including a revision of the questionnaire to make it compatible with international reporting and its translation into five languages. Around 17,000 people were trained in use of the new questionnaire. So far the system remains sustainable. New head teachers are being trained, and, following two years of funding by UIS, government has taken over the task of printing and distributing the forms. Within the Ministry of Education, the UIS project set up an EMIS unit, installed the software and hardware to compile education statistics, and trained the staff in their use. However, while the staff of the unit can use the software for inputting data and creating reports, an IT specialist is required to make any modifications – for instance, to create a new indicator or report. There is at present no back-up, should the UIS expert leave. 3.92 Annex 7 has more details of other cases, including useful case notes on impact in Niger and Guinea. The point to notice is the intensive nature of the assistance, which works but is costly. UIS is presently reviewing its strategy for statistical capacity building, and the approach that is emerging focuses mainly on somewhat increased UIS field presence in the cluster offices and regional bureaux. This will not however be of a scale to support direct responsibility for projects of the intensity of those done in the past with supplementary external finance. 3.93 Rebuilding education statistics is mainly – probably 70% or more – an administrative task, of organising the movement of large quantities of data through a 46 complex system, and 25%+ statistical, for verifying methodological soundness, accuracy and reliability of figures gathered, and for compiling and analysing the data as they arrive at some central points. We feel that UIS’s strength is in its technical knowledge upstream and by positioning itself there can add value to what in effect should be a sectoral improvement programme. 3.94 There are also synergies still to be built between UIS and NSDSs in-country, and between UIS and Paris 21 concerning some of the software products. UNSD Regional Workshops 3.95 Of direct benefit to countries have been the series of technical workshops which UNSD have been delivering year by year. Based around the fact that 70% of countries are planning a census in 2010 or 2011, they have organised a 10-year programme of training and publications which goes through the census process stage by stage. The theme for 2007 was census cartography. 3.96 In 2007-08 6 regional workshops were held in Lusaka, Bangkok, Trinidad, Rabat, Cairo and Noumea for 200 participants from 78 countries. UNSD keep careful records of customer satisfaction which has been high immediately after the event and remains high 6 months later, with so far all participants saying they have used the materials from the workshop and three-quarters claiming that they have been able to implement some of the recommendations. The workshops are a mixture of information, options and country exchanges, with the commercial sector invited to present their products. As well as going through the recommendations, UNSD leave time to work with individual countries on specific issues. They also advise on technical assistance. 3.97 Advice on new and existing technologies has been valuable, with UNSD assisting countries to take a pragmatic view. GIS and GPS, for example, may not be a sensible investment if they are only to be used for census, nor might investing in expensive optical scanning equipment. UNSD are able to give full and unbiased information to countries on the costs and benefits, which it is to be hoped will lead to less money being wasted during this round. UNECE Gender Statistics 3.98 This programme, run by the World Bank Institute and UNECE, started in 2003 but received new impetus from DGF funding from FY07 onwards. It contributes to MAPS action 5 - improving the MDGs - but in the documents we have seen has never referred to, or been connected with, MAPS itself. For pre-existing reasons, the partner countries for what could be seen as a pilot programme are mostly in the CIS and former Yugoslavia. 3.99 The first workshop funded by DGF was for training of trainers, and was held in Almaty in April 2007 for Bosnia-Herzegovina, Kazakhstan, the Kyrgyz Republic, Moldova, the Russian Federation, Serbia, Tajikistan, Turkmenistan and Uzbekistan. Follow up workshops were held in five countries which are now developing policies and practice for engendering their national statistical systems. A community of practice has been formed, sustained and strengthened with the assistance of WBI and UNECE, 47 and since the national statistical offices take ownership of the programmes, sustainability is likely. 3.100 The gender dimensions of statistics are often overlooked - they are more than a sex-disaggregation of survey data - but can be developed fairly cheaply. Using UNECE/WBI training materials, statisticians from participating countries are training others. This programme has the potential to make a difference, though it is in its early stages and there are few concrete results as yet. 3.101 Other regions are interested in the programme which could easily be extended with the right partners. The main issue, we think, is to make more of a connection with the other parts of MAPS. The Question Bank, for instance, may benefit from a gender commentary. Countries may wish to put gender into their NSDS. The UN Census Bureau has already indicated its interest. IIID. International Evaluation Data 3.102 In the sections above, we have set out the principal activities funded through MAPS and attempted to show the improvements in the quality of statistics which are likely to come about as a result. The majority of actions, however, do not in themselves guarantee success. Statistical improvement will come about through a sustained programme of institutional and technical development, which will take time. And improvements we can see now are quite possibly the result of actions which started before MAPS. 3.103 Given these caveats, we turn first to the expectations. In the various DGF agreements, we have a mixture of performance indicators, most of which are to do with activities rather than outcomes. This is a weakness, which has been recognised. The scaling up initiative framework document contains a much broader and more searching set of indicators of capacity than has hitherto been collected systematically. So the expressed high-level expectations are of limited use for this evaluation. [They can be summarised as: countries are able to improve the performance of their statistical systems; or, increase the capacity of national and international statistical systems to collect, compile, disseminate and use statistical data; or simply, better data for better results.] 3.104 A performance indicator, then, for successful statistical capacity building could then be: “an observable improvement in what the statistics office (or system) can produce and maintain unaided”. The most important place to look for evidence is therefore the countries themselves. For baseline “before-the-event” data we might have an analysis in the PRSP or NSDS; for improvements we need to look at the published data and have them quality assessed, preferably by the producer of the data. Websites are valuable for what is available now, but cannot throw light on what it was like some years ago. 3.105 For the remainder of this section we shall examine in turn the main international sources of data for indications of improvement. These will include: 48 • The World Bank Country Statistical Information Database • The UN MDG website • The IMF Reports on the Observance of Standards and Codes • UNdata for consumer prices and external trade 3.106 The World Bank CSIDB has scores for statistical capacity measured in a number of ways. Table 3.3, in the Statistical Annex, summarises the various factors which contribute to the score for a particular country. It will be noted that some of the indicators measure whether or not a particular survey has been done during a certain period. If survey data are available for 2000, for example, but no later, then that would trigger a change in that indicator’s contribution to the index from 10 in 2004 to 0 in 2007. But if the survey in question was donor funded, some would reasonably question whether the capacity of the statistics office has changed for the worse. 3.107 Table 3.4 gives the scores for IDA countries for the year 2004 to 2007, with a qualitative assessment of the direction of change, if possible. In sub-Saharan Africa, four countries experienced a general decline in reported capacity - CAR, Zambia, Burkina Faso and Cote d’Ivoire - while in Eritrea and Gambia there was sudden drop from one year to another. In one third of SSA countries (13) the scores were so variable between the years as to make a judgement of direction impossible. But in 7 countries there was a steady improvement, and in 6 more a step improvement. The remaining 7 saw no significant change. 3.108 Outside sub-Saharan Africa only two other countries saw a decline - Djibouti and Nicaragua. 14 of the 41 saw improvements, in 7 there was no real change, while in 18 the scores are too variable to judge. It is hard to see any logical connections between changes in the CSIDB score and the presence or not of MAPS activities. 3.109 This is at the level of overall score. We note, in fact, that the CSIDB methodology makes no distinction between national and international data for some indicators, and the section on data collection is mainly concerned with surveys and censuses. We might consider then that the section on statistical practice might give us potential candidates for measuring capacity. UNESCO reporting, trade price indices and the industrial production index all require capacity to be maintained. It is unfortunate that it is the base year of the CPI which is recorded, and not whether or not it is actually calculated. Table 3.5 shows the scores for these three indicators and highlights where there has been any change in the three years from 2004 to 2006. Very little can be deduced. 3.110 Turning now to the UN MDG website, as at May 2008 this had been last updated (for the majority of the indicators) in August 2007, with data for 2006. In a few months time we should be able to access data for 2007. 3.111 The way data are typically shown is demonstrated in Table 3.6 for the four countries visited by the consultants. The webpage is extremely eye-catching and colourful: it is unfortunate that this cannot be downloaded “as is” – the download consists of cells containing data and cells containing footnotes. We see from the table the improvements in basic education data, and the effect of the introduction or reintroduction of surveys such as DHS. There is no consistent picture of Capacity improvement, however. 49 3.112 In fact, an examination of the MDG website for SSA IDA countries shows that many indicators are periodic - every 5 or 10 years, which implies that they are survey- or census-based, which is not necessarily a measure of national capacity. Some indicators are estimates: it therefore could be argued that a move from UN estimates to use of country data would demonstrate increased capacity, but reporting methodology is against that - infant and under-five mortality, for example, which are crucial MDG indicators, are estimated every fifth year for all countries, including developed ones. 3.113 Education data are one area where potentially the MDG website might show increased capacity, and a closer examination of the SSA IDA countries shows that this is the case. A majority of countries have moved from estimates to country data; and many have broadened the coverage of education indicators from net primary enrolment to the others. Table 3.7 shows, for sub-Saharan African countries, the change in availability and source of three indicators - gross or net primary enrolment, national poverty estimates and births attended. The move from estimated data to country data for the education indicator is clear. 3.114 It would be good if some of the health indicators could be used as measures of capacity: unfortunately, the TB indicators for SSA IDA countries in all but 2 cases are identical for the period 2001 to 2006 and are clearly linked to the estimates generated by the DOTS programme. Figures for immunisation against measles are all shown as estimates. Births attended data do show developments from year to year - but as Table 3.7 shows, just as many SSA countries had data in 2002 and not in 2005 as vice versa. Again, this is likely to be linked to the existence of a survey. 3.115 For indicators which link to housing and slums, Table 3.8 shows what is available worldwide. The majority of developing countries can provide at least two data points from which the direction of change can be measured on the three indicators - improved drinking water, improved sanitation and slum population. This says little about capacity. 3.116 The third potential source of information on capacity is the IMF, whose Reports on the Observance of Standards and Codes for Data Dissemination contain an analysis of country practices against the data quality assessment framework (DQAF). Unfortunately, for the SSA IDA countries, such reports are few and far between, and in only two cases has there been more than one assessment, so as a measure of change they are not very useful. For the record, ROSC’s exist for Data Dissemination in the following SSA IDA countries: Burkina Faso (Mar 04), Cameroon (Aug 01), Chad (Aug 07), Gambia (Dec 05), Kenya (Oct 05), Malawi (Feb 05), Mozambique (Mar 03 and Aug 05), Niger (Jun 06), Senegal (Dec 02), Tanzania (Mar 04), Uganda (Aug 99 and Jul 06) and Zambia (Feb 05). 3.117 We look finally at UNdata, the database tool developed by the UN Statistics Division to bring all the UN databases into one portal. Statistics are available as follows: on agriculture (FAO), education (UNESCO), employment (ILO), energy, environment (UNFCCC), health (WHO), human development (UNDP), industry, ICT, national accounts, population, refugees (UNHCR), trade and tourism (UNWTO). 50 3.118 One could imagine, perhaps, that production of the consumer price index and external trade statistics could be a reasonable measure of internal capacity. For sub- Saharan African IDA countries the relevant extracts are presented in Table 3.9 for consumer prices and Table 3.10 for external trade data. As regards consumer prices, the only sign of improvement is in Guinea-Bissau, which has begun to report. In those countries where the index only existed for the capital city, it has not been extended to the whole country. The ILO database, though updated in April 2008, only contains country data to 2004, which is not a good sign of effective reporting, though the IMF manage to have data available up to 2006. 3.119 It is a slightly more interesting story for external trade, which shows if anything a decline in the number of countries reporting - or being reported on. The lack of data since 2004 for Guinea, Rwanda and Nigeria would warrant explanation. 3.120 To summarise, then, we have shown that with few exceptions - notable being the UIS database and its quality and timeliness assessments - the international reporting system does not lend itself readily to monitoring capacity. Too many of the indicators are derived from surveys; and too many data points are estimated - for what reason, is not always clear. IIIE. Specific Answers 3.121 The terms of reference ask specific questions of the evaluation team regarding outcomes, impact and their sustainability. The analysis in the preceding sections has, we hope, provided the background and justification for the comments which follow. To what extent has the MAPS partnership supported actions at the national and the international level? Extremely well. The international activities have been particularly innovative, with a firm logical basis and structure. The national activities have been necessarily more opportunistic, since success is more likely if the activity is demanded by the country and not imposed. This is the ethos of MAPS. To what extent has the MAPS partnership helped increase investments in statistics? Usefully, but it is early days to draw any strong, broad conclusion. The MAPS partnership has widened and strengthened the interest in strategy-based statistical capacity building. We have seen that in countries where there is a strong NSDS, this has often led to increased national budgets for statistics. Recently the African Union has indicated its willingness to provide additional support for statistics. Donors have come forward, and are still coming forward, to provide funding. To what extent has the MAPS partnership contributed to increased statistical capacity in countries? Technical capacity has been improved, and is likely to be sustainable. Less improvement in institutional capacity has been seen, but the potential for an increase in capacity due to MAPS activities and advocacy is huge. 51 To what extent has the MAPS partnership contributed to an improvement in the availability of key statistical data? Again, in a small way which is visible internationally. In selected countries, education, gender and urban indicators have been improved. Improvements from non-MAPS activities such as those of the IMF continue. To what extent is a review of the impact of the program on statistical capacity and statistical outputs meaningful, given that MAPS is a relatively new program and institutional changes typically take some time to achieve? The question answers itself, in recognising that institutional changes can take time and benefits may not be seen until well after the initiative has ended. Many of the changes began before MAPS got started (much of the legislative reform in Francophone Africa, for example, and some setting up of autonomous statistical bodies), and so in that sense MAPS has joined in a process and given it new impetus and structure. But statistical capacity building is, and has always been, a matter of institutional development with the added complication of statistical quality. MAPS, with its country ownership of national strategies, is a clear improvement on what was attempted in the past and should stand a better chance of success given time. Are these outcomes and impacts on the ground likely to be sustainable? Yes, if activities are properly funded and part of a country-owned and country-led strategy. To what extent do the MAPS partnership activities have measurable performance indicators – of outputs, outcomes and impacts? How useful are those indicators for assessing the effectiveness of the activities? The indicators have been mainly focused on activities rather than outcomes and impact. The process of this evaluation - the discussions with Paris 21, UIS and other institutions - has led to some small improvement in reporting, but each part of the MAPS programme now needs to look systematically at what it is really trying to achieve in the medium term and search for innovative indicators. Those in the scaling-up initiative are a very useful set. Systems then need to be put in place, preferably by recipient countries, to monitor progress. To what extent have the indicators improved since 2005 when the DGF MAPS program started? The measures of output and activity have become more realistic in terms of what can be expected over relatively short periods of time. 52 This page is blank 53 IV. Cost-Efficiency of MAPS 4.01 In contrast to the previous chapter, there is little to say on this topic which has any concrete foundation, and it is unlikely that there would be a great deal more to say even with several weeks of additional time. Given the paucity of information on impact and outcomes regarding statistics and statistical capacity, it is hardly surprising that references to cost savings or cost efficiencies are practically non-existent. We have in all the documents one quotation: the development of the new education questionnaire by UIS cost about $200,000, and will be of use to 140 countries for about the next 10 years. We also know that UIS’s new error reporting system will free up perhaps 94 person- days a year and DIVA about 315 hours a year for other work. 4.02 What we can underline are those areas where MAPS activities have the potential to • make things cheaper; • reduce waste; and • allow work to be done more efficiently; though in the same way that MAPS alone cannot be held responsible, or take the credit, for statistical capacity building, neither can it do so for costs. 4.03 At the outset, we must note that the DGF grant amounts did not appear to be decided as a result of any competitive tendering or detailed budgeting. One cannot then in retrospect say whether the amounts allocated were reasonable. But reading what was achieved by Paris 21, UIS, UNSD, UNECE and UNHabitat, with relatively small amounts of money, it is tempting and perhaps acceptable to conclude that it would have cost a lot more doing it through a profit-making body. 4.04 We can see potential cost savings to countries in many areas. First, the technological advances. The software which has been developed is free, open-source and runs on existing hardware. There is no necessity to purchase new machines or buy special software. Connected with this is the fact that all the necessary manuals, guides, recommendations, advice, etc are available on-line. In the past they sometimes had to be purchased and certainly copied many times. 4.05 The strengthening of South-South cooperation, building on easy-to-use technology, reduces costs for training and other development activities. Advice does not always have to be purchased from western consultants; and can also be delivered through the web as more materials are available. 4.06 Paris 21 in particular has used its regional network of advisers in multiple roles, saving them money and statistics offices time. They have worked in the margins of other meetings to maximise the impact of scarce resources. 4.07 We can see the potential to reduce waste in a number of areas. The biggest potential waste of money for a statistics office is a badly done census - it is also a loss of hard-won credibility - and the work of UNSD attempts to minimise the risk of failure by providing appropriate advice when it is needed. This includes, importantly, advice on when not to invest heavily in new technology. One interesting example of direct cost 54 saving comes from Ethiopia, where CSA estimates that their printing of paper copies of reports has been reduced by 70% as a result of data now being available on-line. 4.08 In the area of saving existing resources, the IHSN/ADP programme of giving new life to old surveys must be given credit. In Bangladesh, for example, all survey micro-data prior to 2000 are stored on magnetic tape and at present inaccessible - MAPS is working to try to rescue the data. 4.09 Finally, we can see a number of cases of cost efficiency where existing surveys have been made more valuable - the new slum indicators, the addition of questions to MICS and DHS, the gender dimension. The Question Bank will allow surveys to be developed more quickly and in line with international requirements, thus saving everyone’s time. ~~~~~~ 4.10 The terms of reference ask five specific questions, which, due to the lack of hard information, can only be answered in very broad terms. First, we are asked if the programme has cost more or less than planned, and how it measured up to its own costing schedule. Given that we feel the budgets were unplanned, we merely note that the recipients have worked within the budgets given. Expenditure, actual and planned, has been regularly reported to the Advisory Board. The strong links between some of the MAPS partners has led to a sharing of costs for specific activities - a workshop, for example, or a consultant - and they also use their own resources and those from other donors, so it is not possible to disentangle precisely who funded what without a great deal of effort. It may be felt that such effort would be nugatory, since this work is just a component of MAPS and, beyond it, of the overall global activity in statistics for development. 4.11 Second, we are asked how actual costs compare with benchmarks from similar programmes or activities, and if there are obvious cases of inefficiency or wasted resources. The first part we cannot answer with the evidence so far available. For the second part, we see no obvious cases of waste or inefficiency. 4.12 Third, we are asked what would be the implications of scaling the programme up or down in terms of costs, cost-effectiveness or efficiency. We feel that a scaling down at this stage could damage the programme, which to be fair has only really been running for 3 years, given the delay in starting funds flowing. MAPS is gaining a good reputation and complementary funding is beginning to appear, but it is too soon to scale down. 4.13 But nor is it necessary, in our opinion, to scale up expenditures on the package of programs presently underway. What is needed, and this was told to us, is reliable funding for several more years, not additional money. The main thing which could be done with more funds is greater in-country capacity building, which is indeed much needed but mostly requires a substantial investment over a long period, which is not appropriate for DGF and the GPP. 4.14 Fourth, we are asked what would be the cost of replicating the programme’s activities in a different environment. In different geographical regions it may be a little 55 more or less expensive depending on the partner; but the technology is completely transferable. Indeed, ADP, IHSN, NSDSs are already branching out over most of the world. In an environment without Paris 21, then, it is hard at the moment to see the programme surviving without creating a clone: as we said earlier, Paris 21 brings a welcome and much-needed independence to statistical capacity building. 4.15 Fifth and finally we are asked how the costs affect the results and sustainability of the programme. At present the programme would not sustain itself without core funding from somewhere. There is still too much to do of a global nature which could never be funded from national or regional programmes. If it stopped now, a lot would be lost. 56 This page is blank 57 V. Country-Level Perspective 5.01 The purpose of this chapter is to synthesize the main findings that emerge from the field work undertaken in the second phase of the evaluation (August-September 2008). That work extended our contacts beyond the partner agencies to at least a small sample of the countries which are intended to be the main ultimate beneficiaries of MAPS activities. 5.02 As noted in the report on the first phase of the evaluation, these visits were planned to meet a general purpose – of providing an opportunity for open and direct expression of views about the whole initiative from some of the developing countries. And they were intended also to meet three more specific purposes, of enabling the evaluators to gain (a) more of a feel as to the degree of country ownership surrounding the National Statistical Development Strategies developed, (b) some understanding of obstacles that might be arising in strategy implementation, and (c) some knowledge of country experience with the specific assistance provided to them by individual MAPS partners. 5.03 The findings from the country visits significantly clarified and deepened our understanding of the way MAPS initiatives have been impacting on the countries, but they were largely consistent with the assessments, and the general outlook of cautious optimism, that emerged from our discussions with the partner agencies in April-May. Points directly relevant to some of the more technical statistical discussions in Chapter III have been taken into account in the final editing of that text. Brief reports on each of the country visits undertaken are given at Annexes 3 – 6. This chapter looks at the various interventions in the context of each country’s overall statistical development, with a comparative perspective to try to ensure derivation of sound conclusions from such a small sample of countries. 5.04 On the basis of the comprehensive information gathered from the partner agencies about African countries’ progress in preparation of statistical strategies and plans, and their participation in other MAPS activities, the Phase 1 Evaluation Report identified 7 countries as potentially most suitable for review aimed at meeting the above-mentioned objectives. Mainly for reasons of travel feasibility, including securing of visas where necessary, we finally requested support for the work from Ethiopia, Malawi, Mali and Niger, all of which kindly agreed to receive the brief one-person missions envisaged. 5.05 All four countries are among those with the lowest per capita incomes in Sub- Saharan Africa, but they are largely at peace and politically stable. In terms of statistical capacities, the first two fall within the top quarter of all SSA countries according to the scoring system developed by the Development Data Group, while Mali and Niger lie about the middle of the overall distribution. 5.06 The four countries present rather contrasting pictures in regard to statistical development over the last 8-10 years, and it is quite surprising to find these contrasts 58 also reflected in the relative progress they have made in generating indicators for the MDGs that are based on country data rather than broader estimates or international modelling exercises. Comparison among the four tables 3.6, one for each country, shows that each has been making some gradual improvements, especially on health and to a lesser extent on education, where coverage was already stronger. 5.07 But Ethiopia indicates perhaps the largest improvement, with country-data-based estimates becoming available seemingly for the first time on five important health indicators. Niger demonstrates a strong record of up-to-date country-data-based reporting through these years, and is the only one among the four to have recent figures for adult literacy by sex and age. Malawi has caught up a bit on coverage of some health indicators which are also more current. Mali has apparently country-data-based estimates for the first time for a number of important health indicators, but mostly dating back to 2001 and without coverage since. Unlike the other countries, Mali seems not yet to have been able to generate a country-data-based figure for the proportion of children under 5 years old sleeping under bed-nets. 5.08 Although not one of the countries yet has a fully system-wide NSDS under full implementation, considerable care has been devoted to the development of such plans, and there are indications of quite strong ownership of the plans in three of the countries. Ethiopia’s Central Statistical Agency, a federal government body, has been following Paris 21 guidelines closely in preparation of the country’s first system-wide NSDS, now nearing completion with wide government backing. CSA and the federal government have shown a good record in implementation of the more limited statistical plan that has been underway since 2004. 5.09 Niger’s carefully prepared NSDS was formally approved by the government only in January 2008 and awaits stronger commitment of foreign support for full implementation, but the statistical system has made steady progress since 2000 with strong government support and the NSDS seems to be genuinely owned by the political leadership and the ministries. Malawi had traditionally a probably somewhat stronger statistical system than the other countries and it is currently implementing a plan that seeks to restore good coordination between the NSO and the statistical work of some of the most important ministries. Mali issued in March 2008 a further improved version of a nationwide statistical plan involving essentially the same institutional reforms as put forward in 2000, but it is unclear whether sufficient consensus and ownership have now been achieved to enable them to go ahead. 5.10 More than any generic obstacles to plan implementation, the experience of the different countries does highlight the large amount of time required for plan preparation, including the very important up-front stages of creating stronger arrangements than have existed in the past for effective interaction between producers and users of statistics and closer coordination among the different producer organizations. Even with the good organization, and the high priority assigned to the matter, in Niger, the process took a minimum of 18 months, and with preliminaries and time for approvals, more than two years. And if the planning exercise reveals the need for significant legal reform, time required is likely to be substantially longer. 5.11 These time requirements, combined with the fact that the four countries reviewed were chosen mainly to try to capture experience of the more advanced 59 statistical reformers in Sub-Saharan Africa, raise substantial doubts about the feasibility of reaching the already scaled down objective of ‘enabling all low-income countries to improve the performance of their statistical systems by 2010.’ On the other hand, nothing emerges from the experience reviewed to recommend skimping on the preparation steps that have been recommended by Paris 21. And the fact that at least one of our cases (Niger), and possibly more, appear to be reaching the stage where they could make good use of grant financing beyond the scale that it has hitherto been available for statistical work underlines the timeliness of the scaling-up initiative originated by the World Bank, UK and the Netherlands. 5.12 As regards the programmes and initiatives undertaken by the MAPS partners with the aid of DGF financing, reactions from those concerned in the countries visited were generally enthusiastic and indicated significant effects, achieved or in prospect. Systems for collecting primary and secondary education statistics in all four countries have clearly benefited from UIS advice and exchanges. Initial UIS efforts on education expenditures and school management have attracted much interest from younger staff and would appear worth expanding. 5.13 UNSD census-related work has clearly demonstrated its value in the two countries that have already carried out their census for the ‘2010 round’ (Ethiopia and Malawi) and is expected by the national Statistics Directors to do the same in the other two countries. ADP has been very active in the three most needy countries, leading all of them into effective archiving of past surveys and laying foundation for better management of future surveys. There is no evidence that these efforts have already affected research work or policy studies underway in or on the countries, but this is only a matter of time. 5.14 Paris 21’s active and sympathetic promotion of statistics in service to development, and of the institutional structures responsible for generating and disseminating them, is greatly appreciated by those in charge of statistical work in the four countries. A wide range of benefits is acknowledged, including links to the outside world and the international community of statisticians, help on publicity and publications about countries’ own plans (and any necessary translation thereof), organization of regional workshops, networking among institutions and people with particular knowledge and experience, conducting of useful research and issuance of guidelines and other advisory documents, and help in broadening and deepening relations with donor agencies. Paris 21 is particularly valued in cases where the local environment for statistical work is difficult. Suggestions for the Future 5.15 The country reports draw attention to a few points that may warrant further consideration by the MAPS Partnership in the future development of its work: • There is a widespread perception that capacities in policy departments of key ministries to analyse statistics for diagnosis of problems in public services or in broader economic trends are not developing fast enough to enable countries to get full value from statistics produced. MAPS might be able to pilot solutions, such as a very practically focused training programme that could be replicated on a country basis by others. 60 • Many IDA countries are still struggling with the creation of better links between the demonstrated performance of government-supported programmes in one period and the budget provided for the next period, as a means to stimulate better management for results. A Paris 21 task group on this subject, or on adaptation of statistical products to specific user needs more broadly, could produce recommendations of great practical value on appropriate statistics for such purposes and ways to make them available on the very tight time schedule involved. • Indications are that better integration of statistics-producing units into a national system, as promoted by an NSDS, is increasing the need for fully trained statisticians and the gap that seems to have arisen – now in Francophone as well as Anglophone countries – between need and availability of funds for scholarships for such training. Since the subject is so important to achievement of MAPS objectives, the Partnership should perhaps seek to stimulate promising solutions from the broader aid community. • The history reviewed indicates one or two occasions when countries faced quite specific dilemmas or obstacles to progress in the development of their statistical systems, and very timely and focused advice might have made a large difference to the pace of progress they could achieve. The MAPS Partnership should consider creating standing mechanisms that could enable provision of rapid, tailor-made advice whenever such circumstances arise. • A number of instances have come to light where host agencies in the developing countries find themselves unable to make small modifications or extensions to the computer software provided under MAPS programmes and feel that limited additions to the training that was provided to their technicians would have enabled them to avoid this problem. MAPS partner agencies should give careful consideration to these possibilities when they decide the coverage of training provided in support of new software, and also to measures to assure as convenient back-up facilities as reasonably possible. 61 VI. Assessment of Governance and Management Structure and Patterns 6.01 The designers of MAPS were at pains to bring about stronger communications and partnership between the stakeholder groups and agencies that would be involved, but to rely as much as possible on existing reporting and accountability arrangements and minimize the creation of additional bureaucratic superstructure. It was finally decided to create two new coordinating bodies, which came into existence toward the end of 2005. 6.02 The principal new body was an international MAPS Advisory Board, which would provide strategic direction, promote coordination and monitor implementation of MAPS and particularly of the components being financed by DGF. The Board would include very experienced policy makers and statisticians, from international agencies, from a few OECD countries and from some developing countries, with the Bank’s Vice President Operations Policy and Country Services serving as Chairperson. Membership of the Board would be by invitation. The Board started operation with 16 members but 4 additional members were appointed in 2006 in order to increase the representation of Asia, Latin America and Europe. 6.03 The Board was expected to meet once a year, which is what it has normally done, at a location to which many members expected to be coming for other gatherings. Sessions have lasted half a day. This gave the opportunity for the members to hear verbal presentations on MAPS progress from Bank staff and from Directors of partner agencies. Debates have been lively and have provided increasingly valuable advice and feedback as participants became more familiar with the programs and possibilities. The Board is expected to be asked to play a very similar role in future for the Statistics for Results Facility in addition to MAPS. It has been suggested that the value of the Board’s discussions might be further increased by greater clarity in the minutes as to which issues raised were, in the event, discussed, and what follow-up action the Board expected from the MAPS Secretariat in light of the conclusions of the discussion. 6.04 The other new body created in 2005 was an internal Statistical Capacity Building Committee (SCBC), proposed by the DGF Council to ensure better coordination among statistical capacity-building initiatives from different parts of the World Bank, and to oversee implementation of the DGF grant that had been approved, and preparation of applications for any future annual renewals. As initially constituted, the committee included eight senior representatives from the Bank networks and regions involved in the results agenda and statistical capacity building, meeting under the chairmanship of the Director Country Services. 6.05 The committee’s terms of reference expressed its mandate in the following terms: “The function of the SCBC will be to promote the coordination of statistical capacity building activities within the World Bank as a key component of the overall results agenda. It will advise the Vice-Presidencies involved in implementing different components of the Statistical Capacity Building Program and will report, as required, to the Senior Management Team and the Executive Board.” It was at that time expected to meet on two or three occasions in a year. 62 6.06 In practice the Committee has met only with similar frequency to the Advisory Board, and focused its attention very largely on the DGF grant to MAPS. The Committee has strongly upheld the focus of the MAPS program on capacity building for generation of statistics that would meet national and international needs and have wider effects in terms of enhanced development results. It advised the DGF Council against addition to the program of projects which the Committee considered not to meet these standards. 6.07 An important question raised in the Terms of Reference for the current evaluation is the extent to which the governance and management structures established for MAPS facilitate the effective participation and voice of different stakeholder categories in major decisions. Consistent with the structure and organization of the World Bank more broadly, decisions, as decisions, are the undivided responsibility of the assigned staff-members and managers of the Bank, acting under the authority of the President and Board of Executive Directors. But the Bank gave great attention for MAPS to the wide range of outside stakeholders – statistical and aid agencies, both international and national, developing countries, and data users – whose views needed to be brought to bear, and to securing high-quality and well-balanced representation of these interests in the Advisory Board. 6.08 The composite structure mobilized, led by the management of the Development Data Group, with firm guidance from the DGF Council and the Statistical Capacity Building Committee on grant allocations, and advice on main directions and issues from the Advisory Board, with considerable reliance on largely pre-existing senior governance structures in the cooperating agencies, can be considered a generally efficient and effective solution. It has certainly contributed, both in general and in many specific instances, to achievement of stronger inter-agency and inter-country cooperation for the development of official statistical capacities in developing countries. 6.09 There would appear to be two main ways by which participation of interested stakeholder groups in the shaping of main decisions could be somewhat further increased. Both concern the operation of the Advisory Board. Representation of non- statistician data users, and especially of those with policy-making experience in developing countries, has been weak in Board composition, and weaker still in actual Board attendance. The MAPS Board appears to compare poorly in this respect with similar bodies such as the Steering Committee of Paris 21 or the Governing Board of UIS. 6.10 Second, the Board would seem to need a broader and more forward-looking invitation to consider future alternatives and help shape the MAPS program. This, and more time to do the work (a half day seems to be only a third or less of the time allowed for main meetings of UIS and Paris 21 governing bodies), would give better opportunity for the participants’ varied perceptions and experiences to be brought usefully to bear on major decisions. 6.11 Accountabilities within the various cooperating agencies for program delivery and expenditures seem to be clearly understood and acted upon, by project officers to the managers of the agencies in which they work and those agencies’ supervising boards, as well as to the World Bank staff member responsible for following the grant, 63 and through him or her to the higher management of the Bank, the SCBC and the DGF Council. 6.12 The Terms of Reference inquire about the degree of acceptance of responsibility within MAPS to stakeholders not directly involved in programs’ governance and past activities, and the extent to which potential participants or beneficiaries under the various programs are given equal opportunities to take part. The programs have all been organized through inter-governmental organizations, and bodies under their wing, which have long-established responsibility and well-tested procedures for serving all the countries in their constituency and attracting the interest of relevant bodies within the countries. They have generally well-conceived websites, to which specialized subsidiary sites have been added, as necessary, for programs initiated. 6.13 Participation in workshops and seminars organized in quite large numbers by bodies such as UNSD for 2010 Census Round, UIS, Paris 21 for strategic planning and ADP activities has been very wide, and there is no evidence of favoritism within the groups of countries targeted in each case. The UNECE initiatives in promoting more attention to gender issues in statistical work in countries of the former Soviet Union and South Eastern Europe have been concentrated quite intensively in some eight countries but that reflects the somewhat experimental nature of the program and the fruits of considerable prior effort to identify relevant interest and potential in the statistical institutions of a larger number of countries in the region. UNECE, WBI and others have also continued to give considerable publicity to the program’s activities, to attract broader interest in the region and in other regions, which seems indeed to be developing. 6.14 Public reporting of the progress of MAPS does appear to have suffered some neglect since the first year of operations, not for any intention to avoid transparency but rather due to shortage of time and resources. That MAPS has not achieved the widespread recognition of Paris 21 does not worry us at all since MAPS only intends to mobilize effective actions among a limited number of agencies and governments. Consequently it may well have been a wise decision, in face of resource constraints, to drop the originally publicized intention of preparing an annual report on activities which the Advisory Board would send on to the UN Statistical Commission and the World Bank’s Board of Executive Directors. The agencies benefiting from the DGF funding do appear to give good publicity on their web-sites to the additional activities being undertaken, since that has obvious value for them. 6.15 But the relevant pages of the World Bank web-site seem to have had little updating since announcement of the original FY2006 DGF grant approvals for MAPS components, and it is hard even to find accounts of what actually happened at the Hanoi Roundtable of February 2007, in follow-up to Marrakech. The interest of concerned outsiders should be encouraged and catered to, at least to the extent of keeping basic news about MAPS and the relevant DGF approvals fairly up to date on the Bank’s web- site. 6.16 A few observers appear to believe that much more attention should be given to broader publicity and active promotion of the partnership concept within the statistical community and between it and others. One underlying thought is that the Advisory Board members should be doing more on their own, and among the organizations with 64 which they regularly work, to further promote the spirit of partnership that we have already noted as a valuable contribution from the MAPS activities. 6.17 It has also been suggested that the Advisory Board, or the Bank on its behalf, might seek an official mandate from the UN Statistical Commission to fulfil this role of promoting cooperation among the statistical community in support of development. The parallel is made with the UNSC’s assignment of responsibility to the Bank for managing the International Comparison Project (ICP) which also fostered substantial inter-institutional cooperation and brought about considerable progress in methodologies and standards for comparison of income-levels among countries. 6.18 The idea is that, with a more formal mandate, the Advisory Board would be encouraged to look more at experience and potential of partnerships generally in the field of statistics in relation to development, and the Board’s views and recommendations would be taken much more seriously by statistical agencies and institutions. It would have become more than merely an advisor to the World Bank. 6.19 As regards efficiency and probity, indications are that the normal operating procedures and disciplines of the Bank and the other agencies, as well as the dedication of staff attracted to this kind of work and the quality of leadership appointed, together ensure attainment of high standards. Managements are cost-conscious and supervising bodies help channel available resources to uses yielding higher development return. The external Boards themselves are not very costly since they normally cover travel costs for participants coming from developing countries only; all Board members contribute their time without any remuneration from MAPS or the other agencies served. 6.20 The most important potentials for improving the contribution of governance and organization to the achievement of MAPS objectives lie in adjustments to the roles of the Advisory Board and of the Statistical Capacity Building Committee. Especially with the addition of the Statistics for Results Facility, it is important to shift the main focus of the Board to providing strategic direction, the objective originally identified, by taking the discussions upstream to issues of identifying priorities in improvement of statistical work to get better overall development, and best ways to advance those priorities. 6.21 Subjects of focus for Board discussions would include: priorities among sectors, among types of country, among different data sources, and numerous concrete issues such as how best to help statistical development in fragile states, optimal design of a program to help developing countries carry out censuses, most effective divisions of labor between execution of surveys and reinforcement of administrative data collection, cost accounting for statistics, ways to build cooperation between central statistical agencies and statistical divisions of line ministries, solutions to implementation problems in strengthening statistical services. 6.22 At the time of its design, in 2005, no fixed term was set for appointments to the Advisory Board, but it is clear from the papers that the maximum length envisaged for a normal term was no more than three years. Because the Board first met in October 2005, this three-year period would be completed later on in the current year, and nearly two-thirds of existing Board members would therefore be eligible from this point of 65 view for retirement. A few would need to be extended because of the functional positions that they hold or the great importance of the experience accumulated about MAPS. 6.23 But sufficient places should be vacated to invite a few new members, including in particular the two or three people that are needed with extensive experience in policy- making in developing countries and in getting hold of the statistical information and analysis required for the purpose. Since a person of the high quality needed, such as a former Finance Minister with plenty of hands-on experience, may be too busy to accept a multi-year appointment, the alternative may be considered of having one or two Visiting Board Members (serving for only a year or two) to animate relevant discussions. 6.24 As regards the Statistical Capacity Building Committee, consideration needs to be given to the alternative of returning now to the larger mandate that seems to have been in mind in 2005 but was subsequently dropped: coordination of statistical capacity building activities in service to the results agenda, quite generally in the Bank. 6.25 Experience has repeatedly shown that successful development of sectoral statistical systems, like statistical systems more generally, depends significantly on maintenance of excellent links in two directions: with sectoral users, to ensure that their priorities, in terms of coverage and timeliness of data collected, are given principal emphasis in the conduct of work; and, secondly, with other parts of the country’s statistical system, responsible for the complementary data – for example on population, economic activity or pollution – that is often urgently needed to pursue analysis and diagnosis of trends and underlying causes but is very hard to pre-identify with any specificity. 6.26 Recent research on results measurement for IDA 15 suggests that sectoral lending operations have been giving considerably increased attention to country capacities to monitor key sector outcomes. Many of them envisage provision of support to reinforce country systems,2 and some of course include financing for major MIS developments. Such work is often in effect building the country system which generates the information required to estimate the indicator defined by efforts such as those financed by DGF with bodies like the UIS, UNSD and potentially IHSN – and sometimes such bodies are also assisting applications at the country end too. In the report on DGF allocations for FY 2008,3 DGF staff urged greater attention to the scope for constructive synergy between some of its operations and Bank/IDA lending operations. 6.27 Thus the mandate given to the Committee in 2005 seemed to be preparing the way for cooperation and partnerships to respond most effectively to opportunities that were newly opening up for sound capacity building through IDA operations as well as MAPS activities. It seemed slightly cumbersome to require that the Program for Education Statistics (as the already existing DGF-supported UIS program was called) get an endorsement from the SCBC in addition to that from the Education Sector Board out of whose tentative share of the DGF budget it would continue to be financed. But 2 IDA: Operations Policy and Country Services 2007 (op. cit.) p. 28. 3 World Bank 2007. “FY08 Development Grant Facility Budget and Review of Global Programs” Board Document R2007-0092 (May 25, 2007) p. 3. 66 such a requirement rather precisely reflected the importance of both user and co- producer links mentioned above. Moreover, the dual sponsorship and deliberation seems to have given rise to no significant delays or bureaucratic problems in actual application the last few years. 6.28 The issue may have become topical again currently insofar as the DGF hesitated to apply in 2008 the same policy as it applied in 2005. We understand that a first-year application for a DGF contribution to cover a small part of the costs of the Health Metrics Network’s flagship program was approved, on the basis of sponsorship by the Health, Nutrition and Population Sector Board, without requiring passage through the SCBC or formal inclusion under the MAPS umbrella (despite the fact that the HMN is in fact already actively involved with MAPS through IHSN and Paris 21). Indications from our small country studies and other sources are that health-related statistics typically remain weaker than those for most other major sectors, with one of the more important problems being the population denominators used, where good coordination with central statistical sources is crucial. 6.29 There do seem to be strong reasons for emphasizing links of health statistics with the rest of the statistical system as well as with the “in-house” users in the medical establishment. The SCBC should be able to make a valuable contribution to achieving this through reviewing annually the proposed grants to HMN out of the Health, Nutrition and Population Sector Board’s tentative share of DGF grants and seeking to ensure similar coordination in other operations, whether DGF- or IDA-financed, affecting statistical services for the health sector. 6.30 It is to be hoped, then, that DGF would consider returning to its 2005 policy when the second-year grant for HMN comes up for consideration. As the range of DGF-supported projects that can benefit from participating in the MAPS Partnership diversifies it will be appropriate, as also foreseen in 2005, to expand membership of the SCBC to include a wider range of experienced sector specialists. 67 VII. Main Suggestions for Further Action 7.01 This chapter responds to four questions about how to improve MAPS partnership performance that were included in the Terms of Reference for the evaluation. Second, it attempts to draw up a first and very tentative picture regarding the level of DGF support that would be desirable in the coming years, if possible, for the programs reviewed, treating each individually. Improving the Performance of the MAPS Partnership 7.02 The most effective way to build greater political commitment to statistical development in support of countries’ overall development is to achieve more cases of full or partial success and to document them appropriately, tracing the linkages from better information to better decisions to more successful development. This means that it would be worthwhile for the MAPS Partnership to try to bring about, in collaboration with national governments and outside sources of technical and financial assistance, some demonstration cases of success. 7.03 For example, Niger could probably be one very good candidate among the poorest countries if it is now able to mobilize the foreign support needed to implement its NSDS and succeeds at the policy level in accelerating recent progress on poverty reduction and the MDGs. It would be important, of course, to work with several countries with potential for such success in view of the many unforeseeable factors that can arise to obstruct progress. 7.04 Other broader work in the same direction can also make useful contributions. Increasingly systematic coverage of the adequacy of countries’ statistical services in Country Assistance Strategies prepared by the World Bank for IDA countries should deepen awareness of the links between information and performance, and encourage attention to identification of steps forward that have already been made and of new ones that could be promoted. Reflection and research could help to draw up better frameworks for thinking about which improvements in statistical services generate the highest benefits in different stages and conditions of development. 7.05 MAPS effectiveness at the country level now depends increasingly in the short term on successful implementation of National Statistical Development Strategies that are being generated by so many of the countries of greatest concern and are expected gradually to be completed by most of those remaining. In this period, and perhaps especially for the next three or four years, MAPS partners – and the other providers of financial and technical assistance – should be able to benefit greatly from six-monthly progress reports that Paris 21’s regional representatives, especially those in Africa, might be asked to prepare on all NSDSs already under implementation in countries of their constituency and the problems and obstacles arising. Countries would be strongly encouraged to prepare their own contributions to the reports compiled by the Paris 21 regional staff. 7.06 The Advisory Board, perhaps with wider participation from the partners, should review with concerned World Bank staff and others what initiatives might be envisaged to help overcome implementation problems. A regular and objective comparative 68 progress report to the Advisory Board on NSDS implementation could in itself be a useful stimulus and aid to countries for more effective implementation actions, to make sure that the country did not show up poorly compared with others; this could apply too to budgetary provisions for statistical services, which are highly likely to turn out short of plan projections in some countries despite the fairly modest scale of needs. 7.07 But other types of implementation obstacle may be sufficiently common among different countries to merit multi-country cooperation in trying to overcome them – such as focused training efforts to increase ministry capacities for use of statistics in the design of policy improvements, effective arrangements to fill the backlog in supply of trained statisticians, identification from different countries’ experience of the factors which distinguish fully effective statistical coordination bodies from others, or development of better practices for management and monitoring of NSDS execution. 7.08 The most promising general way to improve the results of MAPS grant-funded activities is probably to seek larger effect from them on the design and implementation of the related larger-scale projects (statistical or statistics-using) that are normally financed by other aid suppliers through loans, credits and grants. The standards and frameworks, concepts and methods, that are being developed by the MAPS projects have to be applied and used in large part by the local structures of social or economic reporting and in national surveys and censuses. 7.09 The planned development of SWAps specifically for statistics, as proposed for consideration in connection with the Statistics for Results Facility, could have a beneficial effect on coordination at this level, enabling donors to align any help they would offer for statistics with the country’s overall strategy for the service. Another helpful factor could be the recommendation of last year’s extensive evaluation of European Commission aid to statistics in developing countries that this sizeable aid program be somewhat more closely aligned with programs of the worldwide multilateral agencies.4 7.10 All the above suggestions underline the importance of sustaining the effort to strengthen MAPS programs’ development objectives and their follow-up, and particularly to give more explicit attention to the higher, outcome-level objective aimed at in each case. That objective will normally consist of improved decisions, and consequently better development, in the field concerned. In the case of NSDSs prepared in accordance with Paris 21 guidelines, these issues have benefited significantly from the emphasis on early and extensive user-producer dialogue; there is consequently greater likelihood of expanding production of the many different types of official statistics in line with needs and abilities to use them. 7.11 Main initial emphasis in monitoring a well-prepared NSDS is therefore probably best concentrated on the often difficult process of simply ensuring timely availability of the inputs required in the numerous sectors to be covered, and the progress of outputs. This will be valuably supplemented by introduction and regular updating of the improved Country Statistical Capacity Indicator that the Bank is currently developing in connection with the new Statistics for Results Facility. More searching review, of how well the official statistical services are responding to a country’s priority needs, and of 4 ADE s.a. 2007. “Evaluation of the Commission Support for Statistics in Third Countries. Brussels. 69 the instances where the statisticians did have or could have had decisive effect on development decisions, may be worthwhile in many cases about Year 3 or 4 of NSDS implementation. The Bank may be able to benefit from participation in the work of a Reflection Group on Evaluation of Statistical Capacity Building that Paris 21 is considering establishing. 7.12 Last among the specific questions raised by the Terms of Reference was whether the DGF initiatives have been with the right agencies and at the right level within them. The evidence from our review indicates a strongly affirmative answer. The partner agencies all have a lead role in setting statistical standards and frameworks which are important for the development of official statistics in the developing countries (beyond the more purely economic statistics that have traditionally been handled mainly by the IMF and the World Bank). In their areas of responsibility, these partner agencies play a unique role, which is important in itself for the world’s management of economic and social issues and can also have an amplified value through its effect on practices at the country level. Possible Future DGF Support for Programs Started 7.13 When MAPS began, broad indication was given that, provided the individual programs performed up to expectation, funding could be expected to be renewed annually for five years, i.e., through FY 2010. No commitment of any sort was made for later years. Most of the very large number of non-statistical activities assisted historically by DGF appear to have received support for a few years, until their task was completed or they had attracted sufficient other supporters or become self-supporting. A few however have continued to receive DGF funding for many years, recognizing that they were providing an important public good, closely enough related to its mandate that the World Bank needed to remain involved in providing some of the required public support. 7.14 We try here to give a perspective for the next five years, to FY 2013, starting from the commitments already made for FY 2009 and reflecting what we have learned in the course of the evaluation. 7.15 The program which presents the greatest difficulties in deciding appropriate structure and funding over the coming years is the highly successful, innovative International Household Survey Network and its accompanying Accelerated Data Program. There are many options, and important management decisions will need to be taken over the next 12 or 18 months as to which course to take. An interesting plan, developed in response to our request by those directly in charge of this work, proposes a major effort to respond to the larger than expected scale of demand that the initiative seems to have generated, in a very wide variety of countries, ranging from the poorest to some of the richest. 7.16 The techniques so successfully applied to the archiving and design of household surveys would be systematically developed also for the many other types of survey and administrative data collections, developing appropriate guidelines, software, and specialized question banks. It is envisaged that IHSN and ADP activities would continue alongside one another under Paris 21 supervision, but DGF funding would be concentrated (for five years starting in 2011) entirely on IHSN, introducing and 70 developing the new techniques and standards. ADP would have a larger scale of program, assisting countries in the application of the new approaches, especially for the design of new surveys and data systems (and particularly to meet PRSP- and MDG- based needs better than in the past), but it would be financed out of countries’ own budgets or donor contributions to the financing of their NSDS. 7.17 As proposed, the combined load of IHSN/ADP on the DGF would fall from somewhat above $3 million in 2009 to about $2 million in 2011 and following years. But it may be optimistic to assume that ADP could transition so rapidly from being very largely core-funded to total dependence on country-based contracts. Another variant of the plan, though it is unclear how much it would reduce the financial contribution required from DGF, would be to transfer the network secretariat, survey cataloguing, software clearance, and perhaps question-bank maintenance functions – under the umbrella of IHSN – to a mainline international official body such as the UNSD. 7.18 All responsibility for development of new tools, guidelines or question banks would be placed with ADP, still housed within Paris 21 where, amongst other advantages, it is in direct touch with the best central source of information about countries’ NSDSs and their implementation. Under such an arrangement ADP would probably require a minimum of $800,000 p.a. for development of new guidelines, software and question banks, plus perhaps $500,000 core funding (but falling quite rapidly to lower levels) to help with its generation and management of country assistance projects. 7.19 While much further work has to be done to refine planning for these important programs and there are good prospects of finding other mechanisms within a few years for financing substantial parts of the activity, it seems at this point unwise to count on any reduction of desirable DGF contribution from current levels before 2013. But the desirability of transferring more permanent functions of IHSN to UNSD earlier than that should be studied, both because UNSD’s role in clearance and certification of products will anyway be increasingly important given the volume of useable products emerging and in prospect, and because UNSD will no doubt be able to find more permanent ways of funding costs that are no longer in the developmental category suitable for DGF. 7.20 Phase-out can, however, be foreseen for some other elements of the MAPS program. DGF support for the UN Habitat program was never scheduled to extend beyond FY2008, and good progress seems to have been made by Habitat in finding other, bilateral and private-foundation sources of funding. DGF support for the UNECE program regarding gender-related statistics, always envisaged as a short-run effort to foster indigenous reform groups in the ex-Soviet countries, is due to be completed in FY 2009. Together these changes would save nearly $500,000 of annual DGF commitments. 7.21 The DGF support which has enabled the UNSD to provide extensive training services and technical assistance to poor countries in support of the current Census Round clearly has to be maintained through the upcoming peak years of census execution. It would be highly desirable that it extend at about the same level of $1 million per year into the immediately following years of compilation and analysis of census results, dissemination of data and documentation, including of the cartographic aspects which should help reduce costs of other surveys in coming years. Lessons of 71 the 2010 Census Round should also be drawn, in preparation for the following decade. But with adequate advance planning, it should be possible to complete all these tasks before the end of FY 2012. 7.22 It is hard, on the other hand, to see much possibility of reducing DGF funding for MAPS’ two other major programs within the coming years: UIS, receiving in FY2009 a little less than $2 million, and the Paris 21 core program about $1 million. Both programs are producing mainly goods and services for the poorer countries (for which those countries would otherwise have to pay), but with a strong public-good dimension; the services are of considerable importance for the Bank’s work; and prospects of obtaining additional public financing to replace that which has been provided by DGF are not particularly good. 7.23 UIS has much work yet to do to build its country coverage of education finance – a job which took some ten years for OECD in the much easier context of its member countries – and to generate more data that are suitable for use in the crucial learning outcomes research. Paris 21 has to sustain a major effort over the next five years to bring the strategic planning initiative to successful fruition in implementation in a rapidly increasing number of countries, the large majority with quite weak administrations. 7.24 The trends described and prospects foreseen indicate that, even with a constant MAPS-DGF budget in real terms, a significant margin gradually increasing to $1.5 million in 2013 could be available for allocation once again to these or other sectors (The previously mentioned DGF contribution to the Health Metrics Network is understood to be financed out of the putative Health Sector share of DGF funds). Some of the relevant candidates that would probably be widely supported are renewals of past programs – such as launching an effort on gender issues that would meet the needs of statistical services in the countries of the Middle East and North Africa region, or further efforts to fill the large backlog in statistical concepts and institutional arrangements for the urban areas which are expanding so rapidly in developing countries. 7.25 Another approach, very consistent with the MAPS philosophy, could be to identify MDG indicators – and the sectoral statistical services that collect the underlying data and therefore heavily affect the quality of the estimates for the indicator – which most need effort. It would be a task of further clarification, mobilization of partners, and improved data collection effort. An important candidate, suggested by a leading member of the Inter-Agency and Expert Group on the MDGs, is the hunger indicator, for which FAO is responsible and that is part of Goal 1. An initiative in this area, focusing in significant part on the relatively problematic area of agricultural statistics, could be considered currently timely even from broader points of view. 7.26 We do believe that the scope for good use of DGF resources in helping to strengthen statistical capacities to achieve development results fully warrants extension of the past level of support for MAPS beyond 2010. Through this support, the Bank is helping to maintain a substantial level of activity by the partner agencies for developing countries, and the combined Bank-partner contributions in turn underpin much larger levels of financing for statistical development by the wider MAPS Partnership, some parts of it through other collaborative programs with the Bank. In this situation, it is 72 hard to make a convincing case for shifting more of the MAPS financing from DGF to other donors interested in statistics. 7.27 While we have been given to understand that, on present prospects, the DGF would be in a position at least to consider continuation of support as proposed, we also recognize increasing uncertainties about the scale of activity that DGF may be able to undertake. All recipients of the DGF funds do need to be aware of these risks. And those programs which would be most severely disrupted by any sudden reduction in DGF support would be well advised to redouble efforts to locate potential alternative sources: this would seem to us to apply in particular to ADP, which probably needs to increase relations with potential bilateral funding sources, and UIS, whose notable accomplishments of the last few years should enable it to attract levels of co-financing more comparable to some of UNESCO’s best established affiliates. 73 Statistical Annex Table 3.1: MAPS activities in IDA countries in increasing order of CSIDB 2007 Assessment Table 3.2: Main stages in the introduction of strategic planning for statistics (NSDS) showing Progress achieved by IDA member countries of Sub-Saharan Africa 2004-08 Table 3.3: CSIDB definitions and scoring Table 3.4: CSIDB scores for IDA countries, 2004-07 Table 3.5: CSIDB score for potential indicators of capacity - 2004 to 2006 Table 3.6: Availability and source of selected MDG indicators at September 2008 for Ethiopia, Malawi, Mali and Niger Table 3.7: Changes in availability and source of selected MDG indicators in SSA IDA countries, around 2002 and around 2005 Table 3.8: Water, sanitation and slum MDG indicator data availability Table 3.9: International reporting of basic CPI data for IDA SSA countries Table 3.10: International Reporting of Basic External Trade data for IDA SSA Countries Table 3.1: MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment 2005 Census dates and UN CSIDB CSIDB No of UIS TA & WBI/ECE NSDS ADP Gross plans Habitat score change surveys in other Gender Status tasks PER 2000 TA 2007 2004-07 IHSN cat assistance 2010 Round partners avail’y Round rec’d Africa Liberia 17 ▬ D 1&3 9 RW … - 2008 Somalia 21 ▲ N 9 … - (…) √ Guinea-Bissau 29 ◊ P 10 … - (2008) CAR 33 ▼ D 11 TA/cWB/scb ** 2003 (2013) Eritrea 34 ▼ P 6 RW √ - (2009) √ Sudan 34 ◊ P 13 √ - (2008) √ Angola 37 ▬ I&P 8 TA/scb … - (2010?14) Congo, DRC 39 ▬ P 3 9 -2** - (2010) Burundi 41 ▬ D 8 √ - (2008) Sao Tome & Principe 44 ▬ D 9 √ 2001 (2011) Sierra Leone 46 ▲ D 8 TA/scb … 2004 (2014) Chad 52 ◊ I 18 √ - (2008) Ghana 53 ◊ I&D 1 25 TA/EU +1 2000 (2010) √ Gambia 54 ▼ I 1 13 RW -1 2003 (2013) Comoros 56 ▲ D 11 ** 2003 (2013) CSIDB change 2004 to 2007 NSDS status UIS symbols UIS data Census symbols no discernable change ▬ I = implementing sv = site visit √ data for 2005 date = census taken improvement in score ▲ P = planning TA = tech ass’ce … no data (date) census expected that year variable scores ◊ D = designing cWB = coopn with WB * nat’l estimate (…) census expected this Round decline in score ▼ N = no plan RW - regional workshop ** UIS estimate - no census taken or planned X = no information scb = formal project +1 refers to 2006 WEI country -1 refers to 2004 For full explanation see end of table EU = FTI project -2 refers to 2003 75 Table 3.1 (ctd): MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment 2005 Census dates and UN CSIDB CSIDB No UIS TA & WBI/ECE NSDS ADP Gross plans Habitat score change surveys in other Gender Status tasks PER 2000 TA 2007 2004-07 IHSN cat assistance 2010 Round partners avail’y Round rec’d Congo Brazzaville 56 ▲ I 2 √ 1996 2007 Guinea 56 ▬ D plans 20 TA/cWB/EU √ 1996 (...) Kenya 57 ◊ I 1 40 sv √ 1999 (2009) √ Zimbabwe 57 ◊ D 20 WEI -2 2002 (2012) Cape Verde 58 ▲ I 12 √ 2000 (2010) 1,3, & Niger 58 ◊ I 31 sv/TA/scb √ 2001 (2011) pilot 2 Togo 58 ▲ D 11 cWB √ - (2009) Mauritania 60 ◊ I&D 22 EU √ 2000 (2010) 1&2 Mali 61 ▲ I 31 sv/TA/scb √ 1998 (2009) pilot Zambia 61 ▼ P 27 √ 2000 (2010) √ Nigeria 62 ▲ I 1 32 EU √ - 2006 √ Burkina Faso 63 ▼ I 19 sv/TA/cWB/scb √ 1996 2006 Tanzania 63 ◊ D plans 29 sv/scb/EU +1 2002 (2012) √ Benin 64 ▲ D 15 cWB √ 2002 (2012) CSIDB change 2004 to 2007 NSDS status UIS symbols UIS data Census symbols no discernable change ▬ I = implementing sv = site visit √ data for 2005 date = census taken improvement in score ▲ P = planning TA = tech ass’ce … no data (date) census expected that year variable scores ◊ D = designing cWB = coopn with WB * nat’l estimate (…) census expected this Round decline in score ▼ N = no plan RW - regional workshop ** UIS estimate - no census taken or planned X = no information scb = formal project +1 refers to 2006 WEI country -1 refers to 2004 For full explanation see end of table EU = FTI project -2 refers to 2003 76 Table 3.1 (ctd): MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment 2005 Census dates and UN CSIDB CSIDB No UIS TA & WBI/ECE NSDS ADP Gross plans Habitat score change surveys in other Gender Status tasks PER 2000 TA 2007 2004-07 IHSN cat assistance 2010 Round partners avail’y Round rec’d Madagascar 64 ◊ I&D 26 RW √ - (2009) Uganda 65 ▲ I 2 pilot 32 EU √ 2002 (2012) √ 1&2 Cameroon 67 ▲ I&D 17 * - 2005 pilot Malawi 67 ◊ I&D 25 √ 1998 (2008) Ethiopia 68 ◊ I&D 2 pilot 37 sv/TA/RW/EU +1 - 2007 √ Cote d'Ivoire 69 ▼ D 27 -2** 1998 (2008) 1996/ Lesotho 69 ◊ I&D plans 17 sv/TA √ 2006 √ 2001 Mozambique 69 ▲ I&D 1&3 21 sv/TA/RW √ 1997 2007 Rwanda 71 ▲ I 22 RW ** 2002 (2012) √ Senegal 74 ▬ I 1 22 √ 2002 (2011) √ East Asia 1995/ Kiribati 32 ▬ X 7 * 2005/(2010) 2000 Solomon Islands 32 ▬ X 0 √ 1999 (2009) Timor-Leste 36 ◊ N 6 √ 2004 (2010) CSIDB change 2004 to 2007 NSDS status UIS symbols UIS data Census symbols no discernable change ▬ I = implementing sv = site visit √ data for 2005 date = census taken improvement in score ▲ P = planning TA = tech ass’ce … no data (date) census expected that year variable scores ◊ D = designing cWB = coopn with WB * nat’l estimate (…) census expected this Round decline in score ▼ N = no plan RW - regional workshop ** UIS estimate - no census taken or planned X = no information scb = formal project +1 refers to 2006 WEI country -1 refers to 2004 For full explanation see end of table EU = FTI project -2 refers to 2003 77 Table 3.1 (ctd): MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment 2005 Census dates and UN CSIDB CSIDB No UIS TA & WBI/ECE NSDS ADP Gross plans Habitat score change surveys in other Gender Status tasks PER 2000 TA 2007 2004-07 IHSN cat assistance 2010 Round partners avail’y Round rec’d Vanuatu 46 ◊ I 9 ** 1999 (2009) Papua New Guinea 47 ◊ P 7 -2** 2000 (2010) Samoa 51 ◊ X 17 ** 2001 2006/(2011) Tonga 52 ▲ I 7 ** 1996 2006/(2011) Myanmar 54 ▬ X 10 √ - (…) Laos PDR 64 ▬ I 1 16 √ 1995 2005 Vietnam 72 ▬ I 1 34 EU √ 1999 (2009) Cambodia 73 ▲ I 13 √ 1998 (2008) Mongolia 81 ▲ I 1 24 √ 2000 (2010) Indonesia 88 ◊ I 1 86 WEI ** 2000 (2010) Europe and Central Asia Bosnia-Herzegovina 54 ▲ I 15 … - (2011) √ Uzbekistan 64 ◊ P 12 -1 - (…) √ Tajikistan 69 ◊ I 18 √ 2000 (2010) √ Azerbaijan 74 ◊ I&P 17 √ 1999 (2009) Georgia 83 ▲ P 29 √ 2002 (2010) CSIDB change 2004 to 2007 NSDS status UIS symbols UIS data Census symbols no discernable change ▬ I = implementing sv = site visit √ data for 2005 date = census taken improvement in score ▲ P = planning TA = tech ass’ce … no data (date) census expected that year variable scores ◊ D = designing cWB = coopn with WB * nat’l estimate (…) census expected this Round decline in score ▼ N = no plan RW - regional workshop ** UIS estimate - no census taken or planned X = no information scb = formal project +1 refers to 2006 WEI country -1 refers to 2004 For full explanation see end of table EU = FTI project -2 refers to 2003 78 Table 3.1 (ctd): MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment 2005 Census dates and UN CSIDB CSIDB No UIS TA & WBI/ECE NSDS ADP Gross plans Habitat score change surveys in other Gender Status tasks PER 2000 TA 2007 2004-07 IHSN cat assistance 2010 Round partners avail’y Round rec’d Albania 86 ▲ I 20 -1 2001 (2011) Moldova 86 ▲ D 36 * 2004 (2012) √ Kyrgyz Republic 90 ◊ I 23 √ 1999 (2009) √ Armenia 93 ▲ X 16 √ 2001 (2011) Latin America and Caribbean Haiti 35 ◊ D 3 9 … 2003 (2013) √ Grenada 48 ◊ P 0 ** 2001 (2011) Dominica 50 ◊ P 1 plan 9 * 2001 (2010) Guyana 54 ◊ D 1 plan 15 √ 2002 (2012) St Lucia 56 ▬ P 1 plan 11 √ 2001 (2010) St Vincent 61 ▲ P 7 √ 2001 (2010) Honduras 65 ◊ I&P 1 39 EU ** 2001 (2012) Bolivia 66 ◊ I&P 1 25 -1** 2001 (…) Nicaragua 75 ▼ I&P 1 plan 11 √ 2995 2005 CSIDB change 2004 to 2007 NSDS status UIS symbols UIS data Census symbols no discernable change ▬ I = implementing sv = site visit √ data for 2005 date = census taken improvement in score ▲ P = planning TA = tech ass’ce … no data (date) census expected that year variable scores ◊ D = designing cWB = coopn with WB * nat’l estimate (…) census expected this Round decline in score ▼ N = no plan RW - regional workshop ** UIS estimate - no census taken or planned X = no information scb = formal project +1 refers to 2006 WEI country -1 refers to 2004 For full explanation see end of table EU = FTI project -2 refers to 2003 79 Table 3.1 (ctd): MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment 2005 Census dates and UN CSIDB CSIDB No UIS TA & WBI/ECE NSDS ADP Gross plans Habitat score change surveys in other Gender Status tasks PER 2000 TA 2007 2004-07 IHSN cat assistance 2010 Round partners avail’y Round rec’d Middle East and North Africa Djibouti 37 ▼ I 10 √ - (…) Yemen, Republic of 56 ◊ I 1 18 √ 2004 (2014) √ South Asia Afghanistan 38 ▲ I 5 √ - (2008) Bhutan 41 ▲ I&D 3 … - 2005 Maldives 66 ▬ P 10 √ 95/2000 2006/(2011) Sri Lanka 73 ◊ D 1 24 WEI … 2001 (2011) Nepal 78 ▲ I 25 +1 2001 (2011) Bangladesh 80 ▲ P 1 35 EU -1 2001 (2011) India 83 ◊ D plans 71 WEI ** 2001 (2011) √ Pakistan 86 ▲ D 35 EU √ 1998 (2008) CSIDB change 2004 to 2007 NSDS status UIS symbols UIS data Census symbols no discernable change ▬ I = implementing sv = site visit √ data for 2005 date = census taken improvement in score ▲ P = planning TA = tech ass’ce … no data (date) census expected that year variable scores ◊ D = designing cWB = coopn with WB * nat’l estimate (…) census expected this Round decline in score ▼ N = no plan RW - regional workshop ** UIS estimate - no census taken or planned X = no information scb = formal project +1 refers to 2006 WEI country -1 refers to 2004 For full explanation see end of table EU = FTI project -2 refers to 2003 80 Table 3.1 (ctd): MAPS activities in IDA countries in increasing order of CSIDB 2007 assessment Symbols and sources used in table CSIDB: World Bank Country Statistical Information Database. For detailed scores see table 3.4. Assessment of direction of change between 2004 and 2007 by authors: ▬ = no discernable change; ▲ = improvement in score; ◊ = variable scores; ▼ = decline in score. NSDS Status: Paris21. For more detail on sub-saharan Africa and definition of symbols used see table 3.2. P = Planning; D = Designing; I = implementing; N = No plan; X = no information ADP: Paris21. Tasks undertaken under the Accelerated Data Programme. IHSN: IHSN website (International Household Survey Network) count of surveys listed May 2008. (www.internationalsurveynetwork.org ) UIS: source UIS. Type of assistance given: sv = site visits; TA = technical assistance; cWB = cooperation through World Bank programme; RW = regional workshop; scb = formal statistical capacity building project; EU = education fast track initiative, UIS provide TA under EU programme; WEI = world education indicators country. UIS: quality of primary enrolment data for 2005: √ = on time; +1 = 2006 available; -1 = 2004 available; -2 = 2003 available; … = no data; * = nationally-made estimate; ** = estimate made by UIS Census: source UN Statistics Division for 2010 Census Round (2005 - 2014) “date” = census taken; (date) = census expected that year; (…) = census expected this round; - = no census taken or planned WBI/ECE: countries with a programme on gender statistics with the World Bank Insitute or the UN Economic Commission for Europe. UN Habitat: source Habitat; countries who have received assistance from Habitat for urban and slum indicators. May - August 2008 Table 3.2: Main stages in the introduction of strategic planning for statistics (NSDS) Progress achieved by IDA member countries of sub-Saharan Africa 2004-08 Country Situation in 2004 Situation in 2008 N P D F I Name of Plan E N P D F I Name of Plan Somalia + + Congo Dem + + (NSDS) Republic Eritrea + + (NSDS) Guinea-Bissau + + (NSDS) Sudan + + (NSDS) Central African + + (NSDS) Republic CSO Strategic Plan Zambia + + (NSDS) 2003-07 Sao Tome & + (not funded) + (NSDS) Principe Programme d'Activites Chad + + + (NSDS) Statistiques 2002-07 Plano Estatistico Angola + Nacional de Medio + + (NSDS) Prazo 2002-06 Schema directeur de la Burkina Faso + + + NSDS 2009-12 statistique 2004 - 2009 Medium term statistical Ghana + + + (NSDS) programme 2001-2005 Plan d'actions prioritaire Burundi + pour le dev't des + (NSDS) statistiques 2004-07 Guinea + Statistical Master Plan + NSDS 2008-13 Sierra Leone + NSDS + NSDS 2008-12 Schema directeur de la Cote d'Ivoire + + NSDS 2008-10 statistique 2001-2005 Programme Trienniale Cameroon + + (NSDS) 2003-05 Medium term statistical Ethiopia + programme 2003/4 - + + (NSDS) 07/8 Lesotho + CSO Plan 2002-2005 + + (NSDS) Key: N = No plan P = Process started D = Design stage F = Final stages I = under Implementation E = Existing plan 82 Table 3.2 (ctd): Main stages in the introduction of strategic planning for statistics (NSDS) Progress achieved by IDA member countries of sub-Saharan Africa 2004-08 Country Situation in 2004 Situation in 2008 N P D F I Name of Plan E N P D F I Name of Plan Senegal + + NSDS 2008-13 Togo + + NSDS 2007-12 Liberia + + (NSDS) Tanzania + + NSDS 2008-13 Comoros + + NSDS 2008-12 Schema directeur de la Benin + + NSDS 2007-12 statistique 2005-06 Statistical Master Plan Gambia + + SMP 2007-11 2007-11 NSO Strategic Plan 2002- Malawi + + + NSDS 2007-11 06 Mozambique + Strategic Plan 2003-07 + + NSDS 2008-12 Madagascar + Plan directeur 2003-07 + + NSDS 2007-10 Schema directeur de la Mauritania + + + NSDS 2007-12 statistique 2000-06 Niger + + NSDS 2008-12 Schema directeur de la Mali + + SDS 2006-10 statistique 2001-05 Rwanda + NSDS 2001-06 + NSDS 2007-11 Uganda + UBOS Corporate Plan + NSDS 06/7-10/11 Key: N = No plan P = Process started D = Design stage F = Final stages I = under Implementation E = Existing plan 83 Table 3.2 (ctd): Main stages in the introduction of strategic planning for statistics (NSDS) Progress achieved by IDA member countries of sub-Saharan Africa 2004-08 Country Situation in 2004 Situation in 2008 N P D F I Name of Plan E N P D F I Name of Plan Djibouti + NSDS 2006-10 + NSDS 2006-10 Programme Pluriannuel Programme Congo Brazzaville + + 2005-09 Pluriannuel Statistical Agenda for Statistical Cabo Verde + + Development 2006-10 Agenda Nigeria + NSS Masterplan 2005-09 + NSS Masterplan Kenya + NSDS 2004-08 + NSDS 2004-08 Totals 12 5 3 6 13 15 1 11 7 11 4 Key N = No strategic plan P = Process Started - Decision made to prepare a strategic plan, and resources are being assembled to carry out the planning work D = Design - NSDS preparation under way. F = Final stages - being made ready for approval, or waiting for eg legislation or funding to be agreed I = Being implemented - NSDS under active implementation. E = Existing plan being implemented Source: Authors’ assessments, on basis of evidence available in the Paris 21 secretariat and the Development Data Group of the World Bank, plus studies of statistical office websites and available strategic documents 84 Table 3.3 CSIDB definitions and scoring Statistical Practice Score National accounts base year Annual chain linking is adopted or the base year is within the last 10 years 10 Otherwise 0 Balance of payments manual in use Countries adopting the latest edition (BPM5) 10 Otherwise 0 External debt reporting status Data were used as reported (actual) or data were preliminary and included an element of staff estimation (preliminary) 10 Data are staff estimates (estimate) 0 Consumer price index base year Base year is within the last 10 years 10 Otherwise 0 Industrial production index Index is available monthly 10 Otherwise 0 Import and export price indexes Index is available monthly 10 Otherwise 0 Government finance accounting Consolidated accounts 10 Otherwise 0 UNESCO reporting Country reported at least 3 times in the last 4 years 10 Otherwise 0 National immunization coverage Government official estimate on measles vaccine coverage is consistent with the WHO/UNICEF estimate 10 Otherwise 0 Special Data Dissemination Standard Subscribing countries 10 Otherwise 0 Data Collection Population census Country had a population census at least once in the last 10 years 20 Otherwise 0 Agricultural census Country had an agricultural census at least once in the last 10 years 20 Otherwise 0 Poverty survey (eg LSMS, HBS, HIES) Survey conducted at frequency of 3 years or less 20 Survey conducted at frequency of 5 years or less 10 Survey conducted at frequency of over 5 years 0 Health survey (eg DHS, CWIQ, MICS, LSMS, PS) Survey conducted at frequency of 3 years or less 20 Survey conducted at frequency of 5 years or less 10 Survey conducted at frequency of over 5 years 0 85 Table 3.3 (ctd): CSIDB definitions and scoring Vital registration system coverage Country is judged to have complete registries of vital statistics by UNSD 20 Otherwise 0 Indicator Availability Income poverty (Proportion of population below US$1 a day at 1993 ppp) Periodicity of the indicator is less than 3 years 10 Periodicity of the indicator is less than 5 years 6.7 Periodicity of the indicator is more than 5 years 3.3 Otherwise 0 Child malnutrition (Prevalence of underweight children under-five years of age) Periodicity of the indicator is less than 3 years 10 Periodicity of the indicator is less than 5 years 6.7 Periodicity of the indicator is more than 5 years 3.3 Otherwise 0 Child mortality (Under-five mortality rate) National or international estimate is available for reference years 10 Otherwise 0 Immunization (Proportion of one-year-old children immunized against measles) Periodicity of the indicator is annual 10 Otherwise 0 HIV/AIDS (Incidence of HIV among people aged 15-49) National or international estimate is available 10 Otherwise 0 Maternal health (Births attended by skilled health staff) Periodicity of the indicator is less than 3 years 10 Periodicity of the indicator is less than 5 years 6.7 Periodicity of the indicator is more than 5 years 3.3 Otherwise 0 Gender equality (Ratio of the gross enrollment rate of girls to boys in primary and secondary education levels in both public and private schools) Indicator is observed for 5 out of the 5 latest years 10 Indicator is observed for at least 3 out of the 5 latest years 6.7 Indicator is observed for 1 out of the 5 latest years 3.3 Otherwise 0 Primary completion rate (or survival to last year of primary) Periodicity of the indicator is less than 3 years 10 Periodicity of the indicator is less than 5 years 6.7 Periodicity of the indicator is more than 5 years 3.3 Otherwise 0 Access to water Primary estimates are observed for at least 2 out of the 6 latest years 10 Primary estimates are observed for 1 out of the 6 latest years 2 Otherwise 0 Per capita GDP growth Periodicity of the indicator is annual 10 Periodicity of the indicator is less than 1.5 years 6.7 Periodicity of the indicator is more than 1.5 years 3.3 Otherwise 0 Source: World Bank CSIDB database http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK: 20541648~menuPK:1164885~pagePK:64133150~piPK:64133175~theSitePK:239419,0 0.html 86 Table 3.4: CSIDB scores for IDA countries, 2004-07 CSIDB score 2004 2005 2006 2007 Direction of change Africa Liberia 17 17 18 17 no change ▬ Somalia 13 17 17 21 improvement ▲ Guinea-Bissau 28 30 37 29 variable ◊ CAR 40 38 32 33 decline ▼ Eritrea 38 38 38 34 sudden drop ▼ Sudan 35 25 30 34 variable ◊ Angola 33 37 35 37 no change ▬ Congo, DRC 38 38 37 39 no change ▬ Burundi 42 40 40 41 no change ▬ Sao Tome & Principe - 42 42 44 no change ▬ Sierra Leone 27 37 40 46 improvement ▲ Chad 57 55 58 52 variable ◊ Ghana 55 57 52 53 variable ◊ Gambia 60 53 53 54 sudden drop ▼ Comoros - 50 57 56 step improvement ▲ Congo Brazzaville 40 40 43 56 improvement ▲ Guinea 57 55 55 56 no change ▬ Kenya 65 53 55 57 variable ◊ Zimbabwe 68 68 53 57 variable ◊ Cape Verde - 53 52 58 step improvement ▲ Niger 58 60 63 58 variable ◊ Togo 48 48 52 58 improvement ▲ Mauritania 55 55 62 60 variable ◊ Mali 53 55 57 61 improvement ▲ Zambia 70 65 58 61 decline ▼ Nigeria 40 52 52 62 improvement ▲ Burkina Faso 67 67 65 63 decline ▼ Tanzania 65 65 62 63 variable ◊ Benin 53 57 57 64 improvement ▲ Madagascar 62 53 57 64 variable ◊ Uganda 60 67 67 65 step improvement ▲ Cameroon 48 48 65 67 step improvement ▲ Malawi 67 60 63 67 variable ◊ Ethiopia 63 63 60 68 variable ◊ Cote d'Ivoire 75 75 70 69 decline ▼ Lesotho 70 67 62 69 variable ◊ Mozambique 63 68 68 69 step improvement ▲ Rwanda 53 53 53 71 step improvement ▲ Senegal 75 75 75 74 no change ▬ 87 Table 3.4 (ctd): CSIDB scores for IDA countries, 2004-07 CSIDB score 2004 2005 2006 2007 Direction of change East Asia Kiribati - 30 30 32 no change ▬ Solomon Islands - 32 30 32 no change ▬ Timor-Leste - 32 30 36 variable ◊ Vanuatu - 42 50 46 variable ◊ Papua New Guinea 48 52 50 47 variable ◊ Samoa - 48 45 51 variable ◊ Tonga - 43 43 52 step improvement ▲ Myanmar 53 55 53 54 no change ▬ Laos PDR 63 63 62 64 no change ▬ Vietnam 75 75 75 72 no change ▬ Cambodia 58 63 65 73 improvement ▲ Mongolia 70 80 80 81 step improvement ▲ Indonesia 88 85 85 88 variable ◊ Europe and Central Asia Bosnia-Herzegovina 37 38 48 54 improvement ▲ Uzbekistan 63 62 58 64 variable ◊ Tajikistan 63 72 68 69 variable ◊ Azerbaijan 75 77 77 74 variable ◊ Georgia 72 73 73 83 step improvement ▲ Albania 80 80 83 86 improvement ▲ Moldova 67 77 80 86 improvement ▲ Kyrgyz Republic 92 93 93 90 variable ◊ Armenia 82 88 90 93 improvement ▲ Latin America and Caribbean Haiti 37 37 32 35 variable ◊ Grenada - 50 47 48 variable ◊ Dominica - 52 45 50 variable ◊ Guyana - 58 50 54 variable ◊ St Lucia - 57 57 56 no change ▬ St Vincent - 58 60 61 improvement ▲ Honduras 62 55 58 65 variable ◊ Bolivia 68 70 70 66 variable ◊ Nicaragua 82 78 75 75 decline ▼ 88 Table 3.4 (ctd): CSIDB scores for IDA countries, 2004-07 CSIDB score Direction of 2004 2005 2006 2007 change Middle East and North Africa Djibouti - 45 38 37 decline ▼ Yemen, Republic of 57 55 58 56 variable ◊ South Asia Afghanistan 15 25 28 38 improvement ▲ Bhutan - 38 38 41 improvement ▲ Maldives - 67 67 66 no change ▬ Sri Lanka 78 72 72 73 variable ◊ Nepal 65 73 77 78 improvement ▲ Bangladesh 73 78 80 80 improvement ▲ India 82 75 77 83 variable ◊ Pakistan 73 80 80 86 improvement ▲ Average 64 65 64 66 no change ▬ Source: World Bank CSIDB database and authors’ assessment, August 2008 http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK: 20541648~menuPK:1164885~pagePK:64133150~piPK:64133175~theSitePK:239419,0 0.html 89 Table 3.5: CSIDB score for potential indicators of capacity - 2004 to 2006 Production index Import / Export prices Reporting to UNESCO 2004 2005 2006 2004 2005 2006 2004 2005 2006 Angola 0 0 0 0 0 0 0 0 0 Benin 0 0 0 0 0 0 1 1 1 Burkina Faso 0 0 0 0 0 0 1 1 1 Burundi 0 0 0 0 0 0 1 1 1 Cameroon 0 0 1 0 0 0 1 1 1 Central African Republic 0 0 0 0 0 0 0 0 0 Chad 0 0 0 0 0 0 1 1 1 Congo, Dem. Rep. 0 0 0 0 0 0 0 0 0 Congo, Rep. 0 0 0 0 0 0 1 1 1 Côte d'Ivoire 1 1 1 0 0 0 1 1 0 Eritrea 0 0 0 0 0 0 1 1 1 Ethiopia 0 0 0 0 0 0 1 1 1 Gambia, The 0 0 0 0 0 0 1 0 1 Ghana 0 0 0 0 0 0 1 1 1 Guinea 0 0 0 0 0 0 1 1 1 Guinea-Bissau 0 0 0 0 0 0 0 0 0 Kenya 1 0 0 1 0 0 1 1 1 Lesotho 0 0 0 0 0 0 1 1 1 Liberia 0 0 0 0 0 0 1 0 0 Madagascar 0 0 0 0 0 0 1 1 1 Malawi 1 1 1 0 0 0 1 1 1 Mali 0 0 0 0 0 0 1 1 1 Mauritania 0 0 0 0 0 0 1 1 1 Mozambique 0 0 0 0 0 0 1 1 1 Niger 0 0 0 0 0 0 1 1 1 Nigeria 1 1 1 0 0 0 0 1 1 Rwanda 0 0 0 0 0 0 1 1 1 Senegal 1 1 1 0 0 0 1 1 1 Sierra Leone 0 0 0 0 0 0 0 0 0 Somalia 0 0 0 0 0 0 0 0 0 Sudan 0 0 0 0 0 0 1 1 1 Tanzania 1 1 1 0 0 0 1 1 1 Togo 0 0 0 0 0 0 1 1 1 Uganda 0 0 0 0 0 0 1 1 1 Zambia 0 0 0 0 0 0 1 1 1 Source: World Bank CSIDB database, May 2008 http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK: 20541648~menuPK:1164885~pagePK:64133150~piPK:64133175~theSitePK:239419,0 0.html 90 Table 3.6a: Ethiopia - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Population below national poverty line, total, % Population below national poverty line, urban, % Population below national poverty line, rural, % Children under 5 underweight, percentage Children under 5 severely underweight, percentage Total net enrolment ratio in primary, both sexes Total net enrolment ratio in primary education, boys Total net enrolment ratio in primary education, girls % of pupils reaching last grade of primary, m&f % of pupils who reach last grade of primary, boys % of pupils who reach last grade of primary, girls Primary completion rate, both sexes Primary completion rate, boys Primary completion rate, girls Literacy rates of 15-24 years old, both sexes, % Literacy rates of 15-24 years old, men, % Literacy rates of 15-24 years old, women, % Ratio of literacy rates, 15-24 years old Gender Parity Index in primary level enrolment Gender Parity Index in secondary level enrolment Gender Parity Index in tertiary level enrolment Children under five mortality rate per 1,000 Infant mortality rate (0-1 year) per 1,000 live births Children 1 year old immunized against measles, % Maternal mortality ratio per 100,000 live births Births attended by skilled health personnel, % Contraceptive use among currently married women 15-49 years old, any method, percentage Antenatal care coverage, at least one visit, % HIV prevalence rate, women 15-49 years old HIV prevalence rate, men 15-49 years old AIDS deaths Condom use 15-24 years old, women,% Condom use 15-24 years old, men, % Antiretroviral therapy coverage % Children under 5 sleeping under bed nets, % Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 91 Table 3.6a (ctd): Ethiopia - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Tuberculosis incidence per 100,000 population Tuberculosis prevalence per 100,000 population Tuberculosis death rate per 100,000 population Tuberculosis detection rate under DOTS, % Tuberculosis treatment success rate under DOTS % Proportion of land area covered by forest, % Prop'n using improved drinking water sources, total Prop'n improved drinking water sources, urban Prop'n improved drinking water sources, rural Prop'n improved sanitation facilities, total Prop'n improved sanitation facilities, urban Prop'n improved sanitation facilities, rural Slum population as percentage of urban Slum population in urban areas Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 92 Table 3.6b: Malawi - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Population below national poverty line, total, % Population below national poverty line, urban, % Population below national poverty line, rural, % Children under 5 underweight, percentage Children under 5 severely underweight, percentage Total net enrolment ratio in primary, both sexes Total net enrolment ratio in primary education, boys Total net enrolment ratio in primary education, girls % of pupils reaching last grade of primary, m&f % of pupils who reach last grade of primary, boys % of pupils who reach last grade of primary, girls Primary completion rate, both sexes Primary completion rate, boys Primary completion rate, girls Literacy rates of 15-24 years old, both sexes, % Literacy rates of 15-24 years old, men, % Literacy rates of 15-24 years old, women, % Ratio of literacy rates, 15-24 years old Gender Parity Index in primary level enrolment Gender Parity Index in secondary level enrolment Gender Parity Index in tertiary level enrolment Children under five mortality rate per 1,000 Infant mortality rate (0-1 year) per 1,000 live births Children 1 year old immunized against measles, % Maternal mortality ratio per 100,000 live births Births attended by skilled health personnel, % Contraceptive use among currently married women 15-49 years old, any method, percentage Antenatal care coverage, at least one visit, % HIV prevalence rate, women 15-49 years old HIV prevalence rate, men 15-49 years old Condom use 15-24 years old, women,% Condom use 15-24 years old, men, % Antiretroviral therapy coverage % Children under 5 sleeping under bed nets, % Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 93 Table 3.6b (ctd): Malawi - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Tuberculosis incidence per 100,000 population Tuberculosis prevalence per 100,000 population Tuberculosis death rate per 100,000 population Tuberculosis detection rate under DOTS, % Tuberculosis treatment success rate under DOTS % Proportion of land area covered by forest, % Prop'n using improved drinking water sources, total Prop'n improved drinking water sources, urban Prop'n improved drinking water sources, rural Prop'n improved sanitation facilities, total Prop'n improved sanitation facilities, urban Prop'n improved sanitation facilities, rural Slum population as percentage of urban Slum population in urban areas Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 94 Table 3.6c: Mali - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Population below national poverty line, total, % Population below national poverty line, urban, % Population below national poverty line, rural, % Children under 5 underweight, percentage Children under 5 severely underweight, percentage Total net enrolment ratio in primary, both sexes Total net enrolment ratio in primary education, boys Total net enrolment ratio in primary education, girls % of pupils reaching last grade of primary, m&f % of pupils who reach last grade of primary, boys % of pupils who reach last grade of primary, girls Primary completion rate, both sexes Primary completion rate, boys Primary completion rate, girls Literacy rates of 15-24 years old, both sexes, % Literacy rates of 15-24 years old, men, % Literacy rates of 15-24 years old, women, % Ratio of literacy rates, 15-24 years old Gender Parity Index in primary level enrolment Gender Parity Index in secondary level enrolment Gender Parity Index in tertiary level enrolment Children under five mortality rate per 1,000 Infant mortality rate (0-1 year) per 1,000 live births Children 1 year old immunized against measles, % Maternal mortality ratio per 100,000 live births Births attended by skilled health personnel, % Contraceptive use among currently married women 15-49 years old, any method, percentage Antenatal care coverage, at least one visit, % HIV prevalence rate, women 15-49 years old HIV prevalence rate, men 15-49 years old Condom use 15-24 years old, women,% Condom use 15-24 years old, men, % Antiretroviral therapy coverage % Children under 5 sleeping under bed nets, % Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 95 Table 3.6c (ctd): Mali - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Tuberculosis incidence per 100,000 population Tuberculosis prevalence per 100,000 population Tuberculosis death rate per 100,000 population Tuberculosis detection rate under DOTS, % Tuberculosis treatment success rate under DOTS % Proportion of land area covered by forest, % Prop'n using improved drinking water sources, total Prop'n improved drinking water sources, urban Prop'n improved drinking water sources, rural Prop'n improved sanitation facilities, total Prop'n improved sanitation facilities, urban Prop'n improved sanitation facilities, rural Slum population as percentage of urban Slum population in urban areas Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 96 Table 3.6d: Niger - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Population below national poverty line, total, % Population below national poverty line, urban, % Population below national poverty line, rural, % Children under 5 underweight, percentage Children under 5 severely underweight, percentage Total net enrolment ratio in primary, both sexes Total net enrolment ratio in primary education, boys Total net enrolment ratio in primary education, girls % of pupils reaching last grade of primary, m&f % of pupils who reach last grade of primary, boys % of pupils who reach last grade of primary, girls Primary completion rate, both sexes Primary completion rate, boys Primary completion rate, girls Literacy rates of 15-24 years old, both sexes, % Literacy rates of 15-24 years old, men, % Literacy rates of 15-24 years old, women, % Ratio of literacy rates, 15-24 years old Gender Parity Index in primary level enrolment Gender Parity Index in secondary level enrolment Gender Parity Index in tertiary level enrolment Children under five mortality rate per 1,000 Infant mortality rate (0-1 year) per 1,000 live births Children 1 year old immunized against measles, % Maternal mortality ratio per 100,000 live births Births attended by skilled health personnel, % Contraceptive use among currently married women 15-49 years old, any method, percentage Antenatal care coverage, at least one visit, % HIV prevalence rate, women 15-49 years old HIV prevalence rate, men 15-49 years old Condom use 15-24 years old, women,% Condom use 15-24 years old, men, % Antiretroviral therapy coverage % Children under 5 sleeping under bed nets, % Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 97 Table 3.6d (ctd): Niger - availability and source of selected MDG indicators at September 2008 98 99 00 01 02 03 04 05 06 07 08 Tuberculosis incidence per 100,000 population Tuberculosis prevalence per 100,000 population Tuberculosis death rate per 100,000 population Tuberculosis detection rate under DOTS, % Tuberculosis treatment success rate under DOTS % Proportion of land area covered by forest, % Prop'n using improved drinking water sources, total Prop'n improved drinking water sources, urban Prop'n improved drinking water sources, rural Prop'n improved sanitation facilities, total Prop'n improved sanitation facilities, urban Prop'n improved sanitation facilities, rural Slum population as percentage of urban Slum population in urban areas Country data Adjusted country data Estimated Global monitoring data Modelled Type not known Source: http://mdgs.un.org updated 14 July 2008 98 Table 3.7: Changes in availability and source of selected MDG indicators in SSA IDA countries, around 2002 and around 2005 % below national Gross or Net Primary % of births attended by poverty line Enrolment Rate skilled personnel ~2002 ~2005 ~2002 ~2005 ~2002 ~2005 Angola - - - - A - Benin - - E/C C A - Burkina Faso A - C C A - Burundi - - C C - - Cameroon A - - - - A Cape Verde - - C C - - CAR - - - - - - Chad - - C - - A Comoros - - - - - - Congo DRC - - - - A - Congo Brazzaville - - - C - A Cote d'Ivoire - - C - - A Djibouti - - E C A - Eritrea - - C C A - Ethiopia - - C C - A Gambia - - E - - A Ghana - - C C - - Guinea - - C C A - Guinea Bissau - - E - - - Kenya - - C C A - Lesotho - - C C - A Liberia - - - - - - Madagascar - - C C - A Malawi - - - C - A Mali - - E C - - Mauritania - - E C A - Mozambique - - C C A - Niger - - C C - - Nigeria - - - E A - Rwanda - - E/C C/E - A Sao Tome & P - - E C A - Senegal - - C C A - Sierra Leone - A - - - - Somalia - - - - A - Sudan - - - - - - Tanzania A - C C - A Togo - - C C A - Uganda A - - - A - Zambia - A C C A - C = Country data A = Adjusted country data E = Estimated data - = no data Source: UN mdg database www.mdgs.un.org May 2008 99 Table 3.8: Water, sanitation and slum MDG indicator data availability Countries with at Number of Number of least 2 data MDG indicators by world region Total countries countries points, number of with at least with at least excluding countries 1 data point 2 data points modelled data by agency Indicator 7.8 Proportion of population using an improved drinking water source (Oct 2006) Developed regions 48 32 30 30 Countries of the CIS 12 12 7 7 Developing regions 163 145 110 110 Northern Africa 6 5 4 4 Sub-Saharan Africa 50 48 37 37 Latin America and the Caribbean 46 39 26 26 Eastern Asia 6 4 3 3 Southern Asia 9 9 8 8 South-Eastern Asia 11 10 7 7 Western Asia 15 13 10 10 Oceania 20 17 15 15 Indicator 7.9 Proportion of the population using an improved sanitation facility (Oct 2006) Developed regions 48 24 20 20 Countries of the CIS 12 12 5 5 Developing regions 163 140 104 104 Northern Africa 6 5 5 5 Sub-Saharan Africa 50 47 36 36 Latin America and the Caribbean 46 38 27 27 Eastern Asia 6 3 1 1 Southern Asia 9 9 6 6 South-Eastern Asia 11 10 6 6 Western Asia 15 11 8 8 Oceania 20 17 15 15 Indicator 7.10 Proportion of urban population living in slums (July 2006) Developed regions 48 1 1 1 Developing regions 163 106 105 105 Northern Africa 6 5 5 5 Sub-Saharan Africa 50 43 43 43 Latin America and the Caribbean 46 31 31 31 Eastern Asia 6 3 3 3 Southern Asia 9 8 8 8 South-Eastern Asia 11 7 7 7 Western Asia 15 9 8 8 Source: UN mdg database www.mdgs.un.org May 2008 100 Table 3.9: International Reporting of basic CPI data for IDA SSA countries 98 99 00 01 02 03 04 05 06 07 Angola C C C C C C C X X Benin C C C C C C C X X Burkina Faso C C C C C C C X X Burundi C C C C C C X X X Cameroon W W W W W W X X X Cape Verde W W W W W W W X X CAR C C C C C C C X Chad C C C C C C C X X Comoros Congo DRC X X X X X X X X Congo Brazz'e C C C C C C C X X Cote d'Ivoire C C C C C C C X X Djibouti Eritrea included with Ethiopia Ethiopia C C C C C C C X Gambia C C C C C C C X Ghana W W W W W W W X X Guinea C C C C C C C Guinea Bissau C C X X Kenya C C C C C C C X X Lesotho W W W W W W W X X Liberia Madagascar W W W W W W W X X Malawi W W W W W W W X X Mali C C C C C C C X X Mauritania W W W W W W W X X Mozambique C C C C C C C X X Niger C C C C C C C X X Nigeria W W W W W W X X X Rwanda C C C C C C C X X Sao Tome & P Senegal C C C C C C C X X Sierra Leone C C C C C C X X X Somalia Sudan X X X X X X X X X Tanzania W W W W W W W X X Togo C C C C C C C X X Uganda W W W W W W W X X Zambia W W W W W W W X C = ILO database, capital city only W = ILO database, whole country updated April 2008 X = data for more recent years available in IMF database updated March 2008 ILO at http://data.un.org/Data.aspx?d=ILO&f=srID%3a4620 IMF at http://www.imf.org/external/pubs/ft/weo/2008/01/weodata/weoselgr.aspx 101 Table 3.10: International Reporting of Basic External Trade data for IDA SSA countries 98 99 00 01 02 03 04 05 06 07 Angola Benin IER IE IE IE IER IER IER IER Burkina Faso IE IE IE IE IE IE IE Burundi IE IE IE IE IE IE IE IE Cameroon IE IE IE IE IE IE IE IE IE Cape Verde IER IER IER IER IER IER IER IER IER CAR IE IE IE IE IER IER IER Chad Comoros IE IE IE Congo DRC Congo Brazz'e Cote d'Ivoire IE IE IE IE IE IE IE IE IE Djibouti Eritrea IER Ethiopia IE IE IE IE IE IE Gambia IER IE IE IE IE IE IE IE IE Ghana IE IE IE IE IE IE IE IE IE Guinea IE IE IE IE IE Guinea Bissau Kenya IE IE IE IE IE IE IER Lesotho IE IE IE Liberia Madagascar IE IE IE IE IE IE IE IE IE Malawi IER IER IER IER IER IER IER IER IER Mali IER IER IER IE IE IE IE Mauritania IE IE IE IE I I I Mozambique IE IE IE IE IE IE IE IE IE Niger IER IER IER IE IE IE IE IE Nigeria IE IE IE IE IE IE Rwanda IE IE IE IE IE IE Sao Tome & P IE IE IE IE IE IE IE IE IE Senegal IE IE IE IE IER IE IE IE IER Sierra Leone IE Somalia Sudan IER IE IE IE IE IE IE IE Tanzania IE IE IE IE IE IE IE IE IE Togo IER IER IER IER IER IER IER IER Uganda IE IE IE IE IE IE IE IER IE Zambia IE IE IE IE IE IE IE IE IE I = Import data E = Export data R = Re-export data Source: Extracted from UNdata - last updated November 2007 UNSD Commodity Trade Statistics database, series Trade of Goods All Commodities http://data.un.org/Data.aspx?d=ComTrade&f=_l1Code%3a1 102 This page is blank 103 Annex 1 List of Persons Interviewed for the Evaluation Washington, D.C. (April 14-18) World Bank Jeff Gutman Vice-President, Operations Policy and Country Services Kyle Peters Director, Country Services Aysegul Akin-Karasapan Director, OPCS Delivery and Results Management Shaida Badiee Director, Development Data Group Robin Horn Education Sector Manager, Human Development Network Misha Belkindas Cluster Leader, International Statistical Programs, Development Data Group Neil Fantom Team Leader, Statistical Development and Partnership Team, Development Data Group Sulekha Patel Team Leader Socio-Demographic Data Team, Development Data Group Anju Sharma Operations Officer, Global Programs and Partnerships Donald Bundy Lead Specialist, Human Development Network – Education Mila Freire Senior Adviser, Finance Economics and Urban Department, Sustainable Development Network Lucía Fort Senior Gender Specialist, Gender and Development Group, Poverty Reduction and Economic Management Gulnara Febres Senior Operations Officer/Manager WBI program for Central Asia, PREM, WBI/World Bank Graham Eele Senior Statistician, Development Data Group Lianqin Wang Senior Education Specialist, Human Development Network – Education Olivier Dupriez Team Leader, Household Survey Development Team, Development Data Group Adriana De Leva Consultant to Development Data Group Mustafa Dinc Grant Administrator, Trust Fund for Statistical Capacity Building, Development Data Group Barbro Hexeberg Senior Economist, Development Data Group Naoko Watanabe Statistical Officer, Development Data Group Perinaz Bhada Urban Environment and Climate Change Thematic Group, Finance Economics and Urban Department International Monetary Fund Rob Edwards Director, Statistics Department William E. Alexander Deputy Director, Statistics Department 104 New York (April 21-22) UN Statistics Division, Department of Economic and Social Affairs Paul Cheung Director Francesca Perucci Chief, Statistical Planning and Development Section Jeremiah Banda Chief, Demographic and Social Statistics Branch Jean-Michel Durr Chief, Demographic Statistics Section Srdjan Mrki´c Chief, Social and Housing Statistics Section Montreal (April 23-25) UNESCO Institute for Statistics Albert Motivans Head of Section, Analysis and Information Brian Buffett Chief of Section, Data Processing, Standards and IT Services Nafiou Inoussa Coordinator – Africa, Statistical Capacity Building Programme Bertrand Tchatchoua Programme Specialist, Statistical Capacity Building Said Belkachia Programme Specialist (EFA), MDG/Gender Focal Point Said Ould A. Voffal Education – Survey Operations Juan Cruz Perusia Assistant Programme Specialist, Education – Survey Operations Paris (April 28-30) Paris 21 Secretariat, Development Cooperation Directorate, OECD Antoine Simonpietri Manager Christophe Duhamel Deputy Manager François Fonteneau Project Officer, ADP and IHSN Geoffrey Greenwell Technical Project Officer, ADP & IHSN Mark McConaghy Statistics Adviser, International Household Survey Network Tony Williams Development Statistics Consultant Gérard Chenais Paris 21 Consultant Samuel Blazyk Paris 21 Secretariat Eric Bensel Paris 21 Secretariat Jean-Marc Landais Paris 21 Secretariat Jenny Galleli Paris 21 Secretariat General Directorate of International Cooperation and Development, Ministere des Affaires Etrangeres, France Jean-François Divay Chargé de Mission 105 Malawi (August 25-29) National Statistical Office Charles Machinjili Commissioner of Statistics Mrs Mercy Kanyuka Deputy Commissioner Jameson Ndawala Assistant Commissioner (Economics) Masozi Kachale NSS Project Officer Angela Msosa Chief Statistician (Technical Services) Willie Kachaka Chief Statistician (Demography), Strategic Plan drafting team Deric Zanera Chief Statistician (Demography) Malawi Yute Chief Statistician (Publications), Strategic Plan drafting team Mylen Mahowe Principal Statistician (Demography) Ishmael Gondwe Statistician (Agriculture) Benjamin Banda Statistician (Agriculture), Strategic Plan drafting team Simeon Yosefe Statistician (Technical Services) Centre for Social Research Dr C Chilimampunga Director Statistics Association of Malawi and Chancellor College Statistics Department Jupiter Simbeyi Head of Statistics Department Norwegian Embassy Britt Hilde Kjolas 1st Secretary DFID Charlotte Duncan Governance Adviser Ministry of Education, Science and Technology Enoch Matale Assistant Statistician, head of Statistics Unit, Planning Department Ministry of Health Central Monitoring, Evaluation and Research Division Planning and Policy Department Kelvin Saukila Chief Statistician Chris Manyamba Principal Statistician Ministry of Agriculture and Food Security Isaac Chirwa Principal Statistician, Planning Department 106 Ministry of Labour and Vocational Training Patrick Kabambe Secretary for Labour Brian Ng’oma Chief Planning and Research Officer Gracious Hamuza Statistician, head of statistics section Ministry of Justice and Constitutional Affairs Misha Kalongwe Chief Economist, Planning Unit Chris Manyamba ex Statistician Reyneck Matemba Parliamentary Draftsperson Amos Konyani Assistant Statistician Ministry of Economic Planning and Development Ben Botolo Director of Monitoring and Evaluation World Bank Country Office Tim Gilbo Country Manager Khwima Nthara Senior Economist Thomas Munthali Economist Bamako (August 27-29) Ministère de l’Economie, de l’Industrie et du Commerce Direction Nationale de la Statistique et de l’Informatique (DNSI) Mr. Seydou Moussa Traoré Directeur Mr. Balla Keita Chargé d’Exploitation et d’Analyse des données, Division Méthodes et Analyse Cellule Technique – Cadre Stratégique de Lutte contre la Pauvreté (CSLP) Mr. Mamadou Magassa Chef Unité Développement Institutionnel et Gouvernance Ministère du Développement Social, de la Solidarité et des Personnes Agées Observatoire du Développement Human Durable et la Luttle Contre la Pauvreté au Mali (ODHD/LCPM) Mr. Zoumana Bassirou Fofana Coordonnateur Mr. Bourema Fassery Ballo Expert Statisticien Mr. Dramane Traoré Expert Economiste Ministère de l’Education Cellule de Planification et de Statistique Mr. Tidiane Niambele Sous-Directeur Mme. Joseline Caseli Coopération Française – AFD 107 Mr. Modibo Diallo Mr. Sekou Traoré Statisticien Cellule de Planification et de Statistique Développement Rural Mr. Diakité Kotié Chef Intérimaire Mr. Diallo Broulaye Membre de l’Equipe PSA Délégation de la Commission Européenne Mr. Pierre Beziz Chargé de programmes Banque Mondiale Mme. Claire Harasty Economiste de la Pauvreté Programme des Nations Unies pour le Développement Mr. Alassane Ba Economiste National Addis Ababa (September 1-4) Central Statistical Agency Yakob Mudesir Deputy Director-General, Operation, Methodology and Data Processing Yasim Mossa Deputy Director-General, Economic Statistics Gebeyehu Abelti Deputy Director-General, Population and Social Statistics Mageru Haile Head, Industry, Trade and Services - Chairman of NSDS Task Force Alemayehu Gebretsadik Head IT Department Eleni Kebede IT Team Leader Ministry of Education Adamu Gnaro EMIS Expert World Bank Country Office Eyerusalem Fasika Research Analyst Niamey (September 1-4) Institut National de la Statistique Mr. Abdoullahi Beidou Directeur-Général Mr. Ghalio Ekade Secrétaire Général 108 Direction de la Coordination et du Développement de la Statistique Mr. Ibrahim Soumaila Directeur Mme. Mariama Awal Mr. Kabirou Samaïla Responsable Annuaire Statistique Direction des Statistiques et des Etudes Economiques Mr. Mahamadou Chekaraou Directeur Mr. Tassiou Almadji Chef Comptabilité Nationale Direction des Enquêtes et des Recensements Mr. Omar Habi Directeur Mr. Souleymane Alzouma Direction des Statistiques et des Etudes Démographiques et Sociales Mr. Sani Oumarou Directeur Mr. Soumana Harouna Mr. Gapto Mai Moussa Mme. Hawa Omar Chef du Centre de Formation et de Perfectionnement Mme. Julienne Aitchedji Chef de la Division Informatique Mr. Hamado Sawadogo Expert principal, projet Union Européenne Mr. Ali Malaï Chef ONAPAD Mr. Ousseini Hamidou Chef UAAP Ministère de l’Economie et des Finances Mr. Abdou Soumana Secrétaire Général Mr. Seydou Yaye Directeur Général, Direction Générale de l’Evaluation des Programmes de Développement Cabinet du Premier Ministre Secrétariat Permanent de la Stratégie de Réduction de la Pauvreté Najim Elhadj Mohamed Coordonnateur Mme. Aminata Takoubakoye Chargé du Suivi-Evaluation Système d’Alerte Précoce Mr. Harouna Hamari Coordonnateur Assemblée Nationale Mr. Samoussi Tambari Jakou (Maradi) Président Commission Comerce Mr. Moussa Zanghou (Agadez) 109 Ministère du Développement Agricole Mr. Harouna Ibrahima Directeur, Direction de la Statistique Mr. Ali Doulaye Coordonnateur, RG Agriculture, Recensement Général de l’Agriculture et du Cheptel Ministère de la Santé Publique Dr. Souley Rabi Maïtournam Directrice des Statistiques de la Surveillance et de la Riposte aux Epidémies, Système National d’Information Sanitaire (SNIS) Ministère de l’Education Nationale Mr. Yacouba Djibo Abdou Statistician World Bank Mr. Ousmane Diagana Country Manager Délégation de la Commission Européenne Mme. Anja Nagel Chef de Section Economique UNICEF Mr. Souleymane Ousmane Database officer 110 Annex 2 Summary status of recent statistical planning in Sub-Saharan Africa IDA countries September 2008 This commentary is based on documents taken from the Paris21 Knowledge Base (P21KnB), supplemented by information from the P21 network and what can be found on country statistical office and World Bank websites. It reflects the personal view of the consultants. Angola National Medium-Term Plan for Statistics, 2002 - 2006. This document describes a statistical system centred round the NSO and the Central Bank. It is very short - 17 pages - and entirely forward looking. There is no analysis of weaknesses or description of the current situation. A large increase in budget and staff is foreseen and explained in a separate document. There is no description of how the Plan was formulated. Funding partners are sought. P21 add that the Plan was not implemented. Angola joined the PALOP workshop on NSDSs in May 2005, and this leads to the second document, the Road Map for a National Statistical Development Strategy for Angola. This is simply a proposal for a way forward and there is no information on later progress. Benin According to P21, there is a 10 year statistical programme in support of the PRSP (2005-2014) and a Statistics Master Plan for 2005-2006. Neither are available on P21KnB or the NSO website. Benin joined the Bamako workshop in Feb 2005, and produced a Road Map for an NSDS. This is of the usual form. P21 add that this has led to an NSDS for 2007-2012 being produced, which was adopted by the National Council for Statistics in January 2008, and a communication to the Council of Ministers is ready. A 3-year implementation plan is being prepared. We have not seen a copy of the NSDS. Burkina Faso The Statistical Master Plan 2004-2009 was to all intents and purposes an NSDS, though not influenced by P21. Legislation and the institutional framework were in place in 2000 (as for Chad), so the plan hoped for some quick action. It is reasonably good on objectives and process, on the SWOT, on the principles. It is not so strong on demand analysis - it is more a wish-list which is not prioritised - and requires external finance. Due to this the action plan, though very detailed, is theoretical. In looking forwards it focuses to a great extent on the NSO, though the early analysis involves the other parts of the NSS. It mentions an earlier plan which was first discussed in 1994, was finally prepared in 1999 for 2000-04, but was never implemented as it was too “heavy”. 111 The World Bank StatCap document repeats the analysis. Burkina Faso was a pilot country for StatCap along with Ukraine, and Part 1 of the programme runs from 2004- 2009. A second phase is foreseen if all goes well. The Master Plan was revised in September 2007 with a new action plan for 2007-2009. Much of the text repeats the original plan, but progress in each area has been added. Financing of the Plan, despite StatCap, has been less than required. Further to this document is the National Statistical Programme 2008 which is mainly a list of unfunded activities for that year. P21 add that Burkina Faso have requested support for a full NSDS to cover the period 2009-2012. Burundi There is a Plan for a Road Map (undated - about 2003) on the P21KnB. P21 add that Burundi has not had to date a statistical strategy, but are in the process of preparing one, having taken part in NSDS workshops in Addis and Ouagadougou in 2006. The WB website mentions a statistical capacity building project running from 2006 to 2008, totalling $200,000, with no other details or documents. There are no relevant documents on the Stats Burundi website. Cameroon There are no Plans on the P21KnB. P21 adds that while there has been no explicit statistics strategy for Cameroon, there have been some partial work programmes. One was the 3-year programme for statistical operations 2003-2005, no copy on file. There are no stats projects mentioned on the WB site and nothing on the Statistics Cameroon site about development plans. The Road Map 2005 sets out an NSDS development plan in the usual format, with finalisation and presentation to donors foreseen for mid-2006. P21 add that this did not happen - the draft NSDS is being revised at the moment. Cape Verde A 32-page summary in English of the Development Plan for the Statistical System, 2006-2010, is available on P21KnB. This document, though only a summary, appears to follow NSDS principles and was the basis for a Round Table discussion with donors in May 2006. There are no later documents on the database, nor on the statistics office website, which is comprehensively set out. There is talk of a StatCap project, but this does not appear on the WB website. 112 Central African Republic There is a Road Map on the P21KnB, undated but finalised in September 2006 with the usual content. P21 add that work had begun on designing an NSDS, but that it is blocked for the present time though there is a wish to re-launch the process. No further information is available on websites. Chad The PPAST (Multi-year programme of statistical activities 2002-07) is an interesting example of a Plan. The situational analysis is thorough and does not hide the major problems faced by the statistical system. The legislative and organisational aspects pre- date the Plan, which sets out the vision and mission of the NSS. It is cross-referenced to the PRSP. It is to some extent a wish-list of activities, but the problems in achieving each one are set out with the expected results and a strategy to achieve them. Due to lack of funds, a detailed work programme only covers the first two years. P21 add that implementation of many components were delayed to 2005, when a revised Plan was written for the years 2005-2008 (not on websites). Chad now wishes to prepare a full NSDS. The Road Map presented at the Bamako meeting in 2005 started the process. Comments in the Road Map confirm that the 2002-07 Plan is at best only partially operational. Comoros The P21KnB contains a very brief Draft Road Map (end 2004) which has no detail. This was followed in May 2005 by a document Producing an NSDS which contains some of what might be expected in a NSDS plan - some problem analysis, for example, but which is more of a scoping document. It has a huge list of indicators, but no budget or priorities. There is some attempt at a SWOT analysis but no indication that anyone has been consulted. P21 add that an NSDS has now been designed for the period 2008-2012. Congo-Brazzaville A Multi-year Statistical Development Programme 2005-09 is available, published in October 2004, which P21 say is now operational though there have been delays with the legal aspects. It was written by a national team with external advice, and at the start and finish a large number of producers and users of data were involved. It has high-level support. It is blunt about the challenges facing statistics in Congo and analyses well the weaknesses of the different parts of the statistical system. The plan envisages a slow and gradual development process, done internally with the assistance of donors. There are targets, strategies to achieve them and results expected. There is also a small section on risks. It is similar to the Plan for Chad. 113 Cote d’Ivoire The P21KnB contains just a Road Map essentially for the third NSDS, to cover 2006 - 2011 and presented at the 2005 Bamako seminar. Cote d’Ivoire has prepared two earlier plans: the first was partly implemented, but the second was not officially adopted. The internal situation of the country prevented much progress. There are no other documents on websites. P21 add that an NSDS for 2008-10 has been prepared. Democratic Republic of Congo The P21KnB contains the Road Map for an NSDS for 2001-11, prepared in November 2005. It is more detailed than other roadmaps, and could form the basis for a strategic document. A number of isolated statistical activities have taken place during 2000 - 2005, and the plan brings these together. No other documents are available and there does not appear to be a country statistical office website. P21 add that the NSDS design has reached the diagnostic stage. Djibouti The Statistical Master Plan 2006-10 for Djibouti was prepared in 2005 and is currently being implemented. It is closely linked to, and inspired by, the 2003 PRSP. It follows the recommendations for NSDSs, with a detailed analysis of the current situation and a moderate analysis of its weaknesses. It is not clear to what extent the development programme was prioritised, but it is clear that there was a great deal of discussion between users and all parts of the NSS. Eritrea There are no documents in the P21KnB. P21 comment that a Master Plan for the Development of Economic Statistics exists, but has no further information except that Norwegian cooperation recommended in 2007 the preparation of an NSDS. The Eritrean statistics office does not appear to have a website. Ethiopia Ethiopia has very recently started the process of designing an NSDS for the full statistical system, not just for the Central Statistical Authority. This is being designed using NSDS guidelines to a great extent. It follows on from a CSA Strategic Plan 2005- 2009 (which is only available in Amharic), and which the CSA say is of limited relevance. The Medium Term Statistical Programme 2003/4 - 2007/8 is operational. During 2001 and 2002, a detailed analysis of data requirements was followed by an assessment of the current data, identification of data gaps and then a programme to fill these gaps, prioritised to some extent over the five years of the Plan. These analyses covered 13 sectors, from population to the macro-economy via agriculture and construction. 114 Where it does not follow NSDS guidelines is in areas such as mission, vision and values; an overarching strategy; a detailed human resource analysis; and stakeholder consultation. The partner in the Plan preparation was the Ministry of Finance and Economic Development (MoFED), who were seen by the CSA at the time as the ultimate client for all statistics and who could speak for the whole user community Gambia The P21KnB contains a copy of the Summary Strategic Plan to improve the Statistical System (2002) taken from a report by a WB consultant, which is mostly a brief statement of intent. It was followed in 2004 by a Statistics Master Plan again written by a consultant. This is very good on priorities and has a detailed work programme to support these. It also covers in detail the necessary human and physical resources required. No consultation was done - the recommendations of the plan were to consult within the statistics office and then externally. The PRSP for 2007-11 states that the master plan will be made operational. The statistics office website contains no further information. P21 add that an NSDS is being finalised in 2008. Ghana There are no relevant documents for Ghana on the P21KnB. P21 add that there is a Medium Term Statistical Programme 2001-2005, and that the NSDS process was launched in September 2008. Guinea Guinea does not have an extant statistics development strategy. A Roadmap was prepared for the Bamako workshop in February 2005, and P21 add that an NSDS for 2008-13 has just been prepared. There is no further information on the DNS website. Guinea-Bissau Guinea-Bissau does not have an extant statistics development strategy and there are no documents on the P21KnB. P21 add that a Road Map was adopted at the PALOP workshop in Bissau in May 2005, and that an NSDS is being prepared, though the process for the present has halted. Kenya A Strategic Plan for the National Statistical System 2004-2008 was designed in 2003 and is being implemented. It is in line with NSDS principles and has high level support. An Implementation Plan was subsequently developed to put more detail on that aspect of the Plan. P21 add that a follow-on NSDS is being developed. Lesotho P21 report that a NSDS for the period 2008/9 - 2012/13 is under development and that the final report was expected in July 2008. Phase 1 of the work covered the Bureau of Statistics and 9 ministries and authorities, and Phase 2, expected to resume in January 2009, will cover the remaining parts of the statistical system. Although a work 115 programme for the implementation of Phase 1 has been drawn up, it hinges on a donor financing conference planned for November 2008. There are no relevant documents on the P21KnB or the LBS website. Liberia There is no prior statistics development strategy for Liberia and no documents on the P21KnB. P21 add that an NSDS was developed following the production of a Roadmap by a consultant in 2006, and was delivered to Government in January 2008. The office does not have a website. Madagascar The Strategic Plan for a rapid and sustainable development of the Statistics System 2003-07 is a relatively short document which follows some of the lines of an NSDS but without the detail. It describes the weaknesses of the current system, and sets out a costed work programme, but says nothing about priorities, the sources of finance, or indeed the process by which the plan was developed. A Roadmap for a full NSDS was prepared for the workshop in Addis Ababa in December 2004. P21 add that an NSDS for 2007-10 is ready and an action plan is being finalised. Malawi Malawi is implementing its third Strategic Plan for the NSO 2007-11 which is a fully fledged NSDS for the NSO only, following all the guidelines. In addition, they have recently finalised a first attempt at a partial NSDS for the national statistical system, covering the NSO and 6 key line ministries, for 2008-12. This follows to some extent the recommendations for bringing sectoral plans into an overall strategy, the main deviations being the extent to which line ministries obtained prior high-level support, and the amount of prioritising of activities. Mali Mali’s first Statistical Master Plan 2001-2005 (January 2001) is a typical document of its time, describing the statistical system and statistics by sector. It was not an NSDS, in the sense that there was no real analysis of strengths and weaknesses though there was a problem analysis. There was early consultation of users but a poor response, and a useful summary of donor interest and interventions. Although a Mission, Vision, Values section was not specifically done, the Plan does set out strategic lines of action. The Plan was not costed, but external funding was to be sought. Lamine Diop adds in his review paper that it was not formally adopted by Government. In 2005 the Plan was updated to apply to 2006-10, with retention of the same reform proposals, and it was approved by the Council of Ministers in 2006. A somewhat further elaborated document, covering 2008-12, with the first two years mainly for institutional reform, was presented at the joint Mali development roundtable in June 2008. A partner group around statistics was launched in Mali in October 2007 under 116 experienced leadership of Sweden and the EC. Paris21 Secretariat considers it one of the best examples of partners coordination on statistics in Africa, and is hopeful that it may enable the country finally to move from the long drawn out planning process into actual implementation. Mauritania Mauritania has a Statistical Master Plan 2000-2006 for which the P21KnB has a number of documents. The analysis of the existing situation is thorough, and links the statistical system to national and international priorities (for example GDDS). It covers the whole NSS. A second document is an extremely interesting analysis of various scenarios for a future structure for the system, and a third contains a detailed action plan with costs. A relatively positive mid-term evaluation was carried out at the end of 2004. This is a good example of a statistics strategy which predates the international initiatives. An NSDS 2007-2012 followed this earlier plan, which closely follows the NSDS recommendations, with a SWOT analysis, mission, vision and values statements, and a prioritised programme of work. There is a strong analysis of the remaining weaknesses of the NSS - this is the subject of an earlier specialist report; good references to the structures and systems which still need to be established; and links both to the current PRSP and international statistical requirements and recommendations. Unfortunately, a sizeable funding gap needs to be closed before the Plan can be fully implemented. P21 add that the NSDS was presented to the Consultative Group for Mauretania in December 2007. Mozambique The existing Strategic Plan for the National Statistical System 2003-2007 follows to some extent the structure of a traditional NSDS. It does seem to rely on expectations of donor support, and it is not stated what the preparation process was nor how, or whether, any prioritising was done. It is a brief document. The P21KnB contains reference to a RoadMap for an NSDS done for the Bissau conference in May 2005, and P21 add that an NSDS for 2008-12 has been produced. This is not on the INE website. Niger The first NSDS for Niger was finalised in September 2007, together with an implementation plan for 2008-12. It is a classic example of a strategy for the statistical system. Although concentrating mainly on the statistical aspects, it does analyse in reasonable detail the human and physical resources required, and in fact foresees a comprehensive human resource strategic plan being developed. Substantial donor contributions will be required, or a very large increase in government funding. The process by which the NSDS was developed is not described, though the fact that the analysis includes all parts of the NSS would suggest some amount of discussion. It is not clear though at what point external stakeholders were consulted. The analysis of the demand for statistics is linked to the 2006 (second) PRSP and the MDGs. 117 Nigeria A Statistical Master Plan for the Nigerian National Statistical System 2004/5-2008/9 is under implementation. It is a comprehensive document which follows NSDS guidelines. It builds on earlier partial plans, and donor projects operating in parts of the system, but is the first complete plan for the NSS. Consultation was thorough within the Federal Office of Statistics, but external stakeholders were only involved at the start and finish of the process. The analysis of strengths and weaknesses is good, as are the lessons taken from previous attempts to improve the system. The SWOT analysis is very detailed, and leads naturally into a plan for improvement with the required overall vision for the NSS. The Plan is at Federal level and links into NEEDS (the Nigerian PRSP). Further plans for the State Statistical Systems are under discussion. Rwanda A summary Strategic Development Plan for the NSS dated November 2002 is on the P21KnB. It is not clear what its status was, but some of it must have been implemented since the new structure and legal basis are all in place. The NISR Strategic Plan 2007-11 is a full NSDS, backed up by the legal and institutional framework required, and fully funded by government and donors. Although a shorter document than many, it covers all the aspects of an NSDS with a tough analysis of the challenges, and a detailed action plan with targets and deadlines. Although the analysis is of the whole NSS, the action programme focuses on the NISR - but the new law does state the responsibilities of, for example, the line ministries and the part they must play in the statistical system. Sao Tome and Principe A Plan for the Development of the National Statistical System 2004-07 was produced in October 2003, but P21 add that it was not funded. Sao Tome attended the May 2005 meeting in Bissau and produced a road map, and will seek funding for an NSDS. P21 add that the diagnostic phase of the NSDS process has now been achieved. Senegal The Statistics Master Plan 2008-13 was developed during 2006 and 2007 and is the first strategy of its kind for the country. It was preceded by a reform of the legislation and the creation of an autonomous national statistical agency. The SMP was developed through a thorough consultative process not only looking at statistics by sector but also analysing the human and physical resources and the organisational structure which would be required. It is linked to the PRSP, to the UN Fundamental Principles, and to various national and international initiatives. The SMP follows for the most part the recommendations for an NSDS, though it is not so clear how (or indeed whether) there has been prioritising of activities, or if the required budget is attainable. It is also a little brief on monitoring and evaluation. It is 118 also not clear whether some final approval is required of the Plan before it is implemented. P21 add that a financing round table will be organised soon. Sierra Leone An NSDS 2008-12 was finalised in June 2008 following some years of preparation and consultation, revision of the legislation and the setting up of Statistics Sierra Leone. The process was well supported by donors including the TFSCB. The document follows standard NSDS guidelines, and is prioritised and budgeted. Implementation will require large external financing for some years. Somalia P21 report that a statistical office has been set up in Somalia, and they wish to develop a strategic plan. Sudan The Central Bureau of Statistics website has a powerpoint presentation describing a strategy developed in 2003, but no documents are available. Some revision of legislation and organisation has taken place. Tanzania Tanzania launched the design of a Statistics Master Plan in 2006, which was expected to be completed in March 2008, according to the latest progress bulletin on the NBS website, which is for December 2007. P21 add that they have been told that a StatCap loan is being negotiated for its implementation, and that the official launch is planned before the end of 2008. Togo A Roadmap was prepared for the Bamako workshop in 2005, forecasting a statistics strategy to be prepared by 2006 for the period 2007-12. P21 add that the NSDS is finalised but not yet officially adopted, and that the TFSCB is supporting the process. No information is available on the Statistics Togo website. Uganda The Uganda Bureau of Statistics Corporate Plan 2002-07 is similar in scope to an NSDS, with its mission, vision and values derived from the new Statistics Act which established UBOS, and a clearly stated methodology for prioritising new data demands. As well as establishing a core programme of statistical production, it refers to building relations with users, and the need for donor funding while GoU funding is being realigned. The only thing missing is the SWOT analysis. This was followed by the Plan for National Statistical Development 2006/7 - 2010/11, which covers the whole statistical system and follows NSDS guidelines. (In fact it was produced with assistance from P21). It was created with full stakeholder participation and has high-level support in government. The mission, vision, values statement is 119 linked through to a set of actions with measurable targeted objectives. Implementation is under way with donor support. The Uganda strategy, and the way it is being implemented, is regarded as one of the models for NSDSs in Africa. Further information from the website was not available, since the UBOS domain name has expired (September 2008). Zambia The Central Statistical Office Strategic Plan 2003-7 was launched in March 2003 with Presidential support, and an appeal for donors to back it with funds. It is an early example of an NSDS and contains many of the recommended components, including a SWOT analysis, a MVV statement, and a prioritised action plan. Stakeholders were consulted early as part of a user needs assessment. The Plan is strongly anchored in national development strategies, including the PRSP. It covered only the CSO. The plan was not in fact intensively pursued by the CSO, and never got much support from the donor community. No further relevant documents are given on either P21KnB or the CSO website. P21 add that preparation of a new NSS-wide NSDS was expected in 2007. Early in 2008 the CSO still hoped to prepare an NSS-wide NSDS within the year, but there is no information as to whether it has been able to move ahead. 120 Annex 3 Ethiopia: experience with MAPS The National Statistical System in Ethiopia 1. Statistics in Ethiopia are a Federal responsibility, along with Defence, Foreign Affairs and a few other areas. In this, Ethiopia is different from Federal countries such as India and Nigeria. The central statistical service is not therefore trying to coordinate and manage fully fledged statistical services at a lower administrative level. 2. The law is quite clear that no-one may undertake surveys or censuses with national coverage without the prior consent of the Central Statistical Agency (CSA). Administrative data are collected at Warada, then Zone, then Regional level, before coming to the CSA for final compilation. If they wish, Regions (what in Nigeria would be called States) can compile and disseminate data for their own region, and can also undertake regional level surveys, but they cannot make analyses at national level. 3. Unusually among African statistical services, the work of the statistics office has in the main been adequately funded by the government. To cite a few examples: about 90% of the recurrent budget comes from government, and even the 2007 census was about 85% funded by government. Donor funding has traditionally been used for supplementary logistics - mainly equipment, for specific upgrades, and for short-term technical assistance, rather than for a regular survey programme, for example. This puts the CSA in a strong position as regards the donor community. 4. The user community was briefly discussed with the CSA (unfortunately, the key user representative in the MoFED was out of office for the week). The CSA feel that they have a good relationship with their users, and are trusted by them, though the community is still quite small. The experience of Ethiopia with strategic statistical planning 5. The CSA is in the process of writing its NSDS which will include the whole statistical system. Earlier plans focused on the CSA, and did not necessarily have an overall strategic focus. The current plans are the Medium-term National Statistical Programme 2003/4 - 2007/8 which is being extended by another year until the NSDS is operational, and the accompanying CSA Strategy 2005-2009. The latter, I was told, was by way of a gap filler and is not being systematically implemented or monitored. 6. The Medium-Term Programme (MTP) follows some of the NSDS principles. What is perhaps most interesting is that it was a purely Ethiopian construct, with no external support or advice. During 2001 and 2002, a detailed analysis of data requirements was followed by an assessment of what was currently available, identification of data gaps and then a programme to fill these gaps, prioritised to some extent over the five years of the Plan. These analyses covered 13 sectors, from population to the macro-economy via agriculture and construction. 7. Where it does not follow NSDS guidelines is in areas such as mission, vision and values; an overarching strategy; a detailed human resource analysis; and stakeholder 121 consultation - the partner in the Plan preparation was the Ministry of Finance and Economic Development (MoFED), who were seen by the CSA at the time as the ultimate client for all statistics and who could speak for the whole user community. 8. The progress of the MTP was monitored every quarter. 9. Preparation of the NSDS is attempting to follow Paris21 guidelines to the letter. Although two international consultants have been engaged with WB funding to assist with the process, it is the CSA who are managing it. The CSA is fortunate in that the legal aspects are mostly in place - in 2005 there was new statistics legislation; that the new structure of the CSA is already installed; and that there is no reason to think that government funding will not continue. In this respect much of the battle is won. There is high level support for the NSDS, with the Minister of Finance and Economic Development as Chairman of the National Statistical Council. 10. The CSA have the very ambitious target to finalise the NSDS in October and start implementation in January 2009. They also plan to re-engineer all their business processes as part of the implementation strategy. 11. A technical sub-committee has been set up to oversee the preparation, and this in turn established 9 sector sub-committees - sectors identified by the PASDEP (the current Ethiopian PRSP). The process has been as follows: 1. the issue of preparing an NSDS went to the National Statistical Council (composed of deputy line ministers), which accepted the approach and commented that it was already long overdue 2. the technical sub-committee discussed what needed to be done, involving all stakeholders and taking into account past work 3. 33 ministries and institutions were selected, covering the majority of government users. Ministries were then asked to allocate experts to one or more of 9 sectoral technical sub-committees. Key ministries and agencies such as Finance and Economic Planning and the National Bank are represented on all sub-committees 4. each sub-committee prepared a gap analysis in advance of the visit of the two external consultants 5. following the consultants’ first visit, certain assignments were suggested to the task teams: a. to undertake a Monitoring and Evaluation Survey across the board (the current state) b. to review the successes and remaining challenges of the Medium Term Plan c. to undertake a quality assessment of the current statistics using the DQAF as a model d. and to do a user needs survey 6. the consultants also proposed that the sectoral breakdown was revised. 12. A draft of the NSDS is expected early in October, when it will be discussed at a full stakeholder workshop - including NGOs and the private sector. The final version will then be ready by the end of October, when it will go back to the National Statistical Council for high-level endorsement. Amendments to legislation and structure may be recommended. 122 13. The challenge, as always, will be the implementation, particularly since there is no statistical common service in Ethiopia, so the statistical units in line ministries are totally under the control of their ministry. This means that the support of the line ministries is crucial and it is to be hoped that the enthusiasm they showed at the start of the NSDS process is continued into the implementation phase. ADP and IHSN 14. Around 2003-2004, the CSA began to think seriously about data dissemination and archiving and planned to embark on a Central Data Bank project. They wanted to assemble all metadata and data in a one-stop shop. 15. At the time this work was underway, to establish which software tools and technologies to use, which included discussions with Pascal Hughes of the World Bank, IHSN was raising its profile. Following further discussions with Geoff Greenwell and Olivier Dupriez, there was an immediate meeting of minds and a decision to use the IHSN microdata management toolkit to archive censuses and surveys back to 1995. The microdata were available, but a lot of work went in to recreating some of the metadata. 16. Over the last three years, the ADP team has assisted the CSA a great deal. Staff have been trained in the appropriate tools; have gone on regional workshops; people from other offices have visited the CSA to share experiences; and ADP has given solid backup to the CSA when it was required. 17. Current work, in which the CSA is one of the pilot countries, is the development of a National Data Archive with ADP’s help. It will be officially inaugurated in the middle of September. When this becomes operational, it is expected to serve as an example for other countries to follow. 18. The CSA have noted already more on-line access to their data and reports, through the improved website, and have been able to decrease by about 70% the number of hard copies of reports being printed. Census 19. Ethiopia conducted its third census in 2007, and the results are about to be released. For the first time they were able to achieve full geographical coverage, and were able to work with the full cooperation of the Presidents or Deputy Presidents of the Regions. 21. In discussion of the UNSD materials and other support, the CSA made the following observations: • Regarding questionnaire design, the UNP&R2 was the main source of material, modified by the way in which questions had been posed in earlier censuses, and also by user needs • The responsible technical committee used the question bank when designing the questionnaire 123 • For the tabulation plan, they adopted UN recommendations, and added certain tables designed to the same standards • Editing was done entirely to UN recommendations • A rapid assessment of the quality of the age data has been done using UN indices of age preference; and • The CSA is planning to use UN methodologies for projections 22. The regional workshops have been extremely useful, as was the African Census Symposium. The African census community appreciated UNSD’s willingness to take into account certain aspects of census-taking which were felt to be unique to the continent - for example, regarding the treatment of migration and the definition of a household member. 23. The CSA had a study tour to Thailand arranged by UNSD which was most helpful, in particular the exchange of experience to do with using teachers as enumerators. 24. UNSD are closely following the Ethiopian census process, and while there are no formal links, the CSA are appreciative of the fact that they can contact them for advice or views. UNSD is seen as a “friend of the Ethiopian Census”. 25. Regarding the future, the CSA would like to have the software to enable them to calculate maternal mortality and population projections, plus appropriate training. Education Statistics 26. Ethiopia is one of the few countries which were selected for a UIS technical assistance project, set up during a visit from Doug Drew in 2003 and financed by the European Commission. The existing education statistics system was surveyed, and recommendations for improvement made including a revision of the questionnaire to be compatible with international reporting and its translation into 5 languages. 27. Around 17,000 people were trained in the new questionnaire - cascading down from the Federal office, through the regions and zones to the education officer at Warada level and then to the head teachers. This warada education officer will typically collate reports on 50 to 100 schools, and is responsible for training new head teachers to complete the questionnaire. So far this remains sustainable - new head teachers are being trained and, after two years of funding by UIS, government now pay for the printing and distribution of the forms. 28. At the Ministry of Education, the UIS project set up an EMIS unit, installed the software and hardware to compile education statistics, and trained the staff in its use. An IT specialist is in post, and his contract has just been extended by UIS for another 6 months. While the staff of the EMIS unit can use the software for inputting data and creating reports, it requires an IT specialist to make any modifications - for example to create a new indicator or report. There is at present no back-up should the UIS expert leave. 29. In terms of impact, statistics on primary and secondary education are much improved. There is still a problem with the tertiary sector, since data from these 124 institutions were never collected in the past. It is not just a case of introducing a new questionnaire, but of bringing in a reporting system from scratch. 30. At global level, Ethiopia recognises the improvements UIS have made in the system for international reporting. The Education Ministry staff noted: the UIS questionnaire being sent electronically (on which they were trained at a workshop in Zanzibar); UIS’s readiness to adapt the software (the original had no cut and paste facility, but when this was raised as an issue by the workshop participants, it was quickly put right); the instant acknowledgement of receipt of the questionnaire by UIS; and a very good check and query system. 31. There is still a definitional debate going on with UIS (as in Malawi, Ethiopia has 8 years of primary education and 4 years of secondary). 32. The EMIS section is extremely satisfied with the service they receive from UIS, and only desire the ability to modify the software in-house instead of having to rely on external experts. 125 Annex 4 Malawi: experience with MAPS 1. The National Statistical Office (NSO) of Malawi is implementing its third strategic plan, which closely follows NSDS guidelines (except that it is limited to the NSO rather than covering the national system comprehensively) and benefited from cooperation with Paris21 and in-country workshops. Strategic planning is well embedded in the office at all levels. Preparation of the third plan, in fact, was almost a routine exercise of reviewing and updating the second plan. People commented that some of the excitement and chance for blue-sky thinking when the first plan was prepared was missing now: maybe the fourth plan, 15 years after the start of the first, will be an opportune time for a more radical reappraisal. 2. The benefits have been tangible. First, the NSO find that approaches to the Ministry of Finance for funds are much more successful when done with the firm backing of a Plan. The Government of Malawi is clear at the outset what the requested funds are to be used for, and can see whether the commitments have been honoured. 3. Second, many staff of the NSO comment that the existence of the Plan has removed the personal aspects from the work programme. A unit or section is clear what they are to produce, irrespective of who is in charge and their personal preferences. This in turn means that their work has a well understood structure with checks and balances. 4. Third, there is no doubt that approaches to donors for assistance are much more effective and fruitful when they have a strategic basis, though it is true that assistance to the NSO is concentrated at present with just a few donors. 5. The Commissioner of Statistics, who directs the NSO, remarked that other Ministries have shown great interest in the NSO’s strategic plan, having seen the benefits it is bringing to the NSO. 6. Everyone interviewed agreed that NSO has improved over the last 10 years. On the side of the clients, they all noted improvements in reliability, timeliness and relevance. They are more confident in the data, which have better documentation and explanation. The client base in Malawi has been relatively small, but in recent years a large number of very vocal civil society organisations have sprung up with quite different demands to be satisfied. The NSO is engaging in interesting debates with them. 7. Inside the NSO, they too know the situation has improved. There are better and more open management practices. Staff are much better equipped, both academically and technically, to deliver on the Plan. They have seen demand for statistics rise, and have been able to raise their production in response. Computerisation is moving ahead, and, while much more can be done in the way of electronic dissemination, things are moving forward. They have less need for long-term resident experts, and advisers who do come to the office are working within a clearly defined framework to shared Malawian goals. Connections with line ministries are improving. 126 8. So, a steady and careful approach to development of the statistics office is paying off and the strategic plan approach has been instrumental in achieving this. There are of course plenty of outstanding issues. There needs to be more reviewing of plans during the implementation phase. Stake-holders can be involved a lot more than they are now, and earlier. Staff retention is still an issue, as is staffing the reintroduced Statistical Common Service. And the NSO now needs to raise its profile - it has some good products which more people need to know about. 9. The NSO’s strategic plans are well linked to Malawi’s own development plans. Malawi’s PRSP, the Poverty Reduction Strategy 2002, was succeeded by the Malawi Growth and Development Strategy (MGDS) for the period 2007 - 2011. The PRSP envisaged a programme of monitoring and evaluation, which was encapsulated in the PRSP M&E Master Plan. This was finalised in March 2004, more that two years after the PRSP was launched, and is not so much a plan as a framework for monitoring and evaluation data. It did not have a budget per se, expecting the individual sector strategies to include appropriate M&E. The World Bank PRSP annual monitoring reports note with disappointment the absence of data from the Master Plan during the PRSP period, though the NSO and other parts of the statistical system were providing data. The NSO, in fact, introduced its annual welfare monitoring survey (a quick CWIQ), in response to the demands of the PRSP and now the MGDS. 10. It is also important to note the setting up of MASEDA - the Malawi Socio- Economic Database, which is a response to the criticism that there were multiple sources of data and no central repository. It is a development of ChildInfo, and the NSO are a main contributor. The data are updated quarterly and a CD is produced annually. Unfortunately the website does not reflect the latest data - it is updated in India, not by the NSO. 11. Recently, the Economic Planning Ministry has finalised its Joint Programme Support for Strengthening the National Monitoring and Evaluation Systems in Malawi, which is effectively the implementation plan for the M&E Master Plan. Data from the NSO and wider NSS are key inputs, and the NSO is on the planning committee. 12. Despite the seeming plethora of plans in Malawi, it was said that there is only one plan - the Malawi Growth and Development Strategy. The National Statistical System in Malawi 13. Abolition of the Statistical Common Service was a severe blow to the operation of the statistical system (as it was for economists and members of other common services which were abolished at the same time). The statistical common service was reintroduced in 2003, and the formal launch of the Malawi National Statistical System was in 2007, but there is a lot of rebuilding to do of the statistics units in line ministries, both in terms of qualified staff and equipment. 14. To this end, Malawi has produced its first strategic plan for the National Statistical System - only the second country in the area to do so after Uganda. Wisely, they did not attempt to address the entire statistical system for this first plan. Instead they identified 6 key line ministries to participate. These are Education, Health, Agriculture, Justice, Labour and Industry. The goal is that by the end of the plan period 127 (2012), the statistical system will be operating in and between the NSO and the 6 ministries, and plans will be underway to extend it to the remainder of the system. Participating bodies benefited from technical assistance and a study visit to the Uganda Bureau of Statistics, plus of course the Paris21 guidelines which assisted particularly in tightening up the goals. 15. The plan document is interesting to read. The NSO, as coordinators of the operation, were not too prescriptive in terms of setting out common standards for strategies. Instead they allowed each ministry to prepare a sectoral statistics plan, which has been summarised in the overall document. 16. This means that the document is unbalanced, but this reflects the reality of the situation on the ground. It will be a general challenge of implementing NSS strategies to take this into account. Some of the ministries - notably education and health - have well- developed sector strategies and operational management information systems, so the statistical sector strategy can be developed to add detail to the monitoring and evaluation process. In other ministries, such overall sector strategies are not so well articulated and the statisticians have had to work more in isolation to develop their own plans. 17. The high level support, which the guidance recommends mobilizing from the inception of discussions, is also very different between ministries - linked again, quite often, to whether a sectoral strategy is in place. So implementing the plan in some ministries will be relatively simple, whereas in others support for it needs still to be built. Although at the simplest level, this could be thought of as reinstating the statistical common service, the statistics unit of any ministry has ultimately to be funded from that ministry’s budget and therefore has to argue its case against other priorities. 18. Fortunately there is a definite sense in Malawi that monitoring and evaluation have risen up the agenda - and so have statistics. The inability to fully monitor previous development plans is recognised to be unsatisfactory. But it is not just the donor community who are demanding more (or being more demanding). The government at its highest level is far more information conscious than before. A group of economists has been established in the Office of the President and Cabinet specifically to monitor the operations of line ministries through a network of planning departments. And these departments in turn will need to revitalise the regular statistical operations of their ministry. 19. The danger, as has always been the case, is that the urgent need for monitoring information may lead to a focus only on certain aspects of data to the detriment of regular administrative reporting. It is here that the strategic plan for the sector statistics can be really valuable. Whether funding is coming from the ministry’s own budget, or sectoral donors, the statistical units are ready to join the dialogue with a clear strategy backing them up. Other aspects of MAPS 20. Malawi conducted their census in June 2008, which had widespread popular support. The materials produced by UNSD, particularly the Principles and Recommendations Revision 2, form the core of the questionnaire, and Malawi followed 128 the recommendations for vibrant publicity and good documentation, which seem to have paid off. The NSO attended the regional workshops in Lusaka, Kigali and Maputo, and the African Census Symposium in Cape Town, and maintain contact with census offices elsewhere in Africa. Uganda and Mozambique plan to visit Malawi. The UNFPA regional office has been of great help. 21. On education statistics, the statistics unit in the Ministry of Education has strong links with UIS in Montreal. They attended a number of capacity building regional workshops. They find the UIS questionnaire much simpler to complete, though Malawi’s system of 8 years of primary education followed by 4 years of secondary has led to a debate with UIS on definitions - the ISCED classification being 6 and 6. They are using ED*ASSIST, a software package from the AED. 22. Regarding the IHSN, Malawi has around 25 surveys on the catalogue, as a result of earlier World Bank technical assistance in connection with development of the World Bank African Household Survey Database. The NSO have plans to develop a data archive - in fact donors and the Ministry of Economic Planning and Development are speaking of a data warehouse - and it would likely be able to benefit from participating in the IHSN/ADP work on national data archives, data documentation and the micro- data management toolkit. 129 Annex 5 Mali: experience with MAPS 1. Especially over the last ten or twelve years Mali’s statistical services have suffered from a combination of self-reinforcing problems that have reduced the contribution the services were able to make to the country’s development. The initial structure was centralized, with responsibility concentrated in the Direction Nationale de la Statistique et de l’Informatique (DNSI) in the Ministry of Planning. But statistical units came increasingly to be set up in sector ministries, without any clarification of responsibilities between them and DNSI. Neglect of human resource planning, unattractive employment conditions, and reduction of training efforts made statistical skills increasingly scarce relative to demand, so that many of the sector-ministry posts were filled by non-specialists. 2. The almost inevitable result of these trends was that the statistical services were slow to respond to the quite rapidly increasing data needs, and some of the products that came from official sources fell below acceptable statistical quality standards. This, in turn, caused costly loss of time on misunderstandings and corrections and, even more unfortunately, gave ammunition to those decision-makers who anyway did not want to be bothered with empirical evidence and numbers. Disputes about survey findings and indicator values, whether motivated by genuine technical doubts or political considerations, seem still to be more common than in some other countries. 3. Repeated efforts by the country’s statisticians, sometimes assisted by donor agencies, to put the system on a more constructive track have so far had little effect. Implementing a recommendation agreed in 1996, a national consultant financed by UNDP made a thorough diagnosis of the national statistical system in 2000 and wrote a draft Schéma Directeur de la Statistique. Following review by the official coordinating body then in existence, the paper was revised and thoroughly discussed in April 2002 at a national workshop which very largely endorsed it. The hope had been to get the reforms moving to strengthen statistics’ ability to contribute to monitoring of the country’s first internationally supported poverty reduction strategy, Cadre Stratégique de Lutte contre la Pauvreté (CLSP) I, which was then just starting implementation. But the effort ran into the sand and petered out. 4. June 2005 did see two important legal steps towards improving coordination among official statistical bodies: signature by the President of a law recognizing the national statistical system and the principles it should apply, and issuance of a decree creating a Comité de Coordination Statistique et Informatique (CCSI) to advise the Minister responsible for statistics. In 2006 the Council of Ministers at last approved the Schéma Directeur in a slightly updated form that had been prepared by the DNSI. The reform plans there spelled out were quoted at length in the country’s second Poverty Reduction Strategy Paper, Cadre Stratégique pour la Croissance et la Réduction de la Pauvreté, published at the very end of the year. 5. Nonetheless, the reforms which moved forward in 2007 and the first half of 2008 were measures relating to the statistical units serving the ministries rather than to the more fundamental issues concerning the national statistical system that were stressed in these documents. In February 2007 a decree was issued increasing the number of units serving the ministries, called Cellules de Planification et de Statistique (CPS), 130 from 6 to 11. A November 2007 decree provided for the establishment of CPS Coordination Committees, facilitating the service that a CPS could give to several ministries – the CPS Rural Development, for instance, to the ministries responsible for forests and for fishing in addition to the Ministry of Agriculture. 6. No progress was evident, however, on the conversion of DNSI into an executive agency, with greater autonomy, the main change recommended in reform plans since 2000, nor on the establishment of a secretariat to the CCSI, widely recognized as a key step to enable the Committee to be effective and make coordination a reality. The Director of DNSI was left with the same excessive responsibilities as in the past: leading and managing his own directorate in Bamako, supervising its 9 regional offices (Directions Régionales de la Planification, de la Statistique et de l’Informatique, de l’Aménagement du Territoire, et de la Population – DRPSIAP), guiding and assisting the ministerial Cellules de Planification et de Statistique, and being a one-person secretariat to the CCSI. 7. But indications continued to be given to the country’s technical and financial partners that the government was now genuinely behind the reforms proposed in the SDS. Whereas the December 2006 presentation of the country’s new strategy had been strongly critical of actual monitoring efforts under CSLP I and backed SDS proposals, the June 2008 official review of progress in poverty reduction through 2007 called strongly for accelerating SDS implementation. At the July 2008 joint meeting on poverty reduction strategy, the Minister of Economy repeated the government’s commitment to the SDS. The partner countries and agencies welcomed creation of the joint “Groupe Thématique Statistique,” to be chaired jointly by Sweden and the European Commission Delegation on behalf of the donors, and stressed the need for action. 8. The latest version of the SDS, published in March 2008, gives a broad programme for the whole national statistical system for the five years 2008-12, but it divides the period into two parts, with the first part 2008-09 mainly devoted to completion of the institutional reform and to the reinforcement of human resources so urgently required. The focus in these early years should therefore be on introduction of the new legal status for the central institution, implementation of effective coordination, planning of human resources and requisite training, and publication of agreed standards and definitions in each field of work to help overcome the problem of poor quality of data in some areas and consequent hesitancy to rely on any of it. The intention is nonetheless also to maintain a significant level of statistical output. The largest single item would be the national population census, scheduled for April 2009 but, as of August 2008, still with only a small part of the requisite financing committed. MAPS Contributions 9. Despite the small progress achieved in resolving the major managerial problems of the country’s national statistical system, the sector manager – the DNSI Director – is highly appreciative of all the MAPS partner agency efforts and considers that they have made valuable contributions in the slow building up of capacities that has been the most the country could absorb. 131 10. Mali has received extensive technical assistance for the development of its educational management information system over the last twenty years, first from US AID, then from the European Commission, and most recently from the Agence Française de Développement. But the UIS assistance has been a very valuable complement because it has helped the country align its definitions and practices with those of other countries, so as to make its numbers internationally comparable. UIS staff have given the quite specialized training and continued advice to the core team that have been required, and they have helped implement more efficient systems for carrying out the annual school survey. The work on education expenditures from different sources, and that on school management which has been organized by UNESCO’s Dakar office, are both felt to have potential for improving Mali’s policies and practices. But an expensive nationwide schools survey such as US AID sponsored in the mid- 1990s to clarify expenditures may be the only way to make progress on a broad front of the country’s educational establishments. 11. DNSI organized wide review of the IHSN and ADP initiatives and attracted strong interest. As many as 10 different statistical bodies – mostly the CPSs serving different ministries – had participants in the training sessions on the IHSN toolkit that were given in June 2007 in Bamako. More than 50 surveys, many of which had been very little used, were identified as warranting documentation and archiving. Work has now been completed on more than 30 and they are expected to be placed on a NADA (National Data Archive) before the end of the year. Mali is one of the first countries to host ADP Task 2 activities, improvement of new surveys. The work is focused on the Enquête Agricole Conjoncture (EAC), which is the country’s principal source of information on agricultural activity and has been carried out every year since 1992. An ADP-recruited specialist spent a month earlier this year working mainly with the Rural Development CPS, especially on improving survey design and clarifying questions in order to reduce scope for divergent interpretations. 12. DNSI staff participated extensively in the workshops organized by the UN Statistics Division for preparation, and review of drafts, of the revised manual on the planning and reporting of population censuses. They found this work, the manual that emerged (UNP&R2), and the regional seminars on selected themes that have since been held by UNSD, all to be most valuable. They expect to rely mainly on the tools and advice that have been provided from this source in the implementation of Mali’s forthcoming census, and analysis and dissemination of the findings and data gathered. The timely availability of such an up-to-date and comprehensive manual should have a very tangible effect of raising average quality of censuses in the current round and improving comparability. 13. Each of the above-mentioned initiatives has had visible effect on the capacities of one or more parts of Mali’s national statistical system. That effect should be sustainable because it is reflected in tools, procedures, and software that are being increasingly widely used, and the knowledge has spread beyond the few direct participants in the first round of training. It has been harder to get full potential advantage from the Paris 21 efforts in promotion of advocacy and planning because of differences in the phasing of interest (Mali was, in a way, ahead of Paris 21 in recognizing the importance of system-wide strategic planning) and, probably more important, the unreceptive and unresponsive political environment in the country on the particular subject of statistics and objective monitoring. 132 14. The wider contacts that Paris 21 has given to managers of statistical work – through enabling participation for the first time in US Statistical Commission activities, for instance, as well as by invitations to regional seminars with stimulating speakers – have broadened their views and understanding and therefore still helped. Ideas gained from those gatherings as well as from the visits of experienced Paris 21 staff and consultants have been useful for the various revisions made of the development strategy and for the initiatives undertaken in the areas where it was possible to move forward, such as coordination among the statistics-producing bodies. Mali has also much appreciated Paris 21 help on attracting interest from other potential partners, and especially the work in presenting, translating and printing varieties of custom-made publicity and advocacy material about Mali’s statistics. Strengthening the MAPS Contribution 15. The story in this particular case is one of seemingly high need for the sorts of inputs that the MAPS initiative was designed to provide, but low actual effectiveness of interventions at other than a relatively low technical level. The case cries out for concrete suggestions of steps that might have been more effective in contributing to successful growth of statistical services to support development. But the continuing high uncertainty as to whether the long-proposed reform measures are now on the way to effective implementation, combined with the brevity of the MAPS evaluator’s visit and the very limited range of people he was able to see prevent us from putting forward suggestions with any high degree of confidence. 16. Major factors in the inability to move forward seem to have been lack of strong conviction at the political level about the importance of obtaining more reliable flows of statistical data and/or about the feasibility or appropriateness, in Mali’s circumstances, of the executive-agency solution for the main statistics body – as adopted in the large majority of Francophone countries of Sub-Saharan Africa. Change of views at the political level on these issues would seem to be the key ingredients to restoring the self- confidence of the country’s statisticians and the attractiveness of a career in the field such that students doing well in secondary school would be ready to devote their higher studies to this subject. 17. Dealing with such issues of political conviction and will might be considered preferably the province of political or technical leaders of aid agencies or their departments. The likelihood is that several such attempts were made, but that they were politely rejected or anyway not answered with any effective follow-up action within a reasonable period of time. 18. In this situation the appropriate action for the MAPS consortium or one of its members might have been to decide which of the two issues mentioned in para. 16 was the more important block to political action. There are some indications that a major worry at the highest levels at least at certain junctures was that conversion of DNSI to being an autonomous public agency was inconsistent with Malian law or constitution in ways that did not arise in the other Francophone countries. The argument seems at least no longer to be accepted since no references appear to be made to it when the same solution has been proposed in the last two years. But also there is no evidence that the worry was received with sufficient sympathy to justify the kind of rapid but serious 133 research which might have quickly disposed of this obstacle and won additional voices for action. 19. Lack of conviction about the real utility of better statistics for better management of development and poverty reduction, the other likely obstacle at the political level, is a matter that Paris 21 made major efforts to deal with through regional seminars and publications, especially in 2002-04. If this issue were to have been identified as still the main obstacle to broader progress in the particular case of Mali, it might have been worth considering a renewed and focused special effort, drawing on the materials and experience accumulated over the preceding years. 20. The other alternative would have been to accept that Mali’s political circles seemed to have a special hang-up about statistics, which had to be overcome before any major capacity-building initiative could be expected to take root and bear sustained fruit. It would then be reasonable to propose to the government to postpone consideration of any further such initiative in the statistics field at least two years, pending execution of a quite intensive training programme for the younger, policy- related staff in the offices of the main economic/social ministers and their senior civil servants. 21. This training programme would be focused specifically on interpretation of sets of statistics relating to public programmes, identification of causal links and of possible chains of steps toward improvement of performance or solution of problems. A probably foreign professor, experienced in this particular field, would likely need to play a large role in a course concentrated in one- or two-week bites but spread out every month or two over a nine- or twelve-month period. Besides helping to fill a widely perceived gap – in government capacities to make good use of statistics for practical policy-making – the intention of the programme would be to cause members of the government to modify their views and hence create an environment conducive to more effective development of statistical services. 22. One further step that would clearly have warranted more attention even with the posited two- or three-year hiatus in substantial progress on statistics – as also with the longer delay that Mali has in fact inflicted upon itself – would have been maintenance of steady arrangements for training of statisticians. There are strong indications that, equally at lower technical levels (which used to be covered by training courses available within the country) as at the higher levels (aspirants to which would normally undertake 2-4 year training at Abidjan, Dakar or Yaoundé), Mali is lagging seriously behind neighbours in the share of demand for statisticians it is able to meet. A significant reason for low Malian participation at the schools in neighbouring countries is failure to score sufficiently well in the entrance exams, which is attributed in turn to deterioration of secondary education quality standards, probably exacerbated by decreasing ability of Statistics to attract better-performing students, as already explained. It will therefore now, unfortunately, take at least several years to recover the lost ground. 134 Annex 6 Niger: experience with MAPS 1. Official statistical services grew quite steadily in the 1970s and 1980s, although from a very small base. They shared fully in the difficulties that the country and its public services suffered in the 1990s, as a result of the prolonged low price of uranium, recurrent droughts and repeated political crises. But since the successful democratic elections of 1999 they have rebounded strongly in scale and production, responding well to the rapidly growing information needs of government programmes to accelerate development. 2. The new government launched preparation of a Poverty Reduction Strategy Paper (PRSP) in February 2001. A well-ordered process that included extensive consultation with the country’s different regions resulted in a document that government was able to adopt already the following January and use for managing development over the subsequent years. But the experience also brought to light the lags that had accumulated in many fields of statistical work (especially outside the core economic areas) and the uncertainties surrounding poverty estimates and welfare indicators such as those included in the MDGs. 3. In early December 2001 a few of the country’s leading statisticians organized a three-day workshop, under sponsorship of the technical body in the Prime Minister’s Office that is responsible for the PRSP (Permanent Secretariat of the Poverty Reduction Strategy) for producers and users of statistics to review best ways to respond to the new needs. Consensus was reached on recommendations to convert the country’s main statistical body into an autonomous executive agency, to develop and implement an overall plan for statistics development including multi-year programmes, and to strengthen user-producer interactions. A new statistics law was envisaged, as well as a large expansion of resources. Proposals were also made regarding best ways for strengthening monitoring and evaluation of the PRSP. 4. The main recommendations seem to have attracted strong support from the key Government members and civil servants concerned. The recommendations were also generally favoured by Niger’s technical and financial partners, especially the European Commission, UNDP and the World Bank. IDA credit conditionalities of the period sometimes included references to measures related to the statistics reforms, usually at the request of Nigerien parties concerned, to help things move forward. In the event, the key reform law was passed by the National Assembly little more than two years after the workshop, in March 2004. 5. The law effectively converted the long-standing Direction de la Statistique et des Comptes Nationaux (DSCN) within the Finance Ministry into an autonomous agency, called Institut National de la Statistique (INS). It would be the secretariat for the slightly renamed Conseil National de la Statistique (CNS) which would be the policy-setting body for all parts of the statistical system, including, besides INS, the units responsible for gathering and issuing statistics in Ministries and public enterprises. The law also clarified important basic principles of official statistical work. Implementing decrees, for establishment of INS and CNS were issued in September 2004. 135 6. The two top managers of the new body – Directeur Général and Secrétaire Général – have to be appointed by decree of the Council of Ministers upon recommendation of the Minister of Finance. The present incumbents were appointed in August 2005. In addition to taking over day-to-day management responsibilities (the new Director-General came from BCEAO), a major focus of early attention was planning the process of preparing the national statistical development strategy. The main lines of the widely participatory process envisaged were outlined in a draft route map completed in October. Work continued, to refine it and to gather the financial support needed for the work. 7. In January 2006 the Director General announced the staff members he had chosen for all the managerial positions of the new Institute, and it could begin to work more formally. By June, the Minister of Finance, ex-officio Chairman of the CNS, had reached agreement with the Ministries and professional associations to be represented, on their (very predominantly, non-statistician) candidates for membership of the new Conseil National. He issued an official list of all 44 names. INS completed a revised and more detailed road map for strategy preparation, reflecting the financial contributions promised by five foreign agencies as well as from the government budget. 8. The first major step in the process of strategy formulation was execution of the extensive training programme that was designed to prepare the CNS members for their roles in the six committees that would guide, review and approve the work of the staff and consultants working on diagnosis, vision, strategies and action plans in each area. One training week was devoted to planning, and a second to management for results, in both cases animated by Nigerien consultants, with attendance of all 44 members. The committees worked intensively through the later part of 2006 and first half of 2007, reporting as each phase was completed to the full Conseil National, which in turn kept the Council of Ministers informed at important stages. 9. Progress was also made through this period on several other structural initiatives – in addition to maintaining a substantial statistical output of improving quality, and executing several major surveys. Small units were set up within INS, to deepen its analysis of the statistics produced, especially those relevant to poverty: Unité d’Analyse Avancée de la Pauvreté (UAAP, assisted by Canada) and Observatoire Nationale de la Pauvreté et du Développement Humain Durable (ONAPAD, assisted by UNDP). Early in 2007 a Schéma Directeur Informatique was developed and began to be implemented with EU support, including upgrading of INS’s own web-site and development of an INS Intranet, to be gradually extended to incorporate the statistical directorates in the Ministries. 10. Support for government efforts to strengthen local authorities and encourage more local initiative has been another important theme. NigerInfo, based on UNICEF’s ChildInfo software and filing data with GIS coordinates, was started in 2006, to help meet rapidly growing needs for data relevant to local planning. In 2007 the government decided that to strengthen local capacities in the collection and use of statistics, the one or two persons concerned with statistics in the Governor’s office of each of the country’s rural regions should be replaced with a regional statistical directorate that would be a branch of INS. The first three were started in November, and the other four 136 are planned for establishment in 2009, to facilitate execution of the national population census scheduled for 2011. 11. Niger’s first national Statistical Development Strategy was brought to completion in October 2007 with its formal approval at a plenary session of the Conseil National de la Statistique. It had been prepared in the same period as Niger’s second poverty reduction strategy, with close coordination on plans for monitoring and evaluation of both strategies, which cover the same five-year period 2008-12. The statistics strategy came officially into force with its endorsement by the Council of Ministers in January 2008. Most of the relatively large number of surveys (some 12) that had been included in the Plan for INS in 2008 already had firm arrangements for financing, mainly from foreign sources, and they have been proceeding reasonably close to plans. 12. Overall spending for the first year of the statistics plan is likely however to fall significantly short of hopes, probably by 25% and possibly by more. That appears to reflect weaker planning capacities on the part of many of the statistics directorates or units in the line Ministries and large shortfalls on the ambitious plans included for training of statisticians. The longer than expected time required for strategy preparation has exacerbated both problems because it has delayed the date of a donor roundtable specifically on statistics from December 2007 to the currently envisaged December 2008. This meeting will need to make major progress towards identifying a realistic multi-year financial envelope which could show where choices have to be made and how to maintain sensible balance among different resources and purposes. MAPS Contributions 13. Contributions from the MAPS family of international organizations to this very promising development record have been to date mainly from Paris 21’s general promotional function, from Accelerated Data Programme (ADP) support on Surveys, and from the UNESCO Institute of Statistics’ assistance in building capacities for generating education statistics. Niger’s work on its 2011 census will no doubt benefit from the major manual and the regional seminars on key census-related issues that have been developed by the UNSD. 14. As regards the National Statistical Development Strategy, it is likely that it would have been produced whether or not there was any international initiative or outside help. The strong revival of statistical development in Niger since 2000 has been aggressively led by very capable Nigerien professionals. The 2001 consensus on the need for a strategy (para. 3 above) largely predated Paris 21’s emphasis on such an instrument. And the strategy itself was developed and written virtually entirely by nationals. 15. On the other hand, it seems very probable that wider receptivity for the statisticians’ planning efforts, as well as the quality of those efforts, have benefited significantly from Paris 21 advocacy work, publications, regional seminars and personal interactions. The interest in reliable statistics that is widespread at high levels of the government is reported to benefit significantly from the leadership of President Tandja since his original election in 1999, but Paris 21 efforts make available extensive supportive information from international experience. 137 16. By the time that active planning began for preparation of a Statistical Development Strategy, Paris 21 had well started its regional workshops on the subject and produced various materials. Niger’s route map incorporates well thought-out proposals on ways that would be appropriate to national circumstances for dealing with key issues that had been highlighted by Paris 21 such as coverage of the national system as a whole, putting lead responsibility in the hands of a widely representative and politically well-founded council, stronger user-producer dialogue to reach consensus on needs, proper treatment of management issues (three of the committees) as well as sector needs (the other three committees), dividing the work into clear stages with provision to keep political circles informed of the progress of the work, clear prioritisation among improvements recommended, and design of arrangements for systematic and timely annual monitoring of achievements against plans. 17. The strategic plan that has emerged gives a convincing presentation of the progress that has been achieved in the different areas of statistics, the further issues that now need to be resolved, and a broadly prioritised plan of actions to achieve as much improvement as possible in the coming years. Much attention is appropriately devoted to training needs. Paris 21 is giving strong support to preparations for the donor roundtable in December. The plan appears to enjoy wide ownership not only among statisticians but also among senior government officials and political leaders. Consolidated figures on finances of all parts of the statistical system are still unavailable, but government has increased budget funding of INS very substantially over the last five years. 18. Niger was one of the earliest countries to participate in ADP activities. It solicited help in implementation of the third National Household Income and Expenditure Survey which the government had just decided to finance itself (at a total cost of some $2 million equivalent), since the Poverty Reduction Strategy Secretariat attached high priority to the findings but no donor had shown interest in contributing to the costs. ADP provided international technical assistance on sampling design and practice, on design of the questionnaire and related materials, and on data quality controls. The experience has already carried over to benefit the preparation of other surveys in Niger. ADP also financed purchase of laptops to enable direct data entry and a first data quality control in the field. The data gathered are now in process of being assembled and analysed. 19. Thanks to the tools developed by IHSN and two workshops delivered by ADP in Niamey, Niger has been able to recover, document and archive some 15 past surveys and to learn the techniques so as to continue the work on its own. Results already achieved mean that these surveys are now much more accessible for interested researchers than they were before. The workshops included participants from sector- ministry statistics directorates as well as INS staff. Niger expects to develop a national data archive (NADA) in the coming months to store survey meta- and micro-data. It needs to accelerate development of domestic ability to manage and further develop the tools, independent of outsiders. 20. Statistical coverage of the lower grades of education, and especially of primary schooling, has greatly improved over the last five years as a result of special efforts by UIS (especially its staff in UNESCO’s Dakar office), with the aid of financing from the 138 European Commission and others. The definitions and concepts used in Niger have been brought into line with international practice, and intensive training, organizational and equipment assistance have been given for development of sound reporting from schools in the more decentralized environment the country has been developing. Most important for Niger has been the contribution better and more up-to-date data have made to planning of new schools and distribution of teachers. A survey of adult literacy levels under UIS’s Learning Achievements Measurement Programme (LAMP) is planned for execution in 2008-09. 21. Much further work remains to be done in Education, most importantly to deepen understanding of the factors affecting learning outcomes at primary level and ensure progress on this dimension in addition to enrolments. Limited progress seems to have been made so far on application of UIS’s education expenditure survey, establishing the relative importance of different sources, public and private, for covering the costs of different services. The improvements in the coverage and quality of education statistics need to be reinforced at the secondary level and extended to higher levels. Broader Results from Better Statistics 22. The specific initiatives of the MAPS partners that have impinged on Niger are too recent to have brought about broader development decisions (such as major investments or changes in public policies) which could be clearly attributed to them, in part or in whole. But the increased attention that has been given since 2000 to better management of development and faster reduction of poverty, and to good statistics as a tool to these ends, has generated examples of the ways better statistics can help bring about better patterns of development. 23. Even with its improved development performance in the last few years Niger remains in danger of falling short in 2015 on virtually all the MDG targets. This reflects mainly the country’s very limited resource base, its low starting point and the multiple problems of the 1990s. Taking however the MDG indicators for which valid figures were available at the time of preparing the country’s first poverty reduction strategy, in 2001, it is noteworthy that progress by 2006 had exceeded the short-run targets chosen in 2001 for most; only in the case of maternal mortality and primary school completion were there important shortfalls. It is much to be hoped that the sounder information that has become available for indicators which could not be targeted in 2001 will help the short-run targets now chosen for them also to be beaten. 24. The most important effect on Niger’s overall development from recent statistical work may well turn out to be the revision of agricultural development strategy that resulted from the general agricultural and livestock census carried out in 2004-06. This was the first such census for more than twenty years, and the largest single statistical activity of the period, financed almost entirely by the European Commission. It revealed much wider spread of cultivated area than had previously been appreciated, greater rain-fed cultivation, and estimates of overall production significantly above earlier ones. It has been a major factor in shifting focus of future strategy making to small-scale, often privately financed irrigation, exploiting the various types of surface and underground water resources relatively reliably available in the country’s different regions. 139 25. Widespread malnutrition, the sensitivity of food supplies to climatic conditions and pests, and sharp year-to-year variations, have long been serious issues in Niger, subject to extensive statistical and agronomic research with international participation. Early-warning systems in place sounded the alarm toward the end of 2004, but international response possibilities were overwhelmed by the Asian tsunami at the end of December. 26. Several measures have been taken to avoid recurrence of the serious food shortages that hit major parts of most regions of the country in 2005. One important one, sponsored principally by UNICEF and the World Food Programme and carried out by INS, is an annual sample survey of Nutrition and Child Survival, dealing with malnutrition measurement, child feeding practices, and food availability. These surveys still show high rates of malnutrition compared with most other countries, but a substantial improvement. Acute malnutrition of children under 5 fell from a national average of as much as 15% in October 2005 to 11% in October 2007, with no region any longer calling for emergency attention. Chronic malnutrition has fallen from 50% to 36.5%. The improvements reflect changes in child feeding practices as well as maintenance of more stable food supplies, and better use of emergency facilities for helping critically undernourished children. Factors Underlying Success 27. Several factors seem to underlie successful revival of statistics in Niger in the current decade, the promise of further benefits from the reforms carried out, and the environment that was created for making good use of MAPS partners’ contributions. 28. Of prime importance was probably the understanding, among the country’s political leaders in power in this period, of the importance of good management as such, and of sound statistical services as a vital tool for this purpose. This orientation was probably reinforced by the very fact that these leaders were taking over from a lost decade, so that electorate interest in economic improvement was unusually strong. The simultaneous introduction from the international level of the results-focused PRSP mechanism worked in the same direction. 29. Recognition by the political leaders of the advantages, in the prevailing situation of the country’s public service, of establishing the core entity as an executive agency, free to hire and fire staff and to compete for contracts, with a salary scale not limited by civil service rules, seems to have been an important factor in attracting, retaining and motivating many of the best staff. We did not hear of this proposal having been strongly opposed nor of its causing great difficulties for the ministries in their hiring of statisticians on civil service terms, or otherwise complicating relations between INS and the other units within the national statistical system. 30. Open and competitive selection among the candidates for the top managerial posts was used to recruit statistical professionals of exceptional quality and vision, with strong managerial capacities, who have provided active and hands-on leadership and delicately developed the important coordination role with the main sectoral statistical units elsewhere in the government. 140 31. Quite strong and flexible support has been provided by foreign assistance agencies, notably by the European Commission, but also others. They were responding to the generally sound quality and high reliability of INS. They have also helped meet the urgent needs of the social sectors which were further increased by channelling of substantial volumes of aid through sectoral budgetary support. Strengthening the MAPS Contribution 32. From the documents reviewed and discussions held a few suggestions emerge for further or adjusted actions that might be able to be undertaken by MAPS partners agencies to further increase the development results to be obtained from statistics. 33. Better solutions need to be found to the problems of finding financing for the costs (especially the foreign costs) of training of statistical staff. INS faces a particularly large training need at this point because of its increased responsibility, within the framework of the national statistical system as a whole, for assuring qualified staffing of the statistical directorates and units in the sector ministries. In reflection of these needs, the NSDS proposed a large programme of training at different levels in 2008. It has been possible to carry out only a small part of it, and the fifteen or more students who have either already started study at ENSEA (Abidjan), ENSAE (Dakar) or ISSEA or IFORD (Yaoundé), or won entry this year, all remained, as of September 1 2008, without scholarships to enable them actually to attend these schools. 34. There seems to be an increasing imbalance between statistical production (quantity and quality) in Niger and the decision/action-oriented analysis that is made of the figures. INS, with help from Canada and UNDP, has made efforts to strengthen its own analyses of some of the data gathered, and it envisages organizing further training in the area. More attention needs to be given to ways of increasing applied analysis capacities of younger staff in the policy offices of main ministries and their ability to move through diagnosis to tracing out possible solutions. An appropriate series of workshops, with intervening homework assignments, might be organized at the national Université Abdou Moumouni in Niamey, led by a (probably foreign) professor with particular experience in analysing statistics to understand a situation and identify possible public-policy solutions, assisted by Nigerien professors interested in the same function. 35. The link between the demonstrated performance of government departments, programmes and managers and the budgets that they receive for the following period appears to remain quite weak by comparison with leading countries in East Africa, although it is beginning to get more attention in the ministries working with a three-year MTEF (Medium-Term Expenditure Framework). More attention needs to be given to the potentials for increasing efficiency, output and results of programmes by means of this sort and to the processes for gathering the relevant and reliable statistics in a timely way to affect budgets made available. Statistics concerned will be principally those on output of specific types of public services, on use of the services by the population, and on relevant levels of welfare, health or education attained. 36. Niger’s statistical strategy emphasizes the need to reduce reliance on surveys by developing stronger administrative data systems, and INS has already started a pilot programme in systematic civil registration of vital events: births, marriages, and deaths. 141 The strategy also recognizes the high up-front burden and cost of reinforcing administrative data systems. The problem warrants more attention and emphasis from interested aid agencies. 37. Full value from health statistics depends on links with data from many other sources (e.g., demographic, transport, food supply), as in the case of most sectors. The Health Metrics Network is also already involved directly in Paris 21 and the IHSN. WHO-HMN should now become a full MAPS partner. This would improve the correspondence between the coverage of Niger’s SSN and that of MAPS and help to ensure consistency and mutual support between IDA lending support for health statistics development (Health Sector Institutional Reinforcement and Support Project for which a credit was approved in 2005) and MAPS activities. 38. Programmes of technical assistance for building statistical capacities, such as those of ADP and UIS, quite often involve transfer of computer models and of the skill to use them. Problems seem rather commonly to arise in the more advanced uses of the models, and especially when the need develops for introduction of small modifications or amendments to model structure. Careful consideration should be given to the possibilities of training recipients to standards of full self-sufficiency, at least to the level of making small modifications in structure, and of ensuring the availability of some reasonably convenient and inexpensive back-up service in the event of need. 142 Annex 7 Assessment by UIS of Some Main Developments in its Work over the Last Few Years For MAPS DGF evaluation of the Programme of Education Statistics: Additional Inputs, Part I 1. Performance indicators: Timeliness and Coverage of Primary Enrolment Data by Main World Regions (20035 to 2006) Figure 1 shows the percentage of countries reporting enrolment data for the reference years requested in the 2004 and 2007 surveys. The percentage of countries providing data for the requested reference year, or more recent data appears to have decreased during the period 2003-2006. However, it is important to note that the UIS made a policy change in that it would no longer estimate various education indicators (based on previous data and current population data) when countries are not providing timely data – as an incentive to encourage these countries to provide data. Thus, this effort to improve data quality comes at the expense of coverage, as reflected by these indicators. For comparison purposes, the last column includes those countries which would have previously been estimated in order to demonstrate change over time. If the UIS had provided estimates for these countries, the global percentage of countries providing timely data (i.e., for the requested reference year or data one year ahead of schedule) would have increased during the period from 85% to 87% globally. It is notable that improvements in coverage were not observed in all regions. While increases in the percentage of countries reporting timely data increased in East Asia and the Pacific and Latin America and the Caribbean regions (approximately 10 and 5 percentage points respectively), the percentage of countries providing timely data decreased in the Arab States, Central and Eastern Europe by 4-5 percentage points and by almost 9 percentage points in South and West Asia. Figure 1: Percentage of countries reporting timely data for the reference years requested in UIS surveys 2004 (2002/2003) and 2007 (2005/2006) Reference year 2003 Reference year 2006 (without estimates) Reference year 2006 (if estimates had been done) 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Arab States Central and Central Asia East Asia Latin North South and Sub- World Eastern and the America America West Asia Saharan Europe Pacific and the and Africa Caribbean Western Europe 5 Reference years are designated as follows : 2003 (2002/03); 2004 (2003/04); 2005 (2004/05); 2006(2005/06); 2007 (2006/07) – this includes countries for which the school year cuts across two calendar years and for countries where the school year takes place within a calendar year. 143 Source: UIS, 2008 To obtain a clearer picture of the change over time regarding timeliness, Figure 2 shows the percentage of countries reporting data ahead and behind the reference year during the period 2003 to 2006. Globally, the percentage of countries reporting data ahead of the reference year almost doubled from 4% to 8% between the two surveys. Increases were particularly high in Central Asia (22 percentage points), East Asia and the Pacific (9 percentage points) and to a lesser extent Central and Eastern Europe and sub- Saharan Africa (5 and 4 percentage points respectively). While more countries are providing data ahead of the reference year, the percentage of countries reporting data behind the reference year changed less – decreasing slightly by half a percentage point to 7%. Rather decreases seem to be regionally specific to East Asia and the Pacific, Latin America and the Caribbean and sub-Saharan Africa. In contrast increases were reported in South and West Asia and in the Arab States. Figure 2: Percentages of countries reporting data ahead and behind the reference years 2003 and 2006 Data reported ahead of reference year 2003 Data reported ahead of reference year 2006 Data reported later than reference year 2003 Data reported later than reference year 2006 25% 20% 15% 10% 5% na na na na na na na na na 0% Arab Central Central East Asia Latin North South and Sub- World States and Asia and the America America West Asia Saharan Eastern Pacific and the and Africa Europe Caribbean Western Europe Source: UIS, 2008 Notes: 1) Data labeled as reported later than the reference later includes data that is 1, 2 and 3 years post reference year, while data ahead of the reference year represents a one year discrepancy. 2) Reference year 2002/2003 data was collected in the 2004 survey, while reference year 2005/2006 data was collected in the 2007 survey. Finally, Figure 3 demonstrates the percentage of countries that did not report any data from the 2004 and 2007 surveys. Globally, the percentage decreased from 8% to 6%. However, the changes are often more impressive at the regional level. For example, between the two survey periods, the percentage of countries in East Asia and the Pacific decreased by half (i.e., from 18% to 9%) while decreases also occurred in Latin America and the Caribbean and in South and West Asia. In contrast some increases were seen in Central and Eastern Europe and in sub-Saharan Africa. 144 Figure 3: Percentage of countries for which no entry could be included in the UIS surveys 2004 (Reference year 2002/2003) and 2007 (Reference year 2005/2006) due to lack either of raw education data or of useable population figures for the denominator 2002/2003 2005/2006 20% 18% 16% 14% 12% 10% 8% 6% 4% 2% 0% Arab Central Central East Asia Latin North South and Sub- World States and Asia and the America America West Asia Saharan Eastern Pacific and the and Africa Europe Caribbean Western Europe Source: UIS, 2008 Notes: Reference year 2003 data was collected in the 2004 survey, while reference year 2006 data were collected in the 2007 survey. List of countries with no publishable data for any year (indicator=Primary GER) 2004 2007 Comments Antigua and Barbuda Antigua and Barbuda Bosnia and Herzegovina Bosnia and Herzegovina Bhutan Cayman Islands Cook Islands Democratic People's Republic of DPR Korea Korea DR Congo Haiti Haiti Country submit data but inconsistencies with the population estimates Micronesia (Fed States) Monaco Monaco Montenegro Part of former Serbia and Montenegro State Nauru San Marino San Marino Singapore Singapore Country started submitting data recently but inconsistencies with the population estimates Somalia Somalia Tokelau Turkmenistan Turkmenistan 2. Country profiles – what impact is attributable to UIS in the field? The Case of Niger Since 2001, education in Niger has been managed by two ministries (MEBA and MESSRT6). In 2001, MEBA organized a school census in primary schools which allowed the production of a statistical handbook. However the production was very ineffective as the delays were very long thus often rendering the statistics obsolete. Moreover, after the separation of the Minister for Education in 2001, the MESSRT did not set up an effective mechanism for data collection and processing. There was a lack of human, financial and organisational resources for the production of statistical data. In 2003, the UIS began providing significant support to the two ministries using a systemic approach to statistical capacity-building. With support from UIS, Niger set up a technical committee and carried out a wide-ranging diagnostic study that identified its statistical needs and capabilities in the field of education. The results led to the adoption of a data modelling approach developed by UIS to help in the collection, production and analysis of the statistics they needed. The statistical staff in the Ministry of Education were trained in this approach and they then developed a questionnaire that would provide key information on quality and access at the primary level. The goal of this project was to collect data on individual grades, teachers and classrooms for a sample of schools. Information on teachers, resources and classroom conditions were linked analytically to enrolment by age and gender. These data allowed planners to analyze relationships between variables and gain insights into factors influencing key issues such as school progression and gender parity. In early 2004, school directors were trained in completion of the questionnaires, and within a month, the data were recorded, validated and submitted to the Ministry. A school census yearbook for the 2003-2004 academic year was released in May 2004, and the data were used for the first time to guide planning for the following school year – a first for Niger. An important distinguishing characteristic of the Niger case is that, unlike the many countries where our diagnostic studies have shown continued dependence on development partners for covering the costs of the education data production chain, the Niger government has decided to finance the statistics from the State Budget (following the UIS recommendations). UIS support in Niger since the end of 2003 contributed to the improvement of the country’s education statistics, resulting for example in reductions in the time needed to produce data, improved quality of the data analysis, and improved access to data. Niger is working to improve its data collection for secondary, technical and vocational schools as well as at the higher education level. Meanwhile, an in-depth review of sources of education finance data is under way in the Ministries of Finance and Education and at local government offices. This review is looking at both budgetary issues and development of methodology and procedures for producing relevant data. UIS is providing training and mentoring for these analyses. The Case of Guinea For Guinea, as well as for other UIS SCB countries, the starting point has been to establish a national technical committee in charge of follow-up to the project which includes a diagnostic of the statistical data production chain. This was conducted in November 2003 using the UIS methodology and systemic approach. The diagnostic, carried out in collaboration with the national technical committee, addressed information system and data quality deficiencies as it had been demonstrated that Guinea needed more reliable data in order to better manage, monitor and plan education. The diagnostic had also shown that a systematic and systemic approach for collection, production, dissemination and use of the information was needed. Finally, the diagnostic also highlighted the critical mass of competencies and expertise within the Ministry as needing to play a crucial institutional role in achieving these goals. An action plan was elaborated in 2004 to address weaknesses of the system. Activities involved redesigning tools for data collection, processing and analysis. It also included the training of staffs involved in the data collection process at the central level and sub-national levels, as well as the heads of school responsible for the filling of the national questionnaire, which is crucial to improve the quality of data collected at the school level. 6 MESSRT : Ministère des Enseignements Secondaire et Supérieur, de la Recherche et de la Technologie 146 Four years after the implementation of the SCB programme, some good results have been achieved at the technical level as well as at the institutional and organizational levels. ƒ The sub-national staffs in charge of statistics are fully involved in the training on the tools that have been developed; these staffs are able to produce the sub-national yearbook. ƒ In 2005 the pre-primary sector has produced their first yearbook while in 2006, for the first time, higher education and the non formal education sectors have produced their statistical yearbook. ƒ In 2008, for the first time, a dashboard has been produced for higher education. ƒ The Methodology of DQAF has been discussed with the national technical committee. The tools for use at the country level are under development and Guinea will be one of the piloting countries ƒ Amongst other things, a lack of IT human resources, had been noticed during the diagnostic. This issue had been discussed with the education authorities and the World Bank as well. As a result, two IT persons have been recruited in order to easily provide Guinea with ownership and skills transfer and therefore the sustainability of their statistical information system. ƒ In the other hand, the broad composition of the technical committee (staffs from various ministries involved in the education sector, the ministry of finance and the NSO) had allowed a better collaboration between data producers. ƒ As in Niger the development partners have been supportive of the programme. Some have brought financial support for the training of staff by the UIS. This is the case with UNICEF who has financed the training of two staffs from pre-primary education involved in data collection, processing and analysis at UIS Dakar. LAC Region In the LAC region, the UIS has funded technical cooperation in selected countries with Regional Bureau/cluster offices funds (both regular and EXB) as part of the UIS/OREALC regional programme. In Latin America, the main countries supported by the UIS include 1) Honduras for the redefinition of their information system in education and 2) in El Salvador to define their educational indicators system. Asia-Pacific Region The UIS Regional Office in Asia-Pacific is also carrying out capacity-building programme activities in various countries within the region, particularly in Pakistan, Nepal, Sri Lanka and Viet Nam. As a result of a series of capacity building-trainings, a pool of national trainers is currently in place and a significant mass of national capacity has been built at the sub-national level. The government and local developmental partners are also now continuing to carry out these activities in their regular annual work plan leading toward building sustainable capacity. 3. Efficiencies and cost savings The UIS has introduced new software tools in order to improve data quality and overall internal efficiency. These new tools accomplish this by allowing for the reallocation of UIS resources towards more thorough early data verification and analyses. There are two main improvements: the i) Data Capture Infrastructure (DCI) to interface with countries in obtaining data, and ii) internally used tools (e.g., ERS, CALCINDIC, DIVA) to verify and analyse the data received. Data Capture Infrastructure (DCI) The DCI allows the UIS to present it’s constituents with an intelligent document solution to facilitate the capture, distribution and processing of survey information. The infrastructure consists of four main areas/components: Form Design, Form Delivery, Data Capture and Data Validation. The Electronic Lexicon (eLex) is a component of the DCI project that allows the forms to be based on one template, but support all the UN languages. DCI is template driven using a common technology for all surveys thus reducing the need for multiple skill-sets among staff, which in turn leads to less time being required for testing, programming and training. The template driven nature of DCI also allows for flexible redesign of surveys, which reduces the time needed to produce redesigned questionnaires. One of the major efficiency gains attributed to the DCI involves the process of developing forms from one template that can be materialized into various 147 languages and media. For example, each survey can be viewed as paper based, web-based or file-based electronic from one template and process. This eliminates the need to produce separate forms for different media and/or language. Questionnaires are easier to disseminate to users since there is no installation required on their part; and finally, the DCI allows for instantaneous acknowledgement of receipt to respondents. This reduces the time between country response and UIS acknowledgment. Use of the DCI in 2007 has resulted in the UIS being able to produce questionnaires for 8 surveys (instead of 1) without increasing resources. Error Reporting System (ERS) Based on pre-defined rules, the Error Reporting System (ERS) is used to provide reports on inconsistencies and errors in the raw data. Prior to the use of the ERS, Excel templates for each questionnaire needed to be re-created every year even when there were only minor changes since the templates were layout-dependent. The ERS is however based on concepts and is not dependent on questionnaire design layout and therefore can be easily adapted to add new concepts. As the concepts are defined just once, the time spent by professional staff to create and update these is greatly reduced. The use of ERS has resulted in a decrease in the time necessary for processing data. The reports are updated constantly and require less manipulation or intervention from users or the need for copying and pasting data from different sources. An additional feature of the ERS includes the fact that multiple years of data can be verified adding value to the historical series. Using the ERS, managers can identify errors or qualify the data at an earlier stage since reports are flexible. Moreover, the ERS provides better quality reports and information on each stage of data processing. Finally, the ERS can easily be extended to other surveys. A conservative cost estimate to create templates and test them—based on past experience for the 2007 Literacy questionnaire—is approximately 256 hours per year. In addition to these savings, cost savings in data editing were also achieved. Using the ERS instead of using the former tool (excel templates) roughly equals a savings of 400 hours per year. This was based on a controlled test run by a staff member on the 2007 Feature Film questionnaire. Combined, the savings of using the ERS can roughly be estimated at 94 person days per year. Calculate Indicators (CALCINDIC) The Calculate Indicators (CALCINDIC) tool is used to calculate indicators for the sectors. The system knows when raw data is modified, and runs automatically as required, independent of users; thus eliminating the need for large, time consuming batch runs, and user interaction. CALCINDIC provides added efficiency as the staff can react more quickly to user demands for new indicators since they can now program them without the support of Information technology staff. Finally, CALCINDIC also automatically creates documentation for indicators. In terms of savings, it previously took 4 hours for a large batch run of all indicators, years and countries, and just as much time to run regional indicators. Now since the system runs continuously there is no need for this. Furthermore, IT staff’s input was also considerable, and is now no longer required. Finally, the effort required to program indicators has been reduced from weeks to days. As an example, it took 3 days to program all the indicators for the Culture Survey on Broadcasting. This would have taken 3 weeks in the past. Data and Indicators Verification Application (DIVA): The Data and Indicators Validation Application (DIVA) is used to identify and provide reports on inconsistencies and errors in indicators, as defined by the subject matter personnel. With the use of DIVA, data indictors are checked more thoroughly, vigorously, and consistently according to pre-defined criteria. This enables the UIS to provide better quality indicators across all sectors and surveys.. The efficiency inherent to the use of DIVA is that indicators are checked earlier, giving UIS staff more time to react to inconsistent data and allows for earlier feedback to respondents. All data processing (entry, cleaning, estimations, validations and dissemination) processes are streamlined and more transparent. Furthermore, since concepts behind the error reports are defined once, the time spent by professional staff to create and update these is also greatly reduced. DIVA eliminates manual operations, 148 thus reducing the time spent extracting and formatting and allowing more time and focus for in depth analysis. Also, the method of defining the verifications is the same as the one used to define indicators. A conservative estimate of cost savings when statistical support staff use DIVA instead of using the former tool (Excel templates) is approximately 315 hours per year This is based on the following calculation: 35 hours X 9 indicator sets (WB, EFA, MDG, etc.) per year = 315 hours per year7. MAPS DGF evaluation of the Programme of Education Statistics Additional Inputs, Part II 1. Performance indicators: Timeliness and Coverage – Four Key Indicators, World and Sub- Saharan Africa (20038 to 2006) Figure 1 provides the response rate for all countries on four measures of coverage for the survey years 2004 to 2007: gross enrolment ratio in primary, net enrolment rate in primary, completion gross intake ratio to last grade, and gross enrolment ratio in secondary. There was an initial decline on all four indicators between the 2004 and 2005 surveys, which can be attributed to the fact that in the 2004 Survey, the split reference year 2002/2003 as well as the single 2002 reference year data were collected; while in subsequent years countries with a single reference year were reported along with the ending of the split reference year (2004 with 2003/2004), which led to actual improvement in timeliness. Figure 1: Number of countries reporting for four coverage indicators on time or 1 year ahead, Survey years, 2004 -2007 Countries reporting gross enrolment ratio in Countries reporting net enrolment rate in primary primary Total countries On time 1 year ahead Total countries On time 1 year ahead 162 162 162 151 180 185 188 188 136 129 130 158 154 154 119 136 17 8 6 6 10 9 9 9 2004 Survey 2005 Survey 2006 Survey 2007 Survey 2004 Survey 2005 Survey 2006 Survey 2007 Survey Countries reporting completion, gross intake ratio Countries reporting gross enrolment ratio in to last grade secondary Total countries On time 1 year ahead Total countries On time 1 year ahead 166 169 180 182 181 180 151 158 154 148 145 130 126 132 114 122 8 8 8 12 7 6 4 9 2004 Survey 2005 Survey 2006 Survey 2007 Survey 2004 Survey 2005 Survey 2006 Survey 2007 Survey Source: UIS, 2008 7 Based on page 2 of \\uissv6\uis\education\IT\task_force\DIVA\old versions\Why we need a DIVA_2.doc 8 Reference years are designated as follows : 2003 (2002/03); 2004 (2003/04); 2005 (2004/05); 2006(2005/06); 2007 (2006/07) – this includes countries for which the school year cuts across two calendar years and for countries where the school year takes place within a calendar year. 149 Notes: In the 2004 survey, data were collected for the school year beginning in 2002, meaning for split reference years 2002/2003 as well as for the single reference year 2002. In subsequent survey years, data are collected for the school year ending in the following manner: • 2004 for Survey 2005 = split reference years 2003/2004 and the single reference year 2004. • 2005 for Survey 2006 = split reference years 2004/2005 and the single reference year 2005. • 2006 for Survey 2007 = split reference years 2005/2006 and the single reference year 2006. The other main change in the data among the four indicators occurred in the Survey year 2007 where the number of countries reporting coverage indicators on time declined. While this may seem a matter of concern, there are several explanations for these temporary declines including i) the use of the new electronic questionnaire which created some challenges for some countries, ii) the complete reorganization of the UIS, which may have resulted in further constraints including human and organizational, and iii) the fact that for the 2007 Survey the UIS made the decision to no longer estimate indicators based on older data series; in the past these estimations were included in the “on-time” data9. At the same time it should also be recognized the number of countries reporting data at least one year ahead of schedule has increased, albeit slowly over time. Due to the policy decision not to estimate data this resulted in the loss of the following countries in 2007 Survey: Afghanistan, Guyana, Jamaica, Kiribati, Malta, Nigeria, Niue, The former Yugoslavia Rep. of Macedonia, Solomon Islands, Timor-Leste, Trinidad and Tobago, Turks and Caicos Islands, Yemen. In sub-Saharan Africa, the following countries were lost: Botswana, Chad, Comoros, Equatorial Guinea and Nigeria. Figure 2 provides response rates for the same four indicators in the sub-Saharan African region. Figure 2: Sub-Saharan African countries reporting for four coverage indicators on time or 1 year ahead, Survey years 2004 -2007 Gross enrolment ratio in primary Net enrolment rate in primary 41 40 41 39 34 33 33 33 33 30 26 28 Total 24 27 22 23 On time Total 1 year ahead On time 1 year ahead 6 4 4 3 3 4 5 3 2004 Srrvey 2005 Survey 2006 Survey 2007 Survey 2004 Srrvey 2005 Survey 2006 Survey 2007 Survey 9 These countries included Afghanistan, Botswana, Chad, Equatorial Guinea, Guyana, Jamaica, Kiribati, Malta, Nigeria, Niue, The former Yugoslavia Rep. of Macedonia, Solomon Islands, Timor-Leste, Trinidad and Tobago, Turks and Caicos Islands, Yemen 150 Completion, gross intake ratio to last grade of Gross enrolment ratio in secondary primary 38 37 34 35 38 39 37 36 27 32 25 25 29 26 Total 23 Total 18 On time On time 1 year ahead 1 year ahead 3 4 4 4 5 2 3 1 2004 Survey 2005 Survey 2006 Survey 2007 Survey 2004 Survey 2005 Survey 2006 Survey 2007 Survey Source: UIS, 2008 There are several data-related successful stories from sub-Saharan Africa including countries for which the UIS did country visits recently and countries started to submit data regularly. These include: ™ Site visit to Burkina Faso resulted in more timely data since 2005; Mali since 2005, Mozambique since 2004, Ethiopia since 2006, Lesotho since 2005, Niger since 2005; ™ Site visit to Kenya and Tanzania in 2004 resulted in completing all UIS questionnaires. Formal agreement reached on ISCED reporting for Kenya; ™ Technical assistance: Countries which received SCB activity started to produce data with enhanced EMIS: Sierra Leone since 2007, CAR since 2005. Other countries improved data coverage: Ethiopia, Ghana and Tanzania (these countries sent data to UIS one year ahead of UIS survey reference year), Guinea. In addition, at the regional workshops, there were countries which reported missing data and updated ISCED mappings during their participation. These include: ™ Gambia (2005), Marked improvements to 2006 data published by UIS following 3 years of poor data reporting ™ Mozambique (2005): Tertiary statistics reported during the workshop ™ Liberia (2007): Questionnaire A reported following the workshop. Overall improved communication with Liberia following 4 years of missing data. ™ Ethiopia (2007): Formal agreements reached on ISCED classification ™ Eritrea (2007): Finance data reported following the workshop ™ Madagascar (2006): Secondary TVE data submitted for the first time, and accepted after feed back and discussion with country. Publication for the first time of secondary figures and indicators ™ Rwanda (2006): Submission of finance questionnaire for the first time after the workshop and this year. ™ DRC (2006): Submission useful national figures during the workshop. Specific questions 1. How have partnerships (e.g. both funders and technical partners) increased since 2005? It is important to note that in 2005, the UIS was administered by an interim director, who basically oversaw the operations of the Institute without launching any new initiatives in terms of the Institute’s programme or fund-raising strategy. The current director, Hendrik van der Pol, arrived at the Institute in April 2006. Under his leadership, the UIS Medium-Term Strategy for 2008-2013 was formulated and then approved in 2007. With this strategy in place, the Institute has undertaken a more focused approach to secure a stable financial base by establishing partnerships with an expanded and consolidated base of donors. 151 The Medium-Term Strategy for 2008-2013 serves as the basis for a new proposal for core funding, which was submitted to several donors (namely the Governments of Canada, Finland, the Netherlands, Norway, Spain, Sweden and the United Kingdom) in 2008. To date, the UIS has secured new agreements or is in negotiations with all of these donors, with the exceptions of Finland and Spain. As previously mentioned, the aim is not simply to secure new funding but to establish and reinforce working partnerships with these donors. The outline of these partnerships will be more closely defined at the first UIS Donor meeting, which will be held in June 2008. Regionally, the UIS Regional Office has been instrumental in the setting up of the Nepal EFA pooled funds where supervision, evaluation and decisions are based on the Flash Report system introduced by the UIS Regional Office in Bangkok. The Flash Report System is a rapid monitoring reporting system based on input-process-output-outcomes benchmark indicators for the 6 EFA goals. In another example, the EFA Mid-Decade Assessment (MDA) process, led by the UIS Regional Office and UNESCO Bangkok has brought together donors and technical partners in supporting countries to carry out the assessment and prepare their national reports. Aside from Nepal—Indonesia and Sri Lanka are also good examples of countries where education donors worked together and shared funding costs as part of the MDA process. 2. How do we monitor compliance with ISCED standards? Is it sufficiently systematic? In 2007, the UIS revised its approach to ISCED mappings and now fields a stand-alone questionnaire as part of its annual data collection. This helps to produce ISCED mappings of national educational systems. These mappings are being published on the UIS website The UIS systematically validates the ISCED information provided by the countries. For example, prior to publication, all mappings are checked by the UIS staff responsible for the country as well as by the country itself. Furthermore, any changes proposed by the country are discussed with UIS staff to ensure compliance to the international standard (ISCED 97). While an ISCED specific questionnaire has been added to the UIS data collection, the use of bilateral discussions during workshops has been the preferred mechanism to verify compliance with ISCED 97 standards. 3. UIS’s Education Finance Work Current Work: The UIS has been expending considerable effort offering capacity building workshops for sub-Saharan Africa (both Anglophone and Francophone) from 2005 to 2007 in education finance. Eight countries were involved including Angola, Botswana, Burkina Faso, Ethiopia, Lesotho, Mali, Mozambique and Niger. These workshops (4-5 days) included intensive training on education finance concepts and definitions using group activities in order to facilitate the identification of problematic issues. More specifically, the workshops involved i) training on methodology of education finance, ii) working with national statisticians to compile the data from different national sources, iii) mapping of national data into the international framework, and iv) building documentation for countries to reproduce the work. In addition to this, regional follow-up workshops were also held for Anglophone and Francophone sub- Saharan African countries which had the benefit of site visits. One of the goals of these workshops was to facilitate the exchange of best practices and review lessons learned from site visits amongst national statisticians. Secondly, the workshops aimed to enhance the use of finance data at the national level regarding policymaking as well as the publication of key finance indicators in country annual publications. Impact: As a result of these efforts, it was concluded that all participating countries were able to provide data of very good quality on public spending on education -- some of them for the first time in many years such as Botswana, Burkina Faso and Mozambique. Moreover, five of the participating countries are now providing data for the second or even the third (i.e., Burkina and Mozambique) wave without UIS direct assistance. Finally, countries agreed to publish some key finance indicators in their national publications. For example, Ethiopia published unit cost for primary education in their national statistical yearbook. 152 Future Work: In terms of UIS work in education finance, the following planned activities include: ™ A report on education finance in sub-Saharan Africa ™ A follow-up of the site visits programme in the eight project countries ™ Collaboration with UIS’s World Bank partners to sustain the work done in the Country Status Reports including the following five (5) countries: Benin, Burkina Faso, Central African Republic, Guinea and Togo. ™ Collaboration with other organizations: Pole de Dakar, Millennium Challenge Corporation. ™ More international networking on education finance (including with regional experts), ™ More involvement of UIS regional advisors in the countries ™ Similar type of work in other regions apart from sub-Saharan Africa A related objective is the review and redesign of the questionnaire. In particular, the UIS aims to redesign the questionnaire to include an additional tool that would be used by national statistical organisations to collect private expenditure on education. Similarly, UIS also aims to plan additional capacity-building for countries to produce data on private expenditure. 4. Examples of decisions taken at the national level which were linked to the use of comparative data? In Asia, all countries except Afghanistan, Brunei and Singapore, participated in the EFA Mid-Decade Assessment (MDA) and Mid-Term Policy Review and formulated action plans and activities to address remaining unreached and underserved groups. Furthermore, national EFA MDA reports have been prepared by countries, which in turn were used as a basis for the sub-regional EFA MDA reports. The sub-regional MDA reports use UIS data for comparing countries. This process has also led countries to dialogue more closely with UIS through the Regional Office in relation to data issues. Examples of decisions taken at the national level which were linked to the use of UIS comparative data include the following: ™ Given Pakistan’s low HDI rank, officials decided to work on the improvement of its data and work with UIS in strengthening its EMIS leading to the Education Census, which was conducted in 2004-5. ™ Indonesia’s Tsunami Emergency management was assisted by the UIS-AIMS designed rapid survey of the survivors for rehabilitation projects. ™ Cambodia’s Education Quality Assurance system, which is based on the UIS-AIMS method for measuring various aspects of quality. ™ The education and literacy module of Thailand’s MICS survey was designed by UIS-AIMS, enabling the discourse on mother-tongue, education and policy on language of instruction. 153 Annex 8 PARIS 21: NSDS Program Objective of Grant: Mainstreaming strategic planning of national statistical systems, so that all the poorer countries would have plans under preparation (or further advanced) by 2006, and in execution at latest by 2010 The original Marrakech Action Plan adopted this as its first, and in many ways central, objective for the short-run period that it mainly focused on. And it assigned a crucial role to Paris 21’s promotional and educational functions in support of the objective, fully recognizing that its attainment would also be affected by many other factors. Paris 21 remains an organization reputed for its wide connections and generally good rapport with the statistical community in the developing countries, and particularly in Africa. Its services appear to be trusted and appreciated. It is therefore an important contributor to the broader MAPS Partnership and its further development. Paris 21 is supported by a number of mainly bilateral donors. In most years since its inception in 1999 it has also received significant support from the World Bank for its core mandate of promoting understanding and achievement of the larger role statistics have to play if management of countries’ development is to be more results-focused and effective. Paris 21: Total Expenditure and DGF Contribution 2006-08 ($ thousands) 2006 2007 2008 Total Expenditure 2,413 2,999 2,840 DGF Contribution 1,530 1,382 1,418 Dollar/Euro exchange rate used (IFS) 1.25 1.45 1.60 Note: The figures were provided in Euros by the Paris 21 Secretariat and have been converted into US dollars by the evaluation team, using the exchange rates shown. Expenditures shown for 2008 are only for the period up to May 13. Paris 21 pursues its objectives mainly through four categories of activity: Advocacy, Knowledge Base and Planning Methodology, Support for Donor Collaboration, and Regional Programmes. Through its efforts to build understanding of the significance of statistics among senior officials of IDA countries’ governments, Paris 21 had become a very strong proponent of the need for country-owned strategic plans for the development of statistical services. The main theme of the regional workshops that it organized between 2004 and 2006 was the need for preparation of such plans for development of official statistical services, well coordinated between the national statistical agency and line ministries. It has continued to prepare useful promotional materials, important presentations for regional and international high-level meetings, and press/TV statements on occasions such as the African Statistics Day that was devoted to strategic planning for the sector. It has given direct support over the last year to some countries’ own promotional efforts, including country-specific pamphlets and brochures and officials’ presentations of 154 statistical services’ growth needs at Consultative Group meetings and Donor Roundtables. In support of the workshops on strategic planning and in follow-up to them, Paris 21 prepared guides and advisory notes on different aspects of plan preparation and issued them in all relevant main languages. It also assembled a substantial library of good- practice planning material, making it available on its website. Recent years have seen increased attention to issues of plan implementation, and by Paris 21 task groups to related broader issues, including effective collaboration between national statistical agencies and sector ministries in implementation of a national strategy for development of statistics (NSDS), and between statisticians and offices responsible for monitoring and evaluation of development plans and poverty reduction strategies. In regard to collaboration among donors, Paris 21’s most important recent initiatives were probably those in support of the scaling-up initiative that was developed by the World Bank and others in response to developing-country representations at the February 2007 3rd International Roundtable on Managing for Development Results in Hanoi. Paris 21 produced a concept note and guide regarding the possible applicability of a system-wide or SWAp approach to aid for a national system, as has come to be adopted widely for aid-flows from all donors for many countries’ education or health sectors; these ideas have received strong support. Paris 21 and its task teams have invested substantial time in the difficult task of compiling figures on actual aid flows to statistics, two years ago on a pilot basis for Africa (the so-called Light Reporting Exercise) and currently more broadly. The results are valuable, but it is hard to make them comprehensive because of the frequency with which support for statistical services is provided as a sub-component of a broader program. Paris 21 has also given some attention recently to the important question of the adequacy of aid nowadays available for coverage of training expenditures for statisticians, especially in Africa. Paris 21’s links with its developing member countries in connection with development and implementation of strategic plans have been increased by appointment of regional advisers now responsible respectively for the Sub-Sahara African countries (and resident there), for Latin America and for the Arab world, as well as close cooperation with UN ESCAP. Particularly important initiatives of the last 12-18 months have been the support provided to the concept of a Central America Regional Statistical Commission whose creation has now been agreed in the region, organization of the 3rd Regional Forum on Statistical Capacity Building for the Arab Region in Sana’a, and the mounting of the first peer reviews of national statistical services – between three Anglophone African countries, which reportedly found them very worthwhile. 155 Annex 9 Paris 21: IHSN & ADP MAPS Objective: Setting up an International Household Survey Network to get more value from past surveys and higher productivity from future expenditures on surveys The Marrakech Action Plan envisaged creation of a Network comprising the major sponsors of global household surveys (such as MICS and LSMS), the donors who finance them and the national statistical offices which conduct them. The Network would share information and mobilize international support for more efficient approaches to survey work, including better storage and documentation of the data collected. An International Household Survey Network (IHSN), bringing together survey producers, sponsors and users was formed quickly, in September 2004. A management group comprising representatives of major international survey sponsors approved a work program aiming at improvement of survey documentation and storage, and better coordination and harmonization among household survey programs and instruments. In 2005 it was decided to supplement creation of the network with another initiative, entitled Accelerated Data Program (ADP), that would provide assistance to individual countries for implementation of measures consistent with the approaches emerging from the network and, more broadly, for generation of data much needed for policy purposes – whether from survey data already collected or from better structured new surveys. Paris 21, and the OECD Paris headquarters which house it, accepted to add both IHSN and ADP programs to their agenda and to accommodate the small secretariats that would be necessary. DGF funding in the amount of $0.9 million for IHSN and $2.0 million for ADP was approved for FY 2006 but, due to difficulties in finding legal formulae acceptable to both OECD and the World Bank, became available only in April 2006. Until that time the programs could run only with small in-kind contributions from one or two IHSN members. IHSN and ADP: Total Expenditures and DGF Funding 2006-08 ($ thousands) 2006 2007 2008 2006-08 IHSN Expenditures 379 827 306 1,512 DGF Funds Received 920 798 818 2,536 ADP Expenditures 246 2,495 2,336 5,077 DGF Funds Received 2,040 2,127 2,179 6,346 Dollar/Euro rate assumed 1.25 1.45 1.60 NB: The figures were supplied in Euros by the Paris 21 Secretariat and have been converted by the evaluation team into dollars using the exchange rates shown. Expenditures cover only the period up until 13 May 2008, while the funds received should be covering expenditures through 31 December. 156 IHSN has worked to date in three principal areas. Fundamental to all its subsequent work, it early developed a user-friendly IHSN Toolkit for microdata management, drawing on the Data Documentation Initiative (DDI) tool that had been invented in the United States in 2003 as a stable metadata standard for social science datasets. Second, it has built up a central catalog, accessible on its website, listing now a total of some 3,500 surveys carried out in developing countries. Third, under the leadership of a DFID staff-member loaned to the program for two years, substantial progress has been made in the development of an IHSN Question Bank which will serve as a central repository for international recommendations, definitions of indicators and concepts, classifications and questionnaire design. The Question Bank is expected to foster harmonization and comparability over time and between countries, and more generally to help improve survey design. An important recent addition to the tools developed is the National Data Archive (NADA), which offers a searchable web-based survey catalog that allows users to browse and search surveys, to easily retrieve rich metadata, and to request access to datasets that the country decides to disseminate. This cataloguing system complements the IHSN Toolkit and takes full advantage of the DDI metadata standard. Available since December 2007, NADA is already running well in 3 countries, and it is expected that National Archives will have been established in at least 10 or 12 developing countries by the end of 2008. IHSN efforts to improve coordination of future surveys by open exchange of information among sponsors on their plans have not so far proved successful, but a new effort is envisaged, working in the first instance with countries participating in the ADP. ADP’s role was defined from the beginning in terms of three tasks: first, making a country’s existing microdata more widely and easily accessible; second, assessing the quality of the surveys, to help improve future ones; and third, helping to fill pressing gaps in the availability of data needed for policy purposes. The program was expected to start with a pilot phase concentrated on some 5 or 6 countries, which might be increased after a year or so to a dozen. Demand has been much stronger than expected, especially for help in establishing use of the IHSN Toolkit and locating the databases and metadata for past surveys of which often few records remained in the country itself. By April 2008 some 27 developing countries, half of them in Africa, were already active participants in ADP, and 16 further countries had indicated desire to join. Work in a country normally starts with a one-week national workshop to introduce the IHSN Toolkit and the Data Documentation Initiative. Typically work with about 20 participants results in about 10 capable users who can carry out the work of gathering and entering the information from past surveys. In Africa, where the ADP secretariat has been most heavily involved, these workshops are now carried out by people from African neighbors already experienced in ADP. With the much larger number of countries than expected involved, ADP work to date has been heavily concentrated on the first phase of documenting past surveys and making them more accessible. Help has also been given to several countries in the establishment of their own NADA, and to a very few countries in the completion of surveys that were expected to yield important policy-relevant information but on which work had been for one or another reason truncated. 157 The principal hope for early contributions to the supply of information and indicators useful to policy-makers is from deeper analysis of the data collected in past surveys. The work already carried out has confirmed that only very small proportions of many past surveys have in fact been exploited, and often only in quite superficial manner. Deeper re-examination of the data is likely to generate improved estimates and trend information on subjects important to understanding of poverty, agriculture, and availability and use of public services. The most useful results on ADP’s second task, of quality assessment of survey data, are now expected to result from the IHSN Question Bank effort and its intended sequel, of stimulating creation of national Question Banks, synthesizing nationally adapted best practices for different types of survey. That work will automatically extend also to appropriate division of labor, in terms of coverage of different aspects, between household surveys and administrative data systems. These programs have depended heavily on inter-agency cooperation and have generated widespread partnerships for their implementation, both among agencies and among countries. The IHSN management group has included, besides the World Bank and UNSD, also ILO, UNICEF, WHO-HMN, Paris 21 and DFID. The Inter-American Development Bank coordinates ADP activities in Latin America and the Caribbean (where the program is already active in 5 countries, with several others applying), UN ESCAP in Asia and Pacific (with 8 countries now active). The Statistical Commissions of the League of Arab States and of UN ESCWA have collaborated with ADP, and AFRISTAT has contributed to work in Africa. Question Bank work is being undertaken jointly between IHSN and HMN and also with UIS. An important recent development has been the formation, under Ugandan leadership with strong support from ADP, of the African Association of Statistical Data Archivists (AASDA), which is expected to play a catalytic role in changing mind-sets in professional quarters towards better appreciation of the potential benefits of opening national data and experience to wider access, so as to stimulate more analysis and constructive exchange. ADP partnerships continue to expand, recently to include cooperation with France’s INSEE. 158 UNESCO Institute for Statistics Annex 10 MAPS/PES Objective: Improving the Quality of Internationally Comparable Education Statistics Improvement of the reliability and timeliness of developing countries’ education statistics has long been an important concern for the Bank, further enhanced in the 1990s when international targets for educational enrolments began to be agreed and monitored. When UNESCO decided in 2001 to transplant to Montreal its newly established Institute for Statistics (replacing the previous headquarters department for statistics), the Bank began provision of financial assistance to some education portions of the work program. DGF funding for the UNESCO Institute for Statistics (UIS) was brought formally into the framework of MAPS only in 2007. This reflected the desire of the DGF Council to encourage development of more cooperative efforts and synergy in work concerned with building statistical capacities, in line with the broader philosophy of the Marrakech initiative. In contrast to other MAPS components, the annual submission to the Council for this program is still sponsored by the Education Sector Board. UIS was established to foster a culture of evidence-based policy, both nationally and internationally, through the collection and dissemination of high-quality, timely data in education and in UNESCO’s other fields of responsibility. Its principal function in practice has been to help ensure the availability of more timely, complete, and accurate educational statistics at the international level, comparable between countries. The focus has been on primary and secondary education, and the work has had some useful spin-off effects on the quality of the statistics available for planning and management at the national level. These functions are all of such great interest to the Bank that the partnership between the two organizations has been guided more by continuous dialogue than by detailed agreements on the institute’s work program or earmarking of funds to pre-agreed projects. The following figures (in thousands of US dollars) give an indication of the scale of the UIS overall work program and the role of DGF in contributing to its financing. UIS Expenditures 2006-08: Distribution of Overall Total and of DGF Funding ($’000s.) ____2006____ ____2007____ | ____2008____ Total DGF Total DGF | Total DGF Education Statistics 1,929 350 1,936 580 | 2,674 820 Educ’n Stats: Stat. Cap. Building 1,183 310 1,365 200 | 2,313 176 Other Statistics 834 -- 865 -- | 1,316 -- Literacy, Non-Formal Ed. Stats. 1,199 60 1,126 60 | 1,566 60 Cross-Program Activities 2,856 870 2,739 780 | 2,043 600 Governing Board & Administration 3,013 190 3,110 180 | 2,109 144 Fund Raising & Director’s Budget 356 -- 402 -- | 707 -- Total Expenses 11,370 1,780 11,541 1,800 | 12,728 1,800 Selected Sources: UNESCO 4,510 DGF as % of Total Expenditure 15.7% 15.6% 14.1% 159 Important changes in budgetary practices in this period, especially between 2007 and 2008, make comparisons over time difficult. For instance, the large increase in 2008 expenditures on education statistics, and reduction in cross-program activities (largely data processing) reflects mainly institutional rearrangements and transfer of data analysis responsibility to the department responsible for collection of education statistics. Another difficulty with the figures is that they reflect mainly the core budget, while substantial service contracts awarded to UIS for fulfilment for aid agencies are sometimes excluded and sometimes included: the figures in the line for Statistical Capacity Building give a rather distorted picture of growth as a result of exclusion of contract work from the European Commission in the amount of some $900,000 in both 2006 and 2007 and the inclusion of other contracted work amounting to some $1.3 million in the figure shown for 2008. The table does bring out clearly that the DGF contributions have been sizeable, equivalent to as much as 40% of the core funding provided by UNESCO itself (about $4.5 million p.a. in this period, although the precise figure is available only for 2006), and that the DGF contributions have been mainly concentrated on the mainline functions of the institute for the education sector: efficient collection, analysis and dissemination of internationally comparable statistics about primary and secondary education. They have been much less significant in support for field-based capacity building work, quite insignificant in statistical work on non-formal education and literacy testing, and not available for work on other than education sector statistics. The DGF funds have been particularly important to the Institute not only because of the significant contribution they have made to the budget but also because they have been usable for core functions such as development of statistical standards and data processing methods, rather than limited, like much other third-party support, to more narrowly defined projects. Since the move to Montreal, and still continuing over the last year or two as fully a part of MAPS, UIS has made steady and substantial progress in improving the coverage in terms of countries, the quality and international comparability, and the timeliness of the basic statistics (drawn from administrative data) on primary and secondary education. Most data are now reported to Montreal on-line, with quick feedback in response, improved error-detection tools applied, and flexible arrangements for the receipt and electronic publication of data that in turn encourage early supply. Data on education finance unfortunately remain much weaker than those on school enrolments and operations. UIS continues to work more closely with IIEP in Paris in this area than others. It has regularly distributed a high-quality questionnaire on finance since before it moved to Montreal, but the overall response rate still runs less than 60%. If they do respond, most developing countries provide information only on public expenditure, and even that is of very uncertain quality in many. In 2005-06 UIS organized regional workshops in Africa to provide intensive training in financial concepts to selected countries and in 2006-07 it followed those up with one-week national workshops in eight countries. Follow-up sessions for these 8 countries together were held in Paris in June 2007. The efforts yielded very promising results in terms of improved data participants have been able to generate, usually with little further assistance. 160 UIS’s history has seen some tension between different concepts and approaches in the area of capacity building, and it is not easy to distinguish a single path that is clearly most promising. A narrow conception of UIS’s role emphasizes the substantial contribution it can make by developing and improving statistical concepts, standards, methods, procedures and databases. This should at the least be backed too by the regional workshops that are held every year or two to improve liaison with the country statisticians sending data to Montreal, collect and review data, apply ISCED, and resolve weaknesses. It is probably desirably supplemented by experienced UIS representation in the field to keep abreast of individual developing countries’ strengths and weaknesses in education statistics, to participate in diagnostic work and to promote forward movement, whether in the form of country action or donor assistance. The alternative conception, which has been applied briefly in a few cases with valuable positive impact at least in the short run, is a much more intensive direct involvement in the provision of capacity-building assistance for individual countries, work that in the first alternative would be left to donor agencies to contract out, probably largely to private consulting companies. In present circumstances, UIS needs additional financing to carry out any capacity-building efforts beyond its core work of inventing concepts, tools and methods useful for practitioners in the field of educational statistics. Since the experiments with the more intensive approach were felt to have had negative effects at the margin on core functions (due to overall resource constraints), present UIS policy appears to favor, at least initially, securing of sufficient outside resources fully to implement the first alternative described. 161 Annex 11 UN Statistics Division (UNSD) MAPS Objective: Ensuring a Successful 2010 Round of Population Censuses Reflecting the importance of sound population census for results-based management of progress toward national/international goals, as well as the shortfalls of the 2000 census round, support for Census activity was from the start an important element in the Plan of Action proposed for consideration at Marrakech and endorsed by the meeting. Proposals on the matter benefited from discussions that had already been initiated by Paris 21 as well as by the UN Statistics Division. The 2010 World Programme on Population and Housing Censuses was formally initiated at the 36th session of the UN Statistical Commission in March 2005. The goals of the programme were for all countries and areas: to agree on a set of accepted international principles and recommendations governing the conduct of a census; to conduct a census during the period 2005-14; and to disseminate results in a timely manner. The DGF grants have been made to the UN Statistics Division (UNSD), to assist it in actions to promote attainment of these goals, especially by development and dissemination of good-practice materials, and provision of technical support to countries, on census planning and management. UNSD has undertaken much more extensive promotional, informational and training activities in support of Census actions than would have been possible without the DGF grant funds. Present prospects, aided also by somewhat more stable political situations in several countries than ten years ago and increasing recognition among political leaderships of the importance of up-to-date census information, are for the 2010 round to be more successful than that of 2000. Several of the larger African countries which did not participate ten years ago have already carried out the census for the current round, notably Nigeria and Ethiopia, and most others have at present firm plans for execution of a census within the next three years. But it is early days since some 70% of planned censuses usually take place in the middle years of the period (2009-11 in this round) and the UNSD record shows that as of the end of 2007 only 44 out of 232 countries/areas had conducted their census. Particularly important support to the UNSD effort in Sub-Saharan Africa has been provided by Statistics South Africa. The two joined to organize, in Cape Town at the end of January 2006, a first Africa Symposium on Statistical Development which focused specifically on the 2010 census round and attracted participation from 50 African countries. That meeting contributed useful African perspectives that were taken into account in further improvement of the UN Principles for Population Censuses. Maintenance of momentum in Africa has also been assisted by annual statistical Symposia, still sponsored jointly by South Africa and UNSD, first in Kigali in January 2007 and at the very end of 2007, for 2008, in Accra. [Expenditure Table, kindly proposed by UNSD, is not yet available.] Following review and correction of drafts by expert groups, a document entitled United Nations Principles and Recommendations for Population and Housing Census, Revision 2 was adopted by the Statistical Commission at its 38th Session in March 2007, and 162 published (after editing) on the UNSD website in September 2007. October saw launch of the Census Programme’s on-line Resource Center, to serve the community of census managers with current information and news on census-taking worldwide and a knowledge base of technical and methodological documentation. The Resource Center is designed also to keep potential sources of assistance aware of the progress being made in countries’ implementation of their census plans. A database compiling questions from several hundred census-questionnaires has been compiled and has already seen quite extensive use by countries preparing their census and analysts reviewing particular subjects. A further new initiative is development of standard database software, CensusInfo, by which countries can readily make their main census results more accessible to users for statistical analysis. On a number of subjects where good teaching material was lacking, preparation of new or updated guidelines and handbooks has been put in train and supported with expert group review meetings. UNSD has mounted a substantial program of regional seminars and workshops in response to demand from countries for advice on particular aspects of census work. End-of-seminar participant evaluations available for all the recent events of this type give generally very positive assessments, and questionnaires filled six months later suggest that participants have widely used the information gleaned and passed it on to colleagues. A central theme of several workshops in 2007 was census cartography, and the scope for use of GIS and GPS to improve the efficiency of census/survey design. Study has recently been launched on countries’ experience in regard to census costs (both expected and actual), a subject to which attention had been urged in the Marrakech plan as well as earlier Paris 21 discussions. The expanded UNSD outreach to the developing countries, which has been fostered by the DGF grants, has also reinforced inter-agency partnerships, both with UNFPA, the UN agency which was traditionally most involved in the practical aspects of census implementation, and with the regional Economic Commissions, as well as sub-regional bodies and Eurostat. 163 Annex 12 UN Economic Commission for Europe (UN ECE) MAPS Objective: Capacity Building Program on Engendering National Statistical Systems for Knowledge-based Policy Formulation in CIS and SEE Countries This program was added to MAPS in 2006 with strong endorsement from the Bank’s Gender and Development Group as a way to activate a promising program that was emerging from several years’ previous effort. The World Bank Institute (WBI) had been working with the UN ECE to develop collaboration with the statistical offices of the nations which had emerged from the break-up of the Soviet Union and many of the surrounding countries. Grants have been made to UN ECE for periods corresponding to the World Bank fiscal years 2007 and 2008, and a further grant was expected for 2009. Usage of the funds provided (actual for FY07 and revised estimate for FY08) is as follows (in US dollars): UN ECE Expenditures from DGF Grants for Gender Statistics Program 2007-08 (US $) FY 2007 FY 2008 (Actual) (Estimated) Manual (200-page textbook on treatment of gender in production and use of statistics) and other materials 82,398 Meeting to review draft manual (Rome Feb. 2007) 12,358 Training of trainers from 9 countries (Almaty April 2007 and Rome December 2007) 57,104 45,450 National workshops & meetings 2,300 50,452 Advisory services (UNECE staff travel) 15,153 Project Assistant 26,000 57,800 Overhead 25,000 20,000 Subtotal 205,160 188,855 Add: Informal Employment Consulting contract 9,000 Travel of experts 14,400 Grant for analysis 28,000 Workshop on measurement (Kazakhstan) 20,000 Travel of participants 4,000 Overhead 10,000 Subtotal 85,400 Total out of DGF funds 205,160 274,255 Others (UNECE, UNFPA, ILO, FAO et al.) approx. 100,000 100,000 Source: UN ECE 164 The purpose of the DGF grant has been to support countries in the Commonwealth of Independent States (CIS) and South Eastern Europe (SEE) to improve, analyze and disseminate more gender-relevant statistical material, with appropriate sex disaggregation, for purposes of social and economic analysis. The original Training of Trainers under the program, in Almaty in April 2007, was already aimed at forming a Community of Practice among participants (mainly from national statistical bodies). This has also been fostered by creation and maintenance of an interactive web-portal to hold e-discussions, pose questions to the lead experts in the field, and store modules and multimedia shows. Nine countries participated in the Almaty course: Bosnia-Herzegovina, Kazakhstan, Moldova, Serbia, Tajikistan, Kyrgyz Republic, Uzbekistan, Russian Federation and Turkmenistan. The five first-listed of these countries have already since held national follow-up workshops, organized by those who participated at Almaty and focusing aspects of gender-related statistics of greatest importance for the individual country. Other types of follow-up action have been taken in Kyrgyz Republic and Uzbekistan. Participant action plans drawn up at Almaty and follow-up meetings and workshops have led to significant initiatives to strengthen gender-relevance of statistical work, notably on gender-based violence and also on labor market issues in Kazakhstan, on development of gender-relevant MDG monitoring in Moldova, and on inter-agency effort, led by the Committee on Migration, to improve information flow on migrant workers from Kyrgyz Republic, and improvement of the labor force survey in Bosnia- Herzegovina. 165 Annex 13 UN HABITAT MAPS Objective: Strengthening Capacities to Collect and Analyse Urban Indicators for Monitoring MDGs This program at UN Habitat, in Nairobi, was worked up in the months following the Marrakech meeting and became an important element in 2005 plans for MAPS. It has been one of the principal activities initiated in pursuit of MAPS Action Area 5: To undertake urgent improvements needed for measuring progress towards the MDGs. The overall objective has been to strengthen global, national and local capacities to collect and analyse national urban indicators and city-level indicators relevant to monitoring MDG target 11, relating to slums, as well as performance in urban areas on many other MDGs. Evolution of the Applications of DGF Grants to UN Habitat (Monitoring System Branch – MSB) FY 2006 Grant FY 2007 Grant FY2008 Grant Deliverable 1. Build capacity to Better use of higher- collect, analyse and quality urban indicators disseminate indicators by at least 10 NSOs Expenditure Actual US$ 45,000 30,000 % of total for purpose 15% Deliverable 2. Urban indicators Improve coverage of Incorporation of database housing and slum UrbanInfo into routine dwellers work of NSOs Expenditure Actual US$ 35,000 70,000 % of total for purpose 20% Deliverable 3. Information & Incorporation of slum Communication re information in dev. Urban Indicators strategies Expenditure Actual US$ 35,000 % of total for purpose 15% Deliverable 4. Harmonization of water & sanitation definitions among agencies Expenditure Actual US$ 30,000 % of total for purpose 70% Deliverable 5. A monitoring system for security of tenure Expenditure Actual US$ 20,000 % of total for purpose 100% Deliverable 6. Use of geo-spatial data to est’e slums Expenditure Actual US$ % of total for purpose Deliverable 7. Dev. & testing of qual’ve tools to uncover reasons for phenomena revealed by Urban Inequities Svy Expenditure Actual US$ 74,000 % of total for purpose Evaluation 4,000 Project support costs 22,000 26,000 Total DGF 191,000 (25%) 200,000 (9%) 200,000 (10%) All other sources 564,000 1,932,000 1,800,000 166 The original version of the Habitat program was built around 5 main “deliverables” as listed on the left-hand side of the above table. Good progress was made in the first year on the two specific deliverables that had been identified, those numbered 4 and 5 on the table. Activities chosen for use of both FY2007 and 2008 grants consisted of one main area for spreading improvements more widely among countries and agencies, along with one more precise field for research and development on a newer analytical technique or tool. Habitat has reported the latest situation regarding expenditures out of the DGF grants only in a more administrative breakdown, as opposed to the programme-budget style of the table provided above. These new figures indicate total 2006 expenditures of $185,067, very close to those previously reported, but 2007 expenditures of only $83,073. Out of the total of $600,000 received, $301,000 were still available as of 20 May 2008 for expenditures, some already committed. Habitat management considers that the DGF grants have played an extremely important role in enabling its Monitoring System Branch (MSB) to begin the wider application of concepts and methods which have now attracted both widespread interest in the developing world and much increased financial support from other aid agencies and some major private-sector companies. The figures in the bottom lines of the table indicate that, over the three years of the program, the DGF contribution has fallen from accounting for 25% of MSB expenditure in the first year to about 10% in the third year. The Netherlands and Norway have become notably important supporters of MSB work. The advances that have been brought to fruition over these few years – all of them benefiting of course from substantial earlier work – relate partly to slums and the adverse environments faced by slum-dwellers and partly to software for effective storage and management of urban data more generally. Three elements have probably been most important: Definition of slums, Recognition in urban surveys of the likelihood of significant differences between the conditions affecting different groups such as those living in slums as against elsewhere, and Development of UrbanInfo database software. In regard to slums, MSB completed, tested and published in 2006 (with DGF aid, as reflected in the table) its Legislative and Institutional Framework Index (LIFI) which had been developed for monitoring security of tenure. LIFI has since been applied in 6 Latin American cities, Multiple Indicator Cluster Surveys (MICS) in Hama, Syria and Dhaka, and DHS surveys for Mumbai, Kolkota, Dakar and Accra. It has been extensively used in land-related policy discussions for developing countries and is likely to be incorporated in increasing proportions of DHS and MICS applications. Pending accumulation of data on more geographical areas, Habitat, for practical purposes, formulated an alternative interim definition, setting four criteria fulfilment of any one of which would warrant classification of a household as existing in slum conditions. This method has been very widely accepted and used by countries and by international agencies (including UNSD) as the best approach currently possible, but it has not been accepted by some governments and for some uses (including the World Bank’s World Development Indicators). UN Habitat’s emphasis on the importance of disaggregating urban indicators between slum and non-slum populations has been increasingly accepted, and the ‘Urban 167 Inequities Survey’ tool that it has developed as an inexpensive way of achieving such disaggregation has been successfully applied in 12 cities. It shows signs of being adopted more widely and repeated. Most of the work to date has been with city administrations, but some national statistical bodies, as in Brazil, have also been involved. The Japanese Government and Google company are supporting wider spread to new geographical areas. UrbanInfo software appears to be gradually coming into use as a valuable tool to improve country/city capacity to monitor and report on MDGs and other development goals on a more sustained basis. Habitat itself has trained nearly 100 officials from some 20 countries, mostly in Africa, but strong interest is also being shown by some cities in the Arab world and by some national governments, such as Mexico, and by international agencies such as UN ECA. 168 Annex 14 Terms of Reference for an Independent Evaluation of the Marrakech Action Plan for Statistics - Approved at Meeting of MAPS Advisory Board, Thursday 15 November 2007 Introduction The Marrakech Action Plan for Statistics (MAPS) was agreed at the Second International Roundtable for Managing for Development Results in February 2004, as a result of shortcomings in data quality and availability, at global and national levels, for monitoring and measuring development outcomes identified at the first Roundtable in 2002. The plan focuses on six actions, designed to be implemented through partnership with countries and international agencies, to build on existing work programs, and to make short and medium-term improvements in data provision and statistical capacity. The actions and principles of MAPS have been embraced by the international statistical community and many bilateral donors, and were recently reviewed and endorsed at the third Roundtable in Hanoi in February 2007. Since 2004, important partnerships have been developed with UN Agencies and PARIS21, with financial support from the World Bank’s Development Grant Facility. Advisory oversight for MAPS comes from a high-level Advisory Board. The MAPS Unit in the World Bank is now commissioning an independent evaluation to assess MAPS partnership performance over the past three years, and to obtain advice on how its effectiveness, efficiency, and relevance can be improved. Background The second International Roundtable on Managing for Development Results in Marrakech, sponsored by the multilateral development banks in cooperation with the Development Assistance Committee of the OECD, placed high priority on supporting countries to improve their capacity to better manage for development results. Improving national statistical systems was recognized as one of the key areas for action: managing for results requires timely and reliable statistics at the country and global level. Serious problems beset measurement of many key indicators, including the MDG indicators; many countries need greater capacity to produce reliable statistics and make better use of them for effective decision-making. As part of the Marrakech Roundtable, representatives from the international statistical community took part in a series of meetings to identify the main priority areas that need to be addressed if the objectives of better managing for development results are to be achieved. The resulting Marrakech Action Plan for Statistics (MAPS) was adopted, and sets out six key actions. (Box 1). MAPS is intended to present a comprehensive, coherent and costed plan of action focusing on short and medium-term recommendations, which aims to prepare the way for long-term sustainable improvements in national and international statistical capacity. The actions fall into two groups: those directed at improving national statistical systems and those directed at the international statistical systems. But these actions are 169 interdependent: improvements in national statistical systems lead to improved international statistics, while a more effective international system could better coordinate and mobilize resources for improving national statistics. Box 1. The Marrakech Action Plan for Statistics 1. Mainstream strategic planning of statistical systems and prepare national strategies for the development of statistics (NSDS) for all low-income countries by 2006 2. Begin preparations for the 2010 census round 3. Increase financing for statistical capacity building 4. Set up an International Household Survey Network 5. Undertake urgent improvements needed for monitoring the Millennium Development Goals 6. Increase accountability of the international statistical system To support the actions addressing national needs, existing instruments and approaches have been utilized. These include the Bank’s STATCAP financing vehicle (part of regular operational activities) and the Trust Fund for Statistical Capacity Building. Actions addressing international responsibilities have required a partnership effort and have been supported by the Bank’s Development Grant Facility as follows: • International Household Survey Network (IHSN) – The overall aim of the IHSN is to improve the coordination of the activities of international organizations engaged in household surveys. The IHSN objectives are to improve the availability, accessibility and quality of survey data, to avoid duplication of data collection activities, to improve cost-effectiveness of surveys, and to reduce the burden of international survey programs on national statistical systems. Priority areas of work are (i) coordinating international survey programs, by fostering better timing, sequencing and frequency of internationally-sponsored surveys; (ii) fostering adoption of international standards and best practices by harmonizing data collection instruments; (iii) developing and maintaining a central survey data repository; and (iv) developing tools and guidelines for improving survey data documentation, dissemination and preservation. Grants have been made available to PARIS21, who have supported this activity as a “satellite” program. • The pilot Accelerated Data Program (ADP) – The objective is to make rapid improvements in the quality and availability of key statistics in pilot developing countries, particularly those derived from household surveys. The ADP will provide policy makers and other stakeholders with better data and analysis for policy design, monitoring, and evaluation. Intermediate objectives of the Program are to (i) test the ability of national statistical systems to respond to emerging data needs, (ii) assist countries that do not have a coherent long-term survey program to improve the planning and financing of data collection activities, (iii) foster better inter-agency cooperation in country activities, under the umbrella of the IHSN, (iv) build national capacity in micro-data preservation, analysis, and dissemination, and (v) generate updated estimates of key indicators by collecting new data. In achieving its objectives, the Accelerated Data Program should make use of the tools developed through the IHSN (MAPS action 4). At country level, activities should aim to be consistent with the National Strategy for the Development of Statistics, where 170 available. Grants have been made available to PARIS21, who have supported this activity as a “satellite” program. • Preparation of National Strategies for the Development of Statistics (NSDSs) - Statistical development strategies and master plans have proved to be a powerful tool for guiding the development of national statistical programs and increasing political and financial support for investments in statistics; supporting statistical development through good NSDSs is seen as a main pillar of MAPS. Activities which need support include the compilation and dissemination of guidelines for producing an NSDS and direct assistance to developing countries to help them access trust funds and other instruments for providing financial and technical assistance for statistics. A grant has been made available to PARIS21 to support these activities as part of their regular work program. • Urgent improvements in key statistics, including for MDG monitoring – The availability of comparable international statistics are weaker in some areas than in others. Grants have been made available to support UNESCO Institute of Statistics improve education statistics (“Program for Education Statistics”); to help UN- Habitat improve urban indicators; and to support UNECE and UNSD improve gender statistics. • Help countries participate in the 2010 census round - Population censuses are essential tools for policy and planning purposes; data and indicators derived from the census are extensively used as inputs for result-based management and tracking of progress towards national goals (such as those set in PRSPs) and international goals such as MDGs. No other data source provides the level of detail available in the census on location, age and gender, and family size. Developing countries need support to participate in UN 2010 World Program on Population and Housing Censuses in developing countries. The goals of the World Program are for all countries and areas to agree on a set of accepted international principles and recommendations governing the conduct of censuses; to conduct a census during the period 2005 -2014; and to disseminate census results in a timely manner. Grants have been made available to UN Statistics Division to support the World Program in developing countries. To help manage the MAPS program, and to advise the World Bank on its direction, a governance structure was created for MAPS. The main body is the MAPS Advisory Board, which includes representation from developing countries, development banks, donors, statistical agencies worldwide, and UN agencies. The Advisory Board meets at least once a year. The World Bank also created a small MAPS Unit in the Development Data Group to monitor and coordinate activities, and an internal Statistical Capacity Building Committee to review and monitor DGF grant proposal and other activities. Oversight on the Program for Education Statistics is further provided through the Education Sector Board of the World Bank. 171 The MAPS website has more information: www.worldbank.org/data/action Purpose and Scope of the Evaluation The purposes of the evaluation are to (1) Assess the performance of the MAPS Partnership (both efficacy and efficiency) in implementing actions of MAPS, and provide guidance on how to improve this performance; and (2) Assess the extent to which MAPS remains relevant to the statistical capacity improvement agenda and focused on the most important issues. The MAPS Partnership will include the governance arrangements created for MAPS (notable the MAPS Advisory Board), the recipients of grants provided by the Development Grant Facility (notably PARIS21, UN Statistics Division, UNESCO Institute for Statistics, UN Economic Commission for Europe, and UN Habitat), and other key instruments which support MAPS, particularly the Trust Fund for Statistical Capacity Building (TFSCB). The MAPS evaluation will be conducted in two phases. The first phase will review the governance arrangements of t he MAPS partnerships (as defined above) and the impact of the work programs of DGF grant recipients, including the two new programs of PARIS21, the Accelerated Data Program and the International Household Survey Network). The second phase will review the implementation of MAPS more broadly, utilizing the results of the first phase but taking into account activities of other organizations and programs outside of the MAPS partnership defined above. This will include the Trust Fund for Statistical Capacity Building, and broadly the activities of other multilateral and bilateral agencies engaged in statistical capacity building. See “Timetable/Calendar” section below for schedules. 172 The evaluator’s assessment shall address the questions noted below, but need not be limited to these. Where appropriate, illustrations of good (or bad) practice/outcomes should be provided either in the body of the evaluation report or in an annex.10 Efficacy: Outcomes, impacts and their sustainability Assess the performance of the MAPS partnership in achieving its desired results. Strategic Focus: To what extent has the MAPS partnership sustained its focus on the original action plan? Advocacy: To what extent has the MAPS partnership built political commitment for statistical capacity development among developing countries and among multilateral, bilateral and other international development partners? To what extent has the MAPS partnership influenced the strategies and programmes of developing countries and development partners? To what extent have developing countries and development partners influenced the strategies and programmes of the MAPS partnership? To what extent has the MAPS partnership contributed to increased partnerships and coherence of effort at the country level, among local, national and international partners? What evidence is there of the outcomes of those partnerships? Outcomes and Impacts on the Ground: To what extent have actions been initiated at the national and the international level that might not have been initiated without the MAPS partnership? To what extent has the MAPS partnership helped increase investments in statistics? To what extent has the MAPS partnership helped cause an increase in statistical capacity in countries? To what extent has the MAPS partnership improved the availability of key statistical data? 10 The evaluators should refer to the Sourcebook for Evaluating Global and Regional Partnership Programs, Independent Evaluation Group, January 2007. www.worldbank.org/ieg/grpp. 173 To what extent is a review of the impact of the program on statistical capacity and statistical outputs meaningful, given that MAPS is a relatively new program and institutional changes typically take some time to achieve? Are these outcomes and impacts on the ground sustainable? Monitoring and Evaluation: To what extent does the MAPS partnership have effective monitoring and evaluation of activities funded by its financial resources? To what extent is the World Bank exercising effective and independent oversight of the MAPS Unit and MAPS financial resources? To what extent do the MAPS partnership activities have measurable performance indicators – of outputs, outcomes and impacts? How useful are those indicators for assessing the effectiveness of the activities? To what extent have the indicators improved since 2005 when the DGF MAPS program started? Provide guidance on how to improve MAPS partnership performance. How could the MAPS partnership be more effective in building political commitment and increasing investment and coherence for statistical development both at the national and the international level? How could the MAPS partnership be more effective in producing results – both short- and long-term - at the country level? How could the results of MAPS grant-funded activities be improved? Are the interventions by the Development Grant Facility made at the right level through the right set of agencies/organizations? Efficiency or cost effectiveness of the program Has the program cost more or less than planned? How did it measure up against its own costing schedule? How do actual costs compare with benchmarks from similar programs or activities? Are there obvious cases of inefficiency or wasted resources? Do the program benefits outweigh the costs of individual activities? What is the least-cost way of getting the expected results? Were the program’s outputs and outcomes achieved in the most cost-effective way? 174 What would be the implications of scaling the program up or down in terms of costs, cost-effectiveness, or efficiency? What would be the costs of replicating the program’s activities in a different environment? How do costs affect the results and the sustainability of the program? Governance and management of the program To what extent does the governance structure of the MAPS partnership, and the roles played by the Advisory Board, the Statistical Capacity Building Committee, and the MAPS Unit contribute to achieving the MAPS objectives. Legitimacy: To what extent do the governance and management structures permit and facilitate the effective participation and voice of the different categories of stakeholders in the major governance and management decisions, taking into account their respective roles and relative importance? Accountability: Is accountability clearly defined, accepted, and exercised along the chain of command and control? Responsibility: Does the program accept and exercise responsibility to stakeholders who are not directly involved in the governance of the program and who are not part of the direct chain of accountability in the implementation of the program? Fairness: To what extent partners and participants, similarly situated, have equal opportunity to influence the program and to receive benefits from the program? To what extent does access to information, consultation, or decisions of the governing body and management favor the interests of some partners and participants over others, at both the governance and management levels? Transparency: To what extent the program’s decision-making, reporting, and evaluation processes are open and freely available to the general public? Efficiency: To what extent the governance and management structures enhance efficiency or cost-effectiveness in the allocation and use of the program’s resources? Probity: Do all persons in leadership positions adhere to high standards of ethics and professional conduct over and above compliance with the rules and regulations governing the operation of the program? How could the governance and organization better contribute to achieving the MAPS objectives? Relevance: The overarching global relevance of the MAPS To what extent is the strategy and focus of the MAPS relevant to its partners and their country clients to tackle the challenges of improving statistical capacity to support their development efforts? 175 What are the comparative advantages of the MAPS partnership? Is MAPS focusing on the right things? Should the MAPS objectives be reformulated, added or dropped? How could its strategy be improved? Methodology and outputs The evaluation will be conducted by two individual consultants working as a team. One consultant should be an expert evaluator with at least some knowledge of global programs, and the other should be an expert in statistical capacity building. The consultant team will produce an inception report which will describe the evaluation methodology in detail. This will be circulated to the MAPS Advisory Board and to DGF recipients for comments. The methodology should include but not be limited to: - A desk review of MAPS key documents including “Better Data for Better Results: An Action Plan for Improving Development Statistics” (February 2004), work programs and reports of DGF beneficiaries, Advisory Board meeting reports and presentations, evaluation reports, communication products, etc. - Interviews and/or survey questionnaires of Advisory Board members and of representatives of other MAPS partnership constituencies including (but not limited to) the task managers of the grant recipient agencies. Such interviews may include telephone, email, video conference communications, and personal interviews. It is expected that personal interviews will need to be conducted with the largest grant recipients, which are PARIS21 (based in Paris), UN Statistics Division (based in New York), and UNESCO Institute for Statistics (based in Montreal). Both consultants should visit PARIS21, but other site visits should be divided between them. - Personal interviews with MAPS Unit staff at its Washington, D.C. office. - Review of MAPS DGF-funded activities. Field visits should be made to all major recipients (OECD/PARIS21, UN Statistics Division, UN-Habitat, UNESCO Institute for Statistics, UN Economic Commission for Europe). - Assessment in at least two countries of the quality and effectiveness of their National Strategies for the Development of Statistics (countries to be agreed with PARIS21 and the MAPS Unit). - Quantitative methods where feasible. - Any additional sources of information or procedures to obtain views and feedback on the MAPS partnership that the reviewer feels to be necessary in order to accomplish the tasks set forth in these Terms of Reference. 176 The consultant will report their findings and recommendations to the MAPS Advisory Board in a report written in English not to exceed 60 pages, excluding appendixes. The evaluation team shall provide other outputs as noted in the time-plan below. The team leader will be expected to make a presentation of findings and recommendations at the MAPS Advisory Board meeting in 2008. Timing/Calendar September/October 2007 Review of TOR by SCB Committee, IEG and GPP November 2007 Review of TOR by MAPS Advisory Board (AB) and by Education Sector Board (for Program for Education Statistics) November 2007 Short-list consultants November 2007 Circulation of TOR to the DGF grant recipients December 2007 – February Selection and contracting of consultants 2008 March 2008 Inception report that includes the detailed work plan, to be approved by the MAPS Unit in consultation with the Advisory Board March-May 2008 Field work, including visits to grant recipients as appropriate (UN Statistics Division in New York; UNESCO Institute for Statistics in Montreal; PARIS21 Secretariat in Paris), the MAPS Unit in Washington, DC, and Bank program Task Managers for each grant. June, 2008 Presentation of draft report from Phase 1 to Advisory Board, Education Sector Board, and MAPS Unit. This will also provide a chance to redirect the work of the consultants in case there has been any misunderstanding of the content or emphasis of the ToR or any other information August, 2008 Completion of draft report from second phase, including review of TFSCB evaluation and incorporation into findings of first phase. First draft report of complete evaluation submitted to Advisory Board, Education Sector Board, and MAPS Unit. Debriefings with the MAPS Unit. September 2008 Final report submitted to Advisory Board and Education Sector Board November 2008 Consultant presents findings at Advisory Board and Education Sector Board meetings Anticipated field visits The consultants should specify the schedule of field visits in their inception report but the following visits are anticipated: New York (UN Statistics Division), 2 days, during March-May 2008 (one consultant) Paris (PARIS21 Secretariat/OECD), 4 days, during March-May 2008 (two consultants) Montreal (UNESCO Institute for Statistics), 3 days, during March-May 2008 (one consultant) 177 Washington, DC (MAPS Unit), 5 days during March-May, 2008 (both consultants); 2 days, November 2008 (both consultants) Role of MAPS Unit The MAPS Unit will manage the procurement of the evaluation on behalf of the MAPS Advisory Board, which will commission the evaluation. The MAPS Unit will provide consultants with key documents, facilitate contacts with MAPS partners, grant recipients and Advisory Board members, provide temporary office space at the World Bank, as appropriate, facilitate access to World Bank video conference facilities, and ensure independence of the evaluation. Obligations of the consultant The consultants shall: - Inform the MAPS Unit in a timely fashion of all contacts made with MAPS partners and constituents. - Treat documents in confidential manner. - Not publish evaluation results or output without permission of the MAPS Unit. - Return all MAPS partnership documents used in the evaluation. - Report on a timely basis any possible conflicts of interest. More information Further background information on MAPS and the MAPS partnership can be obtained from its website: www.worldbank.org/data/action.