THE WO RLD BANK OPERATIO NS EVALUATIO N DEPARTMENT The Global Development Network Addressing Challenges of Globalization: An Independent Evaluation of the World Bank's Approach to Global Programs Case Study Naveen Sarna Director-General, Operations Evaluation: Gregory K. Ingram Director: Ajay Chhibber Task Team Leader: Uma Lele 2004 This paper is available upon request from OED. The World Bank Washington, D.C. ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. OED Working Papers are an informal series to disseminate the findings of work in progress to encourage the exchange of ideas about development effectiveness through evaluation. The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. Contact: Operations Evaluation Department Partnerships & Knowledge Programs (OEDPK) e-mail: eline@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 http:/www.worldbank.org/oed Acronyms and Abbreviations AERC Africa Economic Research Consortium CERGE-EI Center for Economic Research and Graduate Education ­ Economics Institute CGIAR Consultative Group on International Agricultural Research CIDA Canadian International Development Agency DC Development Committee DFID United Kingdom Department for International Development DGF Development Grant Facility (World Bank) EADN East Asian Development Network EASDN East Asian Development Network EERC Economics Education and Research Consortium ERF Economic Research Forum for the Arab Countries, Iran, and Turkey EUDN European Development Research Network GDN Global Development Network GEF Global Environmental Facility GFATM Global Fund to Fight HIV/AIDS, Tuberculosis, and Malaria GPG Global public good ICEG International Center for Economic Growth IDE-JETRO Institute for Developing Economies IDS Institute of Development Studies IO International organization IEA International Economic Association ISA International Sociological Association ISEAS Institute of Southeast Asian Studies ISPA International Political Science Association LACEA Latin American and Caribbean Economic Association MD Managing Director NIH U.S. National Institutes of Health ODI Overseas Development Institute OED Operations Evaluation Department PACANZ Pacific, Australia, and New Zealand network PATS World Bank's Partnership Approval and Tracking System RNP Regional network provider SANEI South Asian Network of Economic Institutes SDC Swiss Agency for Development & Cooperation UNU United Nations University WIDER World Institute for Development Economic Research WIIW Vienna Institute for International Economic Studies i Table of Contents Acknowledgments............................................................................................................iii Preface............................................................................................................................... iv Executive Summary......................................................................................................... ix 1. Introduction and Context: Global Challenges in the Sector....................................1 Global Challenges in Research and Knowledge Use ..............................................1 2. Program Alignment with Global Challenges and Bank Priorities..........................1 Overview of Program Genesis, Mission, Objectives, and Activities........................1 Two Alternative Models for the Program................................................................2 Alignment with Global Public Goods and Corporate Advocacy Priorities.............4 3. Outcomes, Impacts, and Sustainability......................................................................5 Measuring GDN Outcomes and Impacts.................................................................5 Quality of Monitoring and Evaluation.....................................................................8 OED Findings..........................................................................................................9 Sustainability of Program Impact..........................................................................16 4. Organization, Management, and Financing of the Program.................................16 GDN Partnership with Regional Networks ...........................................................18 Financing of the Program......................................................................................19 Risks and Risk Management ..................................................................................21 5. Role of the World Bank.............................................................................................21 Comparative Advantage and Global-country Linkages ........................................21 Oversight................................................................................................................22 Exit Strategy...........................................................................................................23 6. Findings and Lessons.................................................................................................23 References.........................................................................................................................25 Annex A. Evaluation Framework for Phase 2 Report and 26 Case Studies ..............27 Annex B. Program At a Glance ......................................................................................35 Annex C. Stated Mission, Objectives, Outputs, and Activities....................................37 Annex D. Genesis, Evolution, and Exit Strategies of GDN..........................................38 ii Annex E. Evaluations and Audits of GDN.....................................................................39 Annex F. Governance of GDN........................................................................................40 Annex G. Financing of GDN...........................................................................................41 Annex H. People Consulted.............................................................................................42 iii Acknowledgments This case study of the Global Development Network was prepared by Naveen Sarna, with the assistance of Yianni Konstantopoulos under the overall direction of Uma Lele, team leader of the OED review of global programs. Uma Lele revised the draft in response to internal comments. The review team is grateful to the many people consulted during field visits made to India, the Philippines, Singapore, Thailand, and the Czech Republic to obtain the views of stakeholders involved with the Global Development Network as well as officials in research institutes, World Bank field offices, bilateral and multilateral institutions, nongovernmental organizations, implementing agencies, global program grant recipients, beneficiaries, and academics. The OED team is grateful for the perspectives and inputs from the GDN secretariat, particularly Lyn Squire and Gary McMahon, and of GDN board members, including Bina Agarwal, Richard Cooper, Ulrich Hiemenz, Lal Jayawardhane, Inge Kaul, Jan Kmenta, Samir Makdisi, Sujata Patel, Diane Stone, and Chalongphob Sussangkarn. The team also thanks Isher Ahluwalia, Supriti Bezbaruah, Randy Filer, Ellen Fisk, Fred Gerlach (external evaluator), Carolina Hernandez, Thawatchai Jittrapanun, Nirund Jivasantikarn, Mohsin Khan, Tony Killick, Mario Lamberte, William Lyakurwa, Joyce Moocke, Peter Muth and Frederick H. Gerlach (external evaluators), Caroline Pestieau, Subhorota Ray, Paul Schultz, T. N. Srinivasan, Aravind Virmani, and Chia Siow Yue. Within the Bank the OED global team benefited from discussions with Francois Bourguignon, Paul Collier, Ishac Diwan, Thomas Duvall, Ines Garcia-Thoumi, Alan Gelb, Catherine Gwin, Paul Hubbard, Frannie Leautier, Gobind Nankani, Benno Ndulu, Guillermo Perry, Mamphela Ramphele, Nicholas Stern and Samuel Wangwe and L. Alan Winters. Comments on the draft circulated inside the Bank were received from Francois Bourguignon, Guillermo Perry, and Catherine Gwin. The report reflects comments on the earlier draft of this report from the GDN secretariat. GDN circulated its draft comments to and received endorsement from members of the Executive Committee of the GDN's Governing Body. The report also reflects further exchange with Thomas Duvall and Andrea Emily Stumpf of the legal department and L. Alan Winters of the Development Economics Vice Presidency. Annex H provides a full list of people consulted as well as their respective affiliations. iv Preface The global programs evaluation and its case studies. At the request of the World Bank's Board of Directors, the Bank's Operations Evaluation Department (OED) has been conducting an evaluation of the Bank's involvement in global programs. This is the first independent evaluation the Bank has conducted of its global program portfolio. The Phase 1 Report, entitled The World Bank's Approach to Global Programs, focused on the strategic and programmatic management of the Bank's global portfolio of 70 programs in five Bank Networks (a cluster of closely related sectors) and was presented to the Committee on Development Effectiveness (CODE) in June 2002. The second phase, of which this report forms a part, is based on case studies of the Bank's involvement in 26 global programs (see list on the next page). A meta-evaluation of the Consultative Group on International Agricultural Research (CGIAR), the first of the 26 case studies, was presented to CODE in April 2003. (Both the Phase 1 Report and the CGIAR meta-evaluation are available on OED's external Web site at www.worldbank.org/oed/gppp). OED reports typically contain recommendations only in those reports presented to the Bank's Board or its committees such as the Committee on Development Effectiveness (CODE). While the case studies that underlie OED's Phase 2 Report were not presented to CODE individually, they were distributed in draft to program partners to obtain their feedback, which was taken into account in the final versions of each report before being disclosed to the public. Approval of global programs. Since November 2000, the Bank's involvement in all new global and regional programs has had to be approved at the initial concept stage, based upon the six approval criteria in Annex A, Figure A.1, by the managing director responsible for the Network or Regional Vice Presidential Unit advocating the Bank's involvement. Such approval then authorizes the respective VPU to enter into agreements with partners and to mobilize resources for the program ­ whether from the DGF, trust funds, or the Bank's administrative budget. Both before and after November 2000, the Bank's participation in some high-profile programs ­ such as the Global Environment Facility, the Multilateral Fund for the Implementation of the Montreal Protocol, the Prototype Carbon Fund, and the Global Fund to Fight AIDS, TB, and Malaria ­ has been considered and approved by the Bank's Executive Board. Oversight and management of global programs. Within the Bank's matrix management structure, Network Vice-Presidents are responsible for overseeing and managing their portfolios of global programs. They are responsible for establishing priorities among programs in their Network, for ensuring their coherence with the Bank's strategy for each sector, for sponsoring applications for DGF grants, for managing programs housed inside the Bank, for fostering links to the Bank's country operations, and for promoting synergy among programs within the Network, with the rest of the Bank, and externally with partners. Regional Vice-Presidents are similarly responsible for overseeing and managing the portfolio of regional programs and partnerships in their respective regions. While regional programs are not covered in this OED evaluation, many global programs have strong regional dimensions. These regional issues of global programs are treated in this review as well as the links between the network management of global programs and the Bank's country-level economic and sector work, lending, and policy advice. v List of 26 Case Studies in Phase 2 of OED's Evaluation of the Bank's Involvement in Global Programs Acronym/ Operational Size (US$ Short Form Full Name Start Date millions)1 Environnent & Agriculture 1. CGIAR Consultative Group on International Agricultural Research 1972 395.0 2. GEF Global Environment Facility 1991 387.53 3. MLF Multilateral Fund for the Implementation of the Montreal Protocol 1991 158.6 4. ProCarbFund Prototype Carbon Fund 2000 6.5 5. CEPF Critical Ecosystem Partnership Fund 2000 20.19 6. GWP Global Water Partnership 1997 10.25 7. GIF Global Integrated Pest Management Facility 1996 1.3 Health, Nutrition & Population 8. TDR Special Programme for Research and Training in Tropical Diseases Dec 1975 47.5 9. Global Forum Global Forum for Health Research Jan 1997 3.07 10. UNAIDS Joint United Nations Programme on HIV/AIDS Jan 1996 95.0 11. RBM Roll Back Malaria Nov 1998 11.4 12. Stop TB Partnership Stop TB Partnership July 1999 20.8 13. GAVI Global Alliance for Vaccines and Immunization Oct 1999 124.1 Infrastructure & Private Sector Development 14. WSP Water and Sanitation Program March 1978 12.4 15. ESMAP Energy Sector Management Assistance Programme Jan 1982 7.58 16. CGAP Consultative Group to Assist the Poorest August 1995 12.67 17. infoDev The Information for Development Program Sept 1995 6.07 18. PPIAF Public-Private Infrastructure Advisory Facility Dec 1999 15.61 19. CA Cities Alliance Dec 1999 13.25 Social Development & Protection 20. PostConFund Post-Conflict Fund 1998 10.6 21. UCW Understanding Children's Work 2000 0.56 Trade & Finance 22. IF Integrated Framework for Trade-Related Technical Assistance 1997 2.71 23. FSAP Financial Sector Assessment Program May 1999 10.46 24. FIRST Financial Sector Reform & Strengthening Initiative July 2002 4.64 Information & Knowledge 25. GDN Global Development Network Dec 1999 8.67 26. World Links World Links for Development 1998 6.5 /1 FY04/CY03 expenditures. For the following cases updated, audited data was not readily available so the previous fiscal or calendar year expenditures were used: Global Integrated Pest Management Facility, Water & Sanitation Program, Integrated Framework for Trade-related Technical Assistance. vi Internally and externally managed programs pose different sets of oversight and risk management issues for the Bank, both of which are being explored in the Phase 2 Report. Of the 70 global programs supported by the Bank in 2002, about 30 were managed inside the Bank, 30 were managed by partner agencies outside the Bank, and 10 were self-standing independent and often legal entities. Information on the internally and externally managed programs is not uniformly available. The primary focus of this background paper and the other case studie, is on the Bank and on the strategic role that it has played, and might play in the future, to ensure the development effectiveness of global programs. Yet, since the Bank is only one partner among many, it is difficult to evaluate the Bank's role without assessing the nature, performance, and impact of each global partnership as a whole, and the effect that the partnership has had on the performance of the Bank and vice versa. Moreover, the Bank's role in their governance and management is not as direct and as strong as it is in the case of programs that the Bank chairs and houses. A specific focus of this background paper is on the interface between specific global programs and related Bank activities. The Bank's financial support for global programs ­ including oversight and liaison activities and linkages to the Bank's regional operations ­ comes from the Bank's gross income (for DGF grants), the Bank's administrative budget, and Bank-administered trust funds. Evaluation sources and instruments. The OED evaluation team has conducted a meta-evaluation of previous evaluations of this program and a review of secondary information sources, including program documents, annual reports, DGF documents, trust fund documents, related sector strategies, and literature reviews. In addition, as appropriate for different programs, the team gathered primary source information from stakeholder interviews, surveys of program partners, and field visits to developing countries and program partners. Key interlocutors included Bank managers of task teams and global programs, senior World Bank managers (Network and Regional Vice-Presidents, sector and country directors, and sector managers), and Bank staff involved in the DGF secretariat, DGF council and Trust Fund Operations. Outside the Bank, key interlocutors included staff of partner organizations and developing country partners. Interviews were conducted with members of the GDN board, regional network partners, Bank task managers for global programs, with operational managers, and with Bank task managers. Relationship to other evaluations. An external, fully independent evaluation of the Global Development Network was completed in March 2004. This global case study draws on the results of this evaluation and on the results of internally commissioned evaluations of individual components of GDN, and an internal GDN evaluation report, also completed in March 2004 and intended by the secretariat to be considered in conjunction with the External Evaluation. This evaluation, however, differs from those mentioned above in various ways. First, it draws lessons based on the program specific information that was available from the external and internal evaluations, other documents, and OED team's own interviews. Second, it benefits from the perspective gained through assessing 25 global programs in other sectors, allowing it to identify generic issues raised by GDN, the results of which are synthesized in the Phase 2 Report. Third, it focuses specifically on the role of the Bank in the global program partnership to learn lessons from the information routinely collected and used by the Bank to assess the need for Bank involvement in global programs. vii Many of the processes that systematically and routinely apply to the Bank's country-level investment operations do not currently exist for global programs. In the case of Bank investment operations they include preparation, appraisal, negotiations with borrowers (in the case of global programs this would mean other donors), routine oversight, mid-term reviews, self-evaluations by Bank staff in charge of managing the task and borrowers, and independent evaluations of outcomes and Bank and borrower (in the case of global programs other donor) performance. Unlike in the case of the Bank's country operations, many of the concepts and definitions used in global programs are new, often vaguely defined, and not uniformly interpreted by all concerned parties. Basic information on financing arrangements and on the roles and responsibilities of the Bank as a partner is not always clear or easily available. Hence, this OED evaluation has had to explore new and often untested ground, attempting to clarify concepts and then apply them. These case studies are not intended to be a full-fledged independent evaluation of each global program. Based on the recommendations of OED's Phase 1 Report and the CGIAR Meta-Evaluation, Bank Management and OED are putting in place systematic processes, procedures, and reporting arrangements for global programs. These previous reports, the case studies, and the Phase 2 Report based on their lessons are all intended to assist in this effort. Field visits were made to India, the Philippines, Singapore, Thailand, and the Czech Republic to obtain the views of stakeholders. A detailed list of those consulted is found in Annex H. The objectives of the field visits were to: · Explore the global/national interface in the information and knowledge sector and solicit the views of GDN regional network representatives and the Bank's development partners · Discuss key issues of concern to stakeholders in developing countries · Review the implementation experience and outcomes of the Global Development Network's activities in developing countries. ix Executive Summary GENESIS, OBJECTIVES, AND ACTIVITIES 1. The Global Development The Global Development Network (GDN) Network (GDN) was formally launched in December 1999 with the strong Established: 1999 support of World Bank President James Objectives: · Support multidisciplinary research in social sciences D. Wolfensohn, who has led the effort to convert the World Bank into a · Promote the generation of local knowledge in developing and transition knowledge bank. GDN started with a countries joint effort by the World Bank Institute · Produce policy relevant knowledge on a and the senior managers in the Bank's global scale Development Economics Vice · Build research capacity to advance Presidency. At the time of writing of development and alleviate poverty this review, the GDN (including prior · Facilitate knowledge sharing among to its formal launching) had received researchers and policymakers $30.33 million in support and revenue · Disseminate development knowledge to the public and policymakers from the World Bank. In 2001, GDN Key activities: · Knowledge creation and dissemination moved out of the Bank and became an · Advocacy independent, not-for-profit organization · Supporting national-level policy and based in Washington, D.C. In January institutional reform 2003, GDN's Governing Body (GB) FY03 $10.4 million expenditures: voted to relocate the Secretariat to New Delhi, India. In May 2003, GDN FY04 DGF $4.45 million. Total support of $6.54 million, allocation: including other Bank sources decided to change its status to that of an Governance Independent external entity with close international organization. model: identification with the World Bank Location: Washington, D.C. 2. GDN's stated objectives are to: Governing 17 Gov. Body Members include (7) Members partners: from each of the regions covered by the · Support multidisciplinary regional research networks; (3) Members research in social sciences from Asia Pacific/Europe and NA; (2) Members nominated by International · Promote the generation of local Agencies (UNDP, WB); (3) Members knowledge in developing and nominated by International Professional transition countries Associations (including IEA); (2) Seats to ensure broad regional and scientific · Produce policy relevant representation. knowledge on a global scale Implementing 11 GDN regional networks · Build research capacity to agencies: advance development and Latest First external independent program-level program-level evaluation completed in March 2004 alleviate poverty evaluation: · Facilitate knowledge sharing among researchers and policymakers · Disseminate development knowledge to the public and policymakers 3. GDN has supported activities on two fronts: knowledge generation and knowledge sharing. To develop new knowledge, it has established Regional Research Competitions, a program of small research grants administered by the Regional Network Partners. GDN's x Global Research Projects engage research teams to address key development challenges common across regions on a comparative, cross-country basis. GDN facilitates knowledge sharing via its annual GDN Global Development Conference and electronic tools: a monthly e-mail newsletter, online discussions, and the GDNet Web site. GDN's Global Development Awards and Medals Competition take place at its annual international contest on development research. GDN considers its annual conference of researchers and policy analysts as one of its important outputs for generating and exchanging knowledge. GDN also issues publications out of some of the global programs it supports. GDN DESIGN, IMPLEMENTATION AND FINANCING 4. Conceived as a network of networks, GDN provides funding to regional research networks, called regional network partners (RNPs), for research grants to be allocated to individuals and institutions through regional competitions. The networks are at different stages of development with different needs and capacities. Some regional networks existed prior to the official formation of GDN, namely AERC, ERF, and LACEA. Others, like the East Asian Development Network (EADN) and South Asian Network of Economic Institutes (SANEI), were established with support from the World Bank's Development Economics Vice Presidency to help establish GDN. Through these research network partners GDN supports research of cross-regional interest in countries within each of the regions, and it fosters collaboration among countries and (through its annual conferences and global research projects) across regions. 5. There is lack of agreement on GDN's objectives and policies among its supporters and potential donors. This is not atypical of global programs at their initial stages. GDN's mission statement indicates that GDN itself does not conduct research or policy dialogue, but relies on others, including local researchers, to produce policy relevant outputs. GDN's objectives and policies will continue to evolve as GDN's considerable dependence on the World Bank diminishes and its largely restricted project based donor funding increases and diversifies. The current situation is reflected in year-to-year changes in funding for some key GDN activities. For example, on an accrued basis the share of global research programs increased from 8 percent in 2002 to 27 percent in 2003, and the share of support going to regional research competitions managed by the regional network partners declined from 32 percent in 2002 to 21 percent in 2003. The annual conference costs amounted to 21 percent of total expenses, or about $2 million in 2004, 30 percent higher than the previous year. 6. In FY03 73 percent of the total sources of funds reported by GDN came from the World Bank, including particularly the support for the core budget, which donors have been unwilling to finance. Bank support amounted to $6.54 million including $4.7 million of completely unrestricted DGF funding, $1.19 million for the Education Research component, and another $650,000 in the form of administrative budget, in-kind support, and support for the WDR. 7. To encourage GDN to diversify its funding base on a sustainable basis, Bank support from DGF had declined to $4.45 million in FY04, with $1.18 million for its Education Research Component. The Bank President has assured GDN of continued support, of up to $1.8 million, including underwriting annual administrative costs through 2006. The rest of GDN's funding comes from a variety of donors as restricted funding. GDN's non-World xi Bank financing increased from $1.7 million in FY02 to $4.3 million in FY05. Almost all the increase is project based. GDN's proposed work program for 2005-07 is predicated upon an increase in funding from around $8.5 million to $11-13 million, and it entails expanding GDN's Regional Research Competitions, Global Research Projects, and GDNet. GDN'S EXTERNAL EVALUATION FINDINGS AND RECOMMENDATIONS 8. In March 2004 GDN completed a full-scale, external program-level evaluation (hereafter IEE), as required by DGF. While the evaluation was managed by the GDN secretariat (rather than the GDN board as an evaluation ideally should be), the evaluation was strongly independent and reported its findings to the board orally in New Delhi at the annual conference and in writing in March 2004. The IEE concluded that the GDN concept meets a clear demand in the global market for knowledge and applauded the work of the small but enthusiastic and dedicated program Secretariat. It stressed that most of GDN's programs and activities are worth nurturing and concurred with the positive assessment of GDN activities presented in GDN's own internally commissioned assessments. However, the IEE identified what it viewed as GDN's three major strategic weaknesses: 1. Disagreement among important stakeholders on GDN mission and objectives; 2. Weaknesses in program governance; 3. Poor prospects for the long-term financial sustainability of the program; and the need to reexamine the decisions to locate GDN in India and to adopt an international organization (IO) status. 9. The IEE suggested various measures to address the identified gaps in mission clarity, governance, and financial sustainability, including: · Scheduling GDN global conferences less frequently and using the savings for regional networking; · Introducing competition into fund allocation among regional research competitions; · Appointing a GDN business board and converting its current governing body into an academic board; · Requesting the assistance of the Bank President in raising a sizeable endowment for GDN; · Appointing independent consultants with requisite skills to assess the GDN relocation decision and to determine whether the legal status of GDN as an IO is in the best long-term interest of the Network, and will help achieve financial sustainability. GDN has not accepted all the evaluation recommendations, including the suggestions regarding the establishment of a business board and an academic board, reviewing the planned move to New Delhi and acquiring International Organization status, or considering the merit of an annual conference. But GDN has developed a Business Plan and a Strategic Plan, established Executive, Audit and Programs Committees of the Board, and a Global Development Council to help fundraising from donors and the private sector. It held a Donor's Council Meeting in Paris in June 2004. xii OED FINDINGS Relevance: Are the Program's Objectives Right? 10. OED concurs with IEE's conclusion that the GDN meets a clear demand in the global market place for knowledge and enjoys dedicated staff. The program meets two of the four criteria for Bank involvement in global programs endorsed by the Development Committee in September 2000 (Annex A). It has the potential to add value to the Bank's development objectives, and information and knowledge are global public goods identified by Bank Management as deserving Bank support. Bank action was required to catalyze some research networks and research partnerships in developing countries. However, as in the case of some other partnerships that the Bank has helped establish, the initial Bank financing for GDN has resulted in challenges in mobilizing external resources as discussed below. As for the remaining Development Committee criteria, although there is an emerging international consensus that "global action is required" to support broad-based knowledge generation and sharing, there is lack of consensus on the kind of knowledge generation required and the kind of global action needed to generate knowledge. 11. The World Bank, the IMF and the African Economic Research Consortium prefer to support economic research with a view to ensuring focus and methodological rigor. The Bank reiterated its strong support at the Donors' Conference in Paris to the substantive aspects of GDN research by forming an effective bridge between development research in the international financial institutions and the research in developing countries. Such substantive interaction between GDN, the IMF, and the World Bank is increasing in various ways. Other potential donors such as DFID, SIDA and the Netherlands and some developing countries in East and South Asia, prefer social science research of an applied nature. With the strong demand from GDN's potential donors and some stakeholders in developing countries, GDN- sponsored research has broadened over time, with topics spanning the gamut of social science. Yet, GDN's mission, strategy and research focus - whether it should be largely economic or social science-oriented ­ remain debated. It is unclear how this issue will be clarified, whether through selection of research topics and the disciplines they require to conduct research, through some broadly understood parity among disciplines, or by the donor support a research portfolio can generate. GDN's future direction is likely to determine and be determined by the sources of funding. Similarly, while GDN's efforts initially focused on promoting North-South synergies, there is significant demand among the regional networks for GDN to promote more South-South exchange, collaboration, and synergy, providing them with opportunities to add (and seek) value from a global program supporting knowledge creation and dissemination among developing countries, and countries and regions in transition. It remains to be seen if donors would support such a South-South orientation. Efficacy: Has the Program Achieved Its Stated Objectives? 12. Programmatically GDN is on its way to establishing a distinct identity in each of its areas of activity. It distributed $14 million in 490 grants to researchers through regional networks in developing countries. More than 2,000 scholars from developing and transition countries participated in its competition in 2003. So far, GDN has funded three global research topics: Explaining Growth, Understanding Reform, and Bridging Research and Policy, with xiii the latter two managed outside the GDN secretariat. GDN is successful at tracking its outputs in terms of numbers of conferences, awards, and papers commissioned or published. But it is difficult and probably premature to assess incremental outcomes and impacts in terms of capacity built or policies impacted by the GDN, since its impacts mainly derive from its support for activities of its networks. Regional networks are responsible for country- and regional-level monitoring and evaluation of activities. The attribution challenge is compounded by the existence of some regional networks prior to GDN. Donors and governments were financing them and would have financed some of their research even in the absence of GDN. The issue of net additionality of resources and value added to the networks by GDN is further complicated by GDN's potential competition for resources from the country offices of some of the same donors in a given region and variations in this regard among regions. 13. GDN would also do well to define what constitutes "reasonable evidence" that grant funds are put to their intended use by the regional networks. Internally commissioned external reviews of GDN's regional networks conclude that the regional networks grant research awards in a competitive manner, and that the resulting research has been high quality and relevant. Yet, the OED global evaluation team found that transparency could be enhanced by regional networks providing detailed selection criteria with each call for proposals to minimize the charge of a privileged few benefiting from research grants. GDN could similarly improve competition by making grants available to researchers from countries and institutions with weak research capacity. GDN could also increase its impact on capacity building through training in research proposal writing, and better monitoring of the allocation and the use of resources. The decentralized nature of the GDN network has, however, posed challenges for GDN. Some of the well established existing regional networks of GDN such as AERC have complained about having to develop two different reporting arrangements, one for its board and another for GDN. Networks in Latin America and Eastern Europe have benefited from additional resources and inter-regional networking. Efficiency: Has the Program Been Cost-Effective? 14. The transaction costs of GDN have been high but may be diminishing. GDN design and governance affected its cost effectiveness in a number of complex ways. In FY02, GDN's (direct and indirect) administrative costs amounted to about 25 percent of its budget allocations, including costs of the organization of annual conferences and management and operation of GDNet. Overheads (non-allocable costs) and donor coordination amounted to 6 percent of GDN's budget. DGF grant support as share of program administration has been declining. In FY04 GDN received $600,000 from DGF for operating costs and administration out of a total grant of $4.45 million, i.e.13.3 percent. GDN expects to reduce costs further after the move to New Delhi. 15. Even with the present structure, there is scope for cost reduction. Some regional networks indicated to OED that they have been charging 20 percent of grants for administration, rather than the 15 percent mandated by GDN. GDN requires "reasonable evidence" that the funds were put to their intended use, but GDN neither defines the requirements for this evidence nor expects an externally audited statement from the regional networks. It is uncertain if GDN would be institutionally and financially sustainable if the Bank were to discontinue its support for the program's administrative costs. If more donors were willing to share in the overhead costs, they would perhaps also demand more cost xiv effectiveness in management. But donors prefer funding tied to specific activities. They prefer not to support overhead costs. 16. The design-related issues reflect an inherent conflict of interest, as eight of the 18 board members are nominated by regional networks that benefit from GDN support. Board members endorse decisions that directly affect regional resource allocation. The IEE recommended that competition should be introduced in regional allocation of funds using transparent performance criteria. OED concurs with this recommendation. 17. To improve efficiency, the IIE also recommended converting the current board into an academic board, with a new business board to oversee GDN's business strategy and activities. But the current board structure is a result of an international consultation led by the World Bank at International Economic Conferences, resulting in a strong legitimacy for the chosen structure. However, OED was informed by stakeholders, as were the external evaluators, that senior Bank managers involved in founding GDN conveyed a "take it or leave it" message to potential donors, in terms of how the GDN would be set up. Some saw this as a sign of Bank and GDN secretariat unilateralism, given that many donors have had considerable experience in setting up successful social science research networks, including some in which the Bank currently participates. 18. Whether funding will drive the program, or the program strategy will drive funding is also closely linked to GDN's recent decision to change its status to an international organization with potential ramifications for its governance, financing, and eventual cultural ethos. After considerable consultation and debate, GDN's Board has decided to change its status to an International Organization on grounds that this change will provide GDN with a level of visibility and status that some in GDN felt it could never achieve as a non-profit organization incorporated in India. The new status will bring immediate financial benefits in terms of tax exemptions and privileges for GDN's staff in New Delhi and Cairo, resulting in some savings in overhead costs. It will also provide immunities and legal privileges, making it easier to hire international staff. The IO status will influence GDN's overall character, but how it will influence the research agenda is too early to tell. Some donor signatories to the IO status would provide financial support and have expressed interest in being formally involved in GDN and in influencing the GDN research agenda. According to the proposed new charter, donors have the option of signing the Agreement and placing a representative on the GDN Assembly. Others would form part of the Donor's Council. 19. The decision to move to New Delhi and change the organization's status divided board members. The external evaluation concluded that the decision to locate in India was not fully vetted by the board. OED however concluded that considerable discussion of the options took place before New Delhi was selected. Yet, OED agrees with the external evaluation that the move to Delhi is inextricably linked to the IO status. OED also concurs with the IEE observation that not all board members may have fully appreciated the complexities involved or the full implications of the decision to move to the IO status on GDN's character and research agenda. 20. The GDN board seems to be of a unanimous view that research priorities should be driven by client needs and professional expertise and that GDN should remain free from donor influences in setting them. Yet, moving to IO status could fundamentally change the xv character of GDN's governance and strategy. The move to shift to an IO status is inextricably linked to the move to New Delhi. GDN believes the IO status will provide it with a level of visibility and status, financial benefits, and immunities and legal privileges, including the ease to hire international staff it would not enjoy as an NGO registered in India. Second, GDN management's decision to move to an international organization status requires a revised GDN charter with endorsement from sovereign states. Signatory donors and developing countries have perceived GDN as a World Bank creation. They indicated they would like a larger say in its governance and in defining a broader social science research agenda, and have been reluctant to finance overheads. How the move to the IO status would affect GDN's funding and governance therefore remains unclear. It also remains to be seen whether increased fund- raising efforts will ensure a sustainable flow of funding for GDN, particularly for its core budget. Currently, only IDRC has expressed interest in providing financing for the core budget. 21. The other issue relates to the net addition of donor or country resources GDN would be able to mobilize and the way it would influence the quality of research already being conducted by the regional networks. In several developing regions GDN is competing with regional institutes that receive support from the same set of donors. The Bank's decision to channel funds through GDN rather than to directly fund networks, as it did in the past also resulted in increased costs and some disruption for the AERC, the most established of the networks supported by the Bank. In addition, GDN's advocacy of a multidisciplinary focus for all networks violated the principle of network independence in research priority setting to which the AERC board is strongly committed, causing consternation in the AERC board. GDN secretariat has indicated that these issues between GDN and AERC have been resolved. 22. The Board has decided not to move towards bi- or tri-annual conferences, as recommended by the external evaluation (with the savings used for South-South knowledge sharing). GDN argues that there are scale economies in combining the annual conference, the donor meeting, the meeting of GDN's Board, workshops on the Global Research Projects, and presentations by the finalists for the Global Development Awards. The awards and medals competitions, which account for about 10 percent of GDN resources, are rated as an effective incentive mechanism to encourage research. Tailoring award size to actual research costs in different regions may be another cost saving measure. 23. A concern expressed by several stakeholders to OED is the perception that GDN's orientation and expenditure patterns may not reflect those of a newly formed research network struggling to raise resources. The program's lack of long-term funding alternatives has meant dependency on the Bank. At the Donors' Conference in Paris in June 2004, donors stressed the importance of a larger marketing effort by GDN but also expressed strong support for GDN's policy orientation and work plan, and several offered to consider the proposal for financing. Program staff, however, acknowledge the difficulty of ensuring long- term funding sustainability for GDN, in view of the annual budgetary appropriations of most current donors, although this practice provides a "valuable external check on the relevance and quality of GDN's programs." xvi Bank Performance 24. Since noting the findings of the external review, the Bank has stressed the need for GDN to develop a broader funding base, a sounder financial footing, and to manifest its intellectual and operational independence from the Bank in a credible and concrete fashion. The Bank has also indicated that its financial support and commitment to become signatory to GDN's IO status will depend on the evidence that the Strategic and Business Plans have been effectively operationalized with a view to achieving sustainable change in the financial arrangements. The Bank has recently designated a senior manager from the Development Economic Vice Presidency to carry out independent oversight, and to build bridges with GDN's research activities, while maintaining an appropriate arm's length relationship from the program. OED views this as a very appropriate decision. 25. Bank sponsorship of GDN, the Bank's past processes, and the management style of GDN itself, have promoted a view of GDN as an "offspring of the Bank." The reasons reported to the external evaluators and to OED varied but included GDN's initial "take it or leave it approach," the perceived emphasis on economic research and research closely aligned with the consensus of Bretton Woods institutions, the inadequate link to the research priorities and challenges of developing countries, the high profile participation of Bank managers and Bank-favored economic scholars in GDN conferences, and the style of the secretariat headed by a former senior Bank manager. Although GDN's governing board enjoys strong legitimacy, the program's close linkages with the Bank through its genesis and heavy financial dependence have led to considerable divisions within the Board, exacerbated by a decision by GDN to second its staff to the Bank. 26. Although there is a widespread impression that the Bank's style of operation and culture have had a broad impact on GDN, the Bank has until recently had little oversight of GDN. The Bank board member supported GDN positions in the Bank without addressing the Bank's institutional interest in promoting cost effective development research. As observed by the IEE, GDN suffered from a paradox: the perception of Bank influence and the reality of a hands- off Bank oversight policy. Indeed, in some sense GDN could even be thought of as a victim of benign neglect: with GDN championed by the Bank's President and headed up by a former senior Bank manager, there was little pressure and little interest in the Bank to provide the close oversight GDN needed and deserved. The OED team observed that the GDN board members often lacked access to key information on GDN finances, salary structure, secretariat costs, and that there was a poor relationship between the priority setting process and the fundraising strategy. The constructive discussion of the external evaluation in various fora is already leading to steps to improve governance through such measures as the formation of board committees. 27. Too close a Bank involvement, including underwriting of GDN overhead costs, made it difficult for GDN to develop an independent identity and to raise resources for overhead costs from other donors. The move to New Delhi has been intended to ensure an arm's length from the Bank. Bank membership on the IO status from the outset would continue the impression of close Bank involvement and would potentially present a moral hazard. The Bank would be expected to support GDN financially regardless of its long-term financial viability. This would jeopardize GDN's ability to stand on its own feet while leading to continued questions of its intellectual independence. By maintaining an arm's length xvii relationship from the program's IO status, while continuing to provide GDN with DGF funding on the basis of a diminishing share of total operating costs, the Bank would improve the program's chances of achieving full independence. FINDINGS AND LESSONS (1) GDN faces a number of interrelated uncertainties and related risks well identified by the external evaluation. GDN is diversifying its funding base, operationalizing its Strategic and Business Plans, establishing itself as an International Organization, modifying its governance structure, moving its headquarters to New Delhi, and developing an independent identity. To place GDN on more sound, independent footing, Bank Management has correctly concluded that until GDN has effectively carried out the necessary steps, it is premature to commit to becoming a signatory to GDN as an international organization. First, GDN's external evaluation argued that the move to shift to an IO status is inextricably linked to the move to New Delhi. GDN believes that the shift to IO status will provide it with a level of visibility and status, financial benefits, and immunities and legal privileges, including the ease to hire international staff it would not enjoy as an NGO registered in India. Second, GDN management's decision to move to an IO status requires a revised GDN charter with endorsement from sovereign states. Potential signatory donors and developing countries have perceived GDN as a World Bank creation. They would like a larger say in its governance and in defining a broader social science research agenda, and have been reluctant to finance overheads. How the move to the IO status would affect GDN's funding and governance therefore remains unclear. GDN management believes that funding prospects will be no worse and probably better with IO status than under the alternative status of a non-for-profit organization incorporated in India, and will undertake the move to IO status only if the final version of the Charter preserves in its judgment the organization's independence as a research network. Third, Bank membership on the GDN board, would require endorsement of the Bank's board of governors before which a number of risks and uncertainties need to be resolved. For example, GDN's proposed work program for 2005-07 is predicated upon an increase in funding from around $8.5 million to $11-13 million, and it entails expanding GDN's Regional Research Competitions, Global Research Projects and GDNet. The Bank has rightly concluded that by maintaining an arm's length relationship with the program's IO status, the Bank would help GDN to become financially viable and develop an independent identity. Concurrently, by helping to strengthen its governance and management, the Bank will help improve GDN's chances to obtain support for overhead costs from others. At the same time, the Bank could continue to provide GDN DGF funding on the basis of diminishing share in overall finances. (2) The Bank needed but lacked a well developed strategy to ensure sustainable spin- offs and to manage appropriately the potential risks of a program moving away from the Bank. Although the lessons of GDN and other such programs are leading to a clearer Bank strategy, GDN lost valuable time in the process to ensure its own long-term viability. GDN's sustainability after the Bank disengages from its promised support for the GDN administrative costs in FY06 is still unclear. Some of the IIE xviii suggestions, including developing long-term operating and financing strategies to ensure multi-year funding or developing a sizeable endowment using the Bank's convening power, are being explored by GDN. OED interviews indicated that developing an endowment is unlikely to be a feasible option. (3) The Bank needs to establish independent oversight of global programs. This requires separation of management from oversight, counseling Bank staff participating on program boards as to the institutional expectations in serving on boards, and Bank support for periodic independent external program evaluations. Given its role as a founder and its strong financial support, the Bank needs to ensure that GDN establishes a strong identity as an independent institution that adds value to the Bank's body of knowledge and policies. 1 1. Introduction and Context: Global Challenges in the Sector GLOBAL CHALLENGES IN RESEARCH AND KNOWLEDGE USE 1.1 The development community recognizes that information and knowledge are an increasingly important source of sustainable development. Developing countries will continue to face major challenges unless they acquire the capacity to effectively develop and use information and knowledge. To address policy reforms and bring about institutional change, contextual knowledge suited to country circumstances is important. Whereas the information age provides new opportunities for connectivity, developing countries face a widening knowledge gap. The gap is a result of little or no investment in research, inadequate utilization of existing research, poor analytical capacity within developing countries to effectively generate and apply knowledge, and insufficient collaboration among researchers in developing countries and their developed-world counterparts. The task is to effectively link local, regional, and global knowledge and learning and apply it to problem solving. 2. Program Alignment with Global Challenges and Bank Priorities OVERVIEW OF PROGRAM GENESIS, MISSION, OBJECTIVES, AND ACTIVITIES 2.1 The World Bank launched GDN in December 1999 at the program's first annual conference in Bonn, Germany following a substantial consultative process addressing its governance structure and membership. GDN started as a joint effort of the World Bank Institute and the senior managers in the Bank's Development Economics Vice Presidency, and it has enjoyed strong support of World Bank President James D. Wolfensohn, who has led the effort to convert the World Bank into a knowledge bank. In 2001, GDN moved out of the Bank and became an independent, not-for-profit organization based in Washington, D.C. The program is planning to relocate to New Delhi, India, in 2005 and it is working towards acquiring an International Organization status. 2.2 GDN's stated objectives are to: · Support multidisciplinary research in social sciences · Promote the generation of local knowledge in developing and transition countries · Produce policy relevant knowledge on a global scale · Build research capacity to advance development and alleviate poverty · Facilitate knowledge sharing among researchers and policymakers 2.3 The program works toward meeting its objectives through five clusters of activities.1 · Regional Research Competitions (RRCs)--a program of small research grants administered by the RNPs. By February 2004, GDN had disbursed more than $14 million and awarded over 490 grants through this program. 1. GDN has recently initiated a sixth activity, RAPNet, to give researchers access to statistical databases. 2 · Global Development Awards and Medals Competition--the largest annual international contest for the best research on development produced by researchers from developing and transition countries. More than 2,000 scholars have participated in this competition. · Global Research Projects (GRPs)--major projects involve research teams worldwide that address the most pressing issues of development. GDN has implemented three projects in a comparative, cross-country context--Explaining Growth, Understanding Reform, and Bridging Research and Policy. · Annual Global Development Conferences--international forums for exchanging ideas on sustainable development and poverty alleviation, with the focus on research generated in the developing world. More than 2,500 researchers from over 100 countries have attended GDN's five annual conferences since 1999. · GDNet--the electronic voice of GDN provides researchers and policymakers with access to an online library of scholarly papers and databases of leading research institutes and researchers. The monthly GDNet newsletter currently reaches more than 13,000 subscribers. 2.4 The distribution of GDN funding on an accrued basis among the five different activities for two years shows year-to-year changes (Figure 1) for some of the key activities. These activities and allocations to the projects have been the subject of an external evaluation and board discussion. The noteworthy changes are the large increase in the absolute allocation to global research projects and its increased share in the total. The GDN Secretariat has managed some of these projects centrally. Management of other more recent global projects has been delegated to others. Resources going to regional research competitions declined slightly in absolute terms but substantially in relative terms. The other major increase was in the cost of the annual conference, which, at about $2 million, cost a third more in 2004 than the previous year. 2.5 GDN's interactions with its member research networks ­ called regional network partners, or RNPs ­ focus on supporting research in individual countries within regions; fostering collaboration among countries within the regions; promoting exchange on issues like research methods, quality control, and research findings; and ensuring a focus on policy impact. Regional hubs in Europe, North America, Japan, and the South Pacific region have been established to complement developing country networks. TWO ALTERNATIVE MODELS FOR THE PROGRAM 2.6 Prior to GDN's establishment, two alternative models had been considered for the new global program within the World Bank. One model was to promote open competition among various institutions, with funding going to the most qualified institutions that had submitted research proposals. A second model centered on pre-selected institutions within regions, which would serve as regional hubs and nominate members to the board. 2.7 The second model prevailed. As a result, the concept of designated research institutions serving as regional network hubs for funding and activities became the foundation for GDN's design and programmatic structure. GDN's regional hub approach has helped it to quickly develop processes for regional research competitions and other program 3 activities and move to demonstrate results on the ground while keeping the secretariat staff lean. However, the secretariat costs have not been small and networks are at different stages of development with varying needs and capacities. Some of the regional networks existed prior to the official formation of GDN, namely AERC, ERF, and LACEA, but others, such as the East Asian Development Network (EADN) and South Asian Network of Economic Institutes (SANEI), were established with support from the World Bank's Development Economics Vice Presidency to help set up the GDN. Year-to-year changes in the share of different activities are shown in Figure 2. Figure 1. GDN Expenditures on Program Activities, 2002 and 2003 Global Research Projects Annual Conference Regional Research Competitions Education GDNet 2003 Global Development 2002 Awards Other programs 0.0 0.5 1.0 1.5 2.0 2.5 3.0 $US Source: GDN 2003 Annual Report. Program expenditures do not include supporting services (donor coordination and secretariat expenses). Figure 2. Percent Shares of GDN Activity Expenses, 2002 and 2003 2002 Global Research Projects 2003 Annual Conferences Regional Research Competitions Education GDNet Global Development Aw ards Other Programs 4 ALIGNMENT WITH GLOBAL PUBLIC GOODS AND CORPORATE ADVOCACY PRIORITIES 2.8 The program meets two of the four criteria for Bank involvement in global programs endorsed by the Development Committee in September 2000 (Annex A). It has the potential to add value to the Bank's development objectives, and it requires Bank action to catalyze other resources and partnerships. Moreover, information and knowledge are global public goods, identified by Bank Management as deserving Bank support. As for the remaining two Development Committee criteria, although there is an emerging international consensus that "global action is required" to support broad-based knowledge generation and sharing, as GDN's 2004 external independent evaluation stressed, there is lack of consensus as to the kind of global action needed to generate knowledge and as to the appropriate kind of knowledge most in need of generation. Second, the Bank's comparative advantage in such institution building is in promoting relevant policy and operations research but it has not deployed it fully until recently. Moreover, the Bank initiative in establishing the network has had some costs in developing ownership and financial contributions of other donors. It has also resulted in higher overhead charges by networks for GDN-supported research than GDN guidelines suggest, as demonstrated below. 2.9 Notwithstanding these caveats, evidence to date suggests that coalescing of regional networks and researchers can produce intra- and interregional synergies. While GDN's efforts have largely focused on promoting North-South synergies, there is a significant demand from the regional networks for GDN to promote South-South exchange, collaboration, and synergy, providing GDN with opportunities to add value as a global program in this area. Although GDN-sponsored research may be highly relevant to improving the condition of the poor, in its early years some GDN board members cited what they term GDN's neoclassical economic and elitist approach. They noted that "the GDN may aggravate existing levels of social exclusion, inequality, and disarticulation in the global knowledge economy given the `scientific,' `technocratic' and elite character of the Network." This view was confirmed at the 2004 Donors' Conference in Paris. Some observers also see the program as too closely aligned with the consensus of Bretton Woods institutions and as inadequately linked to the research priorities and challenges of developing countries. At the same time, the World Bank, the IMF and researchers in some networks (e.g. AERC) are concerned about the dilution of the program's focus and the complexity of multi-disciplinary research. GDN has undertaken a large research project for the IMF, drawing on its links to the local researchers. The Bank's Development Economics Vice Presidency has expressed interest in developing bridges with GDN to undertake substantive research of interest to the international financial institutions and has recently announced its decision to hold the Annual Bank conference on Development Effectiveness (ABCDE) in Dakar, Senegal, following GDN's annual conference. GDN's more recent annual conferences and awards suggest that GDN's research support may have been changing in response to the external criticism that it is too closely aligned with the Bretton Woods Institutions generally and with the Bank in particular.2 Although GDN research, networks, and participants were initially predominantly economics-oriented, with the strong demand from the stakeholders in developing countries and GDN's potential donors, GDN-sponsored research has expanded over time, with topics that now span the gamut of social science. The balance of these various cross-currents on 2Stone 2001. 5 GDN's future research scope and agenda will evolve over time and affect its funding prospects. 3. Outcomes, Impacts, and Sustainability 3.1 GDN's performance is of interest in terms of 1) what is known about its outcomes and impacts, and 2) the quality of its monitoring and evaluation activities. MEASURING GDN OUTCOMES AND IMPACTS 3.2 GDN's outputs are relatively easily measurable, and GDN has used internal assessments to inventory its products. The program has a number of performance indicators for the five core program activities used to measure GDN's progress and effectiveness (see Table 1). These performance indicators focus mainly on outputs, and they track, among others, the number of submissions or approvals for papers and awards and numbers of case studies produced. While GDN has been successful at collecting such quantifiable data, it has been less successful at tracking less straightforward indicators, such as the quality of research produced or its impact. Table 1. GDN Performance Indicators for Key Program Activities Program Performance Indicators Progress Activity Regional # of submissions, # of approvals, # of Since 1999 almost 490 projects funded with more Research completed projects. Quality judged by than $14 million. Competitions peer review. Global Research # of country studies and participating 3 global research projects funded. 1st project: 24 Projects researchers as measure of capacity thematic papers and 80 country studies. 2nd building and networking. Published books, project: 10 background papers, 30 country papers, and other referenced work studies. 3rd project: 50 summary case studies in measure the quantity and quality of the 1st phase; 4 background papers, 3 comparative output. Links to policymakers and studies, and call for proposals for additional involvement of policymakers in research studies. projects. Global # of submissions and the quality of the Since 2000, 1,537 participants representing over Development final papers and projects. 100 countries. More than $1 million distributed in Awards awards and travel to finalists and winners. # of Competition applicants rose from 402 in 2002 to 470 in 2003. This represents a lower number of applicants than in the program's first year of operations when 784 applications were submitted. Ninety-three countries were represented by applicants in 2002, decreasing to 78 countries in 2003. Annual Global # and geographical distribution of # of participants increased from 532 in 1999 to Development participants and the range of donor 673 in 2004. # of countries represented remained Conferences support for the conference. almost unchanged, with 100 countries represented in 1999 and 102 in 2004. Poor donor support. GDNet # and geographical distribution of # of subscribers rose from 7,200 in 2003 to subscribers and # of entries into the 13,000 in 2004. # of website hits rose by 66 institute, researcher, and document percent between 2003 and 2004. directories. Source: DGF documentation. 6 3.3 As a result, outcomes and impacts have been and will remain more difficult to measure. The creation and sharing of knowledge does not ensure that research findings will be applied in policymaking or institutional reforms. The causal relationship between knowledge/research and decision making is often unclear and indirect. Additional evaluation challenges include the difficulty of realizing and assessing the benefits of research collaborations, networking, and capacity building, including establishing attribution to GDN and identifying comparators. 3.4 To assess its performance, GDN has also used surveys of conference participants and network members. Yet, GDN's surveys of participants, who are the direct beneficiaries of the program, have the potential to introduce a selection bias, as the differences in the findings of the GDN internal and external evaluation illustrate. For example, GDN's Secretariat conducted a survey during the program's Fourth Annual Conference in January 2003 in Cairo, Egypt. Respondents ranked GDN's global conferences as the program's most valuable undertaking and GDN's online activities as the least valuable. The external evaluators came to the opposite conclusion, namely that the conference was the most questioned GDN activity and that the program's online activities enjoyed widespread support. Conditions, needs, and responses also vary considerably among regional networks. GDN's Value Added 3.5 GDN's value added could be considered in several ways. One of the justifications for organizing global programs is to generate new sources of funding for development activities. Therefore, one important question is whether GDN has helped mobilize substantial incremental resources for global knowledge sharing and research beyond those contributed by the Bank, and whether these resources are coming from new sources or are largely reshuffling existing ODA? Another important question is whether the establishment of GDN has tightened competition for donor financing for research activity and networking? 3.6 For regions, such as Latin America, which have not benefited from external donor research funding, GDN is providing incremental research resources, even if the resources may not be additional to overall ODA funds. In addition, GDN has provided a framework for inter-regional collaboration and for cross-disciplinary collaboration, a contribution researchers seem to appreciate. Researchers in Eastern Europe and Latin America have indicated to OED that they are beginning to turn to researchers in other regions for advice and peer reviews of research proposals. Researchers also indicated that the Bank's support to the South Asia network in anticipation of the establishment of GDN helped to initiate a framework for research collaboration between India and Pakistan, at a time when the environment for such collaboration was not as propitious as it is today. Similarly, EERC was started with World Bank support in 1995 when the environment for independent policy research was not as conducive in Eastern Europe as it is today. 3.7 GDN, however, may face competition for resources from some of the same bilateral donors and foundations that support existing research networks and small struggling local research networks. AERC indicated this to be a potential problem, although currently donor commitment to AERC's principles and practice remains strong. The Bank's current practice of channeling funds through GDN rather than directly supporting AERC and other networks has increased costs, and disrupted research network activities that used to receive support directly from the Bank. For instance, AERC has successfully built African research capacity 7 while allowing an independent research priority-setting process to drive AERC's research mobilization effort. Although AERC continues to receive funding from the Bank, channeling Bank funds through the GDN has distorted the priority-setting process, increased costs and severed AERC's direct link with the Bank. It remains to be seen if GDN will compete with researchers in the South Asia region to obtain funding from national governments and for resources from locally based international donors once it moves to New Delhi. The second challenge is one of leakage of resources from GDN to researchers in industrial countries. Some critics have observed that, at least at its initial stage, GDN gravitated toward researchers in developed countries and researchers in developing countries that already had sufficient access to the policy-making establishments to muster resources they needed for research.3 For example, although the global research project, Explaining Growth, was highly valued by researchers and led to exchange of experiences at workshops and conferences at the regional and global level, some observers say that GDN supported country teams whose members were developing country researchers residing permanently in OECD countries. More recently, though, researchers from developed countries have been barred from participating in GDN-sponsored regional research competitions, rendering the criticism of support for researchers in developed countries obsolete. 3.8 Measuring the value added by GDN also requires examining the objectives and output of other research- and policy-focused networks. At the time of GDN genesis in 1999, a demand and supply analysis of 500 research institutes in developing countries gauged the knowledge service market from the researcher and policy maker perspective to determine whether GDN's various products and services would add value. A second study in 2000 identified and described other institutional sources of products similar to those supported by GDN.4 3.9 The findings of the study, Donor Support for Policy Research Networks, based on a database of 132 research-oriented networks including 68 with a strong policy focus, "strongly suggest that GDN's efforts in this area will provide an important service to the global research and policy community."5 Yet, the study found that GDN is not unique in its global research focus. Out of the 68 policy-focused networks, almost half have a broad development focus. Sixty-five percent of the networks have a global focus, while 35 percent cover a single region. Most networks target a broad audience: 47 percent reach out to at least two of four designated groups ­ academics, nongovernmental organizations and local stakeholders, civil society, and policy makers - with a substantial number targeting policy makers (28 percent) or academic researchers (21 percent). Network service and content vary, ranging from global wide-range service providers to on-line searchable databases. Almost half of the networks combine a single-issue focus with a broader range of services. Multiple- issue service providers make up 13 percent of networks, with those operating at a global level 3. Reviewers have also suggested formulating a strategy for early dissemination of research findings in future GDN-supported global research projects. GDN has acted on this recommendation. In GDN's second global research project, Understanding Reform, launched in 2001, first drafts of 10 thematic research papers were reviewed at a special workshop held during GDN's 2003 conference. They are currently available on GDN's web site. 4. The GDN High Level Committee (HLC) was formed to oversee this work and evaluate the progress and effectiveness of GDN in light of the results of the supply and demand analyses. 5. Global Development Network 2000, p. 6. 8 comprising 4 percent. Jury is still out on whether GDN would be unique without the Bank support for its overhead operations. 3.10 The study found that three other networks resemble GDN in providing a wide range of global-level content and services: the Institute for Development Studies (IDS), the International Center for Economic Growth (ICEG), and the World Institute for Development Economic Research (WIDER). While all three networks place a strong emphasis on research, member institutes undertake much of the research supported by ICEG. ICEG also strongly focuses on disseminating the results of research projects to policy makers. Meanwhile, IDS provides access to extensive information on international development, including up-to-date commentaries on major documents and issues. Some stakeholders have noted that the new European, Japanese, North American, and South Pacific hubs will expand the program's partnership base from the regional networks. The creation of developed country-based research networks would strengthen North-South research linkages and develop cross-border synergies that address overlapping demands for specialized knowledge. GDN could fill the South-South collaboration niche. QUALITY OF MONITORING AND EVALUATION 3.11 GDN has a strong record of self-evaluations: a number of internally managed external evaluations. GDN commissioned assessments of selected program activities, including its Global Development Conferences and Global Awards, which provide insight and information about particular parts of the program. GDN completed in March 2004 an external review of governance, management, funding, resource use, and other aspects of the program.6 Ideally an external evaluation should draw on these internal evaluations but should be completely independent and should be managed by the board and report to the board. The GDN external evaluation was managed by the GDN secretariat, and its findings were first reported orally to the board and donors at the annual conference and, then in writing. The evaluation had a substantial level of independence and it covered most critical issues facing GDN. 3.12 The IEE noted the uniqueness of the GDN concept for meeting the demand for development-related knowledge, as well as the fact that the enthusiastic and dedicated staff of the program's small Secretariat are doing "valuable work," and that most GDN programs and activities are "worth nurturing." Yet, the IEE did not make the case for why the GDN is unique, a statement which the findings of the Donor Support study also contradict. The IEE also differed substantially in some of its assessments from the GDN internal evaluations and recommended actions to correct what it viewed as GDN's three major strategic weaknesses: 1) Disagreement among important stakeholders on GDN mission and objectives; 2) Weaknesses in program governance; 3) Poor prospects for the long-term financial sustainability of the program, and the need to reexamine the decisions to locate GDN in India and to adopt an international organization (IO) status. 3.13 The IEE suggests various measures to address the identified gaps in mission clarity, governance, and financial sustainability, including: 6. Muth and Gerlach 2004; Global Development Network 2004. 9 · Scheduling GDN global conferences less frequently and using the savings for regional networking; · Introducing competition into fund allocation for regional research competitions; · Appointing a GDN business board and converting its current governing body into an academic board; · Requesting the assistance of the Bank President in raising a sizeable endowment for GDN · Appointing independent consultants with requisite skills to assess the GDN relocation decision and to determine whether the legal status of GDN as an IO is in the best long-term interest of the Network and will help achieve financial sustainability. · The IEE focused on governance as a key long-term determinant of both efficacy and efficiency. By governance, OED means the governing board's ability to provide strategic direction and oversight for management, undertake consultative processes, manage risks, and be responsible for program evaluation and audit functions. GDN has not accepted several of the evaluation recommendations, for example, in such areas as establishing a business board and an academic board, reviewing the planned move to New Delhi and acquiring International Organization status, and changing the frequency of annual conferences. But the GDN has developed a Business Plan and a Strategic Plan, established Executive, Audit and Programs Committees of the Board, and a Global Development Council to help fundraising from donors and the private sector. It held a Donor's Council Meeting in Paris in June 2004. OED FINDINGS 3.14 OED concurs with the IEE that a global network such as GDN can fill a gap in knowledge generation and dissemination, provided it helps strengthen research and policy analysis in developing and transition countries. Since it is too early and too difficult to assess GDN outcomes and impacts, the external review argued that GDN's efficiency, impacts and long-term sustainability depend crucially on the quality of GDN governance. OED agrees that governance is crucial for ensuring the program's sustainability, but it finds that program design, implementation and financing, among others, are also important considerations. On some issues OED arrives at different conclusions and derives different implications than those of the external review. The findings and recommendations of the external review have been used to indicate where OED concurs and differs from the external evaluation. Global Conferences: Scheduling GDN Global Conferences Less Frequently and Using the Savings for Regional Networking 3.15 GDN spends over 20 percent of its annual resources on conducting a yearly conference for researchers, inviting well-known development experts from around the globe, including Nobel laureates, to open the events. According to the internal evaluation, the cost per participant has decreased from $3,500 in 2003 to 2,700 in 2004.7 Yet, since the number of participants has increased from 596 in 2003 to 673 participants in 2004, the overall conference costs have also increased. 7. Global Development Network 2004, p.v. 10 3.16 Although these conferences garner significant attention and networking opportunities and receive high marks from participants, a number of GDN Governing Body members and regional network participants OED contacted noted that the high-profile meetings may be too costly for the program to sustain, as they divert limited financial resources away from regional research and limit the attention GDN managers can give to other activities. The fact that the annual conferences take up significant program resources to plan, organize, and stage also suggests that GDN may be better served by a bi- or-tri-annual meeting, as opposed to the current annually scheduled event. OED interviews suggested a willingness of some board members to consider a less frequent conference schedule, with the savings diverted to building South-to-South networking and regional research capacity. However, GDN argues that the costs of GDN secretariat time and effort are fully taken into account in estimating the costs of the conference and there are scale economies in combining the annual conference, the donor meeting, the meeting of GDN's Board, workshops on the Global Research Projects, and presentations by the finalists for the Global Development Awards. It is difficult for OED to second guess the arguments of the secretariat. Ultimately donor willingness to fund overheads and conferences would determine if the return to the conferences is sufficient, when the opportunity costs of the financial resources and GDN secretariat time and effort to organize them are taken into account. 3.17 Some stakeholders contacted by OED expressed the belief that the conferences are primarily geared to economists and are Bank-centric, with involvement of developing countries limited to economists from a few elite institutions well connected with the World Bank. Another frequently heard observation was criticism of inviting observers to conferences with no active role in the proceedings, resulting in little learning for conference participants. The Cairo survey corroborated the perception of an imbalance among disciplines within the GDN community, especially in reference to the annual conference. The overwhelming majority of survey respondents were economists ­ 76.7 percent. Interestingly, the survey found that conference participants with broader social science and multidisciplinary backgrounds were more likely to come from nongovernmental and international organizations. 3.18 GDN has re-calibrated the program to reach out to non-economists within the social sciences and to cover a larger number of micro and sectoral topics of interest to other social scientists. The success of GDN efforts in this area is reflected in increased interest and participation of a variety of social scientists in a range of program activities, including a broadening of the global conferences to actively appeal to the research priorities of political scientists, sociologists, and other social science practitioners in developing countries, as well as to the immediate interests and concerns of policy makers and other government decision makers in countries. One example of this success is the 2002 round of regional research competitions held by the Center for Economic Research and Graduate Education in Eastern and Central Europe, in which applications from non-economists increased by 158 percent, as compared to a 42 percent overall increase in applications. Introducing Competition into Fund Allocation for Regional Research Competitions 3.19 Although regional networks are urged to exercise the same level of care in administering GDN grants they use in administering their own funds, OED observed that GDN does not require a speedy report on regional network use of grants and activities undertaken: no 11 reporting is due back to GDN from the regional networks until 15 months from the time grant monies are expended. This is in part because the established networks have their own reporting arrangements and are not too keen to have yet more layers of reporting. Thus, while GDN requires "reasonable evidence" that funds are put to their intended use, there is no defined standard of evidence required from all the regional networks for the use of grants, nor does the program require externally audited statements from the networks.8 3.20 A GDN commissioned review9 of GDN's regional research competitions notes that the regional hubs ­ to which the program allocates one-third of its resources ­ grant research awards in a competitive manner and have been an effective tool for building regional research capacity. The review concludes that research generated by the competitions has been high quality and policy relevant, with the "potential to contribute to development." The 2004 internal evaluation by GDN's Secretariat noted that the program provides an "effective system of mentoring grantees by experts, special training programs, and `quality assurance' mechanisms to improve research results." 3.21 Notwithstanding these positive reports, it is difficult to ascertain the extent to which regional competitions are broadly reaching researchers and strengthening the capacity of research and policy institutions in developing countries and in-transition economies to undertake high-quality, policy-relevant research. It is not clear whether GDN has specifically adopted the objective of reaching or seeking out little-known institutions and individual researchers with particularly constrained access to resources, including access to computers, or whether a focus on working with pre-selected, relatively well-known institutions fulfills the stated objective. 3.22 Dissemination of both the announcements of competitions and the results of research has been an issue. According to the 2004 internal evaluation by GDN's Secretariat, the program is served by a "reliable and effective grant-giving infrastructure." The internal evaluation observes that grants competition announcements and results are featured on individual RNP Web site and all working papers from GDN-funded research are posted on the Web.10 While it is difficult to identify and measure specific research-policy linkages, the OED team observed that the regional networks seem to be taking steps to reach out to policy makers. For example, the African Economic Research Consortium (AERC) has organized policy forums to support initiatives in long-term development financing and trade negotiation. Meanwhile, the South Asia Network of Economic Research (SANEI) has interfaced with policy makers on various levels, with SANEI Steering Committee members often advising their governments and even going on to serve in influential positions within the public sector.11 However, it is not clear how much of this is an incremental effort caused by GDN, as distinct from the activities of the networks themselves. 8. However, GDN does undergo an annual financial audit at the program level. 9. Craig and Loayza 2003. 10. According to the 2004 internal evaluation, each of the nearly 490 research projects funded by GDN since its inception has produced or is expected to culminate in a working paper. 11. For example, the former head of SANEI, was recently appointed Deputy Governor of the Reserve Bank of India. 12 3.23 The OED global evaluation team concluded that providing detailed selection criteria with each call for proposals could enhance transparency. Competition could improve by making grants available to researchers from countries with weak economic institutions, and capacity building impact could be increased through the addition of a training and monitoring program for researchers. GDN and its regional networks could more routinely publish data which indicate how many applications were received from which institutions and which countries and who were the beneficiaries of the competitive grants. GDN and its regional networks could also hold short-term training courses in proposal writing and indicate which of their activities are a result of GDN financing. Evolution of Awards and Competitions Towards Greater Focus on Intended Beneficiaries and Their Greater Inclusiveness 3.24 According to GDN, its award competition has been one of the program's most visible and successful activities. The competition has reached out to researchers and development practitioners from every corner of the globe, providing encouragement to deserving, often little-known researchers. GDN-sponsored research competitions receive high marks from program stakeholders for their relevance, importance, and positive impact. Many participants report that the competition helped them continue their research, increased the visibility of their work, and made them feel like participants in a broader community of development researchers. GDN management cites evidence that the program has had a significant positive impact on individual awardees, in addition to promoting, supporting, and advancing global policy-oriented research. Accounting for some 10 percent of GDN resource expenditures, OED views the competitions as an effective incentive mechanism for encouraging research in developing and in-transition countries. Accomplishments include: · GDN finalists and winners have been successful in getting policy makers directly exposed to their work, presenting their ideas to relevant stakeholders at seminars and workshops organized by their institutions, country governmental bureaus, and international organizations. · Many of the works have been disseminated in academic workshops and at national and international conferences. Two finalists of the Japanese Development Project Award created documentaries to disseminate their initiatives. · The work of many GDN finalists and winners has received attention from important media, such as The New York Times, The Economist, and TIME, as well as newspapers, TV channels, and radio stations in their home countries. · For many awardees, involvement in GDN competitions has generated access to additional funding sources for key policy-focused research. 3.25 OED interviews suggest that rigorous procedures and strict processes exist for choosing awardees.12 Nonetheless, awards tend to be made to a limited number of institutions, and sometimes to the same individual.13 GDN continued to award medals to 12. The program's external evaluation also noted the existence of rigorous procedures for carrying out GDN research competitions. 13. For example, nearly 33 percent of GDN awards and medals went to India, with 88 percent of this amount going to three institutions in New Delhi and Calcutta. 13 researchers in the developed world until recently, when the program added an eligibility restriction to limit the entries to projects in the developing world. Some interviewees have suggested that a reduction in prize money could be beneficial, as it would increase the breadth of the flow of resources to researchers. The Challenges for Global Research Projects: Optimum Balance Between Centralization and Decentralization, Strategic Vision and Fundraising Strategies 3.26 The program has cooperated with the regional research networks in developing three global research projects. The first, Explaining Growth, reviewed the growth experience of regions over the past 30 to 50 years, resulting in assessments of the economic growth experience in developing and in-transition economies. The second global research effort, Understanding Reform, is following up on the results of the first project in two phases. The third project, Bridging Research and Policy, is a multidisciplinary effort to apply the results, and study implications of real-life developing and in-transition country policies. 3.27 Global Projects have always raised interrelated issues of who sets research priorities and the balance in decision making between the secretariat and the board, and between global and regionally managed projects. They also raise issues of GDN's strategic vision and fundraising strategies, areas in which Board members may be beginning to play a more significant and hopefully a more transparent role with the recent formation of committees with specific responsibilities. Electronic Information Tools Need Sharpening 3.28 Survey results on the perceived value of the program's electronic information tools and services were disappointing. GDNet was evaluated in terms of several online initiatives ­ the GDN Web site, a monthly newsletter, and the program's funding. The survey revealed a low rate of GDN electronic registration among program beneficiaries, with respondents from developing countries in Sub-Saharan Africa and Latin America and the Caribbean reporting the lowest rates. In contrast, respondents from the Commonwealth of Independent States and Eastern/Central Europe reported somewhat greater use of the program Web site and GDN's electronic tools. The survey elicited criticism and suggestions on potential solutions in areas including Web site design and general content; publicity, interactivity, and cooperation; posting of materials in languages other than English, such as French and Arabic; and technical concerns. The program's Web site was being substantially redesigned to address these and other concerns. More effective marketing of the Web site and GDN's other online initiatives will be an important step in enhancing the program's impact. 3.29 Survey findings suggest that more marketing is needed to integrate GDN conference participants into the GDNet community. They should also be informed of the range of benefits of creating an individual profile on GDN's Knowledge Base ­ a searchable repository of knowledge on development from around the world. The circulation of these survey findings presents an opportunity for further expanding usage of GDNet. GDN's Web site received 10,700 visitors in January 2003, while the rate of registration with the Knowledge Base reached 40 profiles per month on average, contributing to a total of 2,500 14 profiles of researchers, all of whom are members of GDN. The number of subscribers to GDN's newsletter rose to around 13,000 in 2004. 3.30 GDN had already concluded that it was premature and perhaps unrealistic, given donor funding cycles, to appeal to the World Bank's President to help raise an endowment fund for GDN until issues of GDN's overall strategic direction, governance and management are sorted out ­ including board responsibilities, GDN's International Organization (IO) status, location, rules and practice of board management, management succession, and Bank oversight. GDN Transition to International Organization Status and the Bank Becoming Signatory 3.31 GDN's Board decided to proceed with the move to the status of an International Organization for three main reasons: 1. It will provide GDN with a level of visibility and status that it argues it could never achieve as a non-profit organization incorporated in India; 2. It brings immediate financial benefits in terms of tax exemptions and privileges for GDN's staff and to its offices in New Delhi and Cairo; 3. It would improve operational efficiency by providing immunities and legal privileges including the right to hire international staff. The Board conducted a protracted discussion of this issue at the Cairo conference of January 2003, and it engaged a legal expert to obtain advice internationally and in New Delhi. GDN staff also examined the experience of several other organizations, such as the Joint Vienna Institute, the International Institute for Democracy and Electoral Assistance, and the International Development Lawyers Organization. The latter reportedly followed the same path as GDN, namely, moving from being a non-profit organization to an international organization. GDN's FY05-FY07 Business Plan sets out the change in legal status, which has been endorsed by ten governments representing eight constituencies on the World Bank's Board. It is expected to be signed by them at the GDN conference in Dakar in January 2005, establishing GDN as an International Organization. 3.32 Notwithstanding GDN's broad consultation with lawyers, directors of research institutes, and managers of international organizations, the external evaluation recommended that GDN "commission competitively selected, independent consultants with relevant credentials to assess whether the relocation and legal-status decisions satisfy the requirements derived from the critical success factors." 3.33 Tax free status, import privileges, ease of obtaining visas and freedom from lawsuits are some of the reasons advocated by GDN management for seeking the new status. 3.34 Some board members questioned whether GDN really needs the IO status to operate effectively. Some also expressed the concern that membership for nations could fundamentally affect GDN's freedom to operate since governments will want to influence the organization, particularly in conducting social science research. Southern board members ­ and even some northerners ­ wondered whether an annual savings of $300,000 in the salaries of GDN staff to the GDN were relevant to fundamental decisions about program status and location. 15 3.35 Some governing board members view the change in status as undermining the authority of the current governing body given that the structure arose through a worldwide participatory process. 3.36 OED concurs with the external evaluation's conclusion that not all board members may have fully appreciated the complexities involved in moving to the IO status. Shifting to the IO status will fundamentally change the character of GDN from a research institute to a forum for representatives of governments and international organizations. For the Bank to serve on the GDN governing board, this shift would require clearance from the Bank's Governors, rather than the Executive Directors. 3.37 OED, however, differs with the external evaluation's assessment that the decision to relocate to India was not fully vetted by the board. OED interviews of the GDN board indicated that, while some members would have preferred other locations, a move to a developing country away from Washington, one with a democratic government and a strong incidence of poverty was thought to provide a wide variety of challenges to which other developing countries could easily relate. Moreover, the country could provide the testing ground for the effectiveness of policy and social science research.14 Unlike other locations considered, India did not offer GDN special privileges it accords only organizations with a legal international organization status. The question still remains as to whether GDN needs such privileges and whether the IO status would affect the decision to relocate. 3.38 The Bank has expressed its willingness to strengthen its research links with GDN, but stressed that GDN needs to develop a broader funding base to be placed on a sounder footing financially and to manifest GDN's intellectual and operational independence from the Bank in a credible and concrete fashion. Furthermore until the Strategic and Business Plans have been effectively operationalized, it would be premature for the Bank to commit to become a signatory to GDN's status as an international organization. Given the widespread expressions of support for GDN, the bilateral donors would also need to pick up more of the funding share. OED concurs with this assessment. 3.39 Too close a Bank involvement, discussed in section 4, including underwriting of GDN overhead costs, made it difficult for GDN to develop an independent identity and to raise resources for overhead costs from other donors. The move to New Delhi has been intended to correct these factors. Bank membership on the IO status from the outset poses a number of risks to the Bank. It would not only continue the impression of close Bank involvement but would potentially present a moral hazard. Other donors would expect the Bank to support GDN financially regardless of its long term financial viability. This would jeopardize GDN's ability to stand on its own feet while leading to continued questions about its intellectual independence from the Bank. The Bank would reduce the risks to itself while improving the program's chances of achieving full independence by maintaining an arm's length relationship initially from the program's IO status, while continuing to provide GDN with DGF funding on the basis of a diminishing share of the total operating costs. 14. The fact that a number of major multinational private companies, including GE, are moving their research operations to India indicates that, despite its dysfunctional bureaucracy typical of a developing country, India is considered an attractive location for multinational corporations seeking skilled human resource base at low cost. 16 SUSTAINABILITY OF PROGRAM IMPACT 3.40 GDN's sustainability has been dependent on the Bank's continued support and the program's ability to diversify and increase funding. The donor interest expressed at the Paris Conference is a positive sign, although core funding remains an issue, particularly since it is unclear if prospects for sustainability of GDN services are improving through cost recovery. A GDN-sponsored survey of 512 research institutions throughout the developing world addressed the cost and the potential for sustainability of GDN-sponsored interventions. Titled "Researching the Researchers," the survey received a response from 202 institutions.15 It indicated that the "scope for cost recovery for activities is limited and on-line delivery at zero cost is preferred to paper delivery at marginal cost for information-sharing services."16 The survey revealed that respondents would change their level of participation in an activity according to the degree to which it is subsidized.17 The same trend was observed for the use of information-sharing services.18 4. Organization, Management, and Financing of the Program 4.1 The OED evaluation considered five core functions of governance: (1) strategic direction, usually exercised by the governing body, (2) oversight of the management unit that is responsible for the day-to-day management of the program, (3) consultation with other stakeholders, both formal and informal and through technical, scientific, or professional advisory bodies, (4) risk management, including various risks identified in Bank work, namely, reputational risks, fiduciary risks, conflict of interest risks, unfair advantage risks, governance risks, and non-performance risks, and (5) evaluation and audit. 4.2 OED also adapted a set of four inter-related corporate governance principles developed by the Business Sector Advisory Group of the OECD, as follows:19 15. The response rate was 39.5 percent. 16. Global Development Network 2003c. 17. Using a scale of 1 (participation extremely unlikely) to 7 (participation extremely likely), the survey revealed that respondents would change their activities based on whether or not the activity was subsidized. Specifically, in respect to the use of activities, the results revealed the following: Annual Meetings: 6.7 (fully subsidized); 3.9 (partially subsidized); 2.3 (no subsidy); Staff Exchanges: 6.6 (fully subsidized); 3.8 (partially subsidized); 2.2 (no subsidy); Training: 6.2 (full subsidized); 3.6 (partially subsidized); 2.3 (no subsidy). 18. With respect to the use of information sharing services, the survey found the following: Information-sharing Services: Using scale of 1 (extremely unlikely to use) to 7 (extremely likely to use): Journal: 6.4 when delivered on-line at zero cost; 4.2 when delivered as paper copy at marginal cost; Archive of Research Abstracts: 6.3 when delivered on-line at zero cost; 3.9 when delivered as paper copy at marginal cost; Data Directory: 6.2 when delivered on-line at zero cost; 3.9 when delivered as paper copy at marginal cost; Directory of Knowledge Institutions: 5.9 when delivered on-line at zero cost; 3.8 when delivered as paper copy at marginal cost; Directory of Development Researchers: 5.9 when delivered on-line at zero cost; 3.8 when delivered as paper copy at marginal cost. 19. The Business Sector Advisory Group found a similar diversity of private sector corporate governance models, with a particular dichotomy between the "shareholder" tradition in Anglo-American countries and the "stakeholder" tradition in continental European countries and Japan. They concluded that, regardless of the model, these four underlying principles were part of a well-functioning corporate governance system, and enshrined these in the OECD Principles of Corporate Governance, endorsed by ministers at the OECD Council meeting at the ministerial level in May 1999. 17 · Clear roles and responsibilities ­ of the officers and bodies that govern and manage the program and of the mechanisms to modify and amend the governance and management of the program in a dynamic context. · Transparency ­ the program provides both shareholders and stakeholders with the information they need in an open and transparent manner (such as decision-making responsibilities, accountabilities and processes, accounting, audit, and material non- financial issues). · Fairness ­ the program does not favor some immediate clients over others (such as Bank staff, participating agencies or program secretariats, specific countries or their agencies, municipal agencies, local authorities, private service providers, NGOs, and community organizations). · Clear accountability ­ of the program for the exercise of power over resources to the program's stakeholders, including international organizations, donors, developing countries, the private sector, and NGOs. 4.3 As indicated earlier, GDN used a broadly participatory, Web-based consultation process to establish its governance structure. Yet, it continues to be perceived as an offspring of the Bank, leading to confusion on the roles and responsibilities of the Bank and GDN's board. The perception of a close relationship is in part due to the fact that the Bank established GDN and continues to provide a large share of its overhead costs. A former senior World Bank staff member serves as GDN's Executive Director and another senior Bank official serves on the GDN board. Other prominent Bank managers or those perceived to be close to the Bank have had high visibility roles in GDN conferences, and the Bank has strongly supported its focus on economic research of interest to Bretton Woods institutions. Therefore, GDN's independence from the Bank is seen by many as key to GDN's credibility and the promotion of knowledge at the national policy level. Relocation to a developing- country locale has been considered to be a positive development. The program's close perceived links to the Bank have led to divisions within the Board on how the GDN/Bank relationship should be handled, as well as divisions within the Bank regarding its relationship with GDN. Yet the Bank has had a largely hands-off relationship with GDN and relatively little oversight. 4.4 The external evaluation observed that GDN's current structure reflects a commitment to policy-relevant, multidisciplinary development research without intervention from governments or donors, but the board's membership neither subsumes the entire "development spectrum" (including policy makers and development practitioners) nor includes representatives of bilateral or private-sector donors. Yet, effective oversight of the business aspects of GDN's operational and financial management is becoming indispensable ­ if only to satisfy donors' requirements of accountability. The board therefore recommended a business board consisting of donors to develop a "business strategy" and an academic board to be responsible for all non-business aspects of GDN. 4.5 OED team's assessment of GDN's governing board is slightly different than that of the external evaluation. The board has strong legitimacy because of its inclusive and 18 participatory selection process.20 Yet, the prolonged illness of its first chairman and the perceived lack of a clear designated substitute to replace him created a perception of a lack of board leadership, a vacuum perceived by many to have been filled by a strong director. Board members have had scant access to information on certain key aspects of governance, including access to information on GDN finances, salary structure, and operating costs of the Secretariat. Several board members expressed dissatisfaction at the limited role the board has played in providing strategic direction and oversight of GDN management, leading to ambiguity in the role of board and management and a weak relationship of the board with the GDN secretariat and the director. GDN PARTNERSHIP WITH REGIONAL NETWORKS 4.6 Each of GDN's regional research networks has its own management structure and program agenda. The regional networks are legal entities, are governed by individual, independent boards and obtain funding from a variety of sources. GDN regional networks are led by organizations with varying institutional experience and expertise. For example, GDN's African research network hub, the Africa Economic Research Consortium (AERC), is generally considered to be a highly successful network in a region where organizational successes have been limited. AERC has successfully kept donor and regional research priorities separate, allowing the institute to maintain a strong focus on its regional research agenda. From its inception, AERC's governance centered around three well-defined components ­ the Donor Board, the Secretariat, and the Advisory Committee ­ with a clear-cut division of labor and responsibilities. After GDN's establishment, though, the Bank began to channel a large proportion of its support for AERC through the new global program, with AERC priorities and activities molded and driven in part by GDN's multidisciplinary focus. 4.7 GDN is still arriving at an optimum level of decentralization that preserves the global program's raison d'être while also allowing regional networks to determine their own priorities, such as whether, and by how much to shift from economic to social science research, and the kind of economic research that should be promoted (neoclassical vs. heterodox, macroeconomic vs. multisectoral). In addition, the regional networks are still identifying their research priorities and developing approaches and strategies for enhancing researcher and institutional capacity-building at the regional level. 4.8 Views differ as to the proper role of the regional research networks and their ideal future structure and focus. Two related issues have emerged. One is the role of the more centrally managed global research programs relative to the decentralized priorities and activities of the regional networks. The share of global research in total funding has increased substantially in 2004. But several board members indicated to OED that they were uncertain as to who sets priorities for this global research, how funds are raised, if funding drives the research agenda, or if the research needs identified by regional networks drive funding. A second issue is the inherent patron/client relationship between GDN's secretariat and the regional networks, rooted in the designated hub model. Some board members view this as a healthy arrangement that allows client representation on the board. With the network 20. The board is comprised of 16 to 18 members, with 7 members nominated by the research community from the developing world, 3 members from the OECD community, 2 from the international community, 3 members representing international professional associations, and 2 open seats. 19 obtaining support from GDN, others see it as undermining program transparency, fairness, and accountability, with a potential long-term effect on capacity building. In addition, with regional board member nominations left up to hub institutions, the nomination process and governance qualifications of regional nominees are often unclear. FINANCING OF THE PROGRAM 4.9 The World Bank has served as GDN's main source of financing and the only source that provides completely unrestricted financing. After furnishing nearly all the program's initial funding, the Bank has provided a total of $30.33 million since GDN's inception. In FY03, Bank support accounted for $6.54 million ­ 73 percent ­ of total GDN financing, provided through a $5.89 million Development Grant Facility allocation and an additional $0.65 from other Bank resources (Table 2). DGF has approved $4.45 for the program for FY04, with $1.18 million for GDN's Education Research Component. The World Bank President has underwritten the program's annual administrative costs of up to $1.8 million through 2006. According to a 2002 memorandum of understanding between the Bank and GDN, the Bank is committed to providing up to $1.8 million a year for five years to support GDN administrative expenses, including staff salaries and benefits, rent, travel, supplies and equipment, hospitality, contractual services, and governing body meetings. Table 2. World Bank Support to GDN since Inception (US$ Millions) Fiscal Years 1998 1999 2000 2001 2002 2003 2004 Total GDN - Core Program 2.00 3.20 3.40 4.76 4.70 4.70 4.45 27.21 GDN - Education Research 1.18 Component 0.75 1.19 /1 3.12 Total DGF Grants 2.00 3.20 3.40 4.76 5.51 5.89 5.63 30.33 World Bank ­ in kind 0.62 0.03 World Bank ­ administrative budget 0.35 World Bank ­ WDR 0.27 Total World Bank Support 6.07 6.54 Total Sources of Funds 8.46 8.91 Percent from World Bank 72 % 73 % Sources: DGF documents and GDN 2003 Annual Report. /1 GDN 2003 Annual Report includes this 2004 DGF grant for the Education Research Component in fiscal year 2003. 4.10 The GDN has had modest success in raising additional, particularly core funds to support its overhead costs ever since the Bonn conference in 1999. Other major donors have made mainly project-based cash and in-kind contributions to the program. They include Australia, Austria, France, Germany, Italy, Japan, Luxembourg, Norway, Switzerland, the United Kingdom, the United States, and the Ford Foundation. At the Donors' Conference in Paris in June 2004 donors stressed the need for a strong marketing exercise for fundraising but expressed strong support for the GDN's policy orientation and work plan, and several offered to consider the proposal for financing. Only IDRC provided firm commitment to core funding. According to IIE, the limited financing prospects have been the result of a lack of a 20 long-term financing strategy, and a board that is inexperienced in fundraising and lacks private sector business backgrounds. 4.11 OED, however, has noted that some individual GDN board members have had substantial fundraising experience and have established major international economic institutions. GDN's perceived close link to the World Bank and a past focus on economics research also contributes to its fundraising challenges. Stakeholders informed OED and the external evaluators that senior Bank managers involved in founding GDN conveyed a "take it or leave it" message to potential donors as to how GDN would be set up. Some saw this as a sign of Bank and GDN secretariat unilateralism, especially given that many donors have had considerable experience in setting up social science research networks including some, such as AERC, in which the Bank currently participates. The board members also stressed to OED that GDN was designed to be free from donor influences in setting research priorities which should be driven by client needs and professional expertise in determining researchable issues. OED concurs with this view. GDN could emulate AERC in combining an independent priority-setting process with a strong advisory board and a strong and committed donor board. 4.12 The French Ministry of Foreign Affairs hosted a donor meeting for GDN in Paris in June 2004, at which GDN discussed the recently completed program evaluation and its business plan and strategy for the next three years in the hope of attracting greater donor support. At the meeting, attended by 23 donor representatives, GDN management noted the establishment of an Executive, an Audit and Programs Board Committee, and a Global Development Council to help fundraising. Many participants expressed support for the policy orientation of the program and interest in providing support but stressed the need for more effective marketing to explain GDN's mission and its value-added to potential donors. Some donors also questioned the Bank's approach to spinning activities off for the donors to fund on a case-by-case basis, after having established a framework with the donors to avoid case- by-case consideration of projects. While GDN has had few successes in raising private sector funds, few official donors offered support for core funding. 4.13 The transaction costs of GDN have been high. In FY02, GDN's (direct and indirect) administrative costs amounted to about 25 percent of its budget allocations, including costs of the organization of annual conferences and management and operation of GDNet. Overheads (non-allocable costs) and donor coordination amounted to 6 percent of GDN's budget. DGF grant support as share of program administration has been declining. In FY04 GDN received $600,000 from DGF for operating costs and administration out of a total grant of $4.45 million, i.e.,13.3 percent. GDN expects to reduce costs further after the move to New Delhi 4.14 In addition, the program allowed regional networks to keep 15 percent of grants to finance grant-incurred administrative expenses. Institutions receiving sub-grants from regional networks are allowed to keep 10 percent for administration. GDN also seems to overlook the increased costs posted by some regional networks. OED team interviews with regional network officials indicate that some networks have consistently charged 20 percent of grants as administrative costs, rather than the 15 percent allowed by GDN. 4.15 GDN's budget is not transparent. Its submission to the DGF has not made clear how much of the externally managed funds - in the form of special projects - reflect total program 21 costs, of which GDN's share may be a small part. This leads to an impression of a large leveraging effect of Bank contributions. In reality, there is little long-term unrestricted funding to support costs of GDN operations and for regional networks. RISKS AND RISK MANAGEMENT 4.16 The Bank and GDN did not adequately identify and manage the risks associated with the program after its spin-off from the Bank, although developments since the external evaluation in this regard are encouraging. From the Bank's perspective, the spin-off runs significant reputational risks, given the widespread perception that the program is close to the Bank and represents its policies. Additional risks of non-performance and disappointment are raised by the lack, for a considerable period, of Bank oversight of the program's performance and its selection of research activities, despite the Bank's pivotal role as GDN founder, donor, and manager. 4.17 Furthermore, the degree of unfair advantage provided to selected institutions remains an unidentified risk, as the program has continued with the approach of pre-selected research institute hubs for regions, rather than following an eclectic approach that would encompass a broader set of research institutes worldwide. 5. Role of the World Bank 5.1 OED evaluated World Bank performance according to the following criteria (see Annex A): · Comparative advantage: Whether the Bank is employing its comparative advantages in relation to other partners in the programs.21 · Global-country linkages: Whether the global program has effective operational linkages to the Bank's country operational work, where appropriate. · Oversight: Whether the Bank is exercising effective and independent oversight of its involvement in the program, as appropriate, for in-house and externally managed programs. · Exit strategy: Whether the Bank is facilitating effective, flexible, and transparent disengagement strategies, as appropriate. COMPARATIVE ADVANTAGE AND GLOBAL-COUNTRY LINKAGES 5.2 OED identified the Bank's comparative advantage at the global level as 1) its global mandate and reach, 2) convening power, and 3) ability to mobilize financial resources; and at the country level as its 4) multi-sectoral capacity, 5) expertise in country and sector-level analysis, and 6) in-depth country-level knowledge. GDN has used its comparative advantage both at the global level and at the country level. The program's annual conference and research activities demonstrate that the program has been able to use the Bank's convening 21. This is also one of the six criteria for approving a global program at the initial concept stage established by Bank Management in April 2000 and one of the eight eligibility criteria for grant support established by the DGF Council in September 1998. 22 power and global mandate and reach. The regional hub approach can enable both program and capacity strengthening, and improve relevance and benefits to developing countries and the Bank in a substantive way if a balance between external and national needs is achieved. For example, by making use of country-level local researchers' experience and knowledge, and, where needed, building their capacity to analyze policy issues of national and international interest, the Bank can use its comparative advantage by providing them with worldwide reach and intellectual and financial resources. To take advantage of the synergy between IFIs and developing country researchers, GDN is implementing a large research project on behalf of the IMF. The Bank's Chief Economist has proposed holding next year's Annual Bank Conference on Development Economics (ABCDE) in Dakar following GDN's annual conference there. The Bank's country and regional offices could similarly promote research involving nationals of its member countries to empirically test the development effectiveness of development approaches followed by the countries themselves as well as those promoted by the international financial institutions. This would improve the effectiveness of advice given and the approaches promoted by the international financial institutions in a number of areas where they have been criticized over the years for pursuing blue print one-size-fits-all approaches.22 It would also encourage developing countries to help shape the research agenda. It is too early to know whether, how soon, and how well GDN will reach a balance between externally and internally prioritized research. 5.3 At least initially the hub approach may have resulted in less direct linkages between the GDN and the Bank's country-level operations. GDN management has stressed the program's aim to strengthen the policy link at the local level. It has also indicated that, by building capacity in developing countries, operational teams could have more local expertise to draw upon. In particular, local GDN-sponsored institutions could provide more effective inputs into their country's Poverty Reduction Strategies. This will depend on the extent to which the program will draw on a wider set of institutions beyond those designated as hubs, help build wider local expertise, and become an effective conduit for operational Bank teams to draw upon local institutions to improve and fine tune their approaches. OVERSIGHT 5.4 Bank oversight for the GDN has been weak. Despite the magnitude of the Bank's commitment to GDN and its support for 80 percent of the program's administrative costs, as in the case of other global programs, expectations as to the role of the Bank staff member serving on the GDN board were not well defined. GDN submitted its annual budget to the Chief Administrative Officer in DEC. The PREM board reviewed the budget before submitting it to DGF. This situation has changed in recent months. The Bank's desire to establish a strong governance, financing and management structure in GDN, to promote sound economic research of interest to the international financial institutions, the clear goal posts of a more diversified and more sustainable funding base and the linkage of the Bank decision to become a signatory to the GDN IO status to its performance in a variety of these areas are clearer. The Bank has also put a senior manager from the Development Economic 22. OED's 2003 Annual Review of Development Effectiveness emphasized the need for strategies customized to country conditions. 23 Vice Presidency in charge of oversight while maintaining an appropriate arm's length relationship from the program. EXIT STRATEGY 5.5 After GDN separated itself from the Bank by becoming an independent, not-for-profit organization in 2001, the Bank continued to provide DGF support to the program. GDN is funded through DGF's Window I, with programs receiving long-term support. DGF's contribution to the program has increased over the years, from $2 million in FY98 to $4.45 million in FY04. While GDN is worthy of Bank support, it needs to address its strategic weaknesses, including mission and objectives, governance structure, financing, and other matters of organizational status. According to DGF documents, future DGF support will take into account GDN's progress in addressing issues, such as governance and diversified funding, that were identified in the external evaluation. Both GDN management and Bank management have stressed the need for the GDN to develop a broader funding base and reduce the Bank's contribution in order to demonstrate its operational and intellectual independence from the Bank. 6. Findings and Lessons 6.1 GDN faces a number of interrelated uncertainties and related risks well identified by the external evaluation. GDN is diversifying its funding base, operationalizing its Strategic and Business Plans, establishing itself as an International Organization, modifying its governance structure, moving its headquarters to New Delhi, and developing an independent identity. To place GDN on more sound independent footing, Bank Management has correctly concluded that until GDN has effectively carried out the necessary steps, it is premature to commit to becoming a signatory to GDN as an international organization. First, GDN's external evaluation argued that the move to shift to an IO status is inextricably linked to the move to New Delhi. GDN believes that the shift to IO status will provide it with a level of visibility and status, financial benefits, and immunities and legal privileges, including the ease to hire international staff it would not enjoy as an NGO registered in India. Second, GDN management's decision to move to an IO status requires a revised GDN charter with endorsement from sovereign states. Potential signatory donors and developing countries have perceived GDN as a World Bank creation. They would like a larger say in its governance and in defining a broader social science research agenda, and have been reluctant to finance overheads. How the move to the IO status would affect GDN's funding and governance therefore remains unclear. GDN management believes that funding prospects will be no worse and probably better with IO status than under the alternative status of a non-for- profit organization incorporated in India, and will undertake the move to IO status only if the final version of the Charter preserves in its judgment the organization's independence as a research network. Third, Bank membership on the GDN board, would require endorsement of the Bank's board of governors before which a number of risks and uncertainties need to be resolved. For example, GDN's proposed work program for 2005-07 is predicated upon an increase in funding from around $8.5 million to $11-13 million, and it entails expanding GDN's Regional Research Competitions, Global Research Projects and GDNet. The Bank has rightly concluded that by maintaining an arm's length relationship with the program's IO status, the Bank would help GDN to become financially viable and develop an independent 24 identity. Concurrently, by helping to strengthen its governance and management the Bank will help improve GDN's chances to obtain support for overhead costs from others. At the same time, the Bank could continue to provide GDN DGF funding on the basis of diminishing share in overall finances. 6.2 The Bank needed but lacked a well developed strategy to ensure sustainable spin-offs and to manage appropriately the potential risks of a program moving away from the Bank. Although the lessons of GDN and other such programs are leading to a clearer Bank strategy, GDN lost valuable time in the process to ensure its own long- term viability. GDN's sustainability after the Bank disengages from its promised support for the GDN administrative costs in FY06 is still unclear. Some of the IEE suggestions, including developing long-term operating and financing strategies to ensure multi-year funding or developing a sizeable endowment using the Bank's convening power, are being explored by GDN. OED interviews indicated that developing an endowment is unlikely to be a feasible option. 6.3 The Bank needs to establish independent oversight of global programs. This requires separation of management from oversight, counseling Bank staff participating on program boards as to the institutional expectations in serving on boards, as Bank representatives have sought, and Bank support for periodic independent external program evaluations. Given its role as a founder and its strong financial support, the Bank needs to ensure that GDN establishes a strong identity as an independent institution that adds value to the Bank's body of knowledge and policies. 25 References Agarwal, Bina. "On the Proposed Governance Structure of the GDN." Careaga, Fernando Loayza. "Global Development Network: Participatory Impact Assessment of the Global Research Project on Explaining Growth." Craig, Barbara. 2002. "Evaluation of Regional Research Competitions: South Asia, Eastern and Central Europe, and Sub-Saharan Africa." Global Development Network, Washington, DC. Craig, Barbara and Fernando Loayza. 2003. "Building Research Capacity Through Competition and Cooperation: Regional Research Competitions in the Developing World." Global Development Network. Washington, DC. Global Development Network (GDN). 2000. "Donor Support for Policy Research in Developing Countries: Report of the High Level Committee." Final Draft. Washington, DC. ______. 2002. GDN Annual Report 2002. Washington, DC. ______. 2003a. GDN Annual Report 2003. Washington, DC. ______. 2003b. "GDN's Future Legal Status: An Electronic Discussion." ______. 2003c. "Researching the Researchers: Establishing Priorities." Washington, DC. ______. 2004. "Internal Evaluation of the Core Activities Supported by the Global Development Network. Report of the GDN Secretariat." Washington, DC. Institute of Southeast Asian Studies (ISEAS). 2001. ISEAS Annual Report 2000-01. Singapore. Loayza, Fernando. 2002. "Evaluation of Regional Research Competitions: Middle East and North Africa, Former Soviet Union, and East Asia." Global Development Network, Washington, DC. Operations Evaluation Department (OED). 2003. Sharing Knowledge: Innovations and Remaining Challenges. World Bank, Washington, DC. 26 ______. 2004. 2003 Annual Review of Development Effectiveness: The Effectiveness of Bank Support for Policy Reform. World Bank, Washington, DC. Romiszowski, Alex. "The Global Development Learning Network: A Review of the First Two Years' Operation." Global Development Network, Washington, DC. Seck, Diery. 1999. "The Governance of the Global Development Network." Global Development Network Conference: Bridging Knowledge and Policy. Bonn, Germany, December 6-8, 1999. Stone, Diane, ed. 2001. Banking on Knowledge: The Genesis of the Global Development Network. Routledge, New York. Stone, Diane, Simon Maxwell, and Michael Keating. 2001. "Bridging Research and Policy." An International Workshop Funded by the UK Department for International Development. Warwick University, 16-17 July, 2001. Muth, H. Peter and Frederick H. Gerlach. 2004. "Global Development Network: Independent Evaluation. Evaluation Report." Global Development Network, Washington, DC. 27 Annex A Annex A. Evaluation Framework for Phase 2 Report and 26 Case Studies 1. The Phase 2 Report and each case study follows a common outline and addresses 20 evaluation questions (Table A.1) that have been derived from OED's standard evaluation criteria (Table A.2), the 14 eligibility and approval criteria for global programs (Table A.3), and the 8 eligibility criteria for grant support from the Development Grant Facility (Table A.4). 2. The sheer number of these criteria, some of which overlap, can be daunting even to an evaluator. Hence the OED evaluation team has reorganized these criteria into four major evaluation issues, which correspond to the four major sections of each report (Table A.1): · The overarching global relevance of the program · Outcomes and impacts of the program and their sustainability · Governance, management, and financing of the program · The World Bank's performance as a partner in the program 3. These four issues correspond roughly to OED's evaluation criteria of relevance, efficacy, efficiency, and Bank performance, appropriately interpreted and expanded for the case of global programs. In the case of global programs, relevance must be measured not only against individual borrowing countries' priorities and Bank priorities, but also in terms of the interplay between global challenges and concerns on the one hand and country needs and priorities on the other. The former are typically articulated by the "global community" by a variety of different stakeholders and are reflected in a variety of ways such as formal international conventions to which developing countries are signatories; less formal international agreements reached at major international meetings and conferences; formal and informal international standards and protocols promoted by international organizations, NGOs, etc.; the Millennium Development Goals; and the Bank's and the Development Committee' eligibility criteria for global programs. While sponsorship of a program by significant international organizations may enhance "legitimacy" of a global program in the Bank's client countries, it is by no means a sufficient condition for developing country ownership, nor for ensuring its development effectiveness. "Relevance" and ownership by the Bank's client countries is more assured if the program is demanded by them. On other hand some "supply-led" programs may also acquire ownership over time by demonstrating substantial impacts, as in the case of the internet. Assessing relevance is by far the most challenging task in global programs since global and country resources, comparative advantages, benefit, costs, and priorities do not always coincide. Indeed the divergence of benefits and costs between the global level and the country level is often a fundamental reason for the provision of global public goods. Evaluating the relevance of global action to the Bank's client countries is however important because the global development agenda is becoming highly crowded and resources to finance it have remained relatively stagnant, therefore highlighting issues of selectivity. 4. For the global programs that have been operating for some time, efficacy can be assessed not only in terms of program outcomes but more crucially in terms of impacts on the ground in developing countries. Outcomes and impacts in turn depend on the clarity and evaluability of each program's objectives, the quality of the monitoring and evaluation of results and, where appropriate, the effectiveness of the links of global program activities to the country level. 5. Since global programs are partnerships, efficiency must include an assessment of the extent to which the benefit-cost calculus in collective organizational, management and financing arrangements is superior to achieving the same results by the individual partners acting alone. The 28 Annex A institutional development impact and the sustainability of the program itself (as opposed to that of the outcomes and impacts of the program's activities) are also addressed in this section of each report. 6. Finally, this being an OED evaluation, it focuses primarily on the Bank's strategic role and performance in playing up to its comparative advantage relative to other partners in each program. The Bank plays varied roles in global programs as a convener, trustee, donor to global programs, and lender to developing countries. The Bank's financial support to global programs ­ including oversight and liaison activities and linkages to the Bank's regional operations ­ comes from a combination of the Bank's net income (for DGF grants), the Bank's administrative budget, and Bank-administered trust funds. In the case of the Global Environmental Facility (GEF) the Bank is a trustee and in the case of the Global Fund to Fight HIV/AIDS, Tuberculosis, and Malaria (GFATM), a "limited" trustee. In the case of GEF and MLF the Bank is also an implementing agency. Thus, the assessment of Bank performance includes the use of the Bank's convening power, the Bank's trusteeship, Bank financing and implementation of global programs, and, where appropriate and necessary, linkages to the Bank's country operations. Bank oversight of this entire set of activities is an important aspect of the Bank's strategic and programmatic management of its portfolio of global programs. 7. The first column in Table A.1 indicates how the four sections and 20 evaluation questions addressed in the Phase 2 Report and case studies relates to the eight evaluation issues that were raised by the Bank's Executive Board in the various Board discussions of global programs during the design phase of OED's global evaluation and identified in the OED's Evaluation Strategy paper:1 · Selectivity · Monitoring and evaluation · Governance and management · Partnerships and participation · Financing · Risks and risk management · Linkages to country operations 8. The third column in Table A.1 indicates how the four sections and 20 evaluation questions relate to OED's standard evaluation criteria for investment projects (Table A.2), the 14 criteria endorsed by the Development Committee and established by Bank management for approving the Bank's involvement in global programs (Table A.3), and the 8 criteria for grant support from the Development Grant Facility (Table A.4). 9. The 14 eligibility and approval criteria for the Bank's involvement in global programs have evolved since April 2000 when Bank management first proposed a strategy to the Bank's Executive Board for the Bank's involvement in global programs and include the four overarching criteria endorsed by the Development Committee, and the four eligibility criteria and six approval criteria presented by Bank management to the Bank's Executive Board. Each global program must meet at least one of the four relatively more substantive eligibility criteria and all six of the relatively more process-oriented approval criteria. The first two eligibility criteria relate directly to the Bank's global public goods and corporate advocacy priorities (Table A.3). Although the six approval criteria resemble the topics covered in a project concept or appraisal document for Bank lending operations, unlike for Bank lending operations, there is currently only a one-step approval process for new global 1OED, The World Bank and Global Public Policies and Programs: An Evaluation Strategy, July 16, 2001, page 21. "Partnerships and participation" were originally listed as two separate evaluation issues in the evaluation strategy document. "Monitoring and evaluation" is now interpreted more broadly to include not only an assessment of the monitoring and evaluation procedures of each program but also the findings of previous evaluations with respect to the outcomes and impacts of each program, and their sustainability. 29 Annex A programs ­ at the concept stage and not at the appraisal stage. And new global programs only have to be approved by the Bank managing director responsible for the Network proposing a new program, not by the Bank's Executive Board. 10. While the approval of new global programs is logically separate from and prior to their financing (whether from the DGF, trust funds, or other sources), the eight DGF eligibility criteria for grant support from the DGF (Table A.4) were actually established in 1998. Twenty out of the 26 case study programs and about two-thirds of the Bank's total portfolio of 70 global programs have received DGF grants. Table A.1. Key Evaluation Issues and Questions Evaluation Evaluation Questions Reference Issues Section I. Overarching Global Relevance of the Program 1. Relevance. To what extent are the programs: A modification of OED's · Addressing global challenges and concerns in the relevance criterion (Table sector A.2) for the purpose of · Consistent with client countries' current development global programs. priorities The third bullet also relates · Consistent with the Bank's mission, corporate to managing director (MD) priorities, and sectoral and country assistance approval criterion #1 strategies? regarding a "clear linkage to the Bank's core institutional objectives" (Table A.3). 2. International consensus. To what extent did the Development Committee programs arise out of an international consensus, formal (DC) criterion #4 (Table or informal: A.3). 1. Selectivity · Concerning the main global challenges and concerns in the sector · That global collective action is required to address these challenges and concerns? 3. Strategic focus. To what extent are the programs: The four bullets · Providing global and regional public goods correspond to the four MD · Supporting international advocacy to improve eligibility criteria (Table policies at the national level A.3). · Producing and delivering cross-country lessons of relevance to client countries · Mobilizing substantial incremental resources? 4. Subsidiarity. To what extent do the activities of the DGF eligibility criterion #1 programs complement, substitute for, or compete with (Table A.4). regular Bank instruments? Section II. Outcomes, Impacts, and their Sustainability 5. Efficacy. To what extent have the programs achieved, OED's efficacy criterion or are expected to achieve, their stated objectives, (Table A.2). taking into account their relative importance? 30 Annex A Evaluation Evaluation Questions Reference Issues 6. Value added. To what extent are the programs adding The first bullet corresponds value to: to DC criterion #1 (Table · What the Bank is doing in the sector to achieve its A.3). core mission of poverty alleviation and sustainable development · What developing and transition countries are doing in the sector in accordance with their own priorities? 7. Monitoring and evaluation. To what extent do the MD approval criterion #6 programs have effective monitoring and evaluation: (Table A.3), since effective communications with key 2. Monitoring · Clear program and component objectives verifiable by indicators stakeholders, including the and Bank's Executive evaluation · A structured set of quantitative or qualitative Directors, requires good indicators monitoring and evaluation · Systematic and regular processes for data collection practices. and management · Independence of program-level evaluations · Effective feedback from monitoring and evaluation to program objectives, governance, management , and financing? 8. Sustainability of outcomes and impacts. To what OED's sustainability extent are the outcomes and impacts of the programs criterion (Table A.2). resilient to risk over time? Section III. Organization, Management, and Financing of the Program 9. Efficiency. To what extent have the programs A modification of OED's achieved, or are expected to achieve: efficacy criterion for the · Benefits more cost-effectively than providing the purpose of global same service on a country-by-country basis programs (Table A.2). · Benefits more cost-effectively than if the individual The first bullet also relates contributors to the program acted alone? to MD eligibility criterion #3 (Table A.3) and DGF eligibility criterion #3 (Table A.4). 10. Legitimacy. To what extent is the authorizing A modification of OED's environment for the programs effectively derived from evaluation criteria (Table 3. Governance those with a legitimate interest in the program (including A.2) for the purpose of and donors, developing and transition countries, clients, and global programs. management other stakeholders), taking into account their relative importance. 11. Governance and management. To what extent are the MD approval criterion #5 governance and management of the programs: (Tables B.3) and DGF · Transparent in providing information about the eligibility criterion #5 programs (Table A.4). · Clear with respect to roles & responsibilities · Fair to immediate clients · Accountable to donors, developing and transition countries, scientists/professionals, and other stakeholders? 31 Annex A Evaluation Evaluation Questions Reference Issues 12. Partnerships and participation. To what extent do DGF eligibility criterion #8 developing and transition country partners, clients, and (Table A.4). beneficiaries participate and exercise effective voice in 4. Partnerships the various aspects of the programs: and · Design participation · Governance · Implementation · Monitoring and evaluation? 13. Financing. To what extent are the sources of funding MD approval criterion #4. for the programs affecting, positively or negatively: (Table A.3). · The strategic focus of the program The third bullet also relates · The governance and management of the program to OED's sustainability · The sustainability of the program? criterion (Table A.2). 14. Bank action to catalyze. To what extent has the DC criterion #2 (Table A.3) 5. Financing Bank's presence as a partner in the programs and DGF eligibility criterion catalyzed, or is catalyzing non-Bank resources for the #4 (Table A.4). programs? 15. Institutional development impact. To what extent has A modification of OED's the program established effective institutional institutional development arrangements to make efficient, equitable, and impact criterion (Table A.2) sustainable use of the collective financial, human, and for the purpose of global other resources contributed to the program. programs. 6. Risks and risk 16. Risks and risk management. To what extent have the MD approval criterion #3 management risks associated with the programs been identified and (Table A.3). are being effectively managed? Section IV. World Bank's Performance 17. Comparative advantage. To what extent is the Bank DC criterion #3 (Table playing up to its comparative advantages in relation to A.3), MD approval criterion other partners in the programs: #2 (Table A.3), and DGF · At the global level (global mandate and reach, eligibility criterion #2 convening power, mobilizing resources) (Table A.4). · At the country level (multi-sector capacity, analytical expertise, country-level knowledge)? 18. Linkages to country operations. To what extent are MD approval criterion #1 there effective and complementary linkages, where (Table A.3) regarding needed, between global program activities and the "linkages to the Bank's 7. Linkages to Bank's country operations, to the mutual benefit of country operational work." country each? operations 19. Oversight. To what extent is the Bank exercising This relates to DGF effective and independent oversight of its involvement in eligibility criterion #6 on the programs, as appropriate, for in-house and "arm's length relationship" externally managed programs, respectively. (Table A.4). Both questions 17 and 18 together relate to OED's Bank performance criterion (Table A.2). 20. Disengagement strategy. To what extent is the Bank DGF eligibility criterion #7 facilitating effective, flexible, and transparent (Table A.4). disengagement strategies, as appropriate? 32 Annex A Table A.2. Standard OED Evaluation Criteria Criterion Standard Definitions for Lending Operations Possible Ratings The extent to which the project's objectives are consistent (1) with High, substantial, modest, the country's current development priorities and (2) with current negligible. Relevance Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). The extent to which the project's objectives were achieved, or High, substantial, modest, Efficacy expected to be achieved, taking into account their relative negligible. importance. The extent to which the project achieved, or is expected to High, substantial, modest, Efficiency achieve, a return higher than the opportunity cost of capital and negligible. benefits at least cost compared to alternatives. The extent to which the authority exercised by the program is High, substantial, modest, effectively derived from those with a legitimate interest in the negligible. Legitimacy /1 program (including donors, developing and transition countries, clients, and other stakeholders), taking into account their relative importance. The extent to which a project improves the ability of a country or High, substantial, region to make more efficient, equitable and sustainable use of its negligible, modest. Institutional human, financial, and natural resources through: (a) better development definition, stability, transparency, enforceability, and predictability impact of institutional arrangements and/or (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements. IDI includes both intended and unintended effects of a project. Sustainability The resilience to risk of net benefits flows over time. Highly likely, likely, unlikely, highly unlikely. Highly satisfactory, satisfactory, moderately Outcome The extent to which the project's major relevant objectives were satisfactory, moderately achieved, or are expected to be achieved, efficiently. unsatisfactory, unsatisfactory, highly unsatisfactory The extent to which services provided by the Bank ensured quality Highly satisfactory, Bank at entry and supported implementation through appropriate satisfactory, unsatisfactory, performance supervision (including ensuring adequate transition arrangements highly unsatisfactory. for regular operation of the project). The extent to which the borrower assumed ownership and Highly satisfactory, Borrower responsibility to ensure quality of preparation and implementation, satisfactory, unsatisfactory, performance and complied with covenants and agreements, toward the highly unsatisfactory. achievement of development objectives and sustainability. /1 This represents an addition to OED's standard evaluation criteria in the case of global programs, since effective governance of global programs is concerned with legitimacy in the exercise of authority in addition to efficiency in the use of resources. 33 Annex A Table A.3. Selectivity and Oversight of Global Programs Selectivity Criteria for Bank Involvement in Global Public Goods: Endorsed by Development Committee (September 2000) /1 An emerging international consensus that global action is required A clear value added to the Bank's development objectives The need for Bank action to catalyze other resources and partnerships A significant comparative advantage for the Bank. Approval Criteria for Bank Involvement in Partnership Initiatives Beyond the Country Level: Established by Bank Management (November 2000) /2 1. A clear linkage to the Bank's core institutional objectives and, above all, to the Bank's country operational work 2. A strong case for Bank participation based on comparative advantage 3. A clear assessment of the financial and reputational risks to the Bank and how these will be managed 4. A thorough analysis of the expected level of Bank resources required, both money and time, as well as the contribution of other partners 5. A clear delineation of how the new commitment will be implemented, managed, and assessed 6. A clear plan for communicating with and involving key stakeholders, and for informing and consulting the Executive Directors. Global Public Goods Priorities /3 Corporate Advocacy Priorities /3 Communicable diseases Strategic Focus for Oversight Empowerment, security, and social · HIV/AIDS, tuberculosis, malaria and of Global Programs: inclusion childhood communicable diseases, Established by Bank · Gender mainstreaming including the relevant link to education Management (March 2003) /4 · Civic engagement and participation · Vaccines and drug development for · Social risk management (including major communicable diseases in disaster mitigation) developing countries a. Provide global public Investment climate Environmental commons goods · Support to both urban and rural · Climate change development · Water · Infrastructure services to support · Forests private sector development · Biodiversity, ozone depletion and land b. Support international · Regulatory reform and competition degradation advocacy for reform policy · Promoting agricultural research agendas which in a · Financial sector reform Information and knowledge significant way Public sector governance · Redressing the Digital Divide and address policy framework · Rule of law (including anti-corruption) equipping countries with the capacity conditions relevant for · Public administration and civil service to access knowledge developing countries reform (incl. public expenditure · Understanding development and accountability) poverty reduction · Access to and administration of Trade and integration c. Are multi-country programs justice (judicial reform) · Market access which crucially depend on Education · Intellectual property rights and highly coordinated · Education for all, with emphasis on standards approaches girls' education International financial architecture · Building human capacity for the · Development of international d. Mobilize substantial knowledge economy standards incremental resources that Health · Financial stability (incl. sound public · Access to potable water, clean air debt management) can be effectively used for and sanitation · International accounting and legal development. · Maternal and child health framework /1 From the Development Committee Communiqué issued on September 25, 2000. Both the Development Committee and Bank Management envisaged global programs as being the principal instrument for Bank involvement in providing global public goods. /2 The Initiating Concept Memorandum in the Partnership Approval and Tracking System (PATS) was initially organized according to these six criteria. /3 These are the five corporate advocacy priorities and the five global public goods priorities (and bulleted sub-categories) from the Strategic Directions Paper for FY02-04, March 28, 2001. Within the Partnership Approval and Tracking System (PATS), global programs are expected to identify, for tracking purposes, their alignment with at least one of these ten corporate priorities. 34 Annex A Table A.4. Eligibility Criteria for Grant Support from the Development Grant Facility 1. Subsidiarity The program contributes to furthering the Bank's development and resource mobilization objectives in fields basic to its operations, but it does not compete with or substitute for regular Bank instruments. Grants should address new or critical development problems, and should be clearly distinguishable from the Bank's regular programs. 2. Comparative The Bank has a distinct comparative advantage in being associated with the program; it does not advantage replicate the role of other donors. The relevant operational strengths of the Bank are in economic, policy, sector and project analysis, and management of development activities. In administering grants, the Bank has expertise in donor coordination, fund raising, and fund management. 3. Multi-country The program encompasses multi-country benefits or activities which it would not be efficient, practical benefits or appropriate to undertake at the country level. For example, informational economies of scale are important for research and technology work, and operations to control diseases or address environmental concerns (such as protect fragile ecosystems) might require a regional or global scope to be effective. In the case of grants directed to a single country, the program will encompass capacity-building activities where this is a significant part of the Country Assistance Strategy and cannot be supported by other Bank instruments or by other donors. This will include, in particular, programs funded under the Institutional Development Fund, and programs related to initial post- conflict reconstruction efforts (e.g., in countries or territories emerging from internal strife or instability). 4. Leverage The Bank's presence provides significant leverage for generating financial support from other donors. Bank involvement should provide assurance to other donors of program effectiveness, as well as sound financial management and administration. Grants should generally not exceed 15 percent of expected funding over the life of Bank funding to a given program, or over the rolling 3-year plan period, whichever is shorter. Where grant programs belong to new areas of activities (involving, e.g., innovations, pilot projects, or seed-capital) some flexibility is allowed for the Bank's financial leverage to build over time, and the target for the Bank grant not to exceed 15 percent of total expected funding will be pursued after allowing for an initial start-up phase (maximum 3 years). 5. Managerial The grant is normally given to an institution with a record of achievement in the program area and competence financial probity. A new institution may have to be created where no suitable institution exists. The quality of the activities implemented by the recipient institution (existing or new) and the competence of its management are important considerations. 6. Arm's length The management of the recipient institution is independent of the Bank Group. While quality an arm's relationship length relationship with the Bank's regular programs is essential, the Bank may have a role in the governance of the institution through membership in its governing board or oversight committee. In cases of highly innovative or experimental programs, Bank involvement in supporting the recipient to execute the program will be allowed. This will provide the Bank with an opportunity to benefit from the learning experience, and to build operational links to increase its capacity to deliver more efficient services to client countries. 7. Disengage- Programs are expected to have an explicit disengagement strategy. In the proposal, monitorable ment action steps should be outlined indicating milestones and targets for disengagement. The Bank's strategy withdrawal should cause minimal disruption to an ongoing program or activity. 8. Promoting Programs and activities should promote and reinforce partnerships with key players in the partnerships development arena, e.g., multilateral development banks, UN agencies, foundations, bilateral donors, professional associations, research institutions, private sector corporations, NGOs, and civil society organizations. Source: World Bank, Development Grant Facility documentation. 35 Annex B Annex B. Program At a Glance Table B.1. General Information Program GDN Operational Date December 1999 Independent legal entity Yes ­ Registered in Washington, D.C., as an autonomous non-profit Authorizing environment Self-authorizing Written charter Yes. GDN has articles of incorporation, by-laws, and a statement on governance. It also has an MOU with the World Bank. Present location Washington, D.C. The program will be moving to New Delhi, India in April, 2005. Internet address www.gdnet.org Program manager Lyn Squire Bank task manager Ines Garcia-Thoumi Sponsoring Bank unit(s) DEC and WBI Reviewing sector board PREM ­ Poverty Reduction Sector Multi-sector Subsector Multi-sector Theme World Bank's overall knowledge and information strategy Bank sector strategy paper Major World Development Reports GPG Priority Information and knowledge CA Priority None FY02 program expenditures $ 6.79 million DGF status Window 1 DGF grant (FY03) $ 4.70 million Bank-administered trust funds Yes TF contributions (FY02) $ 1.17 million (excluding DGF contributions) /1 Person who is immediately responsible for oversight of the program. Sources: The information in this and subsequent annexes and tables has been assembled from the GDN Charter, GDN Web site, the Bank's Partnership Approval and Tracking System (PATS), and other program documents collected by the OED team. 36 Annex B Table B.2. Partners1 and Participants Multilateral International UNDP, World Bank, UNICEF, IMF, PAHO. Organizations Bilateral Aid Agencies Canadian International Development Agency (CIDA), Norwegian Agency for Development Cooperation, Swiss Agency for Development & Cooperation (SDC), and the United Kingdom Department for International Development (DFID). Regional Network Partners in Economics Education and Research Consortium (EERC), East Asian Developing Countries Development Network (EASDN), Center for Economic Research and Graduate Education ­ Economics Institute (CERGE-EI), Latin America and Caribbean Economic Association (LACEA), Economic Research Forum for the Arab Countries, Iran, and Turkey (ERF), South Asia Network of Economic Research Institutes (SANEI), Pacific, Australia, and New Zealand network (PACANZ), African Economic Research Consortium (AERC), and the European Development Research Network (EUDN). Regional Network Partners in GDN-Japan, GDN-North America, and the America and the European Industrialized countries Development Research Network (EUDN). Governments Austria, Brazil, Egypt, France, Germany, Italy, Japan, and Luxembourg. Foundations African Capacity Building Foundation, Ford Foundation, Gateway Foundation, and the German Foundation for International Development (all serve as donors). Private Sector Bloomberg, The Center for International Private Enterprise (US), Deutsche Telekom (Germany), Merck & Co., Nippon Telegraph and Telephone East Corporation, NTT DoCoMo (Japan), Petrobas (Brazil), Picture Tel Corporation (US), Sun Microsystems, Toyo Keizai (Japan), Varig (Brazil), and Yomiuri Shimbin (Japan). Research Institutions Institute for Developing Economies (IDE-JETRO), International Development Research Center (Canada), Institute of Development Studies (IDS), International Economic Association (IEA), International Political Science Association (ISPA), International Sociological Association (ISA), Overseas Development Institute (ODI), United Nations University (UNU), NIH, the Vienna Institute for International Economic Studies (WIIW). 1Partners appear in bold. Table B.3. World Bank's Roles Founder Yes Chair of governing body No If yes, who? --- Member of the governing body Yes If yes, who? Guillermo Perry In-house secretariat No Funding /1 DGF TF trustee Yes TF manager /2 No Lender to the sector Yes Convener in the sector /3 Yes /1 Financial contributions to the program itself, not including BB resources spent on oversight and liaison activities. /2 Involves responsibility for oversight and management of the use of trust fund resources. /3 The World Bank organizes meetings and conferences in the sector on issues related to but outside the scope of the program to advocate change, reach consensus, and/or mobilize resources with respect to emerging issues in the sector. 37 Annex C Annex C. Stated Mission, Objectives, Outputs, and Activities Mission/Goal To address the problems of national and regional development Development · Support multidisciplinary research in the social sciences. Objective · Promote the generation of local knowledge in developing and transition countries. · Produce policy relevant knowledge on a global scale. · Build research capacity to advance development and alleviate poverty. · Facilitate knowledge sharing among researchers and policymakers. · Disseminate development knowledge to the public and policymakers. Outputs/ Knowledge generation and knowledge sharing. Strategy Activities · Global Development Awards. · Regional Research Competitions. · Global Research Projects. · GDN Annual Conferences and Meetings. · GDNet and other Electronic Tools. Source: GDN Official Web site at www.gdnet.org. 38 Annex D Annex D. Genesis, Evolution, and Exit Strategies of GDN GENESIS AND EVOLUTION Jointly recognizing the role of knowledge and information and their importance in development, the World Bank, along with representatives from 22 think tanks in developing countries, convened in 1998 to develop a specific proposal for a new initiative in capacity building for knowledge institutions. This initiative led to the formation of the Global Development Network. A second preparatory meeting, conducted in May 1999, was convened and jointly organized by the World Bank Institute and Development Economics Vice Presidency of the World Bank. Participants in this preparatory meeting included the leaders of each of the seven regional research networks, as well as a number of potential sponsors and partners. The participants at this meeting decided: (1) to launch a survey of research institute needs throughout the developing world; (2) to undertake a global research project involving all seven networks, and (3) to begin preparing for the launch of the Global Development Network. GDN was thus initially a pilot project of the World Bank. The program's official launch took place after extensive consultations at the Bonn meeting on the GDN's objectives, scope and governance structure in December 1999 at an inaugural conference in Bonn, entitled "Global Development Network 1999: Bridging Knowledge and Policy." EXIT STRATEGIES The Global Development Network (GDN) has already spun-off from the World Bank. It became an independent, not-for-profit organization on March 16, 2001 and it physically moved out of the World Bank's Headquarter premises to its new headquarters at 2600 Virginia Avenue in July of 2001. These moves will complete GDN's disengagement strategy. At present, GDN is in the process of relocating to New Delhi, India, as was decided at the Third Annual Global Development Conference. GDN is scheduled to relocate to New Delhi in April, 2005. The program is still debating whether or not to pursue the status of an international organization once its relocation has been finalized. 39 Annex E Annex E. Evaluations and Audits of GDN Table E.1. Recent and Scheduled Evaluations and Audits of GDN Type of Date Commissioned Reported to Conducted by Title evaluation/ completed/ by audit expected Independent GDN Governing H. Peter Muth and Global Development Network: evaluation March 2004 GDN Body Frederick H. Gerlach Independent Evaluation Internal GDN Governing Internal Evaluation of the Core evaluation March 2004 GDN Body GDN Secretariat Activities Supported by the Global Development Network GDN Governing Evaluation of Regional Self-evaluation Body Research Competitions: South of selected Jan. 2002 GDN Barbara Craig Asia, Eastern and Central components Europe, and Sub-Saharan Africa Self-evaluation GDN Governing Evaluation of Regional of selected Jan. 2002 GDN Body Fernando Loayza Research Competitions: Middle components East and North Africa, Former Soviet Union, and East Asia 40 Annex F Annex F. Governance of GDN Legal status GDN is legal entity Governance and GDN Governing Board management bodies GDN Secretariat Regional Networks Location of program Washington, D.C. management unit. Written charter Yes, 1999 and date Other constitutional- Articles of incorporation, by-laws, and a statement on governance; also an MOU with level documents the World Bank Governing Body Name GDN Governing Body Current size 17 Current membership World Bank, UNDP, OECD, and 14 other international/regional networks and research institutions Membership criteria Multilateral and bilateral development agencies, and public and private development organizations, research institutions, and foundations Membership Contribute (in cash or in kind) to GDN's operating budget and/or core fund; this must be responsibilities more than parallel activity and must actually be part of the central program of GDN Commit to participate actively in the GDN, with representatives (a) appropriately placed with their organizations and with the necessary knowledge to support mainstreaming; and (b) provide continuity of representation to the maximum extent possible. Demonstrate a willingness to develop and implement a plan to mainstream activities in the member institution Minimum annual None financial contribution Functions of · To endorse strategic directions and core activities governing body · To conduct reviews · To periodically renew GDN's mandate and funding Meetings Annual Decisions By consensus Chair and Deputy of Akilagpa Sawyer and Richard Cooper Governing Body Functions of chair of · To call and host Annual Meetings and other meetings Governing Body Management Body Name GDN Secretariat Current size 10-member secretariat; with Executive Director Reports to Governing Body Functions Responsible for the day-to-day operations of the GDN To advise GDN Governing Body chair on relevant information To assist with communication among members, staff, and Secretariat (Executive Secretary also serves as secretary for the Governing Body) 41 Annex G Annex G. Financing of GDN Table G.1. GDN, Sources, and Uses of Funds, Fiscal Year 2003 (US$ Millions) Sources of funds Use of funds Financial contributions 10.086 Expenditures 10.370 By restrictions Program services 9.761 Unrestricted 9.291 Annual Conference 2.070 Regional Research Temporarily Restricted 0.796 Competitions 1.943 Global Development Awards 0.685 Global Research By source of funds 10.086 Projects 2.565 Grants, contracts, and contributions 9.804 GDN Net 0.923 In-kind contributions 0.144 Other programs 0.593 Interest Income .035 Foreign currency exchange gain .103 Supporting services 0.609 Others .001 Donor Coordination 0.264 Secretariat (Indirect Costs) 0.345 Beginning of period balance 1.133 Change in net assets -0.284 End of period balance 0.849 42 Annex H Annex H. People Consulted Name Title Organization Caroline Pestieau AERC Board Chair AERC Tony Killick AERC Board AERC William Lyakurwa Executive Secretary AERC Samir Makdisi GDN Governing Board Member American University of Beirut Ellen Fiske Administrative Director CERG-EI Randall Filer President CERG-EI Jan Kmenta GDN Governing Board Member Charles University in Prague. Thawatchai Jittrapanun Head, Faculty of Economics Chualongkorn University, Bangkok, Thailand Samuel Wangwe Executive Director Economic and Social Research Foundation, Tanzania Richard Cooper GDN Governing Board Member, Harvard University Vice Chair Fred Gerlach External Evaluator of GDN Independent Consultant Peter Muth External Evaluator of GDN Independent Consultant Subhorota Ray Fellow Indian Council for Research in International Economic Relations, New Delhi Isher Ahluwalia Director Indian Council for Research on International Economic Relations, New Delhi Carolina Hernandez Director Institute for Strategic and Development Studies, Philippines. Bina Agarwal GDN Governing Board Member Institute of Economic Growth. Delhi University. India Chia Siow Yue Director, and regional coordinator, Institute of Southeast Asian Studies EADN (ISEAS), Singapore Supriti Bezbaruah Administrator EADN Institute of Southeast Asian Studies (ISEAS), Singapore Mohsin Khan Director, the IMF Institute International Monetary Fund Ulrich Hiemenz GDN Governing Board Member Organization for Economic Co- operation and Development. Paris, France Arvind Virmani SANEI Executive Secretary SANEI Lal Jayawardena GDN Governing Board Member Social Scientist Association of Sri Lanka Chalongphob Sussangkarn GDN Governing Board Member Thailand Development Research Institute, Bangkok Thailand Gary McMahon Principal Economist The Global Development Network Lyn Squire Director The Global Development Network Mario Lamberte President The Philippine Institute for Development Studies, Makati City, Philippines Thomas A. Duvall Chief Counsel, Legal Unit The World Bank Andrea Stumpf Senior Counsel, Legal Unit The World Bank