Public Disclosure Authorized Realizing the World Bank Group’s Knowledge Potential for Effective Development Solutions: A STRATEGIC FRAMEWORK MARCH 2021 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized EXECUT IVE S UMM ARY The World Bank Group (WBG) is the reference of choice in development knowledge. Knowledge and its use are vital in the WBG’s mission of supporting poverty reduction and inclusive, sustainable growth. The WBG’s unique comparative advantage lies in the synergies between knowledge and financing: knowledge supports the design of operations the WBG finances, and these operations in turn generate knowledge about what works, informing subsequent operations. The WBG’s convening power as an independent generator and broker of global knowledge allows it to inform development policy makers and take a lead role in setting the agenda for global discussions on development. The challenge is to move beyond current achievements and be ready to assist client countries as they face ever more difficult and complex development agendas. Four emerging trends prompt a renewed effort to build on the past successes in order to ensure the WBG’s continued knowledge leadership. The WBG faces (1) rapidly increasing demands for knowledge-based solutions to address and recover from COVID-19, (2) the emergence of other providers of development knowledge, (3) unique opportunities from advances in development research to use operations to generate knowledge, and (4) increased availability of new technologies to raise the impact of its knowledge work. In this context, the vision is for the WBG to strengthen its role as a solutions bank that combines financing with global knowledge to generate development solutions in a timely, contextualized and integrated way. The agenda to realize this vision is informed by past evaluations on how to bolster the WBG’s knowledge agenda, as well as extensive internal and external consultations. Three main challenges are identified to ensure the WBG’s role as a leader in development knowledge and innovation: (1) enhancing the relevance, quality and impact of WBG analytical work and advice; (2) improving knowledge flows and collaboration across the WBG, and (3) fully realizing the WBG’s potential to learn from operations to ensure its role as a leader in development innovation. To address these challenges and fulfill the vision, the framework is grounded on three pillars: (1) strengthen systems for greater prioritization, evaluation, quality control and effective strategic outreach; (2) strengthen incentives at all levels of management and staff for enhanced emphasis on knowledge; and (3) strengthen human capital through strong recruitment, clearer career paths, and appropriate training, set out in the figure below. Cutting across all three pillars is a need for an outcome orientation and better metrics of performance, as well as greater exploitation of new technologies and platforms. Together, these will enable the WBG to push the envelope to reach its full potential as a solutions bank and strengthen its voice in the global development debate. REALIZING THE WBG’S FULL KNOWLEDGE POTENTIAL Increase relevance, quality, and impact of knowledge products Improve knowledge flows and collaboration across the WBG Realize WBG’s potential to learn from its operations and ensure role as a leader in development innovation PILLAR 1 PILLAR 2 PILLAR 3 Strengthen systems Strengthen incentives Strengthen human capital Cross-cutting 1: Improve outcome orientation and performance metrics Cross-cutting 2: Improve ICT and platforms II TA BL E O F CONTE NTS 1. Introduction and Context.............................................................................................................................. 1 2. The WBG Knowledge Products and their Position in the Knowledge Space........... 7 2.1 WBG Knowledge Products............................................................................................................... 8 2.2 Positioning of WBG Knowledge Products in the Knowledge Space.................... 10 3. Three Challenges to Using Knowledge to Provide Integrated Development Solutions.................................................................................................................................. 13 3.1 Increasing relevance, quality, and impact of knowledge products........................ 13 3.1.1 Prioritizing for quality.............................................................................................................. 14 3.1.2 Ensuring relevance and impact through client engagement and strategic outreach........................................................................................................... 17 3.1.3 Human capital and incentives for knowledge work............................................ 18 3.2 Improving knowledge flows and collaboration across the WBG............................. 19 3.2.1 Institutional incentives for knowledge flows............................................................ 21 3.2.2 Infrastructure for knowledge flows............................................................................... 22 3.3 Improving the WBG’s capacity to learn from its operations....................................... 22 4. The Agenda Ahead............................................................................................................................................ 25 4.1 Pillar 1: Strengthening systems to ensure relevance, quality and impact........... 26 4.2 Pillar 2: Strengthening incentives for knowledge work and collaboration....... 27 4.3 Pillar 3: Strengthening human capital...................................................................................... 28 4.4 Cross-cutting issues.............................................................................................................................. 29 5. Conclusion............................................................................................................................................................... 30 References..................................................................................................................................................................... 32 III Annex 1: Example Boxes........................................................................................................................................ 35 Box 1: Building a Coalition of Support for Human Capital....................................................... 35 Box 2: Knowledge leads the WBG’s 4-decade partnership with China (1980-2020)............................................................................................................................................. 36 Box 3: IFC’s Knowledge for Private Sector Development.......................................................... 37 Box 4: Three Successful programs focused on capturing lessons learned at IFC...... 38 Box 5: Virtuous cycle of knowledge flows in the Bank – Indonesia and Egypt........... 39 Box 6: How lending in Madagascar informed WBG trade facilitation knowledge... 40 Box 7: Engagement approach followed in Indonesia, Afghanistan, and Nigeria...... 41 Box 8: Mozambique’s Energy Sector Transformation: A Coordinated Engagement.......................................................................................................................................... 42 Box 9: Peru: Leveraging Knowledge from the Joint Capital Markets Program (J-CAP) in Support of SMEs.......................................................................................................... 43 Box 10: Data innovations in the World Bank..................................................................................... 44 Box 11: Extracting tacit knowledge to better evaluate place-based policies............... 45 Box 12: Working together – What made the PG-DEC-CMU collaboration in Madagascar a success in terms of embedding knowledge in and capturing knowledge from a project?.................................................................................. 46 Box 13: IFC’s OHADA Investment Climate (IC) Program Evaluation................................... 47 Box 14: WBG Engagement through Country Private Sector Diagnostics (CPSDs).... 48 Box 15: Leveraging the WBG to Deliver a Quality Assessment of Colombia’s Healthcare Sector............................................................................................................................. 49 Box 16: Using artificial intelligence (AI) to overcome size challenge and improve search function................................................................................................................................................... 50 Annex 2: Previous Efforts to Realize the WBG’s Knowledge Potential.................................... 51 Annex 3: Making Knowledge Work at IFC.................................................................................................. 58 IV I N D EX O F FI G U R E S Figure 1: Global Average Rating of ASA and Financing, FY16-19.................................................. 3 Figure 2: The Virtuous Cycle of Knowledge Flows in the WBG..................................................... 8 Figure 3: The WBG’s Position in the Knowledge Space.................................................................... 11 Figure 4: The Knowledge Iceberg.................................................................................................................... 12 Figure 5: Spending on Bank knowledge products doubled over the last decade.......... 15 Figure 6: ASA Expenditure by Funding Source...................................................................................... 16 Figure 7: Realizing the WBG’s Full Knowledge Potential................................................................. 20 V AC K N OWL E D GE ME NTS This document has been produced by a World Bank Group-wide working group convened by, and under the guidance of, the Managing Director for Development Policy and Partnerships (MDDPP), Mari Pangestu. The Working Group includes 24 members representing various units across the Bank (all Practice Groups, one Region, DEC, ECR, HR, IEG, ITS, and OPCS). Members include: William Maloney and Aart Kraay, co-chairs; Emily Adeleke; Enrique Aldaz-Carroll; Hana Brixi; Brenda Barbour; Dani Clark; Richard Damania; Alexandra Drees-Gross; Deon Filmer; Vivien Foster; Roberta Gatti; Stephane Guimbert; Mary Hallward-Driemeier; Rasmus Heltberg; Roumeen Islam; Vivek Kulbhushan Sharma; Rahyab Lari; Elisa Liberatori-Prati; Hoveida Nobakht; Martin Rama; and Edith Umole. The Working Group also benefited from additions from IFC (Anastasia Gekis, Margot Brown, Neil Gregory, Deepa Chakrapani) and MIGA (Laura Alonzo) liaisons. The document has benefitted from extensive internal and external consultations. This involved MDDPP, MDOPS, the Regions, the Practice Groups, DEC, ECR, OPCS, FCV, IEG, HR, ITS, BPS, DFI, IFC, MIGA, the Chief Economists’ Council, the Operations Council, the Meeting of Vice Presidents on Operational Matters (OVP), as well as former Bank Group staff members who have worked in this space. The team thanks Antonella Bassani, Ferid Belhaj, Anna Bjerde, Ousmane Diagana, Makhtar Diop, Annette Dixon, Alison Evans, Qimiao Fan, Manuela Ferro, Stephanie von Friedeburg, Hafez Ghanem, Carlos Felipe Jaramillo, Ayhan Kose, Victoria Kwakwa, Hiroshi Matano, Mamta Murthi, Akihiko Nishio, Sheila Redzepi, Carmen Reinhart, Denis Robitaille, Hartwig Schafer, Axel van Trotsenburg, John Underwood, and Juergen Voegele for valuable advice and feedback. The team also thanks the Board of Directors’ Committee on Development Effectiveness (CODE) for their input and guidance. VI AC R O N YM S AAR After Action Review AI Artificial Intelligence AIMM Anticipated Impact Measurement Monitoring AS Advisory Services ASA Advisory Services and Analytics AUSAID Australian Agency for International Development BB Bank Budget BPS Budget, Performance Review and Strategic Planning CCT Conditional Cash Transfer CD Country Director CEDVP Economics Vice Presidency CEO Chief Executive Officer CEoG Chief Economists of Governments’ Network CMU Country Management Unit COVID-19 Coronavirus Disease 2019 CPF Country Partnership Framework CPSD Country Private Sector Diagnostic CSO Corporate Special Operations CSOs Civil Society Organizations DEC Development Economics Vice Presidency DECRG Development Research Group DFI Development Finance Institution DPF Development Policy Financing DPL Development Policy Loan DPO Development Policy Operation EAP East Asia and Pacific ECA Europe and Central Asia ECR External and Corporate Relations EDM Electricidade de Moçambique EFI Equitable, Growth, Finance, and Institutions Practice Group VII EFO Externally Financed Output EIU Economist Intelligence Unit EM Emerging Markets ESG Environmental Social and Governance ESW Economic and Sector Work FCS Fragile and Conflict-Affected Situations FCV Fragility, Conflict, Violence FDI Foreign Direct Investment FONGIP Le Fonds de Garantie des Investissements Prioritaires FPD Finance and Private Sector Development FY Fiscal Year G7 Group of Seven G20 Group of Twenty GDP Gross Domestic Product GFF Global Financing Facility GL Global Lead GoM Government of Mozambique GT Global Theme HCP Human Capital Project HR Human Resources IBRD International Bank for Reconstruction and Development IC Investment Climate ICR Implementation Completion and Results Report ICSID International Center for Settlement of Investment Disputes ICT Information and Communication Technologies IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation IMF International Monetary Fund IT Information Technology ITS Information Technology Solutions J-CAP Joint Capital Markets Program KM Knowledge Management VIII KMS Knowledge Management Strategy KP Knowledge Package LAC Latin America and Caribbean MALENA Machine Learning Environment (Social & Governance) Analyst MDB Multilateral Development Bank MDDPP Managing Director, Development Policy and Partnership MDOPS Managing Director, Operations MIGA Multilateral Investment Guarantee Agency MIT Massachusetts Institute of Technology NGO Non-Governmental Organization OHADA Organization for the Harmonization of Business Law in Africa OPCS Operations Policy and Country Services PAD Project Appraisal Document PforR/P4R Program for Results PPP Public-Private Partnership PSD Private Sector Development R&D Research and Development RAS Reimbursable Advisory Services RCT Randomized Controlled Trial RD Regional Director ROSC Report on the Observance of Standards and Codes SCD Systematic Country Diagnostic SD Sustainable Development SME Small and Medium-Sized Enterprise TA Technical Assistance TF Trust Fund TTL Task Team Leader UNHCR United Nations High Commissioner for Refugees VP Vice Presidency VPU Vice Presidential Unit WB World Bank WBG World Bank Group WDR World Development Report WEF World Economic Forum IX 1 INTRODUCTION Knowledge can be defined broadly as the advance in understanding that results from the processing of information and the creation and recombination of ideas. One of the World Bank Group’s intellectual founders, John Maynard Keynes, argued that ideas may matter more than anything else in shaping the course of history.1 This underscores the centrality of producing, curating, and disseminating quality knowledge on development to achieving the WBG’s mandate to reduce poverty and promote inclusive, sustainable growth. As President Malpass recently emphasized “Our knowledge work – both global and at the country level – underpins all of our operational engagements and is an important development service in itself.”2 The WBG’s focus on knowledge is not new. Eugene Black in his Annual Meeting address in 1949 highlighted knowledge sharing as one of the most constructive contributions that Bank could make.3 Since the mid-1970s, the WBG has been a leader in development knowledge, pioneering analytical work on issues ranging from economic growth, to poverty and inequality measurement, to governance and service delivery, as well as impact evaluations and randomized controlled trials. Its topical and geographical scope are unparalleled, creating unique possibilities for cross-sectoral insights and cross-regional knowledge flows. The WBG has also pioneered development data, including globally standardized data on households, poverty and debt. Its investments in data curation and dissemination facilitate knowledge creation by researchers everywhere. The WBG’s 1. Keynes wrote “The ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is commonly understood. Indeed, the world is ruled by little else. Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist … I am sure that the power of vested interests is vastly exaggerated compared with the gradual encroachment of ideas.”. The General Theory, Ch. 24 "Concluding Notes" page 383. 1936. 2. https://worldbankgroup.sharepoint.com/sites/news/announcement/pages/thank-you-and-happy- holidays-16122020-130253.aspx 3. https://openknowledge.worldbank.org/bitstream/handle/10986/31726/125459-WP-Black-9-13-1949- Annual-Meetings-PUBLIC.pdf?sequence=1&isAllowed=y knowledge work is highly rated4 and is widely shared, making the WBG the reference of choice for development knowledge.5 WBG knowledge and financing are highly influential in shaping development practice and informing policymakers. A recent study analyzed survey data covering 1,244 public sector officials in 121 developing countries, finding that "analytical and advisory products not only affect the direction of government policy, but also its design and implementation."6 Figure 1 shows that Client Satisfaction surveys reveal a high degree of satisfaction with knowledge products in the form of Advisory Services and Analytics (ASA).7 A recent IEG review of knowledge- based country programs in the WBG found that 84 percent had fully or partially achieved their objectives.8 Areas of WBG strength identified in this review included its ability “to fulfill in a timely manner client requests for state-of-the art advice”; “to act as a trusted knowledge broker”; “to act as a knowledge connector” and “to take a pragmatic approach to important issues that required multisectoral development solutions.”.9 Similarly, IEG has noted that the strength of IFC’s convening power lies in its operational experience and investment expertise, spanning regulatory issues related to private sector development, capacity building and other technical assistance through IFC’s advisory services, and mobilization of private capital, and implementation of IFC’s performance standards.10 The WBG uses knowledge and data to help set the global development agenda. It uses its strong convening power on many development issues, meeting the demands of partners and shareholders, as a highly relevant global actor.11 A seminal initiative was the 2010 Open Data and Knowledge initiative, which greatly enhanced the accessibility of the WBG’s data and knowledge products. More recent examples include the Debt Service Suspension Initiative, the Human Capital Project (see Annex 1, Box 1), and the COVID-19 Crisis Response Approach Paper, which helped shape global conversations in fora including the G7 and the G20, and were highly influential in the WBG’s country dialogue. IFC is known as an agenda-setter on private sector development issues. The WBG has a “seat at the table” because it is often thinking ahead of the curve and client demand, bringing new data and cutting-edge ideas to the global debate. 4. One indicator of this can be found in the rankings produced by Research Papers in Economics (RePEC), a decentralized bibliographic database of working papers, journal articles, books, book chapters and software components, where the World Bank Group ranks first in development economics. The next three are MIT, the National Bureau of Economic Research, and the London School of Economics. 5. One indication of this is that as of October 2020, 45 million Bank documents have been downloaded from the Open Knowledge Repository since its inception in 2012. 6. Knack et. al. 2020:2. This echoes earlier work by Tang (2008) finding that “Stakeholders in IBRD countries indicate a strong preference for the Bank’s nonlending services over its lending services.” 7. This figure reports the average rating across six questions (1) The World Bank worked well with us during this operation; (2) The World Bank brough global expertise to support this operation; (3) The World Bank’s expertise was tailored to my country context; (4) The World Bank provided support at the right time; (5) I would work with the World Bank again for this type of operation; (6) Overall this operation has achieved it’s intended development outcomes. Clients are asked to rate their agreement to each statement on a sale of 1-5, with 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree. 8. IEG (2016a). The review covered Bulgaria, Chile, China, Kazakhstan, Kuwait, Malaysia, the Russian Federation, South Africa, and Thailand. 9. IEG 2016a: xv-xvi 10. IEG (2019). 11. IEG (2020a). 2 FIGURE 1 GLOBAL AVERAGE RATING OF ASA AND FINANCING, FY16-19 Low Rating 1 – High Rating 5 4.45 Average Rating 4.35 4.25 4.15 FY16 FY17 FY18 FY19 ASA Financing Source: World Bank Client Survey. The WBG also serves as a credible broker for global knowledge. Client countries produce knowledge of global interest and the WBG, by virtue of its geographical reach, is well positioned to use and intermediate it. Many knowledge products are co-created in partnership with local analysts, country officials and development partners. Combined with long-standing links to advanced country universities and think tanks, this places the WBG at the crossroads of development debates. The Bank Group’s response to the COVID-19 pandemic illustrates how the Bank Group has harnessed knowledge and financing to rapidly respond to client country needs during the pandemic. A high level of attention by management during crisis times creates a strong incentive for collaboration to facilitate the flow of knowledge across institutions (WB, IFC and MIGA), Regions, Practice Groups (PGs), and the Development Economics Vice Presidency (DEC), enabling an unprecedented scaling up of the operational response to the pandemic. Knowledge extracted from past experiences is rapidly converted into impactful policy notes as well as actionable approaches to meet client needs. The knowledge products and approaches produced during these times receive high visibility and engagement, raising client demand for support in those topics and enhancing development impact. In addition, leveraging experiences from private sector engagements through IFC and MIGA, including during previous crises, adds both impetus and value to private sector responses supporting economic recovery. Finally, thanks to its recognized knowledge, the WBG has had a resonant voice in global policy debates and a prominent role coordinating the international community’s response. 3 The challenge is to face the future with a Strategic Framework that looks to 2030 and beyond. Developing the Strategic Framework for Knowledge calls on the WBG to build the capacity to acquire and disseminate the knowledge that will help our clients meet ever more demanding development needs. Being a solutions bank means being ready to assist countries on ever evolving and more complex agendas. President Malpass has highlighted, for example, the importance of knowledge in assisting countries with green, resilient, and inclusive recovery from the COVID-19 crisis. Countries are seeking timely answers on job creation, longer-term strengthening of health systems that build in pandemic preparedness, and other multifaceted development challenges, all requiring cutting-edge knowledge delivered in useful and imaginative ways. The Strategic Framework is designed to move from the current acknowledged position as a Solutions WBG to the Solutions WBG that is needed for the 2020s and beyond. Four emerging trends prompt consideration of how to build on the above strengths to fully realize the WBG’s knowledge potential: 1 COVID-19 has caused a severe impact on client countries, raising global poverty for the first time in more than two decades, setting back progress towards meeting the Sustainable Development Goals. At no time in recent history has timely development advice – paired with development finance – been more needed to help countries build back better and set their economies on a sustainable and inclusive development path with the private sector playing a leading role in economic recovery. For this increased demand for knowledge- based solutions to be met, the WBG needs to maximize its ability to generate knowledge, adapt it to country context, and enhance its impact. Similarly, for IFC, incorporating the depth of in-house knowledge and lessons learned is essential to benefit clients and to allow the private sector to play a leading role in economic recovery. 2 The WBG is increasingly joined by new players in knowledge areas. Development economics has been transformed from a niche area of academia to the mainstream. Scholarly research on development and on developing countries has become much more abundant, and strong institutions focused on development research and policy have emerged. Even private sector consulting groups have large practices focused on development policy. The WBG needs to ensure its ability to build partnerships and remain at the frontier for expertise and ideas, as well as enhance its role as a global broker in this dynamic environment. 3 The nature of development research has shifted in a direction that provides unique opportunities for the WBG to contribute to the global stock of knowledge. The WBG can do more to learn from and inform its financing operations by taking greater advantage of the unique opportunities provided by its role as an institution that finances and implements development projects 4 while simultaneously housing extensive analytical, advisory, and research capacity. One important avenue is strengthened impact evaluation, building on recent advances in methodology. As recognized by the awarding of the 2019 Nobel Prize in Economics to Abhijit Banerjee, Esther Duflo, and Michael Kremer, research focused on rigorously identifying the impact of development policy interventions relative to well-defined counterfactuals has burgeoned.12 Over this same period, the WBG has been actively engaged in implementing modern impact evaluations of its projects, including through field experiments and randomized controlled trials. 4 New technologies offer potential to raise impact. New information technologies offer the potential for better curation, storage and sharing of knowledge, leveraging the WBG’s role as a knowledge broker, facilitating long distance collaboration, and generating new sources of data for development. At the time that the Bank launched a New Knowledge Partnership in 1996, 1.3 percent of the global population were internet users, mainly in high-income countries. There were 2.5 mobile cellular subscriptions per 100 people, again mainly in high-income countries. Last year there were more than 4.6 billion active internet users, nearly 60 percent of the global population. More than 25 percent of Sub-Saharan Africans are internet users and more than 30 percent in South Asia. In Q2 of FY21 alone there were 28.2 million visits to the WBG website, 52 percent from developing countries and 37 percent by mobile users. Recognizing the importance of knowledge to its mission, the WBG committed to developing a Strategic Framework for Knowledge Generation and Sharing in the context of the 2018 IBRD/IFC capital increase. The March 2018 paper presented to the Development Committee13 stressed the centrality of knowledge to the WBG’s value proposition. WBG management made this commitment in line with the broader conversation about a “Better and Stronger WBG” that operates more effectively and efficiently. The paper states: The WBG’s knowledge and convening role will increase to support the design and sharing of development solutions for greater impact. This includes strengthening ongoing engagements and developing new ones, effectively utilizing the power of new technology and innovations. The WBG effort’s will focus on sharing new research to underpin improved policymaking on emerging challenges; systematically harnessing and sharing knowledge (e.g., South-South exchange) embedded in 12. The citation for the Nobel prize states that “The research conducted by this year’s Laureates has considerably improved our ability to fight global poverty. In just two decades, their new experiment-based approach has transformed development economics, which is now a flourishing field of research.”. Press release: The Prize in Economic Sciences 2019, https://www.nobelprize.org/prizes/ economic-sciences/2019/press-release/ 13. “Sustainable Financing for Sustainable Development: World Bank Group Capital Package Proposal” 2018. The due date for the Strategic Framework was extended to FY21 in the September 2020 IBRD-IFC Capital Package and the Forward Look Implementation Update. A technical briefing on World Bank Knowledge Work was delivered in June 2019 and is the most recent formal engagement on the topic with the Board. 5 financing operations across the income spectrum; supporting innovative approaches for data collection; continuing to strengthen public access to development data; helping countries share experience on maximizing finance for development; and convening the public and private sectors on pressing global challenges. The World Bank will develop a Strategic Framework for Knowledge Generation and Sharing to preserve and enhance its comparative advantage in this area. (p. 21) Past WBG efforts to fully realize its potential as a knowledge institution have focused on two themes. The first is the ongoing effort to put in place key mechanisms and incentives to better discharge the WBG’s knowledge mandate, perhaps best referenced by a series of reports and initiatives around “the Knowledge Bank” as well as several IEG reports. The second is the search of the optimal underlying organizational structure, driven by the goal of removing either regional or thematic silos that could impede the flow of knowledge. Annex 2 provides an overview and key lessons from these previous efforts. The experience of these past efforts has informed the development of this Strategic Framework for Knowledge. This framework differs from past efforts in four respects. First is the renewed emphasis considering the emerging trends on the interdependence between knowledge and financing, on both the public and private sector sides. As discussed in Section 2, knowledge informs the design of financing and knowledge can be drawn from financing operations. Second is this framework’s “whole of Bank” approach that involves all WBG staff to draw knowledge from financing, facilitate the flow of tacit knowledge embedded in staff, and ensure knowledge informs financing. In this sense, it is important to highlight that, while knowledge management, narrowly construed, is an important ingredient in the Strategic Framework, managing the generation and flow of knowledge is the task of all WBG staff and cannot be solely dedicated to a single unit. The third is the opportunity created by the new organizational structure in place since July 2020 and the accompanying revitalized focus on knowledge among WBG senior management. The fourth is its consistency with the heightened emphasis on outcome orientation.14 The vision articulated by this Strategic Framework for Knowledge is for the WBG to strengthen its role as a solutions bank that combines financing with global knowledge to generate development solutions in a timely, contextualized and integrated way, and strengthen its position as the reference of choice in global development debates. With this new Strategic Framework for Knowledge, the WBG is pushing the envelope to better integrate knowledge into the solutions we provide to clients and the global community. The goal is to establish the enabling environment. The paper is organized as follows. Section 2 sets out a taxonomy of WBG knowledge products and shows how the Bank is currently positioned in the knowledge space. Section 3 gives a synopsis the main challenges, based on extensive internal and external consultations. Section 4 presents the agenda ahead, followed by a brief conclusion. 14. Outcome orientation is the organization’s ability to generate feedback on what works, what does not, and why, use this feedback to engage clients and adapt country programs, and ultimately bolster its contribution to country development outcomes (IEG 2020b). 6 2 THE WBG KNOWLEDGE PRODUCTS AND THEIR POSITION IN THE KNOWLEDGE SPACE The WBG is well-positioned to take advantage of the opportunities presented by the emerging trends highlighted above. As Figure 2 suggests, the ability to combine high quality global knowledge with significant finance for development projects makes the institution unique relative to other banks, consulting firms, or research institutions. Quality analytical work informs and is often the gateway to crystalizing client government consensus on key issues. It feeds into the choice and design of WBG financed activities. Conversely, the proximity of WBG researchers to operations provides an unparalleled opportunity to learn from development interventions (see Annex 1, Box 2). Knowledge also flows directly to client countries, in the form of global reports, country reports and advisory work, to improve policies and institutions. Through this open dialogue, WBG staff are well-placed to learn from and synthesize lessons from successful and failed policies. This virtuous cycle of knowledge flows, when fully exploited, enables the WBG to provide integrated development solutions. FIGURE 2 THE VIRTUOUS CYCLE OF KNOWLEDGE FLOWS IN THE WBG KNOWLEDGE FINANCING POLICIES & INSTITUTIONS 2.1 WBG KNOWLEDGE PRODUCTS In the widest sense, all WBG products are knowledge products. Notably, WB-financed operations, IFC financings, and MIGA guarantees incorporate knowledge gained from a wide spectrum of research and analysis inside and outside the WBG. Moreover, they draw directly on staff knowledge gained through education and experience. This section narrows the scope to those activities that are designed explicitly to create, codify, and pass on knowledge. One useful categorization is geographical: country, regional, and global level, while emphasizing that there are feedback loops among all three. 1. Country-level work: At the country level, knowledge work is organized through the Country Partnership Framework (CPF) process. The CPF is based on previous knowledge work, including a Systematic Country Diagnostic (SCD). The CPF sets out the planned WBG support, including priority knowledge work to underpin the goals of the Strategy. This includes core analytical products (such as Poverty Assessments, Country Economic Memoranda, and Public Expenditure Reviews), as well as short policy memos. These often are supported and informed by more fundamental country-specific research and analysis generated by staff, particularly in DEC and the PGs. Other knowledge products support specific goals of the strategy, often providing knowledge necessary for key operations. An example is the FY19 report, Nicaragua: Tracking Teenage Pregnancy, done to inform interventions to reduce teenage pregnancy and develop skills for labor force participation. 2. Regional: The WBG also produces topical and sectoral reports of interest across multiple countries in a region. For example, a report, Sustainable Land Management in the Sahel: Lessons from the Sahel and West Africa Program in Support of the Great Green Wall, drew key findings from projects in twelve Sub-Saharan African countries. 3. Global: Global work is carried out mainly by PGs and DEC on subjects of interest across regions. An important component is the production and maintenance of global data sets 8 such as those in the World Bank Open Data website. It also includes flagship reports such as the annual World Development Report, and the bi-annual Global Economic Prospects report series. IFC’s work on market standards also fits under this category. To better manage its knowledge work, the WBG has established knowledge product categories with distinct governance structures and rules. For the Bank, these categories are Advisory Services and Analytics (ASA), research, and internal knowledge products (KPs). For a more complete exposition, see the World Bank Technical Briefing on Knowledge Work, OM2019-0033, June 2019, and Annex 1, Box 3 and Annex 3 on IFC’s knowledge products. 1. Advisory Services and Analytics (ASA) is the major category for country work but also includes regional and global products. Analytical tasks focus on the creation of new knowledge to help shape client or global development strategies, policies, or reforms. Examples are the country poverty studies that inform WBG programs. Advisory tasks focus on the application of pre-existing knowledge customized as needed. Examples are direct support for institutional capacity building and knowledge exchange. Support for South-South exchanges such as facilitating the transfer of experience of Viet Nam in modernization of electricity transmission to Uzbekistan falls under this category. The ASA category is the largest by number of products annually: more than 1,600 products in FY19 on the Bank side and almost 800 in more than 115 countries in FY19 on the IFC side, for their similar IFC Advisory portfolio. Bank Reimbursable Advisory Services (RAS) are a subset of ASA. These are customized products requested and paid for by non-commercial clients. RAS enable the Bank to support the development priorities of upper and upper-middle income clients, to maintain global cutting-edge knowledge in important technical fields and to create knowledge spillovers and results. In FY19, the Bank delivered 120 RAS products in 35 countries. Through RAS work WBG staff gain knowledge and experience that can be used to assist other countries. 2. Research is the generation of original ideas and novel methodological tools aimed at increasing the understanding of economic and social issues to inform policy dialogue and influence development thinking. It includes data and tools for strengthening development knowledge. It ranges from country-specific research papers, to impact evaluations, to global reports like the World Development Report and Policy Research Reports. This category accounts for roughly seven percent of the annual budget for knowledge products and provides analytical rigor to inform the full range of country, regional, and global work described above. 3. Internal Knowledge Products are targeted to Bank staff in order to support professional development. Examples include, operational guides and toolkits, and knowledge sharing vehicles, for example the Energy Sector Knowledge series. This segment accounts for about five to six percent of the Bank’s annual budget for knowledge work. 9 These are the current knowledge products. Given the fast-moving changes in technology and digital platforms, the list may well evolve as the Strategic Framework is rolled out. Delivery methods will certainly continue to evolve. 2.2 POSITIONING OF WBG KNOWLEDGE PRODUCTS IN THE KNOWLEDGE SPACE WBG knowledge products span a “knowledge space.” This space is depicted in Figures 3.A and 3.B. Figure 3.A distinguishes knowledge products according to whether they are public versus private goods; whether they are fundamental or applied; and whether they are tacit or codified. The blue ellipse in Figure 3.B situates WBG knowledge products in this space along the first two axes, with specific examples of the very wide variety of knowledge products indicated in blue. Other types of institutions and their location in the knowledge space are shown in yellow. • Public good vs. private good knowledge: The vertical axis captures the degree to which the knowledge is a public good with benefits that accrue to the whole development community. Because the benefits are widely shared, any one country or institution will likely not have sufficient incentives to produce it.15 For example, a global dataset like the Income Comparison Project hosted by the World Bank produces benefits for all countries by enabling real comparisons of living standards across countries. Similarly, IFC’s Equator Principles and Operating Principles for Impact Management benefit investors and communities globally. No single country, or single agency, would be willing to pay for this entire effort because many of the benefits accrue to others.16 The provision of a variety of public knowledge goods is thus a critical role of the WBG. At the other extreme, some knowledge products are private goods. Due to the specific subject matter or relevance to just one country, such knowledge products also are produced by the private sector. Reimbursable Advisory Services (RAS) contracted by upper-middle-income countries, or IFC’s EMCompass notes (see Annex 1, Box 4) written to share insights with external stakeholders on IFC’s engagements are an intermediate category: while they are tailored advice for a single country, to the extent that they lead to knowledge spillovers for lower income countries, there is a clear rationale for the WBG to occupy this part of the knowledge space as well. • Foundational vs. applied knowledge: The horizontal axis distinguishes knowledge products by the degree to which they are foundational – such as the big development questions, often the ambit of universities or think tanks – versus the degree to which they 15. Intrinsic to the public good nature is the under-provision of this type of goods in a decentralized free market: the initial generator or collector of knowledge would receive a return to her/his investment below the return to society at large and the amount of knowledge produced by the free market is therefore sub-optimally low, potentially very much so. In the market economy, patents, R&D subsidies and tax write offs seek to align the public and private returns to innovation. The returns to Bank capital effectively provide the subsidy to public goods in the World Bank. 16. There is a “network” externality here. If a single country collects data on its business climate, it is of limited use since there are no comparators to benchmark against. Hence, all countries benefit from the coordinated collection of statistics. 10 are applied – ideas which may not be new, and need to be tailored to a specific situation. Foundational analysis, ranging from individual research papers to flagships such as World Development Reports (WDRs), aims to inform policy discussion and positions the WBG in global debates. More applied products, such as ASA, IFC Advisory Services, IFC Operational Knowledge products and Thought Leadership publications inform financing and anchor the WBG’s dialogue with client countries and its efforts around private sector led opportunities and depicted in the center. Similarly, evaluations performed by IEG and internal evaluations performed by the Bank, IFC, and MIGA (such as completion reports of projects and of analytical and advisory work) generate lessons to inform future advisory work and financing. Moving to the right and more applied might be “nuts and bolts” TA, and data collection or benchmarking such as the Doing Business Indicators. Knowledge feeds into and emanates from all the above. For examples see Annex 1, Boxes 5, 6, 7 and 8. • Codified vs. tacit knowledge. The remaining axis in figure 3A captures the degree to which knowledge is “codifiable” into documents or data that can be impersonally accessed and shared. A significant part of the WBG’s knowledge is tacit – often experience difficult to “download” from an expert’s head – yet important to informing financing and “front- line” policy advice that the WBG provides to its clients. In this sense, it is important to treat codified knowledge products as only a subset of the overall knowledge mission of the WBG. FIGURE 3 THE WORLD BANK GROUP’S POSITION IN THE KNOWLEDGE SPACE 3.A Public 3.B Public Good Universities Think Tanks Tacit Diagnostics Doing Business Research, World Development Indicators, Data Report, Flagships ASA, IFC AS, Policy Dialogue Foundational Applied Financing RAS Source: Chioda, De la Torre, Maloney (2013). Foundational Applied Codified Private Consulting Firms Private Private Good Source: Chioda, De la Torre, Maloney (2013). Figure 4 illustrates how interactions between these varying knowledge products under one roof are one of the WBG’s great strengths. It is critical that economists writing country or sector documents, project TTLs seeking necessary project knowledge, IFC staff leading financing engagements and advisory services, and practitioners offering “nuts and bolts” TA remain linked to the frontier literature to offer clients the best possible advice. DEC, the PGs, Regional Chief Economist Offices, IFC’s Economics and PSD VPU (CEDVP), and IFC operational and 11 support departments have a central role in generating, vetting, curating and disseminating new knowledge through global flagships, such as WDRs, regional and sectoral flagships, and research. Conversely, it is critical that the analytical agendas of these units reflect the emerging priorities and lessons learned from the front lines of client engagement. The figure also highlights that the high-impact quick-turnaround policy notes frequently requested by country authorities cannot simply be cobbled together by generalists relying on sources that the client also has access to. Rather, such front-line policy advice is the tip of a “knowledge iceberg,” resting on a deep installed capacity of research and practitioner knowledge. It is in the context of this practitioner knowledge that IFC is focused on enhancing its available tools to capture the tacit knowledge in-house while also enabling a broader pool of produced knowledge products to benefit project design and further enhance the development impact of IFC’s engagements (see Annex 1, Box 9). FIGURE 4 THE KNOWLEDGE ICEBERG Just in Time Policy Advice Project Knowledge Diagnostic “Nuts & Assessment Bolts” TA ROSCS Country & Convening Sector Services Reports WDR & Data, Regional Research Statistics, Flagships Indicators Source: Chioda, De la Torre, Maloney (2013). In a well-functioning knowledge system, these activities reinforce and are informed by each other. Research and analytical work respond to country requests and also anticipate future policy questions. Policy advice in the form of ASA, IFC Advisory Services and just-in-time notes is anchored in and informed by the most rigorous analytical work. More foundational analytical work versus client-oriented policy work do not compete, but rather complement one and other. All of this suggests a need for close coordination and governance of the knowledge work and mechanisms to ensure the whole is greater than the sum of the parts. 12 3 THREE CHALLENGES TO USING KNOWLEDGE TO PROVIDE INTEGRATED DEVELOPMENT SOLUTIONS The WBG has a strong track record of harnessing the synergies between knowledge and financing to deliver solutions to development challenges. Based on an extensive review effort, including both internal and external consultation, this report identifies three areas in need of strengthening in order to fully realize its potential considering the emerging trends noted above. First, there are perennial concerns about uneven relevance, quality, and impact of some knowledge products. Second, there remains scope to improve knowledge flows throughout the WBG in order to enhance the impact of knowledge on financing and the quality of the policy dialogue. Third, the WBG could do more to fully exploit the potential of learning from its operations, both through more rigorous evaluations as well as greater efforts to capture the tacit knowledge of its staff. 3.1 INCREASING RELEVANCE, QUALITY, AND IMPACT OF KNOWLEDGE PRODUCTS The extensive consultations supporting this document, as well as many of the previous analyses of knowledge work discussed in Annex 2, opportunities to sharpen the relevance, quality, and impact of knowledge products. While many are excellent and highly influential, some knowledge products are seen as having unclear strategic rationale, with limited evidence of their quality and relevance.17 Some areas of the portfolio of knowledge products may be over- diversified, diluting resources needed to ensure quality of individual products, and undermining impact.18 The absence of definitive metrics for relevance, quality, and impact of knowledge products makes any such diagnosis impressionistic. This is true for standard written documents such as ASA and research papers, where measures such as downloads or citations provide only imperfect indicators of relevance, quality and impact. It is even more the case for key front-line knowledge products at the top of the knowledge iceberg (Figure 4), such as quick response notes, as well as for the vast body of tacit knowledge in staff members’ heads. Measuring relevance, quality, of knowledge products is not easy, and impact may appear only over time: a country diagnostic product, such as a country expenditure review with an important finding, may sit unused until the right set of country circumstances make it the basis of truly transformational change. This measurement problem has two implications for the discussion in this section. First, since direct measures of the outcomes of interest (relevance, quality and impact) are limited, this section discusses challenges in the context of three key inputs that need to be in place to ensure these outcomes at the outset in a demanding quality-at-entry process. These inputs are (i) adequate resourcing, (ii) client engagement, and (iii) appropriate human capital. Second, despite measurement challenges, increased client sophistication in the WBG’s knowledge space nevertheless requires sustained effort to continually raise the relevance and quality of all knowledge products. 3.1.1 PRIORITIZING FOR QUALITY Spending on Bank knowledge products has roughly doubled in nominal terms over the last decade. As shown in Figure 5, direct spending on knowledge activities increased from just over $0.5bn in FY10 to nearly $1bn in FY20 . These figures are estimates and should be interpreted with some caution. Tracking resources allocated to knowledge products over time is complicated by internal reforms (such as changes in overhead charges) that have altered the cost of delivery, as well as difficulties in identifying the universe of more and less formal knowledge activities. In addition, these figures do not account for increases in staff costs (the 17. For some examples, see IEG (2016), p.47. 18. One commonly cited indicator is the proliferation of project codes although this is an imperfect indicator. Code inflation can be driven by many factors. Trying to limit the number of activities by limiting the number of codes without addressing underlying incentives has proven ineffective. TTLs bypass it by creating large programmatic codes covering all their activities, which reduces the number of codes but not the number of activities. 19. Notes for Figure 6: Expenditures are reported at activity level by product line; expenditures vary from those reflected in other corporate reports (e.g. QBRR, Budget Document). Knowledge products include research, internal knowledge products and ASA. Research includes thematic research, country-specific research, global reports, datasets and tools, knowledge sharing and impact evaluations. Internal knowledge products are targeted to Bank staff and aim to support profession and practice. They focus on production (synthesis reports), retention (online platforms, operational guides and toolkits) and dissemination (training and knowledge sharing events) of knowledge. 14 main cost item for knowledge products) over the same period. Despite these limitations, this data provides some important insights on how knowledge work is funded, with potential implications for relevance and quality and is not intended as an analysis of use of budget.20 FIGURE 5 KNOWLEDGE SERVICES ROUGHLY DOUBLED OVER THE LAST DECADE Evolution of Knowledge Products, FY10-20 Annual Expenditures, $bn 1.20 1.00 0.80 0.60 0.40 0.20 0.00 FY10 FY12 FY14 FY16 FY18 FY20 Source: OPCS World Bank. The increase in spending has been unevenly financed across funding sources. Figure 6 shows that while spending on ASA (the largest category of knowledge product spending) has risen sharply over the past 12 years, the increase has been driven heavily by external funds in the form of Trust Funds (TFs) and Externally Financed Outputs (EFOs). In comparison to those funding sources, Bank budget (BB) allocated to these activities had been relatively flat, before experiencing an increase in FY18-20, and thus overall resources for knowledge services in this period increased from both sources. While the surge in external funding has enabled an expansion in analytical work, it also points to the importance of ensuring agreed-upon priorities between the Bank and its external funding partners. Trust fund reform is moving in the direction to address the strategic alignment and fragmentation issue. The central innovation under the current phase is the introduction of Umbrella Programs as a flexible way to align and manage development resources. The reforms under Umbrella 2.0, introduced in January 2020, strengthen the link between funding and the strategic priorities of the World Bank, its clients and partners and promote impact at scale. Umbrella Programs are focused around major strategic themes identified by Business Units (primarily the PGs and Regions) and have the potential to convene the Bank and Development Partners to respond to articulated global, regional and country-based knowledge needs. Through the Umbrellas, we are able to ensure the mainstreaming of knowledge generated into future operations and to complement and leverage Bank operations, including through 20. The graph was updated by OPCS for FY20 using the same methodology. 15 co-financing. By channeling donor resources through fewer and larger Umbrella programs (72 have been identified so far), PGs, Regions and DEC are rationalizing their trust fund portfolio and strengthening selectivity and strategic alignment. Trust fund reform efforts are now focusing on improving the integration of the Umbrella program funding with the Bank’s strategy, planning, budgeting and staffing processes. These efforts are meant to address the traditional disconnect between BB and TF funding and support moves towards a “whole-of-finance” approach, allowing management a better line of sight on available funding (sources and amounts), and the basis for decisions around fundraising, partner engagement and portfolio management. This will also help ensure that client demand for knowledge is aligned with resources and staff capacity. The trust fund reform effort of the last few years is an example of an ongoing effort by Management that supports the vision articulated by this Strategic Framework for Knowledge. FIGURE 6 ASA EXPENDITURE BY FUNDING SOURCE Average Annual ASA Expenditures by Funding Source, $m 8 73 80 7 26 Expenditures, $m 71 5 67 402 47 16 1 87 315 4 65 578 199 20 4 1 90 215 FY 09 -1 1 F Y 1 2-1 4 F Y 1 5-1 7 F Y 1 8 -20 BB Trust Funds and EFO RAS These findings highlight the crucial role of strategic prioritization in knowledge work. While total spending on knowledge work has increased substantially, getting the most out of these resources requires a strategic allocation across knowledge activities. These findings also highlight the importance of ensuring agreement with external funding partners on strategic priorities aligned with country needs, given the prominent role of external funds in leveraging limited BB resources. More generally, there is a need to avoid fragmentation and ensure that individual knowledge activities are adequately funded. 16 3.1.2 ENSURING RELEVANCE AND IMPACT THROUGH CLIENT ENGAGEMENT AND STRATEGIC OUTREACH Ultimately, the goal of WBG knowledge work is to inform solutions to development challenges in client countries and provide ahead-of-the-curve thinking on global development topics. Ensuring that knowledge meets client needs requires a broad vision of the global debate and country agenda at both the country and sectoral level as well as prioritization. A promising approach at the country level pioneered in Indonesia, Afghanistan, and several East Asian countries is the “Engagement Approach,” which organizes country programs around multiyear engagements focused on a limited set of development outcomes, selects tasks that help meet them and reviews progress yearly (see Annex 1, Box 7). From here the prioritization of knowledge products and financing becomes clear, as country activities and operations are aligned with the set of development outcomes, increasing synergies and avoiding supply-driven activities. At the global level, the WBG’s research on today’s and tomorrow’s development issues remains critical to maintaining its global credibility and power to convene. The engagement with universities and think tanks plays a critical role in informing research and in its strategic outreach. Strategic outreach is key to ensure the greatest impact. This concept is focused on achieving the desired result, which could range from policy change, informing debate, pursuing advocacy, or generating results and impact on the ground. Impactful products tend to allocate more upstream effort to stakeholder engagement and strategic outreach. As such, TTLs need to have a clear idea of who the audience is, what is the desired result, and how to best reach it at the inception of an activity. As noted by clients, many times the government is aware of what the problems are, but where they need most help is in identifying actionable recommendations and knowing how to implement them. Setting a clear outreach strategy from the outset and taking the extra step from identifying “what” are the problems to “how to” implement the recommendations to solve them would enhance the impact of WBG’s knowledge work (IEG 2013). A key ingredient is ensuring that ECR resources are allocated to the knowledge efforts with the highest strategic priority and that material is tailored to target audiences. ECR promotes major World Bank flagship reports as well as knowledge coming out of the PGs, DEC, Regions and country teams. Outreach and communications strategies for individual products are tailored to the business objectives and intended audience leveraging events, media, social media, and our own platforms. For IFC, flagship publications are coordinated with the Communications and Partnerships VPU following an agreed communications plan leveraging social media, conferences, and other platforms. A more strategic approach to partnerships and global convening initiatives may also be needed to ensure greater impact. As noted by IEG, the WBG is likely to be more effective when “It engages with clear objectives, buy-in and engagement of senior champions; 17 embeds the topic in select country programs; and sustains efforts over time” (2020:xii). A greater involvement of IFC was identified as an opportunity: “[t]here is an increased demand for the convening role of the International Finance Corporation (IFC), especially on how to scale up private sector finance for development.” (2020:x). Increased focus on cooperation and partnerships with governments, civil society, and the private sector can boost the breadth, depth, and efffectiveness of WBG knowledge work. Through strategic outreach, the WBG will need to deepen its efforts to boost capacity for research and analysis across the client income spectrum. A basic goal of the WBG is to help generate client capacity, and the capacity for knowledge generation varies significantly by income level. Examples of existing support include the African Centers for Excellence and the Knowledge Partnership in Malaysia. Under the Strategic Framework, it will be important to expand that support. 3.1.3 HUMAN CAPITAL AND INCENTIVES FOR KNOWLEDGE WORK The sine qua non of knowledge work is the underlying human capital of the WBG staff members who engage in this work. This is true not just for formal written documents and reports, but also for the more “tacit” products like nuts-and-bolts TA where product quality is hard to measure. One piece of evidence pointing to the importance of this tacit knowledge is research showing that the expertise of the TTL is a key determinant of a project meeting its development objectives – as important as conditions in the country in which the project is implemented.21 Human capital for knowledge matters for all WBG staff. WBG staff are not and should not be divided into “knowledge” people and “financing” people. Operations-focused TTLs require high levels of human capital to keep their projects at the knowledge frontier and to distill lessons from them. Similarly, close interactions with country programs keep staff working on knowledge products focused on the relevant issues needing solutions. Raising the quality of knowledge products over time will require raising the level of human capital. This requires ensuring that the WBG has the right skill mix to staff each of the tasks described in Figures 3 and 4, ranging from basic skills in the relevant disciplines, to cutting-edge skills for data analysis and inference, to topics of emerging importance such as climate change or digital technologies. Annex 1, Box 10 highlights examples of such innovative data work at the Bank as a way of illustrating the high returns to investments in this aspect of the WBG’s human capital. Many tasks require specialized advanced skills that need to be maintained over time. As trade and innovation theory predict, the “gains from trade” among specialists and the existence of knowledge spillovers imply that the WBG can become more than the sum of its parts. These 21. Denizer, Kaufmann and Kraay (2013), Bulman, Kolkma and Kraay (2018). 18 benefits of specialization need to be balanced against the emphasis on versatility and fungibility of staff often stressed at the WBG. Wherever this balance is found, it points to the importance of ensuring that knowledge flows well across the institution, irrespective of the specific units to which individuals are assigned at a given point in time (see Section 3.2). The WBG faces challenges in how it recruits and nurtures new talent. This has occurred as the proliferation of new agencies and players in the marketplace for development policy analysis noted earlier makes the job market for development professionals increasingly competitive. For example, anecdotal evidence suggests that there may be diminishing interest in careers at the Bank due to academic salaries rising steadily, as well as due to increased opportunities to pursue careers in development in other organizations. In contrast, the shift towards term contracts and less generous benefits than some competitors may have made the Bank less attractive. Appropriate incentives for maintaining human capital for knowledge work are also required. Previous studies have noted that “[t]here is a perception among Bank staff that lending experience is essential for promotion. At the same time, the world recognizes that knowledge is the foundation of development, and that the Bank should do more in this regard. The incentives for staff to engage in knowledge activities need to be as strong as those for being part of lending operations.”22 IEG (2018) reports that staff working groups in 2012/13 developed reform proposals on incentives for collaboration and knowledge, but there has been limited progress on creating rewards, recognition and accountability for this type of work. These challenges are not confined to certain parts of the organizational structure. Since knowledge should permeate all the WBG’s activities, these incentives are a consideration as much for the PG lead economist as for the project TTL or for the frontline IFC or World Bank specialist delivering nuts and bolts policy advice. 3.2 IMPROVING KNOWLEDGE FLOWS AND COLLABORATION ACROSS THE WBG WBG organizational changes over the last decade underscore the importance of strengthening knowledge flows and collaboration across the institution.23 Three features are particularly important to highlight. First, knowledge is both produced and consumed throughout the WBG’s organizational structure – there are no units that specialize entirely in the one or the other. Therefore, mechanisms for knowledge flows do not simply require staff in certain units to teach those in others – rather they need to facilitate a complex web of knowledge exchange across all functional units in the WBG. While knowledge management, narrowly construed, is an important ingredient in the Strategic Framework, managing the generation and flow of knowledge is the task of all WBG staff and cannot be solely dedicated 22. IEG (2013) 23. See IEG (2019) for further analysis. 19 to a single unit. Second, high-quality knowledge work to meet the needs of increasingly sophisticated clients requires specialized skills from a range of disciplines. Impressive as many WBG staff are, all necessary skills for a particular activity are unlikely to be found in individual “renaissance men and women”. Rather, they require active collaboration and knowledge sharing across large teams of specialized staff. Third, an important body of tacit knowledge at the WBG exists in individual staff members’ heads. Accessing this knowledge requires collaboration, combined with innovative ways to extract this knowledge into codified forms that are more easily accessible to others (see Annex 1, Box 11). As an example of active cross-institutional knowledge flow, MIGA is a major consumer of WB knowledge products. MIGA relies on WB specialists in both macroeconomics and various other specialized fields. For instance, depending upon the project, a biodiversity specialist may be consulted during MIGA’s project underwriting to augment MIGA’s understanding of specialist issues so that MIGA’s environmental and social performance standards (which are harmonized with IFC (who developed the standards) and similar to that of the WB) are met. Similarly, IFC leverages knowledge developed across the WBG, for example through Country Private Sector Diagnostics (CPSDs) and Development Policy Operations (DPOs), allowing for improved decisions on where and how IFC can best support the development of the private sector. Realizing the synergies arising from the interaction of different types of knowledge work requires knowledge to flow efficiently across the WBG and beyond (see Annex 1, Box 12). Ideally, the design of an operation is influenced by frontier thinking on the topic, tailored to the country context. There is evidence that the stock of prior analytic work by the Bank on a recipient country is correlated with the subsequent quality of its financing operations to that country (Deininger, Squire and Basu, 1998; Wane, 2004) and that the quality of prior analytic work matters to the quality of its projects (Fardoust and Flanagan, 2011). In turn, careful evaluation of the impact of the project generates knowledge to inform subsequent work, as discussed in Section 3.3. It is possible a TTL could move on to another project or product as soon as one is completed, missing the opportunity to share knowledge with other parts of the WBG and to repurpose or repackage material and get the most impact from the knowledge work performed, which speaks to the issue of incentives. Despite their importance, lessons from prior analytical work do not always fully inform new knowledge and financing activities. This phenomenon indicates bottlenecks to knowledge flows and has been referred to by the WBG archivists as “institutional amnesia.” For example, some new ASA products begin without a thorough review of the literature and prior WBG knowledge work on the topic. The same is true for some projects. An informal and preliminary review for this paper suggests a high variance in the degree to which the analysis in Project Appraisal Documents (PADs) is formally supported through references to current literature – whether WBG-generated or otherwise.24 24. The review examined the first 3 non-dropped projects approved in FY19 in 3 PGs, finding that none had a complete reference list and the number of references to analytical work varied widely and included zero. 20 3.2.1 INSTITUTIONAL INCENTIVES FOR KNOWLEDGE FLOWS Knowledge flows reflect both the demand and supply of knowledge. On the supply side, generating the needed knowledge points to the need to prioritize resources to critical areas and to ensure relevance, as discussed in the previous section. On the demand side, strong incentives to ensure products are at the frontier are needed to create demand for knowledge. Evidence in Ravallion (2013) indicates that this demand for knowledge is greater the stronger is the human capital of staff, which enables them to be sophisticated consumers of knowledge, and that the demand for knowledge is uneven across the WBG.25 Institutional incentives to share knowledge matter. McKinsey is an example of a consultancy company with strong institutional incentives to share knowledge. The head of a team after project completion will summarize the main takeaways from the project and will circulate them to increase chances of being selected team head again. To some extent, similar incentives exist within the WBG. For example, IFC offers KNovember, the annual flagship knowledge- sharing event that provides a unique platform for staff and teams to identify, create, and share critical knowledge on IFC’s strategic priorities and KNOWbel, a WBG-wide recognition program celebrating teams and individuals who leverage knowledge for better business results and embody knowledge sharing behavior. However, a recurring theme in the consultations for this report is that these incentives could be strengthened, as staff regularly report difficulty in accessing knowledge that already exists in the institution. Institutional incentives matter for knowledge flows outside the WBG as well. Institutional incentives should promote knowledge generation from interactions with clients, academic institutions, CSOs and development partners. The WBG plays an important role in facilitating fruitful South-South exchanges, which are highly appreciated by clients. The provision of policy advice and of technical assistance tends to have a more lasting effect when there is a strong emphasis on capacity building for the client and when partnerships are established with local universities, think tanks and NGOs. Such interactions increase teams’ awareness of the local context and facilitate the generation of local knowledge. Fostering home-grown and home-owned knowledge is critical for the success of any policy reform. For example, the Chief Economists of Governments’ Network (CEoG), originated in the Africa Chief Economist’s Office, and now expanding to the ECA and EAP Regions, provides a platform for bringing together economic advisors to strengthen their skills, knowledge, and network for more harmonized and effective data-driven economic advice. The CEoG has grown to include 40 chief economic advisors to African Presidents and Prime Ministers across 40 countries in the region. 25. Ravallion (2013) analyzed a survey of staff regarding their familiarity with and use of Bank research. Roughly 50 percent reported that they were well-informed about Bank research, either because of the needs of their particular projects or out of general interest. However, 24 percent found they wanted to know more but found it difficult to do so because of difficulties in identifying and accessing knowledge. Surprisingly, 23 percent reported they did not believe their projects required more knowledge. In some sectors, like Poverty, this phenomenon was extremely low. In others, the number was close to 40 percent. While difficult to isolate separately, the responses suggest that more could be done to ensure both demand for knowledge where it is needed to inform operational decisions, as well as to improve the supply of knowledge to ensure its relevance to those same decisions. 21 3.2.2 INFRASTRUCTURE FOR KNOWLEDGE FLOWS Strengthening the information infrastructure to support knowledge exchange: The WBG has made substantial efforts over the last few years to reduce fragmentation of information stored and to introduce modern, cloud-based information management solutions. Close to 700 million files, 1,200 websites, and 3,500 collaborative spaces are now technically managed on a consolidated cloud platform. Efforts across IFC include better intranet design and disciplined content management practices to facilitate internal knowledge dissemination and collaboration. At the World Bank, intranet governance has been introduced and a dedicated team of information professionals works with staff to ensure the digital workplace is transparent, organized and relevant. At the same time, the WBG is also investing in better data governance, which will make large amounts of high-quality data easily accessible to staff for incorporation into knowledge work. Consolidation has led to efficiencies, but the sheer scale of available information highlights the urgent need for better mechanisms for staff to access and use this information as part of knowledge generation and sharing functions. Part of the current difficulty of retrieving information is not technological but related to the heterogeneity of processes across units that leave products without consistent metadata to make the knowledge easily identifiable. Better practices to identify and tag key documents and introduction of innovative new technologies, including machine learning and artificial intelligence, will help staff access and reuse institutional knowledge, as is the standard in many consulting firms. IFC is using advanced text analytics tools to develop an auto-tagger that automatically classifies and tags content with topics, sectors, and keywords allowing IFC internal clients to reduce time in content cleanup efforts, propose new taxonomy terms, and efficiently tag content for the new intranet platform. However, even some basic elements of knowledge curation are still lacking across the WBG. For example, OPCS/Rama (2020) finds that a significant number of ASA outputs cannot be found in any of the three main institutional repositories, and only a small fraction appears in all three. For long term knowledge success, technology and incentives must work together to ensure that staff are able to make their knowledge easily accessible to others across the WBG and beyond. 3.3 IMPROVING THE WBG’S CAPACITY TO LEARN FROM ITS OPERATIONS Given that the WBG’s core business is projects, rigorous evaluation of their impacts is essential to learn from current and past projects in order to inform the design and implementation of subsequent projects.26 Learning from operations requires systematic 26. The WBG Evaluation Principles state “Knowledge management systems should ensure that knowledge from evaluations becomes a key part of the WBG’s operational workflow.” This principle means that incentives and structures should exist to facilitate operational staff and management using evidence from independent evaluations, impact evaluations, and self-evaluations to inform design and decisions. 22 monitoring and evaluation frameworks that yield robust empirical conclusions about the impact of the projects and programs it finances. The WBG relies on several institutional processes for this purpose, most notably the Implementation Completion Reports (ICRs) for projects and their subsequent validations by IEG. In addition to Project Completion Reports and Extended Project Supervision Reports, IFC generates After Action Reviews (AARs), SmartLessons, and EMCompass notes (see Annex 1, Box 4) to capture lessons from projects. However, past assessments have suggested that the WBG could enhance its focus on and attention to learning lessons from its operations and then repackage and replicate them for reuse.27 Formal impact evaluation seeks to identify rigorous measures of impact relative to well- defined counterfactuals. Carefully designed impact evaluations implemented “in real time” in parallel with the project itself can raise project quality by informing evidence-based mid-course corrections, and fill knowledge gaps that then benefit subsequent projects. As an example, in the early days of developing conditional cash transfer (CCT) programs, numerous studies showed that CCTs to families would result in more students in school, yet additional studies were needed to understand why, once there, they learned little. Formal impact evaluations can deploy a variety of tools and are not confined to randomized controlled trials (RCTs) which are suitable for some impact evaluations but not others. Examples of other techniques include model-based counterfactual analysis, as well as synthetic control techniques used to evaluate the effects of IFC-supported reforms to the OHADA Uniform Act on Secured Transactions (See Annex 1, Box 13). Currently, many impact evaluations are carried out in Bank projects, often with leadership from DEC researchers or the regional Gender Innovation Labs. These frequently are implemented as “knowledge consortia” that partner with external academics, Bank researchers, client government officials and incentivized project TTLs, and supported by third party financing. However, fewer than 5 percent of Bank projects since 2010 have been subjected to such formal impact evaluation methods.28 While there is no easy way of concluding what the appropriate share should be, the current share appears low by any standard. There are a variety of reasons why many operations do not incorporate formal impact evaluations. Knowledge created through impact evaluations is a key public good, since it can inform the design of many subsequent lending operations and policy interventions – but individual countries may not be willing to bear the private cost of funding the evaluation. This points to an important role for the WBG to internalize this externality by mobilizing financing for such key knowledge products. In other cases, such as large country-wide macro reforms, creating a counterfactual or control group is not possible. This does not make measuring impact less important -- only more challenging. Nevertheless, the feasibility and usefulness of rigorous impact evaluation can be more systematically addressed in project preparation in order to ensure that the fully Bank exploits its unique comparative advantage to learn from the operations it finances. 27. See IEG (2016b) and IEG (2013). 28. This statistic refers to ASA activities identified as “impact evaluations” that are linked to operational projects. Some projects may have impact evaluations that are not recorded as ASA activities and therefore will not be counted here. 23 Evaluations also help codify quasi-tacit knowledge. In some cases, “tacit” knowledge remains in the TTLs heads not because it is truly un-codifiable, but because there are insufficient incentives to document lessons. Annex 1, Box 11 shows that even less structured focus group methods can be useful to extract tacit knowledge from TTLs. Raising the share of financing operations with in-built formal evaluations may also help to codify and empirically ground much tacit learning. The goals of improving the WBG’s ability to learn from its operations and of ensuring its role as a leader in development innovation link tightly back to the issue of knowledge quality and impact and improving knowledge flows. The more emphasis the WBG places on deploying evaluations to learn from our financing activities the more this creates demand for high quality analytical work, sharpens incentives, and fosters the efficient use of resources. In addition, it creates demand to develop and maintain the human capital of the staff, without which there can be neither thoughtful projects nor evaluation, and to know who in the organization is doing good analytical work in related areas. 24 4 THE AGENDA AHEAD The WBG sees the Strategic Framework as providing the framing to realize its vision to deliver the best knowledge for development solutions provided to client countries and for the global development agenda. This section sketches out the agenda to address the challenges described above. It identifies the broad principles to inform and organize Management’s work ahead. The work centers on three pillars set out in Figure 7. Over the coming years, Management will use these to guide the development of the necessary implementation actions. FIGURE 7 REALIZING THE WBG’S FULL KNOWLEDGE POTENTIAL Increase relevance, quality, and impact of knowledge products Improve knowledge flows and collaboration across the WBG Realize WBG’s potential to learn from its operations and ensure role as a leader in development innovation PILLAR 1 PILLAR 2 PILLAR 3 Strengthen systems Strengthen incentives Strengthen human capital • Strategic prioritization • Management • Selection • Learning from projects attention • Career paths • Quality assurance • Career incentives • Training • Dissemination • Incentives for and outreach connections across • Knowledge WBG and with management partners Cross-cutting 1: Improve outcome orientation and performance metrics Cross-cutting 2: Improve ICT and platforms 4.1 PILLAR 1: STRENGTHENING SYSTEMS TO ENSURE RELEVANCE, QUALITY AND IMPACT Strategic prioritization is critical. Ensuring even greater impact of knowledge in WBG support will require strengthening systems for coordinating and prioritizing at three levels. At the departmental level, Management will explore mechanisms to strengthen coordination among DEC, Chief Economist Offices, the PGs and the Regions to agree on priorities and anticipate future needs. At the country level, strengthening prioritization implies enhancing the attention to knowledge work in the CPF preparation, implementation, review, and evaluation processes. Lessons from initiatives like the Engagement Approach can help strengthen these processes by better identification of synergies across activities and the identification of opportunities to sharpen selectivity. Management will test out mechanisms for more frequent stocktaking of progress in meeting the CPF objectives. The recent move of Practice Group Directors to the Regions will help in sorting out cross-sectoral priorities at the country level. Third, Management efforts will continue to further clarify and strengthen resource allocation procedures related to knowledge work. Steps include ensuring that foundational knowledge work is appropriately prioritized, notably continuing the recent progress in prioritizing funding of core knowledge analytics, and that trust fund resources go to high quality activities aligned with WBG priorities and country programs. An important step toward that goal was the adoption by the Bank in FY18 of a requirement that for all countries, each core ASA product should be prepared at least once every five years. Work towards improving learning from projects is a Management priority. To start, Management will explore three avenues. The first is ensuring that learning from projects is adequately pursued and documented in ICRs. The second is potential actions to improve the mining of existing self-evaluations and independent evaluations of WBG operations, as well operations of other development actors, as sources of operation insights and ensuring that the information gets to staff and clients who need it. The third is exploring the opportunities for implementing more, more rigorous, and more strategic impact evaluations to fill in knowledge gaps on important development topics, supported by adequate financing given its public good nature. Strengthening quality assurance is also a priority. The recent PG realignments provide an opportunity to strengthen the quality assurance for research and ASA products. WBG management will explore how to best ensure quality across a wide range of knowledge products, each of which may require tailored quality assurance processes. One approach is strengthening the tiering into broad categories of knowledge products, assigning appropriate levels of authority and quality control processes for the products in each tier. This is used by IFC and is under consideration by the Bank. At the product level, stronger management of quality-at-entry at the Concept Note stage can ensure projects systematically build on previous literature and have strong linkages with outside work. A key ingredient is “the will to kill” 26 potentially weak and low-priority knowledge products, ideally as early as possible in their life cycle, in order to free up resources for higher-return activities. The WBG will develop ways to strengthen its approach to strategic outreach and engagement in analytical work. Improved articulation of better focused outreach strategies will increase the impact of analytical work. First steps will include the exploration of how best to identify client demand at the outset, set agreed prospective outcome indicators-at-entry, and explore the scope for using new technologies for reaching appropriate audiences and measuring outcomes. It is a priority to appropriately incentivize TTLs to make upstream efforts to focus on stakeholder engagement and strategic outreach. Management will revisit the knowledge management system, broadly construed. The goal is to strengthen the institution’s capacity to efficiently create, vet, curate, and disseminate knowledge, including its 70 years of accumulated lessons. The first step is getting basic things right: ensuring that all knowledge products are properly entered and cataloged in the relevant systems with appropriate meta-data to enable effective searches. More fundamentally, the WBG will work toward forming closer partnerships among development practitioners, information managers, and technologists to align incentives and ensure the necessary mechanisms are in place for easy access to well-curated development knowledge. 4.2 PILLAR 2: STRENGTHENING INCENTIVES FOR KNOWLEDGE WORK AND COLLABORATION Staff incentives will be strengthened through management attention. No incentives created by high-level directives will change staff behavior if the de facto objectives of their managers are different. Clearly verifying the importance of quality knowledge for achieving the WBG’s mission of promoting poverty reduction and shared and sustainable growth requires strong signals from the top, and Senior Management has already signaled its commitment. Research has consistently found that line managers are the key to the successful introduction of a new approach. This Strategic Framework envisions a two-pronged method in ensuring buy-in. The strong signal from Senior Management is one component. The second is accountability. Management will explore ways to strengthen accountability mechanisms at the managerial level for quality knowledge work, building on accountability mechanisms for operations. Management will also address how best to align staff career incentives with the knowledge agenda. As IEG (2013) notes “At a minimum the Bank should review the incentive system for staff, ensure that knowledge contributions are recognized for career advancement; make bringing knowledge to countries a visible priority; and ensure that personal reputations of staff are enhanced by knowledge contributions. This evaluation suggests the need to deploy part of the time of the best staff in the institution to knowledge services.” To this end, three types of incentives will be reviewed for action: the process of setting expectations 27 for performance and promotion, improving the recognition of quality knowledge work as part of the staff evaluation process, and exploring possible work program incentives to engage in knowledge activities. Management will explore incentives for stronger connections across WBG units and with external knowledge partners. Several units in the WBG facilitate knowledge flows inside and outside the institution, and these connections will be strengthened. For instance, DEC and the PGs provide the public good of vetting, curating, and disseminating new academic knowledge for the benefit of TTLs across the institution, while Chief Economist offices offer a bridge between academia, DEC and the regions. These units also partner with local institutions in the co-creation and global diffusion of knowledge. IFC has consolidated its economic analytical work and thematic groups for climate, inclusion and FCS in CEDVP, which serves as a bridge between IFC operations, DEC and the PGs - as well as academia, consulting firms and think tanks. Greater collaboration on joint products like the CPSDs or the Healthcare System Quality Assessment Tool (see Annex 1, Boxes 14 and 15) will also facilitate knowledge flows. 4.3 PILLAR 3: STRENGTHENING HUMAN CAPITAL Appropriate staff selection is key to long-term implementation of the Strategic Framework for Knowledge. Achieving the goals of ensuring staff quality and diversity requires reviewing how recruiting modalities can best identify and attract the appropriate talent. Strengthening the Young Professionals Program and reviewing other intake channels are steps in this direction. Better identifying career paths that are intensive in knowledge work will be explored. These recognized career paths would help counteract the perception that work solely on operational financing is the preferred option to advance in the WBG. Further, as tomorrow’s knowledge leaders are today’s TTLs, a careful monitoring and mentoring of the pipeline will help ensure they grow into this role. IFC has created competencies and career frameworks for economists and strategists, who are key knowledge creators and brokers. These allows a clear alternative to the investment and advisory operational streams for professional development. Training and skill upgrading options will be explored as part of the introduction of the Strategic Framework. Management will review the systems through which staff maintain and acquire new skills, including emphasis on basic WBG “knowledge skills” that translate knowledge into policy advice and operations.29 “Learning by doing” provides highly effective training. Management will explore options such as dedicated staff time devoted to documenting and disseminating knowledge created through operational experiences, supporting not only training but knowledge gathering and sharing goals. The greater emphasis on impact evaluations noted 29. For example, consider training for new staff on how to write policy briefs, how to prepare PowerPoint presentations, how to engage clients to do reforms, as well as working in multidisciplinary streams. Develop a library of case studies of both successful and failed projects to support technical onboarding of operational staff by senior experts. Introduce shadowing of senior “experts” as part of the onboarding of junior operational staff. 28 above also supports the learning-by-doing agenda, bringing operational TTLs and researchers together to implement these evaluations. 4.4 CROSS-CUTTING ISSUES Actions in the three above pillars will be supported by actions on two cross-cutting themes: improving outcome orientation (with a strong emphasis on better metrics of results and performance), and improvements in ICT and technology platforms. Improving outcome orientation and performance metrics is a priority. The WBG’s success relies on its ability to help its clients overcome challenges to achieve the desired development outcomes (influenced by both financing and knowledge), and the institutional shift toward strengthened outcome orientation is vital to this mission. Another recurrent theme in the review was the need to improve measurement of the quality and impact of knowledge products and staff capabilities relative to needs. Advances on these fronts are essential to benchmarking progress as this framework is implemented. However, it is important to recognize that the measurement challenges – particularly for quality and impact – are significant, and that it is unlikely that progress will be measured using a few simple quantitative indicators. Improving ICT and technology platforms is also important for implementing the Strategic Framework. Enhanced opportunities exist for modernizing and digitizing tools to ensure knowledge flows, which feed into the three pillars above. For instance, advances in digital technologies, text analytics, machine learning, and artificial intelligence capabilities, coupled with improvements in knowledge management practices, can help mitigate the long-standing difficulty of finding a WBG document using the WBG’s search engine (see Annex 1, Box 16). New digital modalities of internal and external dissemination offer the potential to increase the impact of knowledge products. The range of training options for staff are enriched by digital offerings.   29 5 CONCLUSION This Strategic Framework for Knowledge presents a vision for the WBG to strengthen its role as a Solutions WBG. The goal is to further advance the WBG’s ability to combine financing with knowledge to generate development solutions in a timely, contextualized and integrated way, and to strengthen the WBG position as the reference of choice in the global development debate. The WBG starts from a strong basis, as the currently acknowledged reference of choice for development knowledge. However, it faces challenges from rapidly increasing demands for knowledge-based solutions to address and recover from COVID-19, taking better advantage of a rapidly-evolving landscape with many new providers of development knowledge, exploiting opportunities from advances in development research to use operations to generate knowledge, and exploring new technologies in the context of development knowledge creation, curation, and dissemination. Our goal is to move from being impactful most of the time to being impactful all the time. The framework is informed by extensive internal and external consultations on the question of how to bolster the knowledge performance of the WBG and by past evaluations of issues around the WBG’s knowledge agenda. That body of work highlighted three key challenges to enhancing the WBG’s role as a leader in development innovation: (1) enhancing the relevance, quality and impact of WBG analytical work and advice; (2) improving knowledge flows and collaboration across the WBG; and (3) fully realizing the WBG’s potential to learn from operations. It sets out a Strategic Framework centered on three pillars. These are (1) strengthening systems for greater prioritization, evaluation, quality control and effective strategic outreach; (2) enhancing incentives at all levels of management and staff for enhanced emphasis on knowledge; and (3) building human capital through recruitment, career path improvements, and strengthened training. Actions in these areas will be supported by the exploration of better metrics of performance, as well as the exploitation of new technologies and knowledge platforms. This document presents an approach to frame how Management may increase the development impact of the WBG’s knowledge. It is not a timebound strategy for implementation with milestones and deliverables. Knowledge is part and parcel of operations and exists in everything the WBG does. Efforts to increase the impact of knowledge should be continuous and adaptive as the environment and conditions in which the WBG operates evolve. The Framework creates links between existing and ongoing efforts of Management (i.e. Trust Fund Reform, Career Mobility and Development Framework, ITS-led upgrades) and brings clearer focus to additional areas for considered actions. Taken together, the sum of the efforts should lead to an even more impactful WBG. The next step is for Management to develop and implement institution-specific actions necessary to realize the vision of the Strategic Framework. The WB and the IFC management will each develop agendas to ensure progress towards the WBG’s knowledge objectives as outlined in the Strategic Framework. MIGA, which as noted above, benefits from Bank and IFC knowledge inputs and cooperation, will contribute to the process. The World Bank’s roll-out will be structured along the pillars outlined in the Framework. Working within the existing structure and with inputs from stakeholder consultations, Management will ensure that short-run and medium-run actions are interlinked, and synergies maximized, as well as monitor its implementation. Management will also seek to build on initiatives that have recently been implemented or are in the process of being implemented, for example trust fund reforms. IFC will develop and adopt an Implementation Plan as soon as its Knowledge Management Strategy (see Annex 3) is approved by the IFC Management Team. This Plan will enable and support the ability of IFC staff, operating in locations across the world and across regions and industries, to consistently capture, curate and reuse knowledge related to its operations, from engagements across the WBG and from other partnerships. Successful implementation of the Plan will allow IFC staff and clients to benefit from the lessons learned and the expertise accrued from these engagements, enhancing both the development impact and long-term financial sustainability of IFC’s underlying portfolios. In addition, Management will inform the Board on the implementation of the Strategic Framework for Knowledge through regularly scheduled updates, including the Quarterly Operations Updates and the Practice Group and Regional Updates, all of which cover analytics, financing, and report on the Bank’s key strategic operational initiatives.   31 RE F E RE N C E S Alavi, M. and D. E. Leidner (2001). “Review: Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues.” MIS Quarterly 25(1) 107- 136. https://www.jstor.org/stable/3250961?seq=1 Bulman, D. J., W. Kolkma and A. Kraay (2018). "Good countries or good projects? comparing macro and micro correlates of World Bank and Asian Development Bank project performance," The Review of International Organizations. https://link.springer.com/article/10.1007/s11558-016- 9256-x Chioda, L., A. de la Torre, and W. Maloney (2014). “Toward a Conceptual Framework for the Knowledge Bank,” World Bank Policy Research Working Paper No. 6623. https://elibrary. worldbank.org/doi/abs/10.1596/1813-9450-6623 Denizer, C., D. Kaufmann, and A. Kraay (2013). "Good countries or good projects? Macro and micro correlates of World Bank project performance," Journal of Development Economics, Elsevier, vol. 105(C), p. 288-302. http://www.sciencedirect.com/science/article/pii/S0304387813000874 Devarajan et al (2018) “A Knowledge Strategy for the World Bank.” Mimeo. Duranton, G. and A. Venables (2018). “Place-Based Policies for Development.” Policy Research Working Paper No. 8410. World Bank, Washington, DC. https://doi.org/10.1596/1813-9450-8410 External Client Satisfaction Survey (2020). Data shared with SFK Working Group by OPSRS. Grant, R. M. (1996) “Toward a Knowledge-Based Theory of the Firm.” Strategic Management Journal 17(S2):109-122. https://doi.org/10.1002/smj.4250171110 Gupta, A. K, and V. Govindarajan (2000). “Knowledge Flows within Multinational Corporations.” Strategic Management Journal 21:473-496. https://www.jstor.org/stable/3094239 Hussain, Kenyon, and Friedman. (2018). “A New Look at Factors Driving Investment Project Performance.” World Bank Policy Research Talk. http://pubdocs.worldbank.org/ en/800781538501913762/Policy-Research-Talk-Friedman-InvestmentProjectPerformance.pdf Independent Evaluation Group (IEG) (2013). "Biennial Report on Operations Evaluation: Assessing the Monitoring and Evaluation Systems of IFC and MIGA (2013)." World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/15793 Independent Evaluation Group (IEG) (2013). “Learning and Results in WB Operations: towards a new Learning Strategy II (IEG 2015). https://ieg.worldbankgroup.org/evaluations/learning-results- wb-operations2 Independent Evaluation Group (IEG) (2016a). “Knowledge-Based Country Programs. An Evaluation of World Bank Group Experience, 2005–11.” World Bank, Washington, DC. https:// openknowledge.worldbank.org/handle/10986/24704 Independent Evaluation Group (IEG) (2016b). “Behind the Mirror: A Report on the Self-Evaluation 32 Systems of the World Bank Group.” World Bank, Washington, DC. https://openknowledge. worldbank.org/handle/10986/24956 Independent Evaluation Group (IEG) (2019). “Knowledge Flow and Collaboration Under the World Bank’s New Operating Model: An Independent Evaluation.” World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/32077 Independent Evaluation Group (IEG) (2019). “The World Bank: An Evolution of the WBG’s Convening Power.” World Bank, Washington, DC. https://ieg.worldbankgroup.org/sites/default/ files/Data/Evaluation/files/WBG_Convening_Power.pdf Independent Evaluation Group (IEG) (2020a). “The World's Bank: An Evaluation of the World Bank Group’s Global Convening.” World Bank, Washington, DC. https://openknowledge.worldbank.org/ handle/10986/33536 Independent Evaluation Group (IEG) (2020b). World Bank Group Outcome Orientation at the Country Level. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/WBG_ OutcomeOrientation.pdf Knack, S.; B. C. Parks; A. Harutyunyan; M. DiLorenzo (2020). “How Does the World Bank Influence the Development Policy Priorities of Low-Income and Lower-Middle-Income Countries?” Policy Research Working Paper; No. 9225. World Bank, Washington, DC. https://openknowledge. worldbank.org/handle/10986/33707 Knowledge Strategy Group (2010). Transforming the Bank’s Knowledge Agenda: A Framework for Action. World Bank, Washington DC. https://hubs.worldbank.org/docs/imagebank/Pages/ docProfile.aspx?nodeid=11768259 Portfolio Management Task Force (the Wapenhans Report) Effective Implementation: Key to Development Impact. September 1992. http://documents1.worldbank.org/curated/ en/793701492973438535/pdf/Effective-implementation-key-to-development-impact-working- papers.pdf Pritchet, Lant (2019). Randomizing Development: Method or Madness? Oxford Blavatnik School of Government. https://d101vc9winf8ln.cloudfront.net/documents/32264/original/RCTs_and_the_ big_questions_10000words_june30.pdf?1565974982= Rama, M. (2020). “Strengthening the management of knowledge products at the World Bank,” World Bank, Washington DC, forthcoming. Ravallion, M. (2013). “Knowledgeable bankers? The demand for research in World Bank operations.” Journal of Development Effectiveness, 5(1):1-19. https://doi.org/10.1080/19439342.2013 .763283 Ravallion, M. (2020). “Should the Randomistas (Continue to) Rule?” National Bureau of Economic Research Working Paper 27554. https://doi.org/10.3386/w27554 Strategic Framework for Knowledge Working Group (2020). “Three GP Pilot review of Project/ Program Documents.” Preliminary evidence collected for this paper. Tang, H. (2008). “Using knowledge to improve development effectiveness: an evaluation of World 33 Bank economic and sector work and technical assistance, 2000-2006.” Washington, DC: World Bank. http://hubs.worldbank.org/docs/imagebank/Pages/docProfile.aspx?nodeid=10587438 World Bank (2009). World Development Report 2009: Reshaping Economic Geography. World Bank, Washington DC. https://openknowledge.worldbank.org/handle/10986/5991 World Bank (2011). “The state of World Bank knowledge services: knowledge for development 2011.” Washington, DC: World Bank. https://hubs.worldbank.org/docs/imagebank/Pages/ docProfile.aspx?nodeid=15560820 World Bank Group (2020). “COVID-19 Crisis Response Approach Paper.” Washington, DC: World Bank. http://documents1.worldbank.org/curated/en/136631594937150795/pdf/World-Bank-Group- COVID-19-Crisis-Response-Approach-Paper-Saving-Lives-Scaling-up-Impact-and-Getting-Back- on-Track.pdf 34 A N N EX 1 : EXAMPL E BOXE S BOX 1 BUILDING A COALITION OF SUPPORT FOR HUMAN CAPITAL The Human Capital Project including the launch of the Human Capital Project is a case study in how communications and stakeholder engagement can help drive the global development agenda and country outcomes. The Bank launched the Human Capital Project (HCP) in 2018 as an initiative to mobilize global support, knowledge, and financing of investments in human capital. It deployed a new cross- country metric anchored in research – the Human Capital Index – to increase the salience for policymakers of investments in the health and education of the young. It also drew on a wide body of operational tacit knowledge to develop a “whole of government” approach to human capital that has been embraced by 78 countries as of September 2020. An active communication strategy was central to transforming this knowledge into policy impact. From the outset the HCP team brought in ECR colleagues to outline a communications and stakeholder engagement strategy to place human capital at the center of the global development agenda. The objectives were to articulate a clear narrative about the importance of investing in people to labor productivity and economic growth; position the WBG as the global leader and a trusted partner for countries; build active support from influential stakeholders; and build political and financial support for development financing platforms (IBRD, IDA and the Global Financial Facility for Every Woman Every Child). The team spearheaded engagement with dozens of stakeholders, providing in-depth briefings to create a Human Capital champions’ coalition, including CSOs, Ministers, parliamentarians and faith-based organizations. This was supplemented by advocacy and public events, and social/media engagement. These efforts paid off as human capital policy issues were prioritized in the G7/G20, and recognition of the importance of this agenda contributed to robust replenishments for IDA, the GFF, and the IBRD capital increase. The Bank’s own human development lending skyrocketed by 70 percent in FY18. This effort also continued with a campaign around learning poverty targets, integrated messaging across sectors, and the continued attention to preserving and promoting human capital during the COVID-19 crisis. Return to text. 35 BOX 2 KNOWLEDGE IS AT THE CORE OF THE WBG’S 4-DECADE PARTNERSHIP WITH CHINA (1980-2020) From the start, knowledge was at the core of the China-WBG relationship and has remained so ever since. The first step in building the relationship, which started forty years ago, was a comprehensive country diagnostic—China: Socialist Economic Development. Only after its completion in 1981 did China access Bank financing. The first Bank project approved was for the modernization of China’s universities. The 1981 diagnostic was followed by a forward- looking assessment, China: Long Term Development Issues and Options (1985). The study was in direct response to a question of Deng Xiaoping to the WBG’s President on what it would take for China to quadruple agricultural and industrial output by 2000, a target which many considered to be implausibly ambitious, but was eventually achieved. More studies followed, including two comprehensive assessments of reform achievements and future challenges, China 2020 (1997) and China 2030 (2013). The latter is notable for being the first diagnostic to be prepared by a joint research team between the WBG and the Development Research Center of the State Council of China. Subsequent reports on Urban China (2014), Healthy China (2016) and Innovative China (2019) followed the same partnership model. The WBG also helped Chinese policy makers connect with international economists and learn from other countries’ experience. The knowledge activities formed the analytical base for subsequent lending operations. In Shanghai, a series of reports, starting from the Shanghai Sector Memorandum (1982) to Two Dragon Heads (2010), Shanghai 2050 (2018) and Metropolitan Governance report (2019) first guided investments in urban infrastructure and utilities and then to the creation of a municipal financing platform. Similarly, the Healthy China report formed the basis for the Health PforR operation in 2018. Lending has also fed back into knowledge work. The Bank’s long-standing engagement in the transport sector, for instance, has motivated knowledge notes on how China built its highspeed intercity railway system and modernized domestic waterway transport, disseminated to other countries through the TRANSFORM platform. The Bank’s experience has influenced Chinese practice in project planning and implementation well beyond the Bank financed portfolio, through new procurement and risk management practices building on the Bank’s framework. Three key factors underpin this fruitful knowledge partnership: (i) clear government demand, ownership, and strategic determination of the knowledge agenda; (ii) high standards of quality, and advice rooted in evidence and free of preconceptions, (which generated trust); and (iii) the ability to access leading experts both within the WBG and outside to buttress the credibility of the WBG’s work and bring additional perspectives. The knowledge partnership has also encountered challenges, such as the limitations on data access which persist to this day. Nonetheless, it is clear, that over the past 40 years, the WBG has been a partner of choice in exchanging ideas and experiences with Chinese policy makers. Return to text. 36 BOX 3 IFC’S KNOWLEDGE FOR PRIVATE SECTOR DEVELOPMENT For IFC, the creation and external dissemination of operational knowledge to its clients and partners is a key component of its thought leadership approach which consists of three complementary elements: (i) convening (ii) content generation and (iii) standard setting. IFC’s convening power allows it to bring together shared knowledge from market participants and to disseminate the content that it generates while also developing standards which gain wide market acceptance. The knowledge that IFC generates comes from combining three sources: • Insights and data generated from IFC’s operational activities, which gives IFC proprietary information from the clients, co-investors and other partners that it works with • Original research generated by IFC staff, World Bank Staff, or by joint IFC-WB teams, often using IFC and WB generated data (enterprise surveys, client surveys, etc) • Research and knowledge captured during IFC’s convening activities IFC’s knowledge generation is complementary to knowledge generated by the Bank, other international institutions, consulting firms, think tanks and academics. IFC combines the shared knowledge from these other generators of knowledge about the private sector in emerging markets. To this end, IFC has established a Private Sector Development Research Network (www.PSDresearchnetwork.com). IFC also generates a range of short and long research pieces which allows IFC staff to turn their knowledge into tangible knowledge publications. Regional departments produce reports linked to conferences focusing on issues specific to their regions. IFC also has a rich history of promoting market standards which mobilize capital for sustainable development. This includes the Equator Principles, which established common ESG standards for project finance lending from over 100 banks; the Green Bond Principles, developed with the International Capital Market Association, which underpin a Green and Social Bond market which has grown to nearly $1 trillion in assets over the past few years; the Blended Finance Principles, which bring discipline and transparency to the growing use of concessional finance by DFIs; and the Operating Principles for Impact Management, which have been adopted by over 100 fund managers and institutions covering over $300bn in assets managed for positive impact. As an example, the IFC PPP team initiated discussions to launch the WBG Scaling Solar program in Uzbekistan. The Scaling Solar framework is an open, competitive, transparent approach that facilitates the rapid development of privately owned, utility-scale solar PV projects, previously deployed in Africa. In May 2018, the Government of Uzbekistan entered into an advisory agreement with IFC to launch 1GW of solar PV auctions under the Scaling Solar framework, starting with the first phase 100MegaWatt project in Navoi. The tender resulted in a winning bid of US$2.679c/kWh, one of the lowest tariffs in emerging markets and much lower than previous electricity prices, making it more available to the poor while also attracting businesses that need electricity to the country. This project reached financial close with IFC financing a payment guarantee from the Bank. A second round of solar tenders has recently been launched, bringing in the lessons from earlier engagements. Return to text 37 BOX 4 THREE SUCCESSFUL PROGRAMS FOCUSED ON CAPTURING LESSONS LEARNED AT IFC After-Action Reviews are an internal knowledge management service designed to help teams learn from success and failures. The output from the lessons help to mainstream innovation within Operations. The team engages in short, facilitated interactions to evaluate key project aspects, identify what to do more of and what to change, and formulates recommendations for future, similar projects. This happens post-first disbursement, at exit or for cancelled projects. Bringing together project teams allows for the capture of lesson that are shared with relevant audiences to improve IFC’s collective learning and knowledge generation. The ability to leverage the results of these AARs allows teams to capture and synthesize the pros and cons of individual projects for the benefit of future decision making. SmartLessons is the IFC program awarding excellence in lesson capture. Aimed at internal and external audiences, the program encourages and incentivizes staff and others to share lessons learned in operations. The output is first-hand, practical lessons that reduce risk, promote validated processes, and improve efficiency and performance. Launched in 2005 in IFC Advisory Services and later expanded to include lessons from IBRD, the SmartLessons collection today numbers over 1,200 lessons. EMCompass notes are short knowledge notes written by IFC staff to share insights and learning with clients, co-investors and other external stakeholders. Launched in 2016, the 100th note will be published shortly. Recent themes include the impact of new technologies on different sectors, and the impact of COVID on the private sector. Return to text. 38 BOX 5 VIRTUOUS CYCLE OF KNOWLEDGE FLOWS IN THE BANK – INDONESIA AND EGYPT The support provided by the Bank on connectivity in Indonesia and local development of lagging regions in Egypt encapsulates how a flagship report can influence ASA, inform lending, and generate greater knowledge. Connecting Indonesia is a challenge because of its size (17,000 islands) and its population (267 million people). Influenced by the WDR 2009 “Reshaping Economic Geography” the Government of Indonesia asked the Bank in 2010 for support to enhance connectivity. The WB team (comprised of Trade and Infrastructure staff) analyzed the links between logistics, spatial development and inclusiveness and identified key pillars for improvement, then partnered with additional infrastructure experts from AUSAID to support the Government’s development of a connectivity master plan. The Government requested inclusion of connectivity reforms as prior actions of the Development Policy Loan series (DPL8 and DPL9), followed by another two-DPL series (2012-2014) on connectivity to support reforms and accelerate implementation. To advance the analysis on bottlenecks to maritime logistics, the Bank developed an analytical program with RAS financing from Indonesia’s largest state-owned port operator Pelindo II. Insights from this RAS informed reforms supported through a further two-DPL series (2016- 2018) to improve Indonesia’s logistics sector. The WB team showcased the Indonesian pilot and connectivity became a main line of work for the Bank. The support helped improve connectivity through the development of the first fully private port in Indonesia and by increased investments in logistics sectors, including freight forwarders and shipping agencies. These improvements are reflected in the Logistics Performance Index (rating from 1 to 5) which improved from 2.76 (75th) in 2010 to 3.15 (46th) in 2018. The WDR (2009) also reframed the policy debate on urbanization, territorial development, and regional integration. The Urban, Disaster Risk Management, Resilience and Land Global Practice developed a diagnostic tool to identify policy distortions and investment priorities. The tool has been carried out in 35 countries resulting in policy reforms and initiating analytical work that informed several operations, including the Local Development of Lagging Regions Program for Results operation in Egypt. This operation helped the Government of Egypt replace the ‘investment-only and centrally planned’ model of the past with the empowerment of local governorates to plan and implement policies, increase accountability, and include citizens and businesses in decision making. The operation (under implementation) has already led to some significant outcomes, like an increase from 13 to 34 percent in the occupancy rate in the industrial zone in the Qena Governorate and improved accesses to quality infrastructure and services for over 6,000 people. Moreover, the implementation challenges and learning through this operation has shaped the recent regional flagship report on Convergence “Five Critical Steps Toward Integrating Lagging and Leading Areas in the Middle East and North Africa” (2020) as well as the forthcoming volume on Space and Productivity, a joint SD/EFI flagship report on place-based policies. Return to text. 39 BOX 6 HOW LENDING IN MADAGASCAR INFORMED WBG TRADE FACILITATION KNOWLEDGE The Government of Madagascar was struggling with getting adequate tax and customs revenue and wanted to implement reforms to combat tax evasion, curb corruption in customs and facilitate trade. The Bank provided support to these reforms through the Madagascar customs component of the public modernization project, implemented from 2016 to 2019. During that period tax collection by customs increased by an equivalent of 1.5 percent of GDP. The WB team (comprised of Governance Global Practice, DEC and CMU staff) and Madagascar customs jointly developed and implemented an inspector performance monitoring system. The WB team also developed a novel analytical methodology to detect collusion between brokers and customs inspectors. It looked for deviations in the assignment of import declarations from a random assignment. It unveiled that the customs IT system was being manipulated to pair certain brokers much more frequently with certain inspectors. Despite being more at risk of tax evasion, these declarations were less likely to be deemed fraudulent by these inspectors, who also cleared them faster. Tax revenue collection would have been 3.8 percent higher in the absence of this collusion. Based on the evidence from the analytical work, several inspectors were sanctioned. The project also designed and implemented an intervention to outsource the assignment of declarations to inspectors (which had been the key conduit to collusion) to a third party. This curbed corruption but also triggered a novel manifestation of IT manipulation, in which officials in the customs IT department concealed certain declarations from this third party. This resurgence of collusion was detected and addressed by local counterparts applying the methodology the Bank developed. The Madagascar engagement shows the valuable role data and research can play in enhancing revenue mobilization. It also attests to the potential of IT technologies to curb corruption, but simultaneously serves as a reminder that such technologies are not a panacea because they can also serve as a conduit to corruption. This example shows how lending, policy advice and cutting-edge research can go together. The methodology and insights from this experience have been captured in a paper “Technology (Ab)use and Corruption in Customs” that earned a revise-and-resubmit from one of the top journals in economics (the Quarterly Journal of Economics). And the practical advice from this work serves as the basis for a new project on improving revenue mobilization in Kenya, while the customs authorities of Somalia and Niger are also currently replicating this approach. Return to text. 40 BOX 7 ENGAGEMENT APPROACH FOLLOWED IN INDONESIA, AFGHANISTAN, AND NIGERIA The engagement approach is based on five principles: Working as a team: with a shared purpose (country outcomes), each member having a role to play, and a collaborative approach (communicate, coordinate, help each other and constructive questioning to find a solution). 1. Organizing ourselves in terms of engagements and taking a structured, iterative and programmatic approach to if and how we engage. The Country Partnership Framework (CPF) typically lays three or four strategic pillars with various development challenges under each pillar. The country program is then organized into 10-15 engagements framed around the 10-15 highest priority development challenges where the WBG can help government make substantial progress over the next four to five years. Each engagement is defined by a set of results indicators and tasks involving a mix of different instruments (ASA, financing, convening, etc.). Engagements must be in priority areas, with clear identification of who requested the help, and committed government partners with capacity to implement. 2. Adopting the role of “personal trainer” to our government counterparts in our engagements. Government partners aspire to “run a marathon”, which we cannot run for them, but we can do everything short of that, to help them get ready and successfully run it. We must get to know our partners—what motivates them, where they need help, etc., and then design the appropriate “training regimen”. But we cannot simply print it out and give it to them; we must be there every day, pushing, reminding, coaching, supporting. 3. A country leadership team composed of staff with the role of engagement leaders. They lead (in partnership with PMs and their teams) the strategic dialogue with government and partners to define the overall direction and content of the engagements – e.g. mix of ASA and financing operations. Provide upstream advice and guidance to the CMU and task teams on the design of individual tasks, to ensure the strategic fit with the overall engagement. Ensure, through ongoing interactions, that the various task teams take an integrated and coordinated approach, maximize the synergies across the different tasks, align (and restructure as needed) tasks with the overall aims of the engagement and respond to client needs. Return to text. 41 BOX 8 MOZAMBIQUE’S ENERGY SECTOR TRANSFORMATION: A COORDINATED ENGAGEMENT The Mozambique energy sector is an excellent example of effective collaboration between IDA, MIGA and IFC. The institutions are working collaboratively to support investments by providing knowledge and expertise for sector financial sustainability. The support includes, i) IDA reform programs; ii) IFC advisory services to the utility, Electricidade de Moçambique (EDM), on how to improve its treasury and financial management functions; and iii) MIGA’s support on mobilization of equity and commercial bank investments in the sector. These wide-ranging activities by the World Bank Group (WBG) in collaboration with the IMF, have led to program opportunities, including IDA lending/grant programs to both the electricity and gas sectors, IFC debt financing to the Central Térmica de Ressano Garcia gas and Mocuba solar projects and an equity investment in the Pande-Temane gas fields. The WBG is currently working on a landmark engagement of about a billion dollars in the Mozambique energy sector through development of the 450 MW power project (Central Térmica de Temane, S.A. or the Project) where complementary products are provided by each of the institutions, including the IDA- funded transmission line, IDA guarantees for the Project, MIGA guarantees for the sponsors of the Project, as well as IFC commercial and concessional debt financing for the Project. The Project is part of an integrated effort to leverage WBG resources and expertise to support private sector investment, in line with the Government of Mozambique’s (GoM) least-cost development plan for the sector and its success has required coordinated and targeted action from all WBG institutions. Return to text. 42 BOX 9 PERU: LEVERAGING KNOWLEDGE FROM THE JOINT CAPITAL MARKETS PROGRAM (J-CAP) IN SUPPORT OF SMES Over recent years, Peru’s government has actively engaged in legislative improvements in support of an enhanced role for local capital markets; one such improvement had focused on legislation supporting the digitalization of receivables. However, this had failed to yield a large increase in the factoring market which when successful, facilitates quicker access to capital for businesses, included SMEs, using outstanding accounts receivables. Peru had the pre- conditions for factoring to become a successful capital markets instrument with a relatively mature mutual fund industry, sophisticated investors base, an adequate market infrastructure, and appropriate regulatory framework. In September 2018, IBRD with the capital markets regulator held several round tables between private and public sector players in Peru while IFC engaged with the government specifically on the issues of digitalization of receivables. Under the IBRD and IFC J-CAP umbrella, a multidisciplinary team of advisory and investment specialists met with relevant public and private sector stakeholders. IBRD brought its knowledge of local capital market instruments, of SME needs, and of the stakeholder ecosystem. IFC contributed its knowledge of the ecosystem and best practices in the factoring market in Peru and other countries. Despite local banks having relegated factoring as a minor business segment, the WBG identified the market’s potential for SMEs and envisioned that Peru’s capital markets could play a larger role by supporting financial entities providing factoring services to SMEs. This engagement provided IFC with valuable knowledge on the sector and led to a project with asset manager Compass, the largest non-bank factoring entity in Peru. In August 2020, IFC committed up to US$21 million equivalent in local currency to be invested in the Compass factoring fund, a multi-sectoral local capital markets instrument. This fund channels resources from institutional investors, which buy quotas of the fund at an attractive yield, to finance the working capital needs of nearly 150,000 SMEs. The latter has been especially important during the COVID pandemic, as the banking sector focused on larger clients and, the support obtained through the factoring fund has been the only funding source for some SMEs. Local capital markets facilitated the mobilization of funds from investors into working capital for SMEs, while providing an investable asset class to institutional investors. IFC’s investment improved the fund’s stability and enhanced its scale potential and interest to other large investors. Additionally, through the IFC investment, Compass agreed to enhance its environmental and social management system and won the local award for Responsible Investor of the Year 2020. Return to text. 43 BOX 10 DATA INNOVATIONS IN THE WORLD BANK DEC is collaborating with PGs and Regions to spearhead innovative analysis of nontraditional data sources using novel techniques to develop knowledge: Prior to COVID-19, the Bank used 15 billion mobile phone records in Senegal to quantify the impact of travelers from high to low incidence areas on malaria infection rates. This provided a simulation-based policy tool for other diseases, such as tuberculosis or COVID-19, that uses mobile phone data to inform targeting of travelers based on their origin and destination, reducing caseloads more effectively. In South Asia teams are using Google searches for consumer products to identify scarce items in communities, and searches for COVID health- related information to identify areas where incidence is increasing. In Indonesia the use of data from the transportation app Waze during the COVID-19 pandemic has permitted to asses if movement restrictions were being abided by or not. In Nigeria Waze data is helping inform the setting up of formal bus lines in Lagos. In Kenya the lack of data on car crashes has been overcome by integrating data from the police, Uber, Weather, Waze and tweets to identify the most dangerous traffic crossings and reduce traffic accidents. In Ethiopia, Malawi, Mali and Uganda the team is leveraging the latest advances in satellite imagery and remote sensing (integrating high-resolution satellite imagery with georeferenced Living Standards Measurement Study surveys and ground-truth measures of crop yields) to generate accurate measures of agricultural productivity of smallholders, which are key for policy design to reduce poverty and food insecurity. In Africa, ECA and LAC regions the use of newly available firm-level administrative tax data is helping estimate the impact of shocks like COVID-19 on firms' profits, employment and survival to help improve targeting of fiscal responses to shocks; as well as improve forecasting of economic activity in real-time. DEC is also innovating beyond money-centric measures of well-being to capture a broader picture of development. Lab-in-the field approaches are being used in the area of mental health combining low-cost electroencephalograms, eye-tracking, and salivary cortisol samples to develop objective measures of stress and correlate them with other economic outcomes. Finally, the WDR 2021: “Data for Better Lives” provides a comprehensive analysis of the profound changes brought about by the pervasive presence of data. It analyzes pathways through which data collected for public and private purposes can be brought together for good – and for ill – and discusses the types of institutional and regulatory structures countries can adopt to maximize the benefits of data for society. Return to text. 44 BOX 11 EXTRACTING TACIT KNOWLEDGE TO BETTER EVALUATE PLACE-BASED POLICIES What have we learned from Bank experience about the design and evaluation of place- based policies such as special economic zones, transport corridors, industrial development in lagging areas of countries and development of new urban centres that absorb a large share of our investment portfolio? An ongoing collaboration between EFI and the GL for territorial development contracted two leading academics, Gilles Duranton (U. Penn) and Anthony Venables (Oxford), to synthesize their academic thinking and Bank experience. They first reviewed the codified knowledge embedded in the PADs of over 25 projects. Then, through interviews and brainstorming sessions, they accessed the tacit knowledge in TTLs memory banks on the initial “context” surrounding their projects, and the decision trees that lead to a particular design. The TTLs enthusiastically contributed their experiences which, in turn became grist for the framework, but they were also candid about the shortcomings of the evaluation process. It emerged that many projects began with only a diffuse economic rationale that even the TTLs who inherited them could not decipher, few clearly identified market failures requiring government intervention, and in many cases client or internal political pressures importantly impinged. Out of this process emerged a robust assessment framework (Duranton and Venables 2018) that can be used by frontline staff assessing place-based projects and is the organizing framework for a pending flagship. The goal is to empower TTLs – intellectually and practically – to better design projects and to challenge, modify, and where necessary reject projects that arise from other pressures. Return to text. 45 BOX 12 WORKING TOGETHER – WHAT MADE THE PG-DEC-CMU COLLABORATION IN MADAGASCAR A SUCCESS IN TERMS OF EMBEDDING KNOWLEDGE IN AND CAPTURING KNOWLEDGE FROM A PROJECT? The following factors helped make the collaboration described in Box 6 particularly effective: 1. Peer to peer mentoring across teams: Senior team members and the TTL extensively mentored DECRG team members, investing a lot of time in explaining how customs functions and what the key day-to-day challenges for different arms of the customs administration are. This in turn enabled the DEC team to tailor their analysis and the design of interventions to be aligned with the programs’ objectives. The CMU facilitated the program by managing the policy dialogue and providing political backing for major reforms. 2. Experimentation (and tolerance for failure): The team adopted a flexible approach. Potential solutions were developed in close consultation with counterparts and subsequently piloted. The effectiveness of these pilots was evaluated and elements of the program that were successful were scaled up. Unsuccessful interventions were modified or abandoned. A crucial aspect of this approach is the willingness to tolerate failures and setbacks as some interventions did not yield the desired outcome. 3. Relationship building and client focus: The team invested heavily in building a good relationship with key counterparts in customs, in the process cultivating political will and trust. This included delivering on quick wins that resolved pressing problems the client faced (including ones not immediately pertinent to the program) and supporting counterparts to present the impact of the interventions in international fora, like the annual congress of the World Customs Organization. 4. Joint fundraising: The CMU, Governance Global Practice and DEC jointly fundraised to finance the research component of the project, and to enable counterparts from customs to attend international workshops and training sessions. Return to text. 46 BOX 13 IFC’S OHADA INVESTMENT CLIMATE (IC) PROGRAM EVALUATION Between 2007 and 2017, IFC’s IC program supported the Organization for the Harmonization of Business Law in Africa (OHADA) in implementing essential reforms to unify business laws across all 17 OHADA-member states. Through this program, IFC and OHADA enacted four Uniform Acts: on Secured Transactions (2010), Commercial Law (2010), Company Law (2014), and Insolvency (2015). The OHADA Uniform Act on Secured Transactions came into force in 2011 in all OHADA member states and introduced new mechanisms for collateral. When evaluated, IFC employed the synthetic control method to estimate if the Act had generated any impact on domestic credit in OHADA member states. Synthetic controls are uniquely well-suited to estimating economy-wide macroeconomic policy impact, and this OHADA evaluation is among the first to apply the method to evaluate multinational legal and regulatory reforms. The evaluation found that between 2011 and 2015, the Act led to additional domestic credit to the private sector of more than $3.8 billion in seven member states – Burkina Faso, Cameroon, Central African Republic, Comoros, Mali, Senegal, and Togo. Results were inconclusive for three countries (Benin, Côte d’Ivoire, and Gabon) due to lack of data; and it was not possible to undertake synthetic control method analysis for seven member states. The results were further validated by in-depth case-studies in Cameroon, Côte d’Ivoire, and Niger. The impact of the reform is particularly encouraging in the context of conflict-affected countries, where mobilizing private resources is hugely difficult. For instance, in the Central African Republic, the reform led to an increase of $33 million in domestic credit to the private sector while in Mali, the number was much higher at $607 million, even though the country’s economy was still recovering from unrest in 2012. Similarly, evaluation evidence suggests that the OHADA Uniform Act on Company Law offered timely support for private equity funding by providing equity funds with modern financial instruments. SMEs have benefitted the most from improvements in business registration and cost savings. The launch of results at the ministerial meeting by the OHADA Secretariat in January 2019 has triggered policy dialogue among member states, including to address issues hindering the impact of the uniform acts, leading to IFC advisory projects to address evident issues. The final report is available on the OHADA website: http://www.ohada.com/content/newsletters/4643/rapport-ohada-ifc.pdf Return to text. 47 BOX 14 WBG ENGAGEMENT THROUGH COUNTRY PRIVATE SECTOR DIAGNOSTICS (CPSDS) As part of IFC’s 3.0 Strategy, IFC has been utilizing CPSDs to provide the analytical underpinnings for the private sector agenda in client countries. These diagnostics identify private sector-led growth opportunities, the main constraints to the private sector, and reforms and policy changes needed to improve private investment. Around 50 CPSDs have been completed or are currently ongoing. CPSDs are jointly undertaken between the World Bank and IFC and are used by SCDs and CPFs to bring the private sector perspectives to address development challenges. They also inform DPOs on specific reforms that would facilitate private investment. For example, the Jobs and Economic Transformation Program for Results (PforR) loan for Senegal, for which preparation has recently started, is in part inspired by the Senegal CPSD recommendations. Horticulture was one of the key value chains proposed in the CPSD and it is one of the main focus areas of the PforR. Recommendations for strengthening the capacity of the FONGIP guarantee fund came out of the CPSD as well. In Kenya, the $750 million Kenya Inclusive Growth and Fiscal Management DPO (May 2019) was informed by the agriculture sector deep dive in the CPSD. Key CPSD recommendations picked up in the DPO include reforms for better targeting of subsidies for agricultural inputs using e-vouchers and biometric digital identification; reducing inefficiencies and leakages in the procurement and marketing of fertilizers; and establishing a warehouse receipt system and a commodities exchange to help farmers get easier access to credit and to reduce post-harvest losses. Meanwhile, in Ecuador, the CPSD analysis of the mining sector spurred an engagement of the WBG with the Government and has contributed to the decision of the country to join the Extractive Industries Transparency Initiative. CPSD analysis has also directly contributed to prior actions incorporated into Ecuador's Third Inclusive and Sustainable Growth DPL. These are prior actions related to facilitating business entry and easing trade and FDI, respectively. CPSD value addition is as much about the analytical output as it is about stakeholder engagement. Through stakeholder consultation, CPSDs bring varied public and private sector perspectives together, creating a common platform for further action. This applies to cross- WBG processes as well as to country-level dialogue where many stakeholders, including government partners, private sector representatives, and development partners are engaged. Return to text. 48 BOX 15 LEVERAGING THE WBG TO DELIVER A QUALITY ASSESSMENT OF COLOM- BIA’S HEALTHCARE SECTOR The World Bank and IFC worked together to develop the Healthcare System Quality Assessment Tool (the Tool). The Tool combined (i) the IFC’s IQ Healthcare Tool, a diagnostic tool that assesses the quality of care at public and private hospitals level and (ii) a system- wide assessment of the sector, including the vision/ strategy of the government, regulations incentive mechanisms, accreditation system and other system-wide aspects. The design of the Tool involved bringing together the expertise of the WB on the public health sector side and the expertise of the IFC on the private health sector side. The jointly developed Tool allowed stakeholders (the government, health insurance companies, healthcare providers and patients) to understand quality gaps in the sector and define actions to raise quality through changes at a facility, government or regulatory level. Once the design of the Tool was finalized, the WB-IFC team picked Colombia as a pilot country and conducted a scoping mission to draft a clear project roll-out roadmap, including documents and interviews, followed by a targeted mission aimed at engaging with all relevant stakeholders. Each team member oversaw one specific aspect of the Tool and were responsible for drafting chapters of the report or facilitating access to the stakeholders on that specific topic (i.e. accreditation system, incentives mechanism, observation from the field). This first joint diagnostic was rolled out in Colombia in 2019 and as a result, the team produced a set of tangible and practical recommendations to improve quality of care. The WB-IFC joint report was well-received by the sector and the dissemination of the findings and recommendations included several events through the country, with active participation of the public and private sector. This effort was the first joint WB-IFC work in the sector in Latin America and has led to lending operations for the WB and investment opportunities for IFC. The WB signed a US$190mn P4R with disbursement tied to selected report’s recommendations (P170298 and P168263). This was the first P4R in Columbia and the first loan in healthcare in Colombia in 25 years. The WB also approved a COVID-19 Crisis Response DPF (P174118) of US$250mn to Colombia. One of the pillars of the Crisis Response DPF is healthcare, and it directly draws from the report’s recommendations. IFC is currently considering investments in private hospitals, both sourced through this work. Return to text. 49 BOX 16 USING ARTIFICIAL INTELLIGENCE (AI) TO OVERCOME SIZE CHALLENGE AND IMPROVE SEARCH FUNCTION Many consulting firms have invested in creating curated knowledge bases, and the Bank has made progress in this area, with repositories like the Open Knowledge Repository. However, the sheer scale and distributed nature of the Bank’s knowledge means that staff often must navigate millions of documents to find what they are looking for. Manual categorization and curation at this scale would be prohibitively expensive for the Bank. ITS is therefore exploring the use of text analytics, machine learning, and AI capabilities to improve the Bank’s search function. This will allow the Bank to detect patterns and extract insights at scale from its large information infrastructure, analyzing and reclassifying past documents to keep historical knowledge relevant in the context of today’s business needs. A promising early example of this is the Knowledge Package (KP) jointly developed by DEC, OPCS, and ITS that leverages machine learning algorithms to automatically recommend relevant knowledge resources to teams working in specific areas. Recently, the team started providing similar service to IFC as well. The launch and use of the IFC KP reduces the time staff spend on research by aggregating similar projects and relevant internal and external documents based on keywords users’ input. For IFC, the KP compiles content and links from IFC/IBRD investment and advisory projects, lessons learned (e.g. Smartlessons), internal documents (e.g. IFC Country Strategies, CPSDs), and external resources (e.g. the Economist Intelligence Unit country report). Providing this information directly to project teams allows the teams to reference relevant information and lessons learned in their design and decision making. Another example is MALENA, IFC’s Machine Learning Environmental Social and Governance (ESG) Analyst which is an AI powered virtual analyst developed to identify ESG risks and generate impact analytics for emerging markets. The project will support IFC operations through enhanced ESG due diligence and differentiated processing. IFC is also testing support for clients through advisory services to deliver analytical capacity at scale for ESG risk analysis. The MALENA AI is being trained using IFC’s historical ESG data. The first algorithm under beta testing is a Natural Language Processing and Sentiment Analysis classification model trained to identify ESG risks and conduct sentiment analysis from unstructured text. Interactive digital tools such as wikis, podcasts, and video sharing can help promote collaboration, knowledge flow, crowd sourcing, and dissemination, and make knowledge more accessible while helping digitally strengthen ease of access to 70 years of development knowledge. Robust information management practices, aligned incentives, high quality content, subject matter expertise, and digital capabilities are essential for a strengthened knowledge management system. Return to text. 50 A N N EX 2 : PR EVIOU S E FFO RTS TO R E AL I Z E THE WBG’ S K N OWL EDG E P OTE NTIA L The WBG has been in the knowledge business for a half-century. Annex 3 presents a summary of the approach to knowledge at IFC, an overview of milestones in the knowledge journey, and the new direction that the Corporation is taking. Introducing the concept of a Knowledge Bank. President Wolfensohn articulated the concept of the Knowledge Bank in 1996 leading to the World Development Report 1998- 1999 Knowledge for Development. It was at this point where a WBG-wide mandate explicitly appears. The creation of the Networks in the 1997 reforms was in fact, designed to facilitate knowledge flows throughout the Bank (Finance and Private Sector Development, FPD, was joint WB/IFC) and support the work of operational staff. The challenge of increasing knowledge flow and quality: Seven years later, the Organization Effectiveness Task Force (2004) noted that knowledge flows remained weak and that the Bank was not capturing the full potential of being a global knowledge-based organization. The 2008 IEG report Using Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Sector Work and Technical Assistance, 2000–2006 reinforced that clients prefer the Bank’s reports and technical assistance to those from other institutions but stressed that the Bank needed to take the results tracking framework more seriously, noting that the existing one had major weaknesses. “This can be seen in the poor quality of information on ESW and TA in the Bank’s administrative database, as well as in the incomplete information in the results tracking framework itself. This in turn reflects the lack of serious monitoring or reporting of these tasks. Finally, there is no systematic follow-up with clients to evaluate the effects of these products.” (p. xvii). These critiques related to the need for better prioritization management of knowledge resurfaced prominently in IEG (2016a). Transforming the agenda: Though strengthening the knowledge agenda was a priority in the 1997 reforms, a major rethinking occurred in 2010 in light of important shifts in the global knowledge economy with the establishment of the Knowledge and Learning Council, comprising members of the Bank’s senior management. This council produced the 2010 Knowledge Strategy—Transforming the Bank’s Knowledge Agenda: A Framework for Action, which informed The State of World Bank Knowledge Services, 2011. In this report, the first in what was envisaged to be a series, the Bank systematically reviewed its knowledge services, advancing concrete proposals for measuring results and modernizing the management of the Bank’s knowledge work. In the preface, President Zoellick stated: 51 This Knowledge Report is timely, because we are witnessing a fundamental transformation in the way the world produces and accesses knowledge. We are seeing an explosion of information from multiple sources, and information increasingly accessible to people across the world. In this context, the World Bank Group has been transforming the way we do our knowledge work, opening our data and knowledge to students, researchers, policy makers, and civil society throughout the world. (2011:v) The Knowledge Paradox and Counterbalancing the Incentive to Lend. But the report also noted a “knowledge paradox”: Surveys show that clients cite knowledge services as the Bank’s most valuable contribution, more than twice as often as financial resources. Yet the Bank’s knowledge work is not seen internally, or by independent evaluators, as having the impact it could. Managers and staff see limited internal support for their knowledge work and some of them feel that such work is undervalued. This gives rise to a knowledge paradox. Most staff feel, despite the growing importance of knowledge work, that the Bank’s main internal incentives are still related to lending. One reason for this apparent contradiction may be the lack of robust and systematic evidence that knowledge work brings demonstrable and measurable returns. By contrast, lending has built-in metrics. Lending volumes and disbursement rates are easily understood and communicated (even though money spent may not be a valid measure of impact). (2011:2) To address this paradox, the Bank set three goals: First, it aimed to establish “consistent standards for governance and the measurement of results. This will apply to each of the nine product lines that make up core knowledge, and aims to strengthen technical quality, relevance, and results, and to put a stronger emphasis on client engagement.” (p 2-3.) Second, it aimed to strengthen connectivity across the core knowledge product lines and to develop a framework for a comprehensive approach to managing knowledge as a portfolio – setting strategic priorities, ensuring complementarities, and creating explicit mechanisms for discussing work programs and priorities among Networks and Regions. Third, Open Knowledge – all aspects of creating, acquiring, sharing, adapting, and applying knowledge were to be made more accessible. (On Open Knowledge the WBG has made great progress.) The diagnostic and indeed much of the analysis continues to be fresh and germane as it was a decade ago, suggesting more can be done to achieve these goals. Raising the Quality of Lending and Knowledge Portfolios. The report’s diagnosis of the paradox echoes the 1992 Report of the Portfolio Management Task Force (The Wapenhans Repor30) that analyzed why the quality of the Bank’s portfolio of project loans had been 30. http://documents1.worldbank.org/curated/en/793701492973438535/pdf/Effective-implementation-key-to-development- impact-working-papers.pdf 52 weakening over time. As stressed here with the knowledge agenda, reforms in the area of quality, incentives and human capital were noted as of critical importance to improving the quality of financing. The driving theme of the report was that “Underlying many of these aspects is the Bank's pervasive preoccupation with new lending” (1992:iii) to the exclusion of ensuring solid design, follow through and tracking impact on the ground. It stressed the need for deep changes in procedures and directives and, among others, that “Country directors and division chiefs must be held as accountable for managing each country’s portfolio performance as for new lending” (1992:18). That said, it stressed that “Such changes alone would accomplish little. They must be made in tandem with a concerted effort, led by top management and firmly supported by the Board, to bring about changes in the Bank's internal values and incentives.” (1992:24). Hence, the three pillars of improvements discussed in this Framework - systems, incentives and human capital - were all anticipated in the lending context, and the need to hold management accountable for progress on all fronts. The 2011 strategy was overshadowed by the 2014 organizational reforms that created the Practice Groups. As noted by IEG (2019) Knowledge Flow and Collaboration under the World Bank’s New Operating Model, the Practice Group model sought to end the perceived fractionating of the Bank into six separate regional entities that made it difficult to move staff and knowledge across regions, and for the World Bank, IFC, MIGA and ICSID to collaborate effectively. The Practice Group model aimed to develop instead state of the art practices informed by the science of delivery of all types of Bank services, including knowledge, concentrated in the central Practice Groups. “Despite the reforms’ many challenges, the operating model has shown value. PGs are thinking and operating more globally. They improve knowledge flow and staff mobility, mobilize expertise for client-facing activities, and sometimes deepen expertise in operationally relevant areas. GTs [Global Themes] provide strategic direction and coherence to cross-cutting priorities” (2019:xvi). The share of staff who state that colleagues across the World Bank Group (IBRD and IDA, IFC, MIGA, ICSID) work together effectively increased from 23 percent in 2013 to 40 percent in 2017 (WBG Employee Engagement Survey). Though important gains were made with the organizational reform of 2014, new problems emerged: An IEG review identified these as follows: Although the objective of overcoming regional silos was sound, the new model created other barriers to knowledge flow and collaboration. Managers on both sides of the matrix incur high transaction costs from routine managerial functions. Further, key knowledge flow and collaboration mechanisms do not perform as intended. Quality assurance processes need enhancement. Budget incentives are not aligned with greater collaboration. Finally, more continuity in making course corrections for organizational effectiveness is needed (IEG 2019:xvi). 53 IEG (2019) further noted that while some PGs had coherent and systematic approaches to managing and investing in knowledge, others did not, and there was much variation in how PGs used and resourced the Global Leads, for instance, with differences in leadership attention and funding explaining much of the variation. The review also noted that “[e]fforts to more closely integrate certain IFC and World Bank advisory services have met with mixed success” (2019:xxi). In short, the modifications to the organizational reforms did not go far enough in designing effective mechanisms and incentives. IEG (2019) concluded that the results in terms of knowledge delivery were disappointing: In hindsight, reform design and implementation succeeded in reducing regional silos as barriers to knowledge flow but did not succeed in enhancing collaboration or changing incentives, behaviors, and organizational culture. Replacing four sector networks with 13 Global Practices and 5 GTs has resulted in greater fragmentation and silos because of the Global Practices’ competitive behaviors, which were driven by the size of their own-managed lending portfolios. A simpler approach to the reorganizations may have been to retain existing groups and structures, perhaps with some adjustments among sector boards, but realign incentives and reporting arrangements. This may have minimized additional fragmentation, confusion over the reforms, and damage to staff morale (2019:49). The 2019 and 2020 modifications to the organizational structure and reporting lines are intended to ensure that global knowledge serves country engagement. There is no one perfect organizational structure, and every possible structure will tend in some ways to favor achieving one objective over another. The current model, which brings the PGs, DEC, and ECR under one Managing Director, seeks to address the unresolved mechanism and incentive issues of the previous two models. The move of PG directors to the Regions is designed to improve incentives for collaboration at the country level. This makes a careful rethinking of the knowledge agenda a timely and critical endeavor. Indeed, one of the lessons is that any proposed change to the structure will alleviate some problems but risks creating new ones. 54 TABLE 1 AN EVOLVING FOCUS ON KNOWLEDGE: 25 YEARS OF THE KNOWLEDGE BANK31 YEAR DESCRIPTION OF EFFORT 1996 President James Wolfensohn articulates the concept of the Knowledge Bank and knowledge networks. The Bank’s “networks” initiative begins with the Human Development Network, and soon grows to include Environmentally and Socially Sustainable Development, Finance, Private Sector, and Infrastructure, and Poverty Reduction and Economic Management. The networks were tasked with addressing emerging development issues and ensuring the flow of knowledge throughout the Bank to support the work of staff. 1998-99 World Development Report: Knowledge for Development is published. The report analyzes the risks and opportunities that the global information revolution is creating for developing countries, and concludes that access to financial, technical, and medical knowledge is crucial to improving the health and living standards of the poor. 2000 The World Bank’s Development Committee report, Poverty Reduction and Global Public Goods: Issues for the World Bank in Supporting Global Collective Action, cites sharing development knowledge as one of five focus areas as the Bank increases its emphasis on producing global public goods and recommends that its knowledge role extend beyond country clients. 2003 The Bank’s management information systems begin incorporating knowledge “products,” with deliverables and predefined milestones and with monitoring of most knowledge products similar to that for lending products. The Independent Evaluation Group (IEG, formerly Operations Evaluation Department) publishes its report, Sharing Knowledge: Innovations and Remaining Challenges, Report No. 25883. 2004 The Organizational Effectiveness Task Force notes that the Bank is not capturing the full potential of being a global knowledge-based organization and recommends building networks to leverage global knowledge and improve the quality of client services. 2003- The Quality Assurance Group reviews economic and sector work and technical 07 assistance in 53 countries, assessing them in the context of country assistance strategies, in consultation with local officials and stakeholders. 31. The entries in this chart from 1996-2011 are taken from the 2011 publication “The State of World Bank Knowledge Services”. Updates from 2012-present by authors. 55 2008 IEG report entitled Using Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Sector Work and Technical Assistance, 2000–2006 is published. It finds that clients prefer the Bank’s reports and technical assistance to those from other institutions, and that country clients generally prefer technical assistance and short reports rather than longer pieces of economic and sector work. It also finds that the results tracking system for knowledge work in the Bank has some serious weaknesses and that more follow- up with clients after delivering products would strengthen impact. 2009 The Knowledge Strategy Group, formed to oversee the preparation of a knowledge strategy, identifies nine product lines as the core of the Bank’s knowledge business, for three audiences: knowledge for clients, knowledge as a public good, and knowledge for internal use. 2010 President Robert Zoellick launches the Open Data, Open Knowledge, Open Solutions, and related policies on Access to Information, resulting in many databases becoming publicly available free of charge, including the World Development Indicators. The data are available under an open license for use and reuse. Under Access to Information, the public has free access to many documents previously restricted to official users. The 2010 Knowledge Strategy, Transforming the Bank’s Knowledge Agenda: A Framework for Action, is adopted. The Knowledge and Learning Council is established to manage knowledge initiatives, including this first Knowledge Report. 2011 Report entitled The State of World Bank Knowledge Services marks the first time that the Bank systematically reviewed its knowledge services, advancing concrete proposals for measuring results and modernizing the management of the Bank’s knowledge work. Six Knowledge Platforms are funded for three years, to support co-generation of knowledge from diverse sources and institutions, by engaging with researchers, policy makers, and practitioners to fill knowledge gaps on developmental issues. 2013 IEG report entitled Knowledge-Based Country Programs. An Evaluation of World Bank Group Experience, 2005–11 was published. This evaluation assessed knowledge-based activities in nine country programs selected from 48 knowledge- intensive programs supported by the Bank Group. The report found that in the sample of countries, the WBGwas more effective when it worked on specific sectors rather than broad topics, designed tasks to address specific client concerns, customized international best practice to local conditions, generated data to support policy making, and formulated actionable recommendations that fit local administrative and political economy constraints. The WBG was less effective when it did not address issues relevant to the client or was unable to follow up consistently with the client on the implementation of advisory activities. 56 2013 A Policy Research Paper entitled Toward a conceptual framework for the Knowledge Bank was published. This paper proposes some basic elements of a conceptual framework to help organize the thinking about policies that can strengthen the knowledge mission of the World Bank. It finds that the range of products coupled with an increasing emphasis on just-in-time advisory services dictate the need for not more generalists, but rather an increased range and depth of very specific and high-quality human capital. However, this increased specialization in turn creates the need for “hinge” actors who can communicate and operate well across different knowledge communities—academics, policy makers, practitioners, etc. The necessary changes in human resource and incentive policies, in particular the critical development of better means of ensuring the quality of knowledge production, are an essential complement to any organizational restructuring. 2014 President Jim Yong Kim implements an organizational restructuring from the networks to a global practice (GP) model. The model partly sought to improve flows of knowledge across regions. 2018 A Knowledge Strategy for the World Bank, by Acting Chief Economist Shantayanan Devarajan is presented internally and highlights many areas for needed improvement. 2019 IEG report entitled Knowledge Flow and Collaboration under the World Bank’s New Operating Model is published. This evaluation assessed how well the World Bank’s current operating model stimulates knowledge flow and how well it enhances collaboration to deliver “integrated solutions” – or multisector and multiservice tasks and approaches – to clients. The evaluation found that certain aspects of the operating model are valuable, especially its enabling of global knowledge flow; these are the effect of setting up GPs that operate more globally than before and Global Theme Groups that provide useful strategic directions and coherence to cross-cutting priorities. However, the model’s structure and processes tended to inhibit collaboration and cause inefficiency, fragmentation, and internal competition. Some GPs lacked coherent and systematic approaches to managing and investing in knowledge. 2020 Organizational restructuring from global practice model to hybrid GP/regional model. Return to text. 57 A N N EX 3 : MA KING K NOWL E DGE WO R K AT I FC As the WBG’s private sector arm, IFC helps developing countries achieve sustainable growth by financing investment, mobilizing capital in international financial markets, and providing advisory services to businesses and governments. There is broad recognition that the private sector is a critical driver of economic growth, and the main creator of jobs in low- and middle- income countries. This is crucially important as the global economy transitions to recovery from the pandemic and reinforces the importance and relevance of IFC 3.0 Strategy, where the creation and recreation of private sector opportunities through IFC’s upstream agenda will be critical. IFC’s unique offering, combining global knowledge and reach, with investment services, advisory services, and asset management remains essential in support of transformational improvements in development For IFC, knowledge is an asset to be used and applied in the delivery of services and products to IFC clients and partners and in the creation of IFC thought leadership which highlights lessons learned, insights and innovation emerging from our work (see Annex 1, Box 3 which highlights further IFC’s knowledge for private sector development). Results from the FY20 Client Survey reaffirm views from previous Client Survey’s and tell us that clients continue to recognize and value IFC’s global, regional and sectoral knowledge in their partnership with IFC. Knowledge drives the internal workings of the Corporation to ensure that our next investment and advisory operation is better than the last because what we have learned has been folded into it. This is and will continue to be at the forefront of our Upstream agenda where lessons learned, sectoral and regional knowledge will be instrumental in building private sector opportunities across client countries (see Annex 1, Box 15 showcasing the example in Colombia healthcare). Our clients and partners benefit from this in-house experience and expertise both tacitly through direct engagements with IFC staff and teams or indirectly through knowledge pieces that we produce. Over recent years, IFC has been increasingly focused on improving the ability of IFC staff to collate and disseminate quality knowledge products for internal and external purposes. While this remains an ongoing initiative, IFC’s efforts are increasingly successful, with a wide array of knowledge and experiences from across regions and industries increasingly available for review and use (see Annex 1, Box 4 on successful efforts within IFC to disseminate knowledge). IFC Knowledge Management Timeline IFC is a generator and holder of valuable knowledge, innovation and insight. The IFC’s knowledge management journey gained momentum in the early 2000’s with the capability formally introduced into the Corporation. Following the implementation of IFC’s Global/Local strategy in 2008, IFC introduced a corporate approach to managing knowledge to ensure that 58 staff in an increasingly decentralized organization can access knowledge from across the world. In 2009, the Global Knowledge Office, as well as a governing body, the Knowledge Strategy Committee, were established to maintain the Corporation’s global knowledge, learn from its operations, and increase efficiencies. Initial investments in information and knowledge capture and organization, SmartLessons and iCollaborate, soon followed. In 2017, IFC established the Economics and Private Sector Development Vice Presidency to lead the creation of evidence and evaluation-based research and thought leadership. Concurrently, the new organizational structure at the time recognized the importance of knowledge creation, ensuring that accountability for development and dissemination of knowledge lay with the Global Industry Directors for each key industry, supported and enabled by the KM professionals within IFC. The importance of global best practice and expertise is a key differentiator enabling IFC’s long- term success in scaling development impact through its private sector engagements in client countries. Moreover, the IFC works across the WBG to leverage its knowledge capabilities and holdings as effectively and efficiently as possible (see Annex 1, Box 8 illustrating the WBG’s engagement in Mozambique’s energy sector transformation). Today, the Global Knowledge and Learning Office continues to ensure that the creation, management and reuse of knowledge at IFC is deliberate, coordinated and strategic. Looking ahead, a new and updated knowledge management strategy is underway with the objective of positioning the Corporation for the future and ensuring knowledge is front and center in IFC’s delivery and engagement with staff, clients and partners. While IFC has made progress over recent years, and a number of best practice examples exist across the Corporation, IFC can and must do more to ensure that the full complement of knowledge is effectively curated, disseminated and updated. As such, IFC has embarked on a journey to significantly retool our approach, ensuring that IFC’s knowledge is strategically relevant, readily at hand, and able to be applied and used in a seamless and reliable way (see Annex, 1, Box 16 showcasing the use of AI in knowledge). IFC Knowledge Management Strategy and Implementation IFC’s 3.0 Strategy was introduced in December 2016. This ambitious retooling of the organization builds on the strengths of our historic approaches, IFC 1.0 and 2.0, and is intended to position IFC at the forefront of developing new and stronger markets for private sector solutions. Over the last number of years, many fundamental building blocks have been put in place, such as the corporate scorecard, Anticipated Impact Measurement Monitoring (AIMM) framework, CPSDs, and more recently, the new Upstream approach. With this fundamental shift in business strategy comes the need to closely review and retool how the Corporation manages one of its most important assets, knowledge. The drive to retool and reposition is the result of long-term organic growth and development of the Corporation, both in size and complexity. In 65+ years, IFC has grown steadily from 12, primarily Washington-based employees in 1956, to a current complement of over 4,100 based in 59 188 countries around the world. Importantly, as part of the WBG Capital Increase commitment, IFC has committed to: (a) more than double its aggregate annual delivery; (b) more than triple annual own-account investment in the poorest and most fragile countries; (c) more than quintuple annual own-account commitments to the poorest of the poor countries (defined as low-income IDA and fragile and conflict-affected IDA countries); and (d) triple its amount of annual climate-related own-account financing and quadruple annual own-account financing dedicated to women and women-owned SMEs. To facilitate these commitments and to deliver against IFC 3.0, IFC has ramped up staffing and efforts around our Upstream agenda and has onboarded a significant new number of new staff to deliver and execute on IFC 3.0. These substantial shifts in the operating environment as well the impact of the COVID-19 pandemic on how IFC has had to operate has reinforced the need to make IFC knowledge more easily accessible, discoverable, and ultimately usable in responding to development challenges and in creating or recreating markets across countries. IFC can do more to leverage and apply the knowledge being generated throughout the Corporation to complement the changing business environment. IFC is therefore developing a Knowledge Management Strategy (IFC KMS or “the strategy” hereafter) that lays out the business need, rationale, and the various components on the why, what and how to reposition and enhance IFC’s knowledge and to enable better stewardship of this valuable IFC asset. The IFC KMS is envisioned to be an enterprise-wide blueprint, with a focus on operations as the engine of the Corporation. Through collaborative consultations across IFC, the KMS is being developed using interviews and small focus groups that were conducted with a range of staff across operations and key supporting functions. These outreach activities included gathering and compiling feedback from the VP-level to the GE+ level, with a strong emphasis on new hires as well as the Corporation’s KM practitioners. Best practice from top private sector organizations, international financial corporations and multi-lateral development banks was also leveraged and has fed into the formation of the KM strategy. These insights will be instrumental in completion of the KM implementation plan. Knowledge CLIENTS AND KNOWLEDGE FINANCING Management PARTNERS 60 The production of knowledge is something that the IFC and WBG has always excelled at – the total volume of material being generated is significant and as noted in the main document, is used widely by academics, policy makers and development practitioners. IFC’s focus is on the operational application of knowledge - folding what we know back into the business is part the corporate business model - using a foundations perspective as illustrated in Figure 1. Quality, strategic relevance, and ultimate impact of the knowledge generated are important questions being addressed as part of the IFC knowledge strategy and implementation plan. However, a strong knowledge management approach is equally as important to bring order, structure and usability to the knowledge being generated. The loss of knowledge or “institutional amnesia” is a real concern in light of the volume of material being generated across the WBG. Only a small percentage finds its way into searchable repositories and unfortunately, the passage of time compounds the problem. COVID-19, the large number of new staff hires, as well as the workforce plan to rebalance the staff curve across the Corporation have reinforced the need for IFC to address any vulnerabilities around knowledge and provide mechanisms to alleviate the traditional heavy reliance on personal networks and the memory-banks of staff for recall. Without a solid, self-service oriented, knowledge management environment and ecosystem, every crisis, every ask, every deep dive can turn into a herculean effort. The traditional approach or default response without these tools is to ask individuals involved, which is highly inefficient and disruptive; and sometimes impossible if the staff are no longer with the WBG. Enhancing and scaling up in house efforts to create this IFC knowledge management environment is at the core of the IFC KMS. Time and continued effort and emphasis from staff across the Corporation is essential to leverage and implement is required to implement best practices from across both private and public sectors. Leveraging Expertise Through Knowledge – Examples from Across IFC As was highlighted in the main document of this paper, while IFC is aware of the continued efforts necessary to fully realize the valuable knowledge available across the Corporation, IFC is not starting from zero. Across the Corporation there are a number of evident examples of excellence and best practice within individual IFC business units or departments. These examples are using new and innovative ideas to ensure that essential knowledge is disseminated across the department, made available more broadly across IFC, or to external audiences. The examples articulated across the main document and in Annex 1 illustrate a variety of areas in which IFC has developed and is disseminating knowledge pieces and where IFC and the Bank are leveraging the depth of knowledge from across the WBG to enable and enhance its impact on the ground. These include the IFC’s thought leadership publications, podcasts, climate or blended finance papers, CPSDs and others. Below are additional examples from across IFC that illustrate the value and importance of best practice knowledge management. 61 Corporate Special Operations (CSO) The Knowledge Management (KM) Team within CSO plays a pivotal role in bridging the knowledge gap by disseminating project-specific, regional, sectoral and thematic lessons learned and insights to improve investment decision making and execution at IFC. What sets these efforts apart is the systemic and systematic approach to continuously harvest tacit and explicit knowledge with a long-term view of mainstreaming knowledge-sharing for all staff. The CSO KM program (hereafter the program) is recalibrated annually to align with IFC’s strategic knowledge needs. Through the program, the team has launched a state-of-the-art suite of knowledge and information tools, and importantly, successfully embedded knowledge functions within its core operations. Knowledge sharing behaviors have now been inculcated among its staff and this is contributing to IFC’s goal of improved risk management capabilities. The program’s knowledge management process centers on closing the loop on lessons and experience from distressed situations, thereby helping improve the structuring of new deals as well as overall effectiveness of portfolio management. The team has recognized the institutional value that can be created from capturing and disseminating the CSO lessons both within and outside of the Corporation. To accomplish this, the team launched an integrated lesson learned capturing tool. With integrated taxonomy and metadata fields, this tool allows CSO Investment Officers to capture lessons learned in a fast and seamless manner either at their desk or on the go while details are still fresh. Taxonomy fields, such as risk factors and mitigation strategies, enable IFC investment staff to easily search and retrieve knowledge for comparable situations, leading to more informed decision making. CSO’s lessons help to preserve IFC’s financial sustainability, protect the development goals of IFC’s investments and safeguard IFC’s brand in the marketplace. With more than 30-35 new resolved projects every year, CSO’s Lessons Learned library, which currently has more than 210 projects, continues to grow rapidly to (i) become a key resource for investment teams, (ii) allow for collaboration across organizational silos and (iii) help de-stigmatize failure by emphasizing that mistakes are learning opportunities. Investment & Credit Risk Department The objective of the knowledge and learning program within the Investment & Credit Risk department is to build technical investment skills and transfer knowledge on investment and credit risk issues to IFC staff when they require it. The department has an expanding portfolio of online, on-demand and just-in-time products to extend learning beyond the classroom to on-the-job competence. The core knowledge and learning program consists of a core, progressive investment curriculum and specialized courses that build investment staff skills and capability in alignment with the IFC 3.0 business strategy and the 2030 goals. The learning by doing component of the program launched in 2013 has rotated 30 investment staff on a 12-month assignment into the department. The program is aimed at high-performing investment officers and provides the opportunity to work closely with a seasoned Chief Credit 62 Officer and to become members of IFC’s Credit Officer team and experience significant professional growth through improved investment and credit skills leading to a deeper understanding of IFC investment business. The program is designed to strengthen IFC’s credit culture by (a) allowing investment staff a “behind the scenes” understanding of the investment and credit review function, (b) fostering stronger relationships and connections between the Credit department and investment staff, and (c) transforming participants into ambassadors for a strong credit culture upon their return into the investment function. Climate Thought Leadership In the climate area, IFC provides thought leadership through reports and conferences such as its Climate Investment Opportunity Reports that identify and quantify investment opportunities across emerging markets in meeting their Nationally Determined Contributions targets on climate. These reports cover a range of sectors and articulate the business case and global best practice in creating markets for private sector investment in climate business. In addition, IFC hosts its annual Climate Business Forum which brings together business leaders to drive climate business across emerging markets. IFC also hosts the ClimateBiz podcast which features CEOs and innovators to share why and how they are moving forward in climate business. IFC embodies knowledge in products and tools such as the EDGE Green Buildings Market Transformation Program-which includes three levels of certification including Zero Carbon Buildings-and the Buildings Resiliency Index. IFC was the first MDB to develop a private sector- oriented approach for climate-smart agriculture and developed a food losses calculator which can estimate the greenhouse gas footprint of wasting a ton of food for over 34 crops in 117 EM countries. IFC also provides technical assistance to local banks in building their green finance business and played a crucial role in building the green bond market, including investing in client bonds and launching two platforms (Amundi’s EGO and Real Economy Green Investment Opportunity Global Emerging Market (REGIO)) that included investment and advisory services. Gender and Knowledge On gender, IFC launched the Moving Toward Gender Balance report outlining for the first time the gender composition and fund performance for emerging markets. This was followed by the issuance of a gender-smart investing guide most recently enabling funds to integrate a gender lens. These reports have led to new approaches on gender lens investing among commercial investors and development finance organizations. Another example is IFC’s research work on women centric ride-hailing and insurance approaches, which inspired the respective industries to re-design their business practices and policies to include more women as both providers and users of transport and insurance services based on a strong business case. To further boost 63 knowledge creation and take-up, IFC regularly convenes country and regional gender focused private sector peer learning platforms, which are time-bound and commitment driven. These platforms create an enormous amount of knowledge as demonstrated in the most recent impact report in Sri Lanka. Fragility, Conflict and Violence Addressing the drivers and impact of fragility, conflict, violence (FCV) is a key priority for the WBG and its development partners. Through decades of experience of operating in fragile and conflict-affected states, IFC has acquired in-depth understanding of the challenges and opportunities of working in these markets and has developed a set of specialized tools and approaches to enable further engagement. IFC utilizes this knowledge and these lessons learned to contribute to the growing body of knowledge on private sector development in FCV countries and to provide guidance to help other investors successfully engage in these markets. The lessons learned from IFC’s experience in FCV countries provided a foundation for the development of the WBG Fragility, Conflict and Violence Strategy, endorsed by the Board in 2020, which places strong emphasis on the role of the private sector in FCV countries. Other recent externally-focused publications include Generating Private Investment in Fragile and Conflict-Affected Areas, EM Compass Note: Impacts of COVID-19 on the Private Sector in Fragile and Conflict-Affected Situations and “Private Sector Initiatives in Forced Displacement Contexts: Constraints and Opportunities for a Market-based Approach,” a background paper for a forthcoming UNHCR flagship State of World’s Forcibly Displaced 2021 report. IFC also leverages its knowledge and experience in FCV by co-leading the organization of the DFI Fragility Forum in Oxford—an annual conference that convenes 27 DFIs to exchange knowledge and promote collaboration with the goal to catalyze greater private investment to fragile states. Sector Economics & Development Impact: Subsequent to the creation of the Economics Vice-Presidency (CEDVP) in IFC in 2017, the range and scope of analytical knowledge pieces to support private sector development and guide IFC operations has been significantly ramped up. Several sector-focused knowledge notes, aimed to promote the understanding of trends and help identify areas of opportunity for IFC and its clients, have recently been produced - including e-Conomy, undertaken in collaboration with Google on the sizing of the digital economy in Africa and COVID-19’s impact on pension funds in Africa, as well as others assessing the impact of COVID-19 in infrastructure sectors such as power, transportation, and water. In addition, there continues to be a focus on targeted knowledge publications to support the articulation and implementation of IFC’s approach to market creation – the upcoming IFC flagship on power markets, with the accompanying database tool, (https://www.worldbank.org/en/who-we-are/ifc/power-markets-database), will allow operational teams to assess market stage development in the sector. Other examples include research on affordable housing, economic impact of private equity investments, analysis of the global market for impact investing, and on-going work on the impact of trade 64 finance. Such research across different sectors and themes supports AIMM assessments, economic modeling and provides key insights to design new operations. A key emphasis for IFC is that of both the generation and application of knowledge. CEDVP is spearheading the use of enhanced economic modeling approaches in assessing the jobs and economic growth impact of IFC investments, and to inform its CPSDs. The AIMM visualization tool (http://w0lxpjekins05.worldbank.org:3939/connect/#/apps/681/access), in addition to providing evidence that informs IFC projects’ ex-ante development impact assessments, can be used to undertake development gaps and contextual analysis for upstream and strategy purposes, as well as to communicate with development partners, donors and stakeholders about development challenges that specific projects and programs are designed to address. Return to text. 65