2017 ANNUAL REPORT Influence through EVALUATION 2017 ANNUAL REPORT 1 About IEG How IEG Contributes to the Work of the World Bank Group The Independent Evaluation Group (IEG) is an independent unit within the World Bank Group. It reports directly to the IEG seeks to accelerate the World Bank Group’s World Bank Group’s Boards of Executive Directors, which development effectiveness through independent evaluation. oversee IEG’s work through the Committee on Development We aim to deliver Effectiveness. IEG is charged with evaluating the activities of the World Bank (the International Bank for Reconstruction • independent, credible, and useful evaluations and Development and the International Development Association), the International Finance Corporation (IFC), • impartiality through the employment of rigorous methods and the Multilateral Investment Guarantee Agency (MIGA). that promote triangulation of findings from a cross-section of sources The goal of IEG’s evaluation work is to improve the World Bank Group’s development effectiveness. IEG • new insights through the choice of what we evaluate; evaluations influence the World Bank Group’s ability to how we formulate evaluation questions; and which achieve development outcomes globally and with its methodologies we use, including the use of the latest member countries by providing impartial, evidence-based technology assessments and lessons on drivers of success and failure. • learning through outreach activities that target our key IEG shares these insights through a variety of channels stakeholders, including the Boards of Executive Directors, and engages stakeholders to ensure that lessons are management and staff of the World Bank Group, and the learned and necessary improvements are made. global development and evaluation communities All evaluations discussed in this report are available on IEG’s website: http://ieg.worldbank.org. • evaluation capacity development for World Bank Group staff, clients, and partners. 2 INDEPENDENT EVALUATION GROUP contents Message from the Director-General 2 Influence through Evaluation 6 Influence through Outreach and Learning 17 Staffing and Development 26 The Road Ahead 31 Appendixes 36 MESSAGE from the Director-General Fiscal year (FY)17 has been exciting for us. Development (IBRD), the International Development Association (IDA), the International Finance Corporation (IFC), and the Multilateral In line with our long-term agenda, we have delivered timely, highly Investment Guarantee Agency (MIGA)—adjust their programs relevant evaluations that speak to the current issues. For example, to ensure ultimate success. We also completed two evaluations our evaluations of the World Bank Group’s new country engagement on delivering services to the poor and simultaneously stimulated model and of the relatively new Program-for-Results lending discussions about the assessment of service delivery and behavior instrument informed course corrections for the management of change. For our strategic engagement area on environmental the World Bank Group institutions. sustainability, we put together a synthesis paper of all our findings on resilience to systemic shocks. Likewise, we have influenced debates in our three strategic engagement areas: inclusive and sustainable economic growth, During the year, we enhanced our investments in methodology. investing in people, and fostering resilience to global shocks and With the systematic involvement of our new methods advisor, we threats. We completed an important assessment of how well have seen a greater range of evaluation methods used in our major the World Bank Group is positioned to attain its goal of boosting reports. These investments will further mature in the coming years shared prosperity. The results should help the World Bank Group and be the platform on which IEG can start testing new methods institutions—the International Bank for Reconstruction and in the future. 2 INDEPENDENT EVALUATION GROUP A major pillar of our strategy, outreach (communicating to the right greater understanding, learning, and change within the World Bank people in the right format at the right time), continues to strengthen, Group. Both pilots will be reviewed to assess whether they achieve as evidenced by our increased social media presence, growing greater uptake of evaluation findings and recommendations. subscriber base, and number of report downloads. In addition Internally, we updated IEG’s mandate, worked on developing to sharing findings from evaluations, we use outreach to spark a World Bank Group–wide evaluation framework, and launched discussions among the international evaluation community. In the the IEG Academy as a resource to invest in systematic staff past 12 months, we have run several blog series that attracted a lot development to enhance skills, knowledge, and opportunities of attention, particularly “ReThinking Evaluation” and a collection for professional growth. on program theory. We intend to continue with these series with topics such as ICT for evaluation. Caroline Heider, In response to demands to increase learning and subsequent Director-General, Evaluation change from our evaluations, we also piloted new ways of writing recommendations and new learning engagements—on-demand interventions that draw on existing evaluation evidence to promote 2017 ANNUAL REPORT 3 FY17 Highlights project-level assessments Project performance assessment reports 63 57 (Target versus Actual) evaluations 5 5 Sector and Thematic Evaluations 4 33 Corporate and Process Evaluations project validations 0 00 Country-Focused Evaluations Project validations (Target versus Actual) 475 431 (Target versus Actual) 4 INDEPENDENT EVALUATION GROUP resources expenditures expenditures 35 35 140 30 30 130 25 25 127 128 number of staff 34.1 34 33.6 34.6 US$, millions US$, millions 20 20 120 119 15 15 117 114 10 Target Actual 10 Target Actual 110 5 5 100 • 2013 • 2014 • 2015 • 2016 • 2017 0 0 FY16 FY17 Year knowledge, learning, and dissemination 48 Number of blogs published Number of learning events 58 Number of report downloads 59,625 150,000 Number of unique website visitors in FY17 Number of Access to Information requests received and delivered 67 2017 ANNUAL REPORT 5 Influence through Evaluation An Overview of IEG’s Evaluation Activities During the Year IN LINE WITH ITS MANDATE to evaluate the activities of the World Bank Group, the Independent Evaluation Group (IEG) ensures that its evaluations are aligned with the World Bank Group’s strategic priorities and Forward Look. Our overall work program hinges on three Strategic Engagement Areas: inclusive and sustainable economic growth, investing in people, and fostering resilience to global shocks and threats. Within IEG, the strategic engagement areas provide a better way to coordinate decisions about what and when to evaluate. They also bring a more selective, strategic approach within the organization by ensuring that evaluations link to the most pressing concerns. A results framework underlies the strategic engagement areas, which helps provide a clear line of sight between IEG’s evaluations and its overall mandate. 6 INDEPENDENT EVALUATION GROUP 2017 ANNUAL REPORT 7 Helping the World Bank Group Achieve Its Twin Goals Ef fectively IEG Results Framework OBJECTIVE ONE: What Works and Why? OBJECTIVE TWO: Real-Time Learning Inclusive and Investing in Fostering Helping Clients Solutions Bank Partnerships Sustainable People Resilience to Tackle the Economic Global Shocks Most Important Growth and Threats Development Challenges Strategic Engagement Areas Strategic Pillars IEG Evaluations 8 INDEPENDENT EVALUATION GROUP At the start of the year, IEG identified four priorities: has made a significant effort to incorporate the shared prosperity goal into its various products and services, and across Regions, 1. to deepen the influence of IEG’s work by more closely aligning Global Practices, and World Bank Group institutions. it with the World Bank Group’s strategic priorities; IEG’s findings show that World Bank Group staff face significant 2. to improve the process of engaging with the management difficulties in obtaining timely and good-quality data for performing of the World Bank Group institutions to support greater detailed shared prosperity analysis. Sixty percent of Systematic understanding of our findings and increase uptake of our Country Diagnostic (SCD) reports prepared by World Bank staff recommendations; during the evaluation review period cited low-quality data as a 3. to enhance the quality and credibility of our work; and constraint. 4. to improve learning and knowledge sharing. Three major evaluations completed during the year explored how successfully the World Bank Group supported the creation of an environment for sustainable growth. Significant progress has been made in all four areas. During the year, IEG delivered eight major evaluations and several other products Education linked to the strategic engagement areas. In Higher Education for Development: An Evaluation of the World Bank Group’s Support, IEG reported that most World Bank Group Inclusive and Sustainable Economic Growth higher education projects from 2003 to 2016 were concentrated Under this strategic engagement area, IEG evaluates the extent to in middle-income countries. Among its recommendations, IEG which World Bank Group projects generate growth that creates jobs called on the World Bank Group to carry out a strategic review of and is inclusive, resilient, and sustainable. This area aligns with the its support for higher education and make tactical choices based World Bank Group’s twin goals of eradicating extreme poverty and on overall trends in the sector. Highlighted emerging issues include building shared prosperity through reducing inequality. access, retention, quality, equity, and employability. IEG also encouraged greater coordination within the World Bank Group Shared Prosperity and between it and other stakeholders, including employers, In FY17, IEG evaluated how well the World Bank Group has other donors, and the private and public sectors. incorporated its shared prosperity goal in its knowledge, strategy, and lending work. The evaluation found that the World Bank Group 2017 ANNUAL REPORT 9 Rural Nonfarm Economy Investing in People Another IEG report examined how much the World Bank Group Over the years, IEG has extensively evaluated different dimensions has contributed to the creation of sustainable nonfarm income- of this strategic engagement area with the goal of enhancing the generating opportunities for the rural poor. Most of the world’s poor Bank Group’s effectiveness in serving the poor. To further improve its live in rural areas, and rural nonfarm activities are an important analysis of such projects, IEG developed two new frameworks: one part of their livelihoods. From 2004 to 2014, the World Bank Group for evaluating service delivery in sector evaluations and the other for implemented 1,141 projects, valued at $46.5 billion, that included evaluating behavior change in development operations. Both were support for rural nonfarm economic activities. IEG’s evaluation piloted through evaluations in FY17, that looked at the water and revealed that the World Bank Group’s effectiveness is constrained sanitation services and urban transport projects. by the lack of a well-articulated approach and the absence of a mechanism to coordinate its activities across different units and sectors. IEG WO R K ING PA PE R 2016 / N O.3 Introducing Tax Mobilization a Framework for Evaluating On the institutional governance front, IEG reviewed the World Service Delivery in Sector Evaluations: Urban Transport, Bank’s support for tax administration and mobilization. The study Water and Sanitation, and Nutrition provided findings and lessons on how to strengthen tax policy and administration. It notes that the trade-offs between increasing government revenue and improving tax system efficiency on the IEG WO R K ING PA PE R 2016 / N O.2 one hand and increasing equity on the other hand would be a clear Evaluating Behavior Change in entry point for World Bank involvement, especially under the shared International Development prosperity agenda. An upcoming evaluation in FY19 on public Operations: A New Framework finance for development will build on this work. Other planned evaluations in FY18 and FY19 will look at several new areas, including trade facilitation, creating new markets, fostering regional integration, and renewable energy. 10 INDEPENDENT EVALUATION GROUP Water and Sanitation The World Bank Group has made IEG’s evaluation of the World Bank Group’s support for water a significant effort to incorporate and sanitation services highlighted a number of opportunities, the shared prosperity goal including addressing disparities in access for low-income countries, particularly in Sub-Saharan Africa, South Asia, East Asia and Pacific, into its various products and and Latin America and the Caribbean. IEG also identified the need services, and across Regions, for a robust evidence base for service delivery. The lack of data on Global Practices, and World service delivery is pervasive across World Bank client countries, Bank Group institutions. hindering the formulation of actionable policies, performance targets, incentives for performance, and accountability for results. Urban Transport During the 10-year period from FY07 to FY16, the World Bank using a comprehensive approach that combines upstream (policy Group invested $25 billion in urban transport projects. The World and sector framework) and downstream (operational) measures. Bank Group’s urban transport portfolio provided five main types of Furthermore, weak institutional capacity and coordination remains support: urban roads, conventional bus, bus rapid transit, metro or a critical challenge in the urban transport sector. urban rail, and upstream support focusing on policy and the sectoral framework. Other Evaluations Other evaluations completed under this strategic engagement IEG’s evaluation of the World Bank Group’s support for urban area included learning products on gender in community-driven transport determined that, overall, the World Bank Group has been development, reliable and affordable off-grid electricity services for effective in supporting improved service quality and increased the poor, and a synthesis of the World Bank Group’s support for access, but approaches based on increasing infrastructure capacity public-private partnerships in the health sector. are not balanced with those based on demand management. Although the World Bank Group has achieved localized Upcoming evaluations in FY18 and FY19 will look at essential health environmental mitigation benefits along key urban transit corridors care services, forced displacements, and decentralization and or systems, broader environmental benefits could be achieved by effectiveness in subnational governments. 2017 ANNUAL REPORT 11 Rapid Turnaround Fostering Resilience to Global Shocks and Learning Engagements: Threats Climate Change During FY17, IEG completed a synthesis report compiling key lessons from recent IEG evaluations applicable or related to crisis response and building resilience to systemic shocks. The synthesis Climate change mitigation and drew on past evaluations of the World Bank Group’s responses to adaptation. At the request of the World the food crisis of 2007–08, the global financial crisis of 2008–09, natural disasters, climate change, and pandemics. The report Bank Group’s Climate Change Cross- highlighted the multidimensional nature of resilience in all sectors. Cutting Solution Area, IEG prepared a To better understand the just-in-time lessons-focused synthesis of different dimensions, IEG key findings on the World Bank Group’s is planning to ramp up its Crisis Response and Resilience evaluations work in this support for climate change mitigation and to Systemic Shocks Lessons from IEG Evaluations area. Currently, considerable adaptation from IEG evaluations in the evaluative gaps exist on the topic of the environment, past five years. with the last major evaluation having been completed in 2012. To address this, IEG further engaged with the climate evaluations in FY18 will cover change team to review the screening air pollution, renewable energy, process for climate and disaster risk that and carbon finance. is being rolled out across the World Bank Group’s operations. 12 INDEPENDENT EVALUATION GROUP Corporate Evaluations and Improving Project Outcome ratings for IFC investment projects continued to trend Development Outcomes downward from the prior review period—success rates were down by 3 percentage points to 54 percent for 229 projects evaluated During FY17, several IEG evaluations focused on the World Bank in FY13–15. Group’s corporate processes with the goal of providing early feedback on recent reforms, products, and processes. MIGA projects’ success rate (six-year rolling average)—excluding from the total those projects rated no opinion possible—remained IEG’s flagship report, Results and Performance of the World Bank steady from the prior review period at 61 percent. The success Group 2016, included a thematic chapter on how the World Bank rate when including projects rated no opinion possible in the total Group is managing its operations for better results. The World Bank dropped by 4 percent. Group has a long history of results management and has made good progress in creating a structure and process for measuring Monitoring Management Actions in Response results. IEG’s evaluation recommended that to build on this progress, to IEG Evaluations foundations of results measurement need to be strengthened and The management action record tracks the management of a culture of evidence-based adaptive management and learning the World Bank Group institution’s actions in response to IEG needs to be instilled. The collective challenges of data availability, recommendations in corporate, sector, and thematic evaluations. an inconsistent focus on beneficiary-level outcomes, and the overall For FY17, IEG tracked agreed actions for 150 recommendations quality of results frameworks impede effective results measurement. drawn from 26 evaluations produced between FY12 and This situation warrants attention. Furthermore, an organizational FY15. Although progress continued across all the tracked need for matching staff capacity to the requirements and for recommendations, a significant gap remains between IEG’s strengthening staff skills, competencies, and career paths continues. progress ratings and the World Bank Group’s own assessments of In terms of achieving project development outcomes, the report how much progress has been made. In some cases, this reflects found that the World Bank’s project outcome ratings remained recommendations where the Management of the World Bank Group essentially stable for projects closed during FY13–15. The share of institutions only partially agreed with IEG’s recommendations or projects with an outcome rating of moderately satisfactory or above where they argued that IEG’s recommendations were not feasible (MS+) was 72 percent for FY13–15 and 70 percent for FY10–12, or that they lack the budgets and time to deliver. both below the corporate target of 75 percent by FY17. 2017 ANNUAL REPORT 13 Early-Stage Assessment of New World Bank Group Other Reports Client Delivery Models Other corporate reports evaluated the World Bank Group’s support During the year, IEG completed its early-stage evaluation of for data and statistical capacity, lessons from implementing the World Bank Group’s two new country engagement models, policy-based guarantees, and a review of the World Bank Group’s the Systematic Country Diagnostics (SCD) and the Country experience with implementing joint projects, involving the four large Partnership Framework (CPF). member institutions: IBRD, IDA, IFC, and MIGA. Overall, IEG’s findings found that the new model is working well The next Results and Performance of the World Bank Group report and has resulted in greater IFC and MIGA participation and improved will feature a thematic chapter on environmental management. Other coordination across the different World Bank Group entities reports in FY18 and FY19 will look at how the World Bank Group compared with the country assistance strategy approach used engages citizens, its convening power, IFC’s client engagement previously. The SCDs contributed to enhancing the quality of World model, and early lessons on stimulating knowledge flow and Bank Group country engagements and has been well received collaboration under the World Bank’s new operating model. inside and outside the institutions. IEG’s report found that the SCDs have helped the World Bank Group and its clients identify knowledge and data gaps, which can help improve the evidence base for future policy making. Aligning CPF program priorities to the constraints and priorities identified in SCDs has been challenging. Balancing government priorities with the World Bank Group’s own comparative advantage remains an issue. IEG also found that CPFs did not always effectively address the knowledge and data gaps identified in the SCDs. Although the definition of results chains in CPFs has improved, it remains a work in progress. 14 INDEPENDENT EVALUATION GROUP 2017 ANNUAL REPORT 15 16 INDEPENDENT EVALUATION GROUP Influence through Outreach and Learning OVER THE YEAR, IEG invested significantly in enhancing its knowledge, learning, and communications with the goal of engaging the right audiences with the right channels at the right time in order to increase the use of IEG evaluative insights by key stakeholders. Outreach and Communications Work on revamping the website was completed in FY17 and was very well received. The website remains IEG’s most effective outreach vehicle, attracting over 150,000 unique visitors during the year. Report downloads from the website increased to 59,625, up 86 percent from the previous year. 2017 ANNUAL REPORT 17 IEG Online Dissemination & Outreach—FY17 Highlights Social Media Stats 13,814 followers 1,573 followers Report downloads (up 86%) 59,625 (up 10%) (up 106%) 80,866 followers 385 subscribers 221,357 website (up 5%) (up 44%) sessions (up 19%) 62,446 blog views 150,280 website (down 14%) users (up 16%) Most Read Blogs Rethinking Evaluation: Efficiency, Efficiency, Efficiency 3,729 Rethinking Evaluation: Have We Had Enough R/E/E/I/S? 3,672 Learning from Evaluation: How Can We Stay at the Top of the Game? 3,406 Using Theories of Change in International Development 2,412 What Makes Good Program Theory in International Development? 2,389 18 INDEPENDENT EVALUATION GROUP most downloaded reports Public Disclosure Authorized LE A R N ING PR O DUC T IEG WO R K ING PA PE R INDEPENDENT EVALUATION GROUP 2016 / N O.2 Public Disclosure Authorized Women’s Evaluating Empowerment Behavior Work Program and Budget (FY18) in Rural Change in and Indicative Plan (FY19–20) Community- International Driven Development Development Operations: evaluations Public Disclosure Authorized Program-for-Results Projects A New Framework An Early-Stage Assessment of the Process and Effects of a New Lending Instrument that matter 20 16 a n n u a l r e p o r t Public Disclosure Authorized June 2, 2017 2,142 1,619 1,590 1,450 1,344 Women’s Empowerment Program- for-Results Evaluating Behavior 2016 Annual Report 2017 Work Program in Rural Community Change in International Development Projects Development Operations World Bank Group LE A R N ING PR O DUC T Engagement in LE A R N ING PR O DUC T Situations of Fragility, Conflict, and Violence The Role Learning from of Political IDA Experience: Economy Le s s ons f r om IEG Eva lu ations Analysis in LEBANON AND JORDAN OV E RV IE W Development Policy THE CASE OF THE World Bank Group Engagement The World Bank Group’s Support in Situations of Fragility, Conflict, to Capital Market Development Operations REFUGEE CRISIS and Violence 1,300 1,173 1,066 915 881 Situations of Fragility, WBG Support to Political Economy Lebanon & Learning from Conflict, and Violence Capital Markets Analysis in DPOs Jordan Refugee IDA Experience Shock 2017 ANNUAL REPORT 19 Renewed investments in other channels also yielded strong results. Our monthly newsletter now reaches over 35,000 subscribers, and our social media channels collectively have an audience of over 100,000 followers. The IEG weekly blog continued to perform very well. One of the year’s highlights was a special blog series on rethinking evaluation, which attracted over 12,000 readers and generated over 100 comments. During the year, IEG also hosted 16 dissemination events, including a stakeholder event in Manila on fragility and conflict that was cohosted by the World Bank and the Asian Development Bank. In addition, IEG hosted two successful seminars at the World Bank and International Monetary Fund Annual and Spring Meetings. Through these events, IEG directly engaged over 2,000 participants, including high-level stakeholders, representing governments, civil society, and other international development organizations. Left: Grenada Prime Minister, Keith Mitchell, speaking at IEG Spring Meetings Event Knowledge and Learning can request funding to support deeper analysis of issues IEG has evaluated in the past. Funds can also be used to support learning In line with our goal to improve learning and knowledge activities, such as workshops and seminars, provided they draw management, IEG implemented several initiatives during the year. on existing IEG evaluations. Key among them was a new pilot initiative to increase the use of IEG’s evaluative knowledge among World Bank Group staff. Under The initial response was overwhelmingly positive, resulting in the this new Rapid Results Learning Fund, World Bank Group staff funding of 10 learning engagements. Learning Engagements approved in FY17 Topics Defining and Measuring Project Outcomes Service Delivery: Deriving and Sharing Lessons Learned from the Evaluation Evidence, & Socializing IEG’s SD Evaluation Framework Learning from the Past, Building for the Future: Sharing Lessons Learned from Evaluative Evidence, Combining Evidence with New Approaches, and Socializing IEG’s New Behavior Change Tool (CRI2SP) for Future Evaluations and World Bank Operations Learning from the World Bank’s Support of Student Assessment: Stakeholders Perceptions in Brazil, Laos, and Kyrgyz Republic Developing Resilience Building Results Frameworks and Indicators: Sharing Lessons from IEG’s Evaluative Evidence in Sustainable Development Building Resilience—A Synthesis of Recent IEG Experience IEGSD Collaborative Exchanges on Performance Measurement and Monitoring at the Transport, Agriculture, and Water Weeks Addressing Situations of Fragility, Conflict, and Violence in Middle-income Countries: “What to do and How to do it” with World Bank Operations Learning from the World Bank’s Support of Nutrition in Senegal Guidance for the New “Brief” ICR Process, Developed Using a Participatory, Evaluative Approach 2017 ANNUAL REPORT 21 IEG sponsored 58 learning events, targeting staff across the World A second important initiative involved a partnership with the Bank Group. These included the 2017 Getting to Results Differently World Bank Group’s knowledge management unit, under which Forum, an annual learning event for monitoring and evaluation IEG evidence is included in the knowledge packages provided to professionals in the World Bank Group. This year’s event attracted operational staff at key steps in the project cycle. 387 participants from across the World Bank Group. IEG staff led 13 of the 33 technical sessions at the event. Feedback shows that Overall Stakeholder Perceptions of IEG IEG’s events were well received by stakeholders and highly rated for Remain Positive content, learning outcomes, and overall delivery quality. One of the ways IEG tracks stakeholder perceptions of its work is In addition, IEG implemented several improvements to its internal through the annual client survey. Over 2,000 respondents completed systems to facilitate better knowledge flows within IEG as well as the 2017 survey, which was carried out by an independent research with the rest of the World Bank Group. A new portal for investment firm, ORC International. Overall, results from the 2017 survey suggest completion report reviews was finalized, which will help automate IEG continues to be an influential voice in the World Bank Group and workflows and facilitate easier integration with other World Bank the broader development community. Group knowledge repositories. A majority of World Bank Group Board members (96 percent) say At a corporate level, IEG was part of two World Bank Group–wide they use IEG products, compared with 91 percent and 75 percent efforts to strengthen the use of evaluation in informing World Bank of external stakeholders and World Bank Group staff, respectively. Group projects and operations. One was the preparation of a new Overall, 85 percent of Board members and 84 percent of external World Bank Group evaluation framework. This work was started stakeholders believe that IEG’s work is highly relevant to the World during the year and will set out the World Bank Group’s vision Bank Group’s mission. for conducting and utilizing self- and independent evaluation to Among World Bank Group staff, overall satisfaction with the quality enhance the institutions’ capacity to manage for results, incorporate of IEG products increased from 47 percent to 57 percent—a result lessons learned into new operations and strategies, be more agile that reinforces the importance of our ongoing efforts to improve the and responsive to clients’ needs for higher impact, and set new quality, rigor, and packaging of our evaluation findings. standards for evaluation across the institutions. 22 INDEPENDENT EVALUATION GROUP Evaluation Capacity Development IEG’s mandate includes helping World Bank client countries strengthen their monitoring and evaluation capacity. IEG’s work in this area is delivered primarily through two partnerships that involve other development partners and stakeholders. These are the Centers for Learning on Evaluation and Results (CLEAR) and the International Program for Development Evaluation Training (IPDET). The Centers for Learning on Evaluation and Results Initiative The CLEAR Initiative is a global program that brings together academic institutions and donor partners to contribute to the use of evidence in decision making in developing countries. Now in its fifth year, the CLEAR initiative has six regional centers covering Anglophone Africa, Francophone Africa, South Asia, East Asia, Latin America and the Caribbean, and Brazil. In 2016, the CLEAR centers served nearly 13,500 individuals through activities targeted mainly at government, academic, and civil society clients. CLEAR’s secretariat is housed at IEG, with support coming from various partners including the African Development Bank, the Asian Development Bank, the Inter-American Development Bank, and the Rockefeller Foundation, as well as bilateral country donors such as the Australia, Belgium, Sweden, Switzerland, and the United Kingdom. 2017 ANNUAL REPORT 23 24 INDEPENDENT EVALUATION GROUP As the global hub, IEG manages fund allocations from donors to the The Evaluation Cooperation Group centers and helps provide strategic support in the form of business IEG attended the June 2017 Evaluation Cooperation Group plan development, curriculum design, and cross-center knowledge (ECG) meetings hosted by the International Fund for Agricultural sharing and learning. Development and participated in discussions related to (i) gender equality in evaluation, (ii) evaluation recommendations and In FY17, IEG and its partners helped the CLEAR centers adopt follow-up, (iii) enhancing self-evaluation systems, and (iv) evaluation more strategically selected service offerings and strengthen communities of practice. In each of these areas, IEG will continue to their business models. Donors also extended their support for share its knowledge and expertise with the broader ECG community. the CLEAR trust fund managed by IEG to 2021 and committed additional funding to support CLEAR’s work. Also, a new partnership In FY17 and FY18, IEG will continue to host the website for ECG and was formalized between CLEAR and the International Fund for administer a part-time consultant who manages ECG operations. Agricultural Development, through which CLEAR will receive a $3.5 million grant over three years to be used to design and offer a global monitoring and evaluation curriculum and certification framework. International Program for Development Evaluation Training In March 2017, Carleton University, IEG’s long-term partner in delivering IPDET decided to disengage from the program. As a result, the 2017 summer session of the program was canceled. Plans are under way to find a new academic partner to take on this important program and offer it in 2018. 2017 ANNUAL REPORT 25 Staffing and Development DURING THE PAST YEAR, IEG has continued efforts to strengthen its human resources, revamp its internal structures, and streamline resource allocation. Recruitments to fill vacant staff positions were completed for most senior management and operational roles. In line with World Bank Group practices, over the last five years IEG has reduced the number of senior-level positions, which has freed up resources and allowed us to invest the savings in our front-line staff—the evaluators and other professionals who carry out the bulk of IEG’s work. 26 INDEPENDENT EVALUATION GROUP 2017 ANNUAL REPORT 27 28 INDEPENDENT EVALUATION GROUP Enhancing Our Methods Establishment of the methods Establishment of the methods advisory function has helped IEG advisory function has helped IEG revamp its methodologies and the process for implementing revamp its methodologies and evaluations by ensuring rigorous checks and reviews at every the process for implementing stage of the evaluation cycle. The focus of this work has been on evaluations by ensuring rigorous strengthening our evaluation design and methods; piloting new and innovative evaluation techniques; ensuring consistent and efficient checks and reviews at every collection, use, and presentation of data; and supporting the stage of the evaluation cycle. development of IEG staff evaluation skills. IEG will continue to further strengthen our quality assurance framework to build systematic feedback loops throughout the evaluation process, optimize data collection, analyze and course- correct, and enhance efficiency and quality. IEG Staff Learning Budget Management Overall, IEG slightly exceeded its targets for FY17 by 2 percent. During the year, we launched the IEG Academy, which offers IEG In FY18, IEG will seek to stay on budget. staff a range of carefully designed learning tools. The academy includes courses (delivered face-to-face and online), just-in-time on-the-job coaching, and other resources to support IEG staff in strengthening key competencies. The academy curricula serve task team leaders, evaluators, portfolio analysts, GA–GD staff, and knowledge and communications staff. 2017 ANNUAL REPORT 29 30 INDEPENDENT EVALUATION GROUP The Road Ahead FOR THE COMING YEAR, IEG’s proposed work program is expected to continue building substantive knowledge in the three strategic engagement areas and contribute to enhancing learning and feedback loops. Our proposed evaluation topics strongly align with needs and knowledge gaps related to the World Bank Group’s strategy and the twin goals. In addition, IEG has taken account of the more recent shifts in the World Bank Group’s approach by aligning with the 2017 Forward Look, IDA priorities, IFC’s new strategy (IFC 3.0), and the new cascade approach for maximizing finance for development. 2017 ANNUAL REPORT 31 A New Results Framework a project closes or to look at themes and sectors extending over 10–15 years, meso evaluations will have much earlier entry points In FY17, IEG completed work on reviewing and strengthening its to enable midterm course corrections. results framework in consultation with the Board and management of the World Bank Group institutions. The new framework aligns In addition, IEG will resume its country program evaluations, starting closely with IEG’s mandate and intended outcomes and with the with Mexico and Rwanda in FY18. World Bank Group’s overall corporate scorecard. By creating a clear line of sight between IEG’s mandate and its outcomes, the new As in previous years, IEG will conduct project-level evaluations and framework will allow IEG to better demonstrate its influence and validations, with the goal of assessing project-level performance and contribution. extracting lessons to inform future project design. Emphasis will be placed on better sharing these lessons with staff. Work on operationalizing the framework will start in FY18. On the knowledge, learning, and communications front, IEG will continue to invest in scaling up its programs. Reaching more The External Review stakeholders more effectively, more systematically, and with timely IEG will continue implementing the recommendations of the IEG and relevant insights remains a priority. We will focus on World Bank External Review, which was conducted in 2015 at the request of the Group staff and management as our primary target audience for Board. Specifically, we will continue ongoing initiatives to enhance outreach in the coming year, with the goal of further increasing the the quality, relevance, usefulness, credibility, and influence of IEG’s use of IEG evidence to inform project and strategy design. IEG plans work, building on the progress made in the past two years. to continue supporting the Rapid Results Learning Fund, building on the huge success of the fund’s first year. Evaluation Priorities IEG will also work on finding a new partner to deliver its flagship Following extensive consultations with the Board and management, monitoring and evaluation training initiative, IPDET. In addition, work IEG has finalized a work program that outlines its planned on implementing the program’s revamped curriculum will continue in evaluations for the next three years. The proposed work program the hope of rolling it out as part of the FY18 offering. includes eight major evaluations per year and the introduction of a new product, the meso evaluation, which will be piloted in response The review of existing processes and quality controls will continue, to client demand for smaller, more focused, just-in-time evaluations. as will our broader program to strengthen evaluation methodologies Whereas traditional evaluations are generally commissioned after and promote learning from our evaluations through increased stakeholder engagement. 32 INDEPENDENT EVALUATION GROUP 2017 ANNUAL REPORT 33 FY18 Work Program Major Evaluations Other Evaluation Products Thematic and Sector Evaluations PPARs • Clean World for All • World Bank: 55 • Essential Health Care Services • IFC/MIGA: 8 • Carbon Finance Learning Engagements • Renewable Energy • TBD, based on client demand under • Forced Displacement the new IEG approach to learning Corporate and Process Evaluations Validation Products • Engaging Citizens Country Strategy Completion Report • IFC Client Engagement Model Validations Country-Focused Evaluations • 22 expected • Mexico Project Completion Report Validations • Rwanda • XPSR: 40% coverage (96 projects expected) Systematic Reviews and Impact Evaluations • PER: (20 expected) • n/a • ICRR: 100% coverage (250 projects expected) Results and Performance Report AAA/AS Completion Report Validations • RAP 2017: Sustainability in Operations • PCR: 51% coverage (87 projects expected) Meso Evaluations • Development Policy Operations in IDA countries • IFC Inclusive Business Model • IFC Asset Management Company 34 INDEPENDENT EVALUATION GROUP FY19 Work Program FY20 Work Program Major Evaluations Only (Indicative) Major Evaluations Only (Indicative) Thematic and Sector Evaluations Thematic and Sector Evaluations • Public Finance for Development • Managing Economic Transitions Country Cluster • Creating New Markets Program Evaluation • Fostering Regional Integration • Reaping Digital Dividends • Renewable Energy • Strengthening Bank Stability and Intermediation • Decentralization and Effectiveness of • Role of Cities in Growth and Development Sub-National Governments • Institutional Capacity Strengthening • Resilient Cities • Demographic and Population Issues • Environmental Degradation Corporate and Process Evaluations • WBG Convening Power Corporate and Process Evaluations • Process Evaluation of GPs/CCSAs • TBD Country-Focused Evaluations Country-Focused Evaluations • TBD • TBD Systematic Reviews and Impact Evaluations Systematic Reviews and Impact Evaluations • TBD • TBD Results and Performance Report Results and Performance Report • TBD • TBD 2017 ANNUAL REPORT 35 appendixes IEG Deliverables and Spending (number) Deliverables FY12 FY13 FY14 FY15 FY16 FY17 Planned Actual Percent Total Major Evaluations 8 10 10 7 9 11 8 89 Corporate and Process Evaluations 2 3 4 2 3 4 3 — Sector and Thematic Evaluations 4 5 4 4 2 5 5 — Country-Focused Program Evaluations 1 1 2 1 2 — — — Impact Evaluations 1 1 — — 2 2 — — Learning Products — — — — 6 — — — Learning Engagements — — — — — — 11 — Project Evaluations and Global Program Reviews Project Performance Assessment Reports 40 31 47 39 51 63 57 90 Global Program Reviews 1 2 2 — — — — — Total Validations 414 478 573 571 442 475 431 — Implementation Completion Reports 178 295 361 392 267 270 285 105 Country Assistance Strategy Completion Reports 20 19 22 13 22 20 18 — Project Completion Reports 130 76 87 72 47 96 54 — Expanded Project Supervision Reports 75 80 88 78 88 93 69 — MIGA Validations 11 8 14 16 18 11 14 — Total Spending ($, millions) 32.5 33.8 34.3 33.4 34.1 33.6 34.6 102.9 36 INDEPENDENT EVALUATION GROUP IEG Staffing (number) Staffing FY12 FY13 FY14 FY15 FY16 FY17 Total Staff 110 127 117 128 114 118 By Grade UC 3 2 — 1 2 1 GA-GD 21 23 22 23 23 24 GE 8 6 6 7 8 11 GF 19 22 25 21 27 22 GG 19 26 30 30 30 36 GH 19 22 20 18 17 20 GI and above 7 8 6 3 4 4 Other (ETT, ETC, UC) 14 18 8 12 3 — Photography Page 20, Grant Ellis / World Bank Page 30, Chor Sokunthea / World Bank Page 1, Flore de Preneuf / World Bank Page 23, © Stephan Gladieu / World Bank Page 33, Nugroho Nurdikiawan Sunjoyo / Page 3, Rhoda Baer World Bank Page 24, Curt Carnemark / World Bank Page 7, © Stephan Gladieu / World Bank Page 27, Graham Crouch / World Bank Design Page 15, Arne Hoel / World Bank Page 28, Scott Wallace / World Bank Crabtree + Company Page 16, © Dominic Chavez / World Bank 1818 H Street NW, Washington, DC 20433 http://ieg.worldbank.org ieg@worldbank.org 38 INDEPENDENT EVALUATION GROUP