99727 A N N UA L R E P O R T 20 15 deepening impact The Independent Evaluation Group (IEG) is an independent unit within the World Bank Group. It reports directly to the Boards of Executive Directors, which oversee IEG’s work through the Committee on Development Effectiveness. IEG is charged with evaluating the activities of the World Bank (the International Bank for Reconstruction and Development and the International Development Association), the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA). The goal of evaluation is to influence the World Bank Group’s ability to achieve development outcomes globally and with its partner countries by providing impartial, evidence-based assessments and lessons on drivers of success and failure. Sharing these insights through a variety of channels and engaging stakeholders in a discussion on follow-up actions ensure that lessons are learned and necessary improvements are made. All evaluations discussed in this report are available on IEG’s website: http://ieg.worldbank.org. IEG’s Strategic Goal Contents The Independent Evaluation Group (IEG) seeks Message from the Director-General, Evaluation 2 to accelerate World Bank Group development effectiveness through independent evaluation. Sharing Evaluation Insight, Facilitating Learning 4 We aim to deliver: • Independent, credible, and useful evaluations Systematically Turning Insight into Influence 12 •  Impartiality through the employment of rigorous Improving Our Own Work 18 methods that promote triangulation of findings from a cross-section of sources Appendixes 26 •  New insights through the choice of what we evaluate; how we formulate evaluation questions; and which methodologies we use, including the use of the latest technology • Learning through outreach activities that include the  Boards of Executive Directors, World Bank Group management and staff, and the development and evaluation communities • Evaluation capacity development for the World  Bank Group staff, clients, and partners. As the lead evaluation group in the development community, IEG continued to innovate and adjust to deepen our impact. It has been an extraordinarily busy and productive year for IEG. We had a bumper harvest of deliveries: more than 600 products in all, including seven major evaluations, 39 Project Performance Appraisal Reports, 536 reviews of Bank Group project self-evaluations, and 12 reviews of country operations. We reviewed and updated our two Message from evaluation capacity development programs—Regional Centers for Learning on Evaluation and Results and the International Program the Director-General, for Development Evaluation Training—and took over as chair for the Evaluation Evaluation Cooperation Group. We launched new work to improve the quality and credibility of IEG, including refining our methods and updating our quality assurance standards. And we enhanced our knowledge, infrastructure, learning, and communications work—including launching a new community of practice of data/ portfolio analysts, automating some workflows, and leveraging new communication formats and social media channels. 2 Independent Evaluation Group This year we upgraded the Management Action Record to increase The recently released external review of IEG by an independent our efficiency in following up with World Bank Group management panel will provide further impetus for our continued renewal. All this on our recommendations. We tested and developed new methods work and focus results in the best quality products that will enhance to ensure that our evaluations maintain cutting-edge quality. We our influence in the development world. And that is our ultimate improved our process for validating Bank Group self-evaluation goal: to accelerate development effectiveness through independent and ensured that our work is included in the Bank Group’s Open evaluation. Knowledge Repository. And we joined with other Bank Group staff It’s exciting to be involved in making strategic choices, focusing in a community of practice focusing on results measurement and on the development issues that matter most, and working hard evidence—the Results Measurement and Evidence Stream. to develop skills. We are lucky to work in this profession, which Also important, we committed to enhancing collaboration and provides the opportunity for development and learning as well synergy across IEG. This includes changing the ways IEG is as the opportunity to contribute to the dialogue on critical issues. organized, evaluation teams are staffed, and the work program is As we move further into this International Year of Evaluation, we will developed, even how we determine what evaluations we take on. see even more opportunities for collaboration and knowledge sharing This is truly IEG staff work at its best: identifying game changer from independent evaluation that will help the World Bank Group evaluations that will deepen evidence about World Bank Group succeed in meeting its twin goals. IEG has an important role to play contributions to development effectiveness. in making a difference, and our independence gives us a privileged vantage point from which to do so. Caroline Heider 2015 ANNUAL REPORT 3 2015 IS THE INTERNATIONAL YEAR OF EVALUATION IEG has partnered in this effort with evaluation offices at the United Nations and others in the development community to advocate and promote evaluation and evidence-based policy making. Why? Evaluation can—we believe—enable the development and support of better policies, allowing them to be implemented more effectively. Evaluation looks Sharing Evaluation Insight, at experience and distills what has worked, what has not, Facilitating Learning and why. IEG has adopted the tagline “What Works” to focus on these lessons and to enrich its contribution to development outcomes. This year’s annual report focuses on what has worked for IEG in helping it generate critical evidence to inform and improve the World Bank Group’s decision making and development outcomes. 4 Independent Evaluation Group Insights from Major Evaluations IN FY15, IEG delivered seven major approaches flexibly enables IEG to respond evaluations, 39 Project Performance more rapidly to the evolving business needs of Appraisal Reports (PPARs), a record- the Bank Group. This year’s major evaluations breaking 370 Implementation Completion addressed, among other things, inclusiveness and Results Report (ICR) Reviews, 166 and growth, contributing to the first objective validations of the International Finance of IEG’s Results Framework on deepening Corporation (IFC) and the Multilateral evidence about the results of Bank Group Investment Guarantee Agency (MIGA) programs and activities. self-evaluations, and 12 Completion and Our annual Results and Performance of the Learning Reviews of country operations. World Bank Group (RAP) review focused on In structuring the delivery of more than what the World Bank Group can learn from its 600 products, IEG’s work has focused on efforts to support the Millennium Development three pillars of the new World Bank Group Goals and how these lessons can inform Strategy: help clients tackle the most discussions around the ensuing Sustainable important development challenges; become Development Goals. The five lessons in the the Solutions Bank; and work in partnership. RAP encourage the Bank Group to maintain A results framework specific to IEG’s work a broad perspective on development and along the three pillars guides IEG’s choices evidence-based decision making, to integrate and activities. Working along the World sector best practices in its country model, Bank Group’s strategy allows IEG to address and to ensure clear results chains for its relevant development challenges and foster interventions. an internal culture that values and learns from independent evaluation. The report also points to a number of longstanding performance challenges in Bank IEG’s goal is to accelerate World Bank Group lending operations and demonstrates Group effectiveness through independent that some behavior and process factors evaluation. Selecting evaluation topics and 2015 ANNUAL REPORT 5 strongly influence Bank Group work quality. IEG’s work program is focused on World See the full report on IEG’s website: Bank Group contributions to the twin goals We have indeed benefited “ http://ieg.worldbank.org. of reducing extreme poverty and promoting immensely from [IEG’s] shared prosperity. The Poverty Focus of The FY14 analysis of how the World Bank Country Programs: Lessons from World thorough evaluation and the learns in operational work was followed Bank Experience found that the World this year by a more in-depth evaluation, Bank provides an important public good interaction with IEG and look Learning and Results in World Bank in supporting and reporting global poverty Operations: Part II. The evaluation feeds forward to our continued data and producing high-quality poverty directly into the World Bank Group’s internal diagnostics. It could do better, however, by discussions about its organizational reform, engagement with you when understanding the context and leveraging offering specific recommendations on resources from development partners the new policy framework is how to embed learning and knowledge in and other stakeholders. Specifically, the operations—beyond the structural changes evaluation notes that the Bank Group can approved.” of introducing Global Practices and Cross- continue to play a critical role in supporting Cutting Solutions Areas. The evaluation data and knowledge on how poverty is STEFAN G. KOEBERLE examines how to promote more systematic being reduced. Such data and diagnostics Operations Risk Management learning by staff and clients in World Bank World Bank Group, are necessary to inform interventions. lending. Having identified gaps and grey commenting on the continuing influence Recommendations and the full report can areas, the report suggests some midcourse of the FY14 evaluation The World Bank be found on IEG’s website. corrections, such as providing institutional and Public Procurement. incentives for learning, and provides a For children born into poverty, the equity- baseline against which changes can be enhancing impact of early childhood measured. interventions holds the promise of overcoming social disadvantages and breaking the intergenerational transmission Country and Global Program Lessons of poverty. In World Bank Support to Early Part of the work of achieving inclusive This year IEG began to pilot a new “clustered Childhood Development, IEG found that the growth is laying out a vision for approach” to country program evaluations. Bank fosters an awareness of the rationale transformational private sector collaboration This approach is designed to improve cost to invest in people early, but it lacks a for growth and job creation in support of effectiveness, deepen learning impact, and strategy to support a coordinated approach the twin goals. Two major IEG evaluations broaden country coverage. It also offers in this arena. More emphasis is needed to this year addressed this goal: Investment the potential for significant cross-country include child stimulation and development Climate Reforms and World Bank Lending learning and for countries to learn from each interventions in health, nutrition, and for Financial Inclusion. In the former, IEG other’s experience. social protection. More findings and the found that the Bank Group has successfully The first cluster report, World Bank Group recommendations are in the full report. improved the investment climate in client Engagement in Resource-Rich Developing countries, but the impact on investment, With more than 1 billion people not having Countries, covered Bolivia, Kazakhstan, jobs, business formation, and growth is not access to electricity, universal access is a Mongolia, and Zambia. Each country’s straightforward. Access to financial services daunting challenge. IEG’s evaluation World program is evaluated, and IEG produced has long been believed to lift people out Bank Group Support to Electricity Access a synthesis report to compile the cross- of poverty; IEG’s evaluation found that stressed the urgency for the Bank Group to country lessons. The greatest challenges are the Bank Group has made significant adopt a new and transformative strategy to in three interrelated areas: management of contributions to progress on financial help clients orchestrate a national, sustained resource revenues, growth and employment inclusion, with its support well aligned with sector-level engagement for universal in nonextractive sectors, and inclusive countries’ needs. Yet that support is not access. A major challenge is for the Bank growth and reduction of poverty. The Bank sufficiently systematic or comprehensive. to move beyond its own project financing to Group needs a more consistent approach These reports can be found on IEG’s stimulate private sector investment to close across countries as it strives to stay relevant website. the financing gap for low-access countries. and competitive in adding value through its knowledge and experience. 2015 ANNUAL REPORT 7 Project-Level Evaluations Validation of Self-Evaluation IEG is adjusting the selection criteria for IEG continued validating all self-evaluative For IFC, IEG validated all self-evaluations projects subject to PPARs to ensure closer Completion and Learning Reviews on (166). For closed projects, IEG prepared alignment with the IEG Results Framework. Country Assistance/Partnership Strategies; independent Project Evaluation Summaries Updated criteria include themes related 12 were completed. IEG also continued in lieu of Expanded Project Supervision to future evaluations, high-risk projects, to review all World Bank project completion Reports (78). For Advisory Services, IEG innovative projects, and projects where self- reports. IEG addressed a backlog of ICRs validated 51 percent of Project Completion evaluation ratings significantly differ from that was the result of an unprecedented Reports (72). For MIGA, IEG validations IEG ratings. Prior to this year, IEG produced rise in ICRs in 2012 and cleared a significant of self-evaluations, combined with IEG’s an average of 39 PPARs per year, covering number in that backlog. Through improved independent evaluation, covered 100 12 percent of all closed World Bank projects. data tracking systems (see page 21 percent of MIGA’s mature guarantees (16). This year’s 39 PPARs equalled that average, [ICR/PPAR systems]) and increased training and IEG is committed to moving toward for evaluators and reviewers, IEG decreased 20 percent project coverage over the next the time it takes to process these reviews. three years. Another new initiative is to provide PPARs for private-sector projects, informing IFC Use and Influence of Project Validation of lessons to help the Corporation hone its projects and improve development outcomes. A recent survey indicates that 76 percent of IFC Advisory Services staff use IEG’s project completion report and validation reviews in project preparation. Similarly, 89 percent of MIGA staff report a very high learning value from project reviews that they subsequently apply to underwriting and business development. Use of this information is particularly desirable in challenging environments, where learning can accelerate developmental effectiveness. Source: IEG 2014 Client Survey. 8 Independent Evaluation Group Learning Products These new IEG products, introduced last In addition to these large reports, IEG year, offer ways for IEG to engage Bank continued to deliver a wide range of Group staff in learning. The products brief, focused, and “just-in-time” learning synthesize and disseminate existing IEG products, often at the request of different evaluation work; they use resources such as Just watched your IEG session “ technical units in the Bank Group. Excel and Tableau to present trends to Bank Examples include lessons learned from [on Results and Performance Group operations staff, and they provide a avian influenza for the Bank Group Ebola deep analysis of best practices. Their intent response; summary of findings from water, of the World Bank Group ]… is to build stronger uptake of IEG insights. urban, energy, and transport PPARs; new IFC Sector Highlights that synthesize very interesting and helpful. IEG delivered 15 major learning products lessons from recently evaluated IFC this year. Corporate topics included I made quite a lot of notes operations; and new IEG websites for each lessons from Bank Group partnership work, and environmental and social risk of the new Bank Group Global Practices for my current project.” that provide “one-stop” information on management in development policy relevant evaluations, IEG contacts, and operations. Country topics included SUE HARDING upcoming work relevant for that group. Leadership, Learning, & Innovation, Bank-IFC cooperation at the country commenting on the influence of Results strategy level, selectivity in country and Performance of the World Bank programming, and good practice results Group 2014. frameworks in country strategies. Sector and thematic topics included housing finance, microfinance in Africa, manufacturing, and renewable energy. Influential Evaluations The World Bank and Public Procurement IEG’s 2013 evaluation The World Bank in July 2015. Specifically, IEG impacted the Economic Co-operation and Development. and Public Procurement paralleled the role of the Bank in helping build countries’ IEG organized learning events in a wide Bank’s own review of its procurement own procurement systems; the format and range of countries in the Africa, South Asia, system, undertaken as a prelude for a radical use of country procurement systems as a East Asia, and Latin America Regions, transformation of the Bank’s procurement part of Bank procurement; the principles including events partnered with the policies, for the first time in over 50 years. and mechanisms for the management of African and Latin American Development The IEG evaluation helped shape a wide risk in procurement; and the development Banks, to ensure that both Bank and other range of areas of the proposed new of mechanisms for tracking procurement multilateral development bank (MDBs) procurement policy, notably, a shift in focus efficiency at a Bank-wide level. The World clients understood the nature of choices from the fiduciary aspects of procurement Bank’s new Bank-wide procurement to be made in shaping the proposed new toward the development aspect, befitting tracking system, now being rolled out to field policy. Given the Bank’s leadership role the Bank’s overall role in development offices in all client countries, is a direct result among MDBs, its new procurement policies finance. Senior Bank staff indicated strong of IEG’s report. will impact these institutions and their clients willingness for continued engagement with in addition to the Bank’s own clientele. Bank IEG unusually partnered with management IEG through all stages of preparing the new staff, country clients, other MDBs, and a in global outreach such as to multilateral procurement policy; thus IEG remained a range of global partners have been open to banks’ Heads of Procurement and the World close partner with the Bank even after the IEG’s recommendations and suggestions. Trade Organization. IEG views on the Bank’s evaluation was formally completed. IEG used procurement policies were also sought the evaluation to inform and comment on the by the U.K. Department for International Bank’s proposed new procurement policy, Development and the Organisation for which was presented to the Bank’s Board 10 Independent Evaluation Group Learning and Results in World Bank Operations: Toward a New Learning Strategy World Bank operations staff completed a Task Team Leader Quality Support Toolkit that included checklists for projects, including Quality at Entry, Readiness for Implementation, Implementation Support Quality, and Evaluability. This toolkit is an essential part of guidance to Bank staff on project preparation and implementation support. The IEG team responsible for the Learning and Results in World Bank Operations was instrumental in developing the toolkit. This example illustrates IEG’s work being fed back into operations through the insights gained from its unique vantage point of ex post evaluation. It is an example of IEG’s work contributing to the strengthening of Bank systems for operational quality. 2015 ANNUAL REPORT 11 IEG IS WELL POSITIONED TO CONTRIBUTE to the World Bank Group’s call for assessment and reflection—bringing both a vast knowledge base about the results and performance of development interventions Systematically Turning and a critical independent perspective to nurture and Insight into Influence challenge the Bank Group to do its best work. IEG can do this through its interaction with and outreach to development professionals within and beyond the World Bank Group. Furthermore, IEG’s role as independent evaluator allows us to bring transparency to the Bank Group’s commitment to learn from the past and from insight based on evaluation. 12 Independent Evaluation Group Refining IEG’s Outreach Approach In seeking to increase its impact and A secondary benefit is more synergies videos were featured on the World Bank’s influence, IEG introduced innovations in across IEG from sharing information. intranet page, on IEG’s website, and on IEG’s the design of its learning and outreach YouTube channel. More videos are planned. Along with customized learning and launch events to enhance knowledge uptake. events, IEG also designed associated IEG continued its blog, posting every New communication strategies are sharply outreach material to deliver key messages Tuesday. Topics ranged from best targeted and include more extensive use of and findings from specific evaluations. practices to report findings to evaluation interactive social media. Tracking IEG’s reach Three videos were produced for the professionalization. Readership over the and the impact of each evaluation is easier evaluations Learning and Results in World past year has risen; on average, the blog with this approach and allows us to identify Bank Operations: Part I, the 2013 Results received more than 5,000 visitors per how to best engage audiences so they learn and Performance of the World Bank Group, month. Over the past year, IEG also and apply the lessons from IEG’s work. and The Big Business of Small Enterprises welcomed more guest posts and has (follow-up outreach for 2014 reports). These done a few video blogs as well. Blog Stats 5,001 average visitors per month Strongest month was March, with 8,267 visitors The most popular blogs were n Evaluation in Action: Responding to Ebola (2,019 views) n Thanks for Letting Us Know (1,559 views) Better Together? Seven Guiding Principles for n  Evaluation Partnership Programs (1,558 views) n Professionalizing Evaluation (1,542 views) 2015 ANNUAL REPORT 13 Management Action Record IEG systematically and publicly tracks the In FY15, IEG and World Bank Group progress made by World Bank Group management completed a round of management in implementing actions the ratings and feedback on many of the organization committed to in response recommendations from IEG evaluations. Your IEG report Adapting to “ to IEG evaluations’ recommendations. There are currently 145 recommendations The Management Action Record (MAR) in the MAR. Climate Change: Assessing offers insights into the nature of individual World Bank Group Experience recommendations and the status of action implementation—and progress here is Expanding the Capacity was an inspiration and separately assessed both by Bank Group to Evaluate and Use management and by IEG for four years Evaluation for Decision guidance to look for subsequent to an evaluation’s disclosure. Making Every year, the MAR goes through an alternative ways.” In FY15, IEG further focused its work in update cycle; at the end of that, Bank Group evaluation capacity development along three stakeholders can see, on IEG’s website, PRAVIN KARKI work streams to more effectively strengthen how much progress has been made in Sr. Hydropower Specialist monitoring and evaluation (M&E) capacities, the previous year in implementing actions. World Bank Group, institutions, and systems in the World Bank commenting on the continuing To make the updates happen regularly Group’s client countries. influence of IEG’s third-phase and smoothly, IEG continuously invests in evaluation on climate change. process improvements and clarifications, in relevant information technology systems that facilitate update tracking, and in training for the many staff from across the Bank Group who contribute to the updates. CLEAR: IEG hosts and provides leadership International Program for Development Championing Evaluation Capacity to the global partnership program CLEAR Evaluation Training: Founded in 2001 by Efforts: In FY15, IEG continued to offer (Centers for Learning on Evaluation and IEG in partnership with Carleton University, thought leadership in evaluation capacity Results) that is supported by 10 donors, IPDET offers executive training for evaluators to help promote the relevance of the agenda including, among others, The Rockefeller and evaluation practitioners globally. It among World Bank staff and partners. For Foundation, the Inter-American Development is the only global program with a focus example, in collaboration with the Bank’s Bank (IDB), and the Swedish International on development evaluation techniques. Governance Global Practice, it organized Development Cooperation Agency. In FY15, While the program today is run primarily by and facilitated a high-level dialogue among a midterm review of the program found that Carleton, IEG’s engagement has continued government and academia regarding the CLEAR is relevant, has delivered effectively strategically and by ensuring a focus on optimal ways to build demand and capacity on its program, and has promising centers; development-specific knowledge. Over the for evaluation at the Spring Meetings in it also found that IEG’s role is critical. The years, IPDET has offered a rich learning Washington, DC. It also continued to review highlighted areas for improvement, opportunity to nearly 3,000 participants from produce knowledge regarding the “how to” including the need to purposefully surface about 125 countries, with 30–40 percent of of evaluation. and drive learning about evaluation capacity participants receiving scholarship support. development derived from ongoing activities Given the changing context in development of CLEAR teams around the world. As evaluation, in FY15, Carleton and IEG a result, CLEAR has introduced a new initiated a strategic review to assess and collaborative governance structure, a revised rejuvenate the program and to ensure cost theory of change that emphasizes learning, effectiveness, as well as updated learning and approaches to more systematically link methodologies and the long-term viability performance and learning. In FY15, CLEAR of its business model. The review will be also added a new center in Brazil and completed in FY16. received additional funding from IDB and a renewed pledge from the U.K.’s Department for International Development. 2015 ANNUAL REPORT 15 Engaging M&E Specialists: The Results Measurement IEG Good Practice Awards and Evidence Stream In June 2015, IEG awarded 17 Candid self-evaluation promotes In March 2015, IEG co-led the collaborative teams from across the Bank Group for accountability and learning and provides design and development of a two-day exemplary efforts in designing good the opportunity for teams to make mid- learning and knowledge-sharing event for M&E frameworks and preparing quality course corrections as outlined in the M&E specialists across the World Bank self-evaluations of country strategies, World Bank Group strategy. More broadly, Group with the Results Measurement Bank operations, IFC investment evaluation allows staff to look back at a and Evidence Stream (RMES). This World operations and advisory services, wealth of experience and distill lessons on Bank Group-wide community of practice and MIGA guarantees. what works, what doesn’t, and why. Using event brought together more than 200 staff that knowledge to guide future development “You have to be serious about asking members involved in results measurement. projects will help improve the quality and yourself the hardest question: Is what IEG teams led 14 of 42 peer-to-peer effectiveness of Bank Group investments. we’re doing leading us to the goals that learning sessions. we have set?” President Jim Yong Kim “I see the importance of the project RMES is the staff cadre for results said at IEG’s awards ceremony. “These evaluation reports, which help assist measurement professionals from across awards are elephants, and in fact that’s development outcomes as well as the Bank Group. These professionals so appropriate because evaluation, and effectiveness. Self-evaluation has been provide the Bank Group and its clients especially self-evaluation, is always the a great learning tool for MIGA staff,” said with world-class know-how to maximize elephant in the room…We are the first Keiko Honda, executive vice president the achievement of the Bank Group’s generation in the history of humanity to and CEO of MIGA. goals by making decisions based on be able to think about ending extreme evidence—a natural partnership for IEG. poverty. That is such a compelling goal that we have to be willing to be as self- critical as we need to be to make sure we are on the right path.” 16 Independent Evaluation Group In her closing remarks, Nena Stoiljkovic, vice president of Global Client Services in IFC, congratulated all recipients, telling them, “Your efforts exemplify your commitment to learning in operations, and ultimately will help us improve our development effectiveness.” IEG Senior Advisor and Deputy Director Rick Scobey pointed out that having the awards on June 18 was particularly meaningful, given that “Exactly one year ago, the World Bank Group started RMES,” the collaborative community of practice made up of approximately 300 Bank Group staff who work on M&E and results management. The IEG Good Practice Awards were established 11 years ago, with the purpose of recognizing Bank Group staff for excellent work in preparing self-evaluations. A complete listing of the 2015 awardees is available at http://ieg.worldbank.org/ ieg-awards-2015. 2015 ANNUAL REPORT 17 IMPROVING OUR INTERNAL PROCESSES AND Improving Our Own Work methods will enhance our products and our influence. This year IEG has undertaken numerous initiatives to improve our tools and our work. In FY15, we paid specific attention to systematically enhancing the user experience of IEG’s clients—so as to increase the uptake of our insights. 18 Independent Evaluation Group IEG’s Own Effectiveness: Investing in Internal Knowledge Management, Learning, and Processes IEG has continued to invest in testing, IEG also invested in a number of systems adapting, and developing new methods solutions to facilitate faster and more to ensure that we maintain cutting-edge effective evaluation and validation work. rigor and quality in evaluating World Bank For example, a new system to track and It is good to have you and “ Group development effectiveness. Key steps complete ICR Reviews was developed, as your managers at the helm at include publishing our methodologies on the was a system to automatically track and web to ensure transparency and credibility, improve the production of PPARs. This latter IEG, and we in the evaluation using new software tools to more efficiently system allows all staff to track progress; review documents for evidence, tapping into access all related documents, emails, community want to know more large data sets to deepen the evidence we and notes; receive alerts; and generate about your stewardship of a use in evaluation, and establishing standards custom reports that can feed into PPARs. for data collection and sharing. Furthermore, IEG continued to work with strong evaluation culture that its internal partners to integrate all IEG Within the World Bank Group, IEG uses evaluation reports into ImageBank and the uses evaluation to build a its intranet—IEGNet—as a collaborative Open Knowledge Repository. space, thus ensuring “channels are always better world.” open” vis-à-vis World Bank Group staff. All evaluations are posted here, and there is a TESSIE CATSAMBAS, gallery of interactive self-service reports and commenting on the June 16 What Works data visualizations. Disclosures are tracked Blog, “Building a Better World.” as well, both upcoming and completed. This bridge connects staff around relevant knowledge, enabling collaboration, discussion, and online engagement. Quality Assessment Mechanisms—Learning Our Work Program: from Experience Strategic Engagement Areas IEG invested in and re-released This year IEG continued with two new In FY15 IEG introduced a new approach LessonFinder—a user-friendly interface quality assessment mechanisms that occur for developing our work program, building with a database that brings together several after evaluations are completed. First, on three purposefully selected Strategic hundred project-specific lessons noted by evaluation teams conduct After Action Engagement Areas: Inclusive Growth, IFC and MIGA staff. These lessons draw Reviews to debrief and analyze what Sustained Service Delivery for the Poor, and on Expanded Project Supervision Reports, happened, why, and how the work could Environmental Sustainability. These three Project Completion Reports, and Project have been done better. topics lie at the heart of the new World Bank Evaluation Reports that have been validated Group Strategy and feed into the dual goals Second, we have conducted meta- by IEG. Key words ensure that when staff of poverty reduction and promoting shared evaluations—reviews of our own evaluations begin planning a new project, LessonFinder prosperity. They were selected to maximize by an independent panel, assessing prompts them to access the lessons, the collection of evaluation evidence to the quality, strengths and weaknesses, project documents, and even staff contacts address the overall challenge of meeting the and the overall conduct of a sample of to ensure projects use best practices Bank Group’s goals in a sustainable manner; individual evaluations. This panel uses from previous Bank Group staff work. ensure better coordination and synergy on best practice standards for its review. Two The system, though still in early release, is thematic issues across IEG products; and such evaluations were conducted this year; proving useful to IFC staff. IEG held several create a stronger platform for engaging lessons will be incorporated into all IEG knowledge sessions to increase awareness with Bank Group management on critical evaluation work going forward. of the tool itself. A sustained IEG outreach development challenges. effort has led to more than 500 unique users worldwide. 20 Independent Evaluation Group IEG Handbook on Data Collaboration and Synergy— Analysis and Portfolio Important Working Tools Review Each Strategic Engagement Area generated A staff Task Force on Collaboration and In March 2015, IEG launched its web-based proposals for the coming three years for Synergy examined IEG’s day-to-day Handbook on Data Analysis and Portfolio major evaluations and learning products operations. After a series of internal focus Review. This resource grew out of staff that synthesize existing evaluation evidence. groups and interviews, it outlined a series of suggestions. It provides IEG staff with a Topics also targeted the second objectives measures to enhance collaboration, improve “one-stop shop” of information related to of IEG’s Results Framework, related to efficiencies, and deepen IEG impact. The portfolio management and data analysis, the implementation of World Bank Group task force offered recommendations to which is critical for high-quality evaluation reforms. See page 26 for IEG’s FY16–18 raise staff awareness, introduce incentives work. The handbook provides an up-to-date work program. to collaborate, change work program assessment of what data are available, who development and evaluation teams’ to contact for more information, and where formation, and harmonize processes across to find more resources, including training. departments. The task force work set the Specific chapters cover, among other topics, stage for significant internal improvements an introduction to data analysis and portfolio of IEG operations to come in FY16. reviews, World Bank/IFC data by source and by topic, and user interfaces and data integration. This handbook is an important milestone in IEG’s goal to ensure high and uniform standards of quality across IEG evaluation work. 2015 ANNUAL REPORT 21 IEG Coaching Program IEG Client Survey This internal program consists of technical master classes taught by IEG experts. In this year’s client survey, respondents rated IEG’s impact and influence The goal is to make it a program for on the World Bank Group much higher than in the previous survey in cutting-edge expertise development FY13: 94 percent of respondents said our work influences development and knowledge transfer. Topics include effectiveness to some extent. That is a 20 percent increase among staff. Uncommon Qualitative Collection Methods, The survey was sent to more than 13,000 participants. This year IEG made Survey Methods and Sampling, Investment extra efforts to ensure higher response rates than in the past. The average Risk Analysis, Evaluating Advisory and response rate among all audiences was 18.8 percent. Knowledge Products, and Economic Returns Analysis. With improved knowledge and Overall IEG is doing well; there were consistent improvements in ratings skills, IEG staff will be able to deliver work of our influence. In terms of quality, we achieved a mean score of 4.2 that enables effective development work and out of 6, above the mid-point of 3.5 but not as high as our targets in the truly makes a difference to beneficiaries. Results Framework. It is important for IEG to maintain openness and transparency by facing up to feedback received, and taking action on it. Results are available on IEG’s website. 22 Independent Evaluation Group Looking Forward The Committee on Development Effectiveness (CODE), which oversees IEG’s work, commissioned an independent panel in 2014 to conduct an external review of IEG. The panel was asked to identify IEG’s main strengths and any areas for improvement to enhance IEG’s impact. The final report was released in June 2015. IEG welcomes the comprehensive and insightful report and will be guided by CODE’s decisions on specific next steps. We appreciate the report’s reaffirmation that “IEG is an essential corporate asset and safeguard, and a vitally important global public good.” The panel examined IEG within the larger Bank Group system of CODE, the Board, and management and staff, and identified a set of eight recommendations to enhance IEG’s influence and role as an independent evaluator. Many of the recommendations give further impetus to ongoing initiatives and open the dialogue to push some of them further. A link to the report and IEG’s response are available on IEG’s website. Evaluation is useful only when it generates critical evidence that can inform and improve decision making. IEG is on a path to increase its influence in evaluation quality and impact, with the ultimate goal of improving development effectiveness. 2015 ANNUAL REPORT 23 FY16 1. MAJOR EVALUATIONS 2. OTHER EVALUATION PRODUCTS 3. VALIDATION PRODUCTS Thematic and Sector Evaluations PPARs Country Strategy Completion Report Validations • Tertiary Education • World Bank: 50 • 22 expected • Capital Market Development • IFC: 8 Project Completion Report Validations • Jobs and Competitiveness Learning Productsa XPSR: 40% coverage (103 projects expected) •  Corporate and Process Evaluations  aximizing Development Impact in IDA (Category II) • M • PER: (25 expected)  rocess Evaluation of Systematic • P Lessons from Joint World Bank Group Projects •  ICR Reviews: 100% coverage (340 projects expected) •  Country Diagnostic and Country (Category I) AAA/Advisory Services Completion Report Partnership Framework World Bank Group Transformational Engagements •  • Review of Program for Results (Category II) Validations •  Assessment of World Bank Group’s Domestic Resource Mobilization in World Bank Group •  • PCR: 51% coverage (102 projects expected) Self-Evaluation Systems Instruments (Category I) Mobilizing Private Capital for Development (Category I) •  Country-Focused Evaluations Financial Viability of Infrastructure Delivery (Category I) •  • Fragile Situations in Non-FCS Countries Gender Impacts of Community-Driven Development •  •  Clustered Country Program Evaluation (Category I) on Small States (4 country reports plus • Land Administration (Category I) synthesis report) • Fisheries (Category I) Development Policy Operations: Political Economy •  Results and Performance Report Analysis (Category II) • RAP 2015: Gender Equality Development Policy Operations: Poverty and Social •  Impact Analysis (Category II) Development Policy Operations: Environment Sector •  (Category II) IEG Work Program FY16-18 24 Independent Evaluation Group FY17 FY18 1. MAJOR EVALUATIONS 1. MAJOR EVALUATIONS Thematic and Sector Evaluations Thematic and Sector Evaluations • Shared Prosperity in Country Programs • Public Finance for Investment and Jobs • Rural Non-Farm Economy • Corporate Sustainability and Development • Water Supply and Sanitation • Informal Sector and Development • Urban Transport Services • Sustainable Cities • Environmental Management of Air and Water Resources  • Scaling Up Nutrition • Carbon Finance Corporate and Process Evaluations • Data and Knowledge for Development Corporate and Process Evaluations • IFC Client Engagement Model Process Evaluation of Global Practice/Cross-Cutting Solution Areas •  • Engaging Citizens Country-Focused Evaluations • C  lustered Country Program Evaluation on Dynamic Country-Focused Evaluations Middle-Income Countries • Clustered Country Program Evaluation—TBD Results and Performance Report Results and Performance Report • RAP 2016: Managing for Results • RAP 2017: Working in Partnership 2. OTHER EVALUATION PRODUCTS 2. OTHER EVALUATION PRODUCTS Note: AAA = Analytic and advisory activity; ICR = Implementation PPARs PPARs Completion and Results Report • World Bank: 55 • World Bank: 60 (World Bank); PCR = Project Completion • IFC: 8 • IFC: 8 Review (IFC); PER = Project Evaluation Review (Multilateral Investment Systematic Reviews and Impact Evaluations Systematic Reviews and Impact Evaluations Guarantee Agency); XPSR = Expanded Project Supervision Report (IFC). [TBD in prior year] [TBD in prior year] a. Category I learning products are Learning Products a Learning Productsa sent to the Board’s Committee on  [ TBD in prior year, in consultation with management] [ TBD in prior year, in consultation with management]  Development Effectiveness on a for-information-only basis. Category II learning products are 3. VALIDATION PRODUCTS 3. VALIDATION PRODUCTS discussed by that Committee the Country Strategy Completion Report Validations Country Strategy Completion Report Validations same way major IEG evaluations are. [TBD in prior year] [TBD in prior year] Project Completion Report Validations Project Completion Report Validations  [ TBD in prior year, based on expected project closure] [ TBD in prior year, based on expected project closure]  AAA/Advisory Services Completion Report Validations AAA/Advisory Services Completion Report Validations  [ TBD in prior year, based on expected project closure] [ TBD in prior year, based on expected project closure]  2015 ANNUAL REPORT 25 Appendixes 26 Independent Evaluation Group APPENDIX TABLE A.1 • IEG deliverables and spending FY12 FY13 FY14 FY15 DELIVER ABLES ( NUMBER ) Planned Actual Percent Total Major Evaluations 8 10 10 7 7 100 Corporate Evaluations 2 3 4 2 2 100 Sector and Thematic Evaluations 4 5 4 4 4 100 Country Program Evaluations 1 1 2 1 1 100 Impact Evaluations 1 1 0 0 0 100 Project Evaluations and Global Program Reviews Project Performance Assessment Reports 40 31 47 53 39 74 Global Program Reviews 1 2 2 0 0 n/a Total Validations 414 478 573 571 Implementation Completion Reports 178 295 361 392 Country Assistance Strategy Completion Reports 20 19 22 12 Project Completion Reports 130 76 87 72 Expanded Project Supervision Reports 75* 80 88 78 MIGA Validations 11* 8 15 16 Total Spending ($ millions) 32.5 33.8 34.3 33.4 *Includes validations and independent project evaluations. 2015 ANNUAL REPORT 27 APPENDIX TABLE A.2 • IEG staffing STAFFING ( NUMBER ) FY12 FY13 FY14 FY15 Total Staff 98 107 109 105 By grade GA-GD 22 23 22 23 GE 9 7 6 7 GF 20 22 25 21 GG 20 26 30 30 GH 20 22 20 19 GI and above 7 7 6 5 Note: World Bank Group staff levels are denoted by letters. Representative job titles per level are as follows: GA = Office Assistant; GB = Team Assistant; GC = Program/Information Assistant; GD = Analyst; GF = Professional; GG = Senior Professional; GH = Manager/Lead Professional; GI = Director/Senior Advisor. 1818 H Street, NW Washington, DC 20433 http://ieg.worldbank.org ieg@worldbank.org