99760 Transforming Our World – Aiming for Sustainable Development Using Independent Evaluation to Transform Aspirations to Achievements © 2015 InternaƟonal Bank for ReconstrucƟon and Development/The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contribuƟons. The findings, interpretaƟons, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of ExecuƟve Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominaƟons, and other informaƟon shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages disseminaƟon of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full aƩ ribuƟon to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank PublicaƟons, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Cover Photo: ©Arthimedes/ShuƩ erstock Introduction Background 2015: A Key Milestone between 2000 and 2030. The year 2015 is pivotal in international development. In the lead-up to 2000, the global community came together at various conferences to agree on, for the first time in known history, shared development goals. The eight Millennium Development Goals (MDGs) set 18 targets that were aimed at significantly reducing disease, illiteracy, gender inequality, hunger, and poverty, and improving access to water and sanitation by 2015. Leading up to this point where the era of the MDGs concludes, progress has been monitored and discussions started well ahead of this momentous year to define and meet the more ambitious Sustainable Development Goals (SDGs), building on and bringing to fruition what has been started under the MDG agenda. From MDGs to SDGs: How Far Have We Come? Much progress has been made toward achieving the MDGs. The world reached the poverty reduction target five years ahead of schedule, and progress has been reported in a number of other areas. However, considerable challenges remain: even while declaring success on MDG1, roughly a billion people remained in poverty. A large number of MDG targets will not be met by the end of 2015 (figure 1), and progress remains uneven among the different countries. Moreover, new challenges to progress are emerging deriving from natural and manmade calamities. Figure 1. Progress Toward Achieving the MDGs (percent) 100 80 60 40 20 0 Poverty Hunger Primary Gender Infant Under-5 Maternal Water Sanitation Slums education parity motality mortality mortality Source: World Bank and International Monetary Fund, 2014. Note: Data shows progress toward meeting some MDG targets; 100% = target met. Time for a Historic Shift? The new sustainable development agenda has made two significant shifts. First, the broad-based consultation process has generated goals that have much greater ownership by a global community, a group that is much wider and deeper than at any time before. It brings together not only development practitioners, donors and recipient governments, but also actors from civil society, beneficiaries, private sector, and new organizations that are forming and transforming the development landscape. Second, the SDGs are global goals, not just to be attained by so-called “developing” or partner countries, but by all countries around the world. As is obvious from the related Conference of Parties (COP)21 1|T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t process, sustainability is everyone’s business and so is sustainable development. This shift has implications for the relationship between countries and actors within countries and across boundaries. The vision that the United Nations (UN) General Assembly sets out to endorse in September is ambitious and aspires to transform the world. More Players, More Goals. The upside of greater ownership has come hand-in-hand with the need to accommodate many more goals than before: 8 MDGs have become 17 SDGs. They are complex in that they are interdependent, and in that sense more in tune with the realities of our world, which has always been complex, but is becoming more so in light of a changing climate and its effects on natural resources and human competition for them. As the (draft) outcome document of the upcoming UN summit suggests: “[t]hey are integrated and indivisible and balance the three dimensions of sustainable development: the economic, social, and environmental.” The document goes on to say that the “Goals and targets will stimulate action over the next 15 years in areas of critical importance for humanity and the planet.” These areas are people, planet, prosperity, peace, and partnership. Who Contributes What? The MDGs and forthcoming SDGs define goals and targets, which lend themselves to evaluation. However, there are several obstacles to evaluation, including the evaluability of the goals and targets themselves. In addition, trying to determine the contributions of a single development institution to these goals is hampered by the broad development agenda the MDGs and SDGs set out. There is a broad range of potential interventions and multitudes of agents who can work towards the goals. Deciphering which agent contributed what to the achievement of specific goals is hard, if not impossible. Concepts of mutual accountability have been discussed as part of the development effectiveness discussions from Paris to Busan, but have not achieved a satisfactory conclusion. Shaping a Global Agenda: The World Bank Group’s Engagement The Role of the World Bank in Developing the MDG Indicators. In May 1997, the World Bank, the UN, and Organization for Economic Co-operation and Development (OECD) sponsored a seminar for Development Assistance Committee (DAC) members and development partners to review the concepts, methodologies, and data issues in development indicators. Pulling the various declarations at the UN summits into a coherent framework, they produced a synthesis report that articulated seven International Development Goals (IDGs) with 21 indicators. In 2001, the World Bank hosted a seminar on the IDGs, “From Consensus to Action: Making Cooperation on the Goals More Effective,” which was attended by key agencies to reconcile the IDGs with the goals in the Millennium Declaration. This led to the emergence of a unique set of goals, targets, and indicators—the MDGs—that could be quantified and for which there were reasonable data. Alignment of SDGs and World Bank Group’s Goals. The World Bank Group’s twin goals, adopted in 2013, of eradicating poverty and boosting shared prosperity are mirrored in the SDGs, in particular in SDGs 1 and 8. Although the World Bank Group has not spelled out goals in the many areas relevant to the SDGs, it effectively has worked in most, if not all, domains of the new development agenda. It had been an important partner, providing intellectual leadership and knowledge work, at conferences during the 1990s that eventually resulted in the MDGs. But, already from the beginning, the World Bank Group pursued a more comprehensive approach to poverty reduction than the MDGs by combining private sector–led growth with empowering the poor to participate in development—the twin pillars of World Bank Group strategies since 2000. The World Bank Group has made explicit commitments to support the 2030 agenda. F i n a n c i n g f o r D e v e l o p m e n t |2 Measuring Progress—Learning for the Future Seeing the Agenda Through. The MDGs were unique in that they were setting for the first time commonly agreed goals and measurable targets. The SDGs follow the same with some 169 targets that describe what success looks like by 2030. In addition, the UN General Assembly’s outcome document contacts provisions for follow-up and review to ensure progress is tracked, reported transparently, and evaluated rigorously and based on evidence. This commitment is accompanied with a call for developing in-country capacities for data systems and evaluation. In line with this commitment to learning from the past and taking corrective actions when necessary, this synthesis paper presents findings from independent evaluations of the World Bank Group. Insights from the Past. As it is doing for the other key events of 2015—Financing for Development in Addis and COP21 in Paris (forthcoming)—the Independent Evaluation Group (IEG) presents here findings from evaluations it has undertaken over the past years. This paper draws heavily on IEG’s 2014 Results and Performance report to:  Draw out lessons from World Bank Group engagement with the MDGs and what they mean for engaging with the SDGs, bringing together global goals with institutional agendas;  Showcase World Bank Group experience in the education, health, and water and sanitation sectors, specifically related to the MDGs;  Share select lessons from evaluations that speak to specific SDGs, reviewing experience that has preceded the adoption of the new goals but are drawn from relevant sectors. These insights are presented to help decision-makers and development practitioners build on past successes and avoid preventable mistakes. Additional evaluations are planned over the coming years under IEG’s work program in three Strategic Engagement Areas, designed to address the World Bank Group’s twin goals. Meshing Global Goals with Institutional Agendas This section reviews how the World Bank Group as an institution engaged with the MDGs, both contributing to shaping them and incorporating them into institutional commitments, country strategies, and operational programs. During the 1990s, World Bank Group strategies had incorporated the issues embodied in the MDGs. The World Bank Group formally endorsed the MDGs in the 2001 Strategic Framework Paper and found the MDGs to be a useful results-based framework for assessing development impact and to provide a context for coordination among donors and for engagement with countries. However, the strategy emphasized that meeting the MDGs was a joint responsibility within the international community and expressed concern about the time frames. The World Bank Group’s approach to supporting the MDGs was to strengthen its country focus by using the comprehensive development framework on the one hand, and pursuing private sector-led growth strategies on the other. 3|T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t Providing Intellectual Underpinning to Development Thinking The World Bank’s analytical work has influenced global development Getting at thinking, as well as its own strategies and results agenda. The World Poverty: Data Development Report 1990: Poverty was instrumental for deepening and Diagnostics the development community's understanding of poverty reduction as a core underpinning of any development effort. Since 2000 several flagship publications, such as World Development Report IEG’s 2015 evaluation, The Poverty Focus of Country 2004: Making Services Work for the Poor, advanced the thinking on Programs: Lessons from World Bank Experience achieving the MDGs and significantly influenced the World Bank reviewed how the World Bank had developed and Group’s corporate and sector strategies. used data and analytical capacity to better understand country-specific causes of poverty, and how such analyses were translated into country strategies, Drawing on its strong statistical expertise, the World Bank played a operational programs, and learning from experience. key role in defining the IDGs in the late 1990s and the MDGs in 2001, Data. Data is the foundation for understanding the and pushed forward the Managing for Development Results agenda causes of poverty and informing solutions: it is “the through a series of international roundtables on the use of lifeblood of decision making”. The evaluation found quantitative, time-bound indicators for measuring development excellence in the Bank’s core work on developing impact. Starting in 2004 the World Bank and the International internationally comparable poverty data, but also Monetary Fund produced the annual Global Monitoring Report, identified major challenges. There remain significant shortcomings in the consistency, quality, and which complemented the UN MDG reports by providing analysis of timeliness of the data. Lack of data transparency, the policies and actions needed for achieving the MDGs and related inadequate resources, and in some instances, development outcomes. restricted access to country data inhibit the Bank from completely fulfilling its global role in generating robust One such source of results data are impact evaluation (IE). The and standardized information on poverty. production of IEs in the World Bank Group grew rapidly during FY02– Diagnostics. Diagnostics depend on quality data. 14, as shown in IEG’s evaluation of IEs. Recognizing the role of IEs in Unsurprisingly, when data are not timely or robust, the furthering the knowledge agenda, the World Bank in 2005 created quality of diagnostic work suffers. Overall, the the Development Impact Evaluation Initiative, a global program evaluation confirmed the generally high quality of the Bank's diagnostic work, but found a need for greater hosted by the Development Research Group to help guide policy actionability of the recommendations to help guide choices and improve results. Through the Spanish (now Strategic) policy. Country context, including ethnic and regional Impact Evaluation Trust Fund, there has been a more systematic differences, and factors such as social exclusion and approach to identifying and financing IEs, notably in the human the political economy, are critical for getting at the development network. IEG reviews of IEs for some MDGs— root causes of poverty. Making sure diagnostic work addresses these issues and is picked up in public maternal and child health, nutrition, and water and sanitation— debate and policy-making helps ensure that decision- identified lessons about what works and what is needed to expand makers can make informed choices about the poverty knowledge about the impact of MDG interventions and programs, implications of policies and programs they adopt. as summarized in the second chapter of this paper. However, the Learning. Learning from project experience is 2012 IEG evaluation found that the feedback loop between IE important for strengthening project outcomes. Robust production and World Bank project operations and learning has monitoring and evaluation systems, and feedback been modest, and recommended improved integration of IEs into loops to improve project design and country the design and review of projects to sharpen their results focus. strategies, are key instruments for learning. The report identified significant shortcomings in the design and use of these instruments, by the Bank and borrowers, The World Bank’s knowledge work played an important and growing to learn from experience and improve outcomes. role in supporting the MDGs. In line with the World Bank’s corporate strategies, MDG-related analytical and technical assistance work F i n a n c i n g f o r D e v e l o p m e n t |4 grew in both absolute terms (by expenditure and by number) and as a share of total knowledge work delivered (figure 2). In particular, non-lending technical assistance, which was used to help improve client capacity to design and implement reforms and development programs, expanded rapidly. Figure 2. MDG-Related Knowledge Work b. Composition of MDG-related a. MDG-related knowledge work in total (percent) AAA (percent) 100 FY90–01 FY02–14 80 MDG 1 2 10 MDG 2 12 3 60 MDG 3 14 8 40 MDG 4 0 1 20 MDG 5 0 1 MDG 6 2 3 0 FY00–04 FY05–09 FY10–14 FY00–04 FY05–09 FY10–14 MDG 7 22 23 By Expenditure By Number MDG 8 48 51 Total 100 100 Studies Technical assistance Embedding the MDGs into Country Strategies World Bank Group country strategies are designed to link its institutional priorities with the clients’ own priorities. The 2002 Monterrey Consensus stated up front that “each country has primary responsibility for its own economic and social development.” In endorsing the MDGs, the World Bank Group emphasized the primacy of country ownership and stepped up the country focus in the implementation of its strategies. This heightened country focus was supported by organizational changes initiated in the late 1990s, which included decentralization and matrix management. IEG’s 2012 evaluation The Matrix System at Work found “[t]he World Bank’s country-based model increased client responsiveness by using the Poverty Reduction Strategy Paper (PRSP) as a key country instrument (in International Development Association (IDA) countries) and country budgetary actions under the Heavily Indebted Poor Country Initiative and the subsequent Multilateral Debt Reduction Initiative as sources of funding for the social sectors. In Bangladesh, for example, the entry points for World Bank engagement were largely to align the Country Partnership Strategy objectives with the national goals outlined in the PRSPs, which also formed the basis for collaboration among development partners that led to major sector-wide approaches in health and education. 5|T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t Under the World Bank’s country-based operational model, the MDG themes were incorporated substantially into country programs. In the nine countries reviewed for the Results and Performance report, MDG-related issues were important components of World Bank Group country strategies, even before the introduction of the MDGs. Although the term “MDG” was rarely used (either spelled out or as an acronym), nearly 80 percent of the strategic pillars supported specific MDGs. Virtually all country strategies included objectives in education and health areas, and environmental sustainability also received considerable attention. In this regard, there were no significant changes in the country strategies after adoption of the MDGs since the country strategies were already well attuned to MDG priorities. There were, however, some examples where the MDGs figured Getting to Poverty: Country Strategies prominently in the strategy formulation, such as the Yemen FY03– 05 Country Assistance Strategy, where the World Bank stepped in IEG’s 2015 evaluation, The Poverty Focus of Country to address MDG targets that the country was not expected to meet Programs: Lessons from World Bank Experience (for example, basic education and gender disparity). reviewed how the World Bank had used its data and analytical capacity to generate a better understanding The MDGs had a modest impact on the choice of indicators for of country-specific causes of poverty, and how such results monitoring in World Bank Group country strategies. In the analyses were translated into country strategies and 40 country strategies prepared between FY97–14 for the nine operational programs. countries, the use of MDG-related indicators in the results matrices Country Strategies. The World Bank’s country strategies remained stable between 35 and 38 percent of all indicators for (Country Partnership Frameworks) are determined measuring achievement of the objectives. This is consistent with through discussion with governments, who ultimately the finding that the World Bank’s overall MDG-related set the priorities. In countries with high levels of commitments stayed at about 40 percent of all commitments commitment to poverty eradication, World Bank during FY90–14. For MDG-related objectives, formal MDG country strategies did a good job at translating indicators continued to be less frequently used than indicators that diagnostics into strategic choices. Where countries have not prioritized poverty eradication, the Bank faces addressed some aspects of an MDG objective; even so, there was a tensions between engaging with government priorities modest increase in the use of formal MDG indicators from 11 to 15 versus meeting corporate goals for focusing on poverty percent between FY01–05 and FY10–14. This suggests a gradual reduction. uptake of internationally adopted indicators, where appropriate, for measuring the results of World Bank country strategies. Strategy Implementation. The evaluation found most operational programs to be aligned with priorities for poverty reduction set out in strategies. Not Subject to individual country context, World Bank Group country infrequently, adjustments were made to programs strategies had varying degrees of MDG focus. Measured by the use either for legitimate reasons (such as adjusting to of MDG-related indicators in the results frameworks, the World shocks, capacity constraints, political change), or if Bank Group focused more on MDG issues in the two fragile and government commitment to poverty reduction falters. conflict-affected states (FCSs) than the low-income countries (LICs) If commitment changes, high-quality, timely diagnostics and middle-income countries (MICs) in the sample (figure 3). and technical assistance can be used to identify entry Significant variability existed among the three MICs: in Indonesia, points and lay the groundwork for engagement when MDG issues figured prominently in the World Bank Group program opportunities for greater impact on poverty arise. prior to the adoption of the MDGs, and this emphasis was maintained (though at a lower level) afterward; in Tunisia, the relatively strong MDG focus eroded as time passed, with the most recent country strategy addressing mainly macroeconomic and F i n a n c i n g f o r D e v e l o p m e n t |6 private sector development issues; and in Brazil, attention to the MDGs grew rapidly from a low starting point to make the Brazil program one of the most MDG-intensive in the sample. By contrast, the World Bank Group’s country strategy for Malawi focused on the MDGs as well as growth (agricultural productivity, business environment, and infrastructure) and governance (macroeconomic management, public financial management and accountability) issues. There is no evidence that the World Bank Group considered the MDGs to be less relevant for FCSs or MICs from this sample. Figure 3. MDG-Related Indicators in Nine Country Strategies, FY97–14 (percent) 100 80 60 40 20 0 Nepal Yemen Bangladesh Georgia Malawi Nigeria Brazil Indonesia Tunisia FCS LIC MIC FY97–00 FY01–04 FY05–09 FY10–14 Source: IEG based on World Bank Group Country Assistance/Partnership Strategies and Interim Strategy Notes. Notes: Early CPSs Country Partnership Strategies for Nigeria did not have results frameworks; therefore no outcome indicators were defined. FCS = fragile and conflict-affected states; LIC = low-income country; MIC = middle-income country. World Bank support for the MDGs in the nine countries was followed up through lending (including trust funded projects) and knowledge support (figure 4). The three MICs had a higher proportion of MDG-related lending (42 percent on average) than the LICs (36 percent) and FCSs (34 percent) in the 1990s, and this gap in MDG focus widened after the introduction of the MDGs. To a large extent, this reflects the MICs’ choice to work with the World Bank on MDG-related issues. In Brazil, for example, the World Bank strongly supported MDG 7 throughout the period, as well as objectives in poverty, education, maternal and child health, and communicable diseases. However, in Nepal, one of the poorest countries in the sample, the MDGs were highlighted in World Bank Group strategies, but much of the World Bank’s lending program was directed to transport, energy, and mining. Similarly, the World Bank’s MDG-related knowledge work in the nine countries was high and increasing—from 63 percent in 2000–04 to 74 percent in 2010–14 (by number), consistent with the World Bank’s overall trend. 7|T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t Figure 4. MDG-Related Lending Commitments in Nine Countries (percent) 100 80 60 40 20 0 Nepal Yemen Bangladesh Georgia Malawi Nigeria Brazil Indonesia Tunisia FCS LIC MIC FY90–01 FY02–14 Source: IEG reviews of World Bank Group Country Assistance Strategies, Country Partnership Strategies, and Interim Strategy Notes. Note: FCS = fragile and conflict-affected states; LIC = low-income country; MDG = Millennium Development Goal; MIC = middle-income country. However, the 2012 IEG evaluation The Matrix System at Work found that the country-based model, while having improved the World Bank Group’s responsiveness to client needs, made it difficult to integrate sector and corporate strategies with country strategies. Also, global and regional programs were not well integrated into World Bank Group country strategies and country programs. The IEG evaluation recommended, among other things, rebalancing the relationship between the country and sector units to ensure that the country- focused model is complemented by appropriate attention to global priorities. Linking Results: The World Bank Group’s Role, Approach, and Expected Contributions The World Bank Group strengthened its results orientation, and as time passed, its results monitoring systems and results frameworks used MDG indicators. As part of the institutional effectiveness reforms in the late 1990s, a new system of performance indicators for client responsiveness and drivers of project outcomes was introduced and included baselines and time-bound targets. Results-based country strategies were mainstreamed in 2005, and results based financing is now widely used in many sectors. But, as IEG found when reviewing the quality of results frameworks in country strategies, “[d]esigning an effective operational results chain is possibly the most critical and challenging task in developing the results framework for a country strategy. The results chain is intended to present a logical statement of how planned World Bank Group interventions will lead to the realization of objectives, beginning with inputs, moving through activities and outputs, culminating in outcomes, impacts, and feedback. A well designed results chain identifies risks and makes explicit any underlying assumptions about client (government) or other stakeholder (e.g., firms, civil society organizations, communities) actions. A clearly constructed, logical results chain is critical to accountability, mid-course correction, and learning, and is also integral to exercising selectivity. In IEG’s experience, the operational results chain is often the weakest element in the results frameworks. A F i n a n c i n g f o r D e v e l o p m e n t |8 common weakness is in the reporting of outcomes where, for example, there can often be very little  association or linkage between project‐ and higher level outcomes. In the absence of a clear results chain to  explain why the Bank Group is pursuing any given objective, the results matrix (metrics) is less meaningful,  making it more difficult to evaluate success or to draw lessons for the future.”  Many World Bank Group country programs addressed statistical capacity issues. In most of the countries  reviewed, one or more country strategy documents included dialogues and assistance for statistical capacity  building, although normally these were not highlighted and not reflected in the objectives or the results  frameworks. Support was usually provided through small trust‐funded grants (except for a large International  Bank for Reconstruction and Development (IBRD) loan for Indonesia), but was also included as part of  broader economic management programs. In some cases (Yemen for example), the support for statistical  capacity building was linked directly to results monitoring under the PRSP and the country strategies; in other  cases, the support was linked more generally to the need for improved poverty analyses (for example,  Bangladesh) or improved domestic resource management (Nigeria, for example). Note, however, that the  World Bank provided the least statistical support in the poorest countries in the sample—no specific project  in Malawi, and one recent operation each in Nepal and Yemen.  The World Bank Group did not have a clearly  articulated results chain for the MDGs, nor did it  have a systematic process for assessing or  evaluating its contribution to MDGs. Given the  Feedback Loops are the weakest link in  different views and expectations—both  the poverty and development chain.  externally and internally—on how the Bank  Although good at collecting data on  Group should address the MDGs, a corporate‐ poverty and generating insightful poverty  level results framework and a robust monitoring  diagnostics, these (very similar) capacities  and evaluation system for the MDGs would  were not applied to monitoring and  evaluation. The Poverty Focus of Country  have provided stakeholders and partners with  Programs report reiterates observations  better information on the Bank Group’s  made in many other evaluations about  institutional response and contribution to the  weaknesses in results frameworks,  MDGs. Consequently, there was no effective  indicators and metrics, and measurement  feedback loop that would enable addressing  systems. It also highlights that country and  specific MDG areas that might need special  project results frameworks are not well  attention. Identifying intermediate outcomes  integrated, and recommends that all Bank  that can be monitored would have helped link  projects include baseline data and  the MDGs to World Bank Group activities. The  guidelines for sustained monitoring and  2001 World Bank Group Strategic Framework  evaluation.  emphasized that intermediate goals should take  into account the differences in country and  regional conditions and develop realistic  timeframes. Many World Bank and International Finance Corporation (IFC) projects (both financing and  knowledge services) contributed to the MDGs, but measuring the impact of these projects based on specific  MDG indicators focuses on only one facet of the Bank Group’s work; other intermediate outcomes are  needed to take into account the long‐term nature of the MDGs while clarifying the paths through which the  Bank Group contributes to the MDGs.   9 | T r a n s f o r m i n g   O u r   W o r l d   –   A i m i n g   f o r   S u s t a i n a b l e   D e v e l o p m e n t Engaging Collectively for Achieving the MDGs This section reflects on how, in partnership, the World Bank engaged with key stakeholders in pursuit of the MDGs. Evaluation of that experience has yielded lessons that will be valuable going forward. The SDGs are global goals, not just to be attained by so-called “developing” or partner countries, but by all countries around the world. As is obvious from the related COP21 process, sustainability is everyone’s business and so is sustainable development. This shift has implications for the Bank Group’s relationships with a wide range of stakeholders—for relationships between public and private organizations working in development, relationships between countries and relationships between actors within countries and across boundaries. In engaging with the MDGs, the World Bank adopted the principle that it would step back in cases where other institutions had a clear comparative advantage and would encourage and support leadership roles for partner organizations. The 2001 World Bank Group Strategic Framework recognized that “the alignment of the efforts of the development community with the MDGs presented an unprecedented opportunity to improve effectiveness in poverty reduction.” Applying this principle in the context of other partners’ support programs for the MDGs, the Bank Group engaged in partnerships at several levels in the pursuit of the MDGs. Since the adoption of the MDGs, the World Bank Group experienced an increase in MDG-related trust funds. Between fiscal years 2002 and 2014, a total of $81 billion was channeled through the World Bank alone (figure 5). The expansion of trust funds, in particular the financial intermediary funds, created opportunities as well as challenges. Trust fund reforms resulted in the integration of Bank-executed trust funds and recipient-executed trust funds into regular Bank processes. For the financial intermediary funds, with all but the Global Environment Facility being established after 2000, and with the expressed objective to support MDG issues, each had its own governance structure and operating modalities. This made it challenging to fully integrate and align programs through intermediary funds with World Bank Group country strategies. Figure 5. Trust Funds Support for MDG-Related Sectors, FY02–14 Bank-executed trust funds: Recipient-executed trust Financial intermediary $5.4 billion funds: $30.8 billion funds: $44.6 billion EDU , 8% EDU , 1% HNP, 11% EDU , Others, 15% 20% HNP, ENV, 12% 14% ENV, 13% Others, 70% Others, 65% HNP, 66% ENV, 6% Source: Operations Policy and Country Services Concessional Financing. Note: EDU = Education; ENV = Environment; HNP = Health, Nutrition, and Population. F i n a n c i n g f o r D e v e l o p m e n t | 10 Independent reviews and IEG evaluations identified issues with some partnerships and global programs. IEG, in its work on partnerships and trust funds, has found four common challenges related to selectivity, oversight, linkages to country operations, and results frameworks:  Selectivity: Most donors allocate funds from a fixed envelope for total official aid; trust funds have not increased the size of that envelope. As earmarked pots of money with separate approval and allocation processes, trust funds tend to increase transaction costs for client countries and for the Bank and to impose parallel budgeting and approval processes. That is why the Bank needs to be selective in what trust funds and what governing procedures it agrees to.  Oversight: Evaluations have found weaknesses in governance and transparency in many partnership programs, as well as frictions and conflicts of interest from the multiplicity of roles that the Bank typically performs in partnerships. Yet The Bank has no routine oversight and tracking of partnerships and of how it engages in them.  Links to country programs: The Bank is uniquely placed to help client countries benefit from global programs. However, there are often missed opportunities at the intersection of the Bank’s participation in global programs and its country engagements. There are no explicit agreements on division of labor between the Bank and some major global health programs.  Results are often unknown. Although there has been progress in recent years, many of the partnerships IEG has reviewed lacked clear goals and indicators. It is often hard to attribute results to specific partnerships let alone assess results across the portfolio. IEG identified seven guiding principles that could enhance evaluation of partnership programs, and with that their results-focus and accountability: What Helps Reforms Succeed? 1 To improve the authorizing environment, make sure there is a mutually agreed evaluation policy. Many partners will have different ideas about evaluation. Agreeing, right from the start, what will be evaluated, by whom, when and why is important to ensure evaluations are conducted in a timely manner. If it is a self-evaluation, then arrangements for an independent validation or review are needed to ensure credibility. 2 To improve credibility, ensure evaluation independence. How much independence is necessary? The answer to that question will vary considerably across multilateral development institutions and bilateral donors. Varying standards of independence among partners is often a source of confusion and has a direct impact on evaluations of partnership programs. That’s why it is important that partners develop a shared understanding about what are the acceptable standards for independent evaluation. 3 Invest time in planning the evaluation. Partnership programs often cover a wide range of activities, all of which need to be assessed by the evaluation. Frequently, the partnership arrangements themselves, from governance, to funding and staffing issues, will also be evaluated. Defining the scope and coverage of the evaluation in clearly defined ways is necessary to ensure that partners share expectations, that their most important questions are answered, and that adequate resources are allocated for the exercise. The clearer the boundaries, the more focused the evaluation will be. 11 | T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t 4 Choose criteria that fit the purpose. Evaluations of partnership programs use two broad sets of criteria: one for the development outcomes of the partnership program itself, and the other for its organizational effectiveness to measure how well the partnership is functioning. In assessing outcomes, evaluations build on the standard evaluation criteria established by the OECD/DAC in 1991. For questions of organizational effectiveness, the focus is on governance structures— sharing of voice and power on one hand and of financial burdens on the other—and on whether the partners are in fact getting what they expected out of the partnership. 5 Make sure the evaluation is transparent and that key stakeholders are consulted. It’s not enough for an evaluation to be properly carried out—if it is to have an impact, its findings need to be well received, discussed, and acted upon. Transparency is paramount in this context. If any of the partners or stakeholders do not understand the evaluation process or the criteria against which activities are evaluated, they may reject the findings, however sensible they may be. Consultations with stakeholders—from all different perspectives and interest groups—are essential. They will also help to ensure that the findings don’t come as a surprise. 6 Ensure that recommendations are agreed on and followed-up. As with all evaluations, recommendations should derive from findings and conclusions based on evidence. In partnership programs, these recommendations might be directed more towards one group than another. In general, the wider the scope of these evaluations, the harder it can be to limit the recommendations to those that are the highest priority and where their implementation can be monitored. 7 Plan dissemination in advance. Making a difference with an evaluation can also require thinking beyond the main stakeholders. Broader sharing of the evaluation’s findings through the Internet and various other means is an important aid to learning. Unfortunately, we’ve found that this is something that too few of these evaluations do. MDGs: Education, Health, Water Supply and Sanitation Different sectors within the World Bank responded to the MDGs differently, reflecting analysis of the World Bank’s added value in the respective sector. The responses of the education, health, and water and sanitation sectors to the MDGs were reviewed in the 2014 Results and Performance report as illustrative examples. Sector Strategies. In education, the World Bank shifted its focus from primary education in the 1990s to a broader agenda starting in 2005 to include learning outcomes, education and labor market linkages, and service delivery systems. Given the massive increase in health-related trust funds, the World Bank took a more focused approach in 2007 by emphasizing health systems strengthening. In water supply and sanitation (WSS), the World Bank addressed access issues, but emphasized sustainability of sector investments on the basis of an important lesson from the 1980s where “performance by target” led to pursuing quick results at the expense of bypassing more difficult reforms. Commitments. The World Bank’s support for education, health, and water and sanitation increased relative to overall commitments and became more focused. Primary education focused on IDA countries: average annual IDA commitments to primary education increased by 26 percent between FY90–01 and FY02–14, but IBRD commitments declined by 28 percent. Almost half of Health, Nutrition, and Population (HNP) F i n a n c i n g f o r D e v e l o p m e n t | 12 commitments went to strengthening health system performance, although close to 70 percent of the reproductive health and disease-related (except HIV/AIDS) projects had health system components. In line with the status of the respective MDG targets—the global water supply target has been met but the sanitation target is not likely to be achieved—World Bank lending to sanitation increased almost eightfold, while that to water supply increased by 40 percent. Country coverage increased in all three sectors (table 1), which had operations in almost all countries in the Africa Region and FCS. About two-thirds of the countries covered by maternal and child health (MCH) and sanitation projects were off target, which is consistent with the call to focus on lagging MDGs. Primary education and water supply had a lower proportion of off-track countries in their portfolios: support to primary education in the on-target countries focused on quality issues since the World Bank’s past experience was that global priorities were not the highest priorities in all countries (World Bank 1999). In water supply, World Bank support sought to address institutional strengthening and capacity building. Table 1. Country Coverage in Primary Education, Health, Water, and Sanitation Portfolio, FY 02–04 Countries added Countries covered Countries off-track MDG sectors (number) (number) (percent) Primary education 26 103 54 Maternal and child health 29 92 64 Water 28 106 51 Sanitation 30 107 66 Source: Business Warehouse. The World Bank adopted multi-sector approaches in these three sectors. Public administration was the most prevalent component in primary education and HNP projects. Close to 70 percent of primary education projects approved during FY02-14 had public administration components, with 22 percent having HNP components. In HNP, 62 to 82 percent of projects in various sub-sectors supporting different MDGs had public administration components, with 9 to 27 percent having education components. Although improved water and sanitation were considered critical to health outcomes, a negligible number of HNP projects had WSS components. Nonetheless, because 95 percent of the countries with HNP projects also had WSS projects, the country strategies were seen as the instrument to ensure synergy between HNP and WSS programs. In the case of WSS, there has been an increase in WSS components in projects managed by other sector boards, mainly in Urban Development and Environment. Existing evaluation evidence is inconclusive with regard to the effectiveness of multi-sector approaches in delivering results. More analysis is needed to fully understand how this approach would help in achieving the MDGs. Development outcomes for projects in primary education, HNP, and WSS hovered around World Bank averages, but many faced significant or high risk to development outcomes due to uncertain financial sustainability and institutional weaknesses (table 2). Addressing the financial sustainability risk would require greater attention to improving domestic resource mobilization capacity and, more generally, 13 | T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t establishing stronger linkages between direct MDG support and public sector governance initiatives. During FY02–14, about half of the World Bank’s non-MDG public sector governance commitments were in public expenditure and tax policy, and administrative and civil service reform. Although MDG-related projects included institutional and capacity building components, the effectiveness of these components would need to be improved. Table 2. Risk to Development Outcomes in Primary Education, Health, and Water and Sanitation Projects Primary Risk factors Education MCH HIV/AIDS WSS (N=48) (N=59) (N=48) (N=52) Uncertain Financial Sustainability 29 50 32 26 Institutional/Capacity Weaknesses 27 22 36 33 Lack of Ownership 4 24 4 9 Political/Security Risks 11 7 9 15 Source: IEG reviews of the Implementation Completion Reports of relevant projects approved during FY02–14. Notes: A project may have multiple risk factors. MCH = maternal and child health; WSS = water supply and sanitation. N = number of projects. The World Bank Group followed different sectoral approaches to MDG partnerships. Although trust funds supported all sectors reviewed, there were some notable partnerships that complemented the World Bank’s MDG efforts. IEG’s 2011 evaluations of trust funds and global and regional partnership programs found that on specific issues such as primary education and child immunization, trust funds facilitated substantive coordination of donor funds, but there was no clear evidence that trust fund resources added to global official development assistance, and that half the partnership programs reviewed faced issues of sustainability due to ineffective resource mobilization strategies, poor management, or difficulty in demonstrating results. In response, the World Bank is pursuing reforms of its trust fund activities.  In education, it relied on the Education for All-Fast Track Initiative to channel donor assistance for primary education in low-income countries, with the World Bank managing and supervising the projects. This gave space for the World Bank to channel its own resources into support for education and labor market linkages and accountability and incentive systems to improve service delivery.  In health, disbursement from the health sector financial intermediary funds was equivalent to the total HNP sector commitments approved during FY02–14. The Global Fund to Fight AIDS, Tuberculosis, and Malaria, which was established in 2002 to support the MDGs, disbursed $26.7 billion during 2002–14, about five times the Bank’s disease-related commitments. The Five Year Evaluation of the Global Fund in 2009 found that the external financing of the disease control services, although helped to mobilize existing capacity in the affected countries, had stretched the countries’ generally weak health systems to their limits. IEG’s review of the Global Fund found that country-level stakeholders saw the Global Fund as not well integrated into the donor coordination mechanism in the countries. F i n a n c i n g f o r D e v e l o p m e n t | 14  In Water Supply and Sanitation, the partnerships focused mainly on advocacy, coordination, and knowledge sharing, as with the Sanitation and Water for All partnership. Public-private partnerships were viewed as key to scaling up service delivery in WSS. The World Bank and IFC developed various forms of engagement with the private sector, and several Bank projects supported private sector participation in urban and rural WSS. 15 | T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t MDG 4 and 5: Mother and Child Mortality In 2013, IEG released its report, Delivering the Millennium Development Goals to Reduce Maternal and Child Mortality, which provided a comprehensive and systematic review of impact evaluation evidence in this area, looking beyond the World Bank Group’s experience. The study focused on a first- tier outcome, namely skilled birth attendance, which at the time of adopting the MDGs was one of the targets, and higher level outcomes of mother and child mortality rates. The findings are relevant to the continued pursuit of the MDGs and SDGs. Overall, the evidence base on intervention effectiveness is remarkably thin. Even when data from test settings (with controlled environments) point towards effective solutions, these test results do not guarantee that the same intervention will work in real-life settings. It is therefore paramount that the SDG commitment to rigorous data collection, monitoring, evaluation and informed decision- making be upheld. In spite of mixed evidence, the review indicates that  Countries where mortality rates are highest and rates of skilled birth attendance are lowest are more likely to be impacted by appropriately designed interventions;  Where reported, existing impact evaluations indicated that households with a lower socio- economic status realized larger benefits from these interventions. Countries with a heavier child mortality burden at baseline or fewer births attended by skilled health personnel are more likely to see an impact on those outcomes;  Skilled birth attendance was used as a major intervention in reducing the under-five mortality rate, and combined with other interventions, can improve breastfeeding practices and increase immunization rates and anthropometrics of children;  Reductions in maternal mortality are attributable to integrated mother and child health interventions that promote skilled birth attendance together with complementary and reinforcing strategies. Interventions in non-health sectors were found to consistently and substantially lower infant mortality, which calls for greater coordination and collaboration across sectors The reviews suggests bundling health interventions affecting both provision (supply side) and use (demand side) for both maternal and child mortality, including:  Formal education and community-based delivery of service packages with interventions affecting mothers’ knowledge and information for neonatal mortality  Energy and air pollution, water and sanitation, education, and governance interventions affecting strategy and planning to affect child mortality, especially infant and under-five mortality  Conditional cash transfer or voucher interventions that affect mothers’ ability to pay to improve skilled birth attendance rates  Health worker training in conjunction with providing family services and increasing household health knowledge to improve infant mortality Interventions with longer periods of exposure, which are more likely to report significant effects. F i n a n c i n g f o r D e v e l o p m e n t | 16 New Evaluation Evidence Relevant to Specific SDGs Although the SDGs will be adopted only in September 2015, many of them have been part of the agenda of many broad-based development institutions like the World Bank Group for many years. Past experience, therefore lends itself to draw lessons and ensure progress can be accelerated. Goals and targets are numerous. Some have been dealt with in the two other papers in this series, namely:  Financing for Development, which provides insights on public, private, and pooled funding from both domestic and international sources, and  A forthcoming paper for the COP21, which covers all of the climate-related SDGs. The following offers lessons from evaluations that address specific goals or targets beyond those that have been discussed in the MDG section above, and includes links to the source documents for additional insights. These findings are presented in the numerical order of the SDGs (goals and targets) to make cross- referencing easier. Target 3.2: Road Safety Building safer roads has been a pillar of the World Bank Group’s strategy for its transportation sector since 2008. IEG’s Making Roads Safer aims to accelerate progress towards this strategy with lessons drawn from independent evaluations. With the help of the Global Road Safety Facility, established in 2006 as a global partnership program, the World Bank has promoted, together with the World Health Organization (WHO), the “safe systems” approach which integrates road engineering, user behavior, policing, and accident response into a transportation system. This multifaceted approach calls for:  In-country coordination across departments and agencies concerned with various dimensions of road safety. As the IEG review suggested, this should ideally be done by a body that is adequately financed and staffed and is equipped with a rigorous results framework to measure progress.  Road projects that have not only road safety as their objective, but incorporate specific activities to support it, including actions to improve the behavior of road users and to raise their awareness of safety issues. Attention still focuses more on completing road works, with safety components of projects being addressed too late.  Road safety funding is often well below the 10 percent standard, recommended by the Commission for Global Road Safety. This is also the case for World Bank projects, but with its decision to mainstream safety into its budget allocations for road projects, this type of funding is steadily increasing. Target 3.8: Access to Health Services—Health Finance Providing fair health care for everyone without financial risk means helping countries find effective ways to fund and sustain quality health systems. As IEG’s evaluation of World Bank Group Support to Health Financing found, that it is not just poor countries that struggle to develop health systems that efficiently make use of available resources and provide quality 17 | T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t care in an equitable way to the entire population. The evaluation points to health financing as a key policy instrument in meeting this challenge. The way countries finance health care influences how well the service delivery system performs, including how equitable and efficient it is. Countries decide how to mobilize revenues from public and private sources for financing health care, how to pool revenues and protect populations against financial risk, and how to purchase services wisely from health care providers. IEG found that efforts to improve resource mobilization, pooling, and purchasing for health are more effective and institutionally sustainable when they are coordinated and linked to broader public sector reforms. Elements of an integrated approach include compulsory and less-fragmented pooling in insurance and national health systems as well as strategic purchasing that examines potential adverse effects in a broader public sector context. Where Bank support to purchasing was combined with other health financing functions and linked to the public finance context rather than limited to narrowly defined payment methods, it has been relatively effective. Results-based financing has become more popular as a means to make health services accessible to all. It typically involves a cash payment or nonmonetary transfer made to a government, manager, provider, payer, or consumer of health services after predefined results have been verified. Availability of care increased where countries moved from line-item budgets to activity- or performance-based payments. But without measures to reduce user fees and improve risk pooling, results-based financing is unlikely to lead to greater equity in access to health care services and protection from financial hardship caused by illness. Target 4.2: Early Childhood Development Early childhood development is a particularly strong case to illustrate the need for a multisector approach and spans a number of SDGs, such as target 2.2. on malnutrition in children under 5 years of age, given the strong association between stunting and subsequent delays in children’s development, and target 4.2., which focuses on early learning opportunities such as child care or preprimary education for all children by 2030 to improve primary school readiness. Countries will need implementation support in health, nutrition, social protection, and education programming to improve children’s development (i.e. physical, cognitive, language, and socio-emotional). The 2015 IEG evaluation on World Bank Support to Early Childhood Development noted the need for a strategic framework and organizational structure to support a coordinated approach within the Bank and with its partners. The evaluation also suggested that the impact of the World Bank could be increased by changing its focus on health and survival to include child stimulation and development interventions in health, nutrition, and social protection. Given the important role that parents play in stimulating their children’s development, more emphasis is needed to support families within existing programs in health, nutrition, social protection, and education. IEG’s Systematic Review of Impact Evaluation Evidence on the Lasting Effects of Early Childhood Interventions found that stimulation positively impacted children’s development across a range of domains. The review also noted the need for nutrition interventions to be sustained during the first 1,000 days and beyond to have lasting impacts. F i n a n c i n g f o r D e v e l o p m e n t | 18 Goal 7: Sustainable Access to Sustainable Energy—Access to Electricity The World Bank Group has committed to contributing to the Sustainable Energy for All (SE4All) Initiative to achieve universal access to electricity by 2030. The SE4All challenge is most firmly centered in Sub-Saharan Africa, where 40 of the 51 “low-access” (less than 50 percent population connected) countries are located. Achieving the SE4All commitment for low access countries requires a quantum leap from the present pace of 1.6 million connections per year to 14.6 million per year until 2030. The corresponding annual investment required, including generation needs, is $37 billion. IEG’s 2015 evaluation, World Bank Group Support to Electricity Access, underscores the scale of the challenge, and prods the World Bank Group to reposition its role in the sector by going well beyond the confines of its own direct support of $63.5 billion over a 10-year period. The evaluation finds that the associated development outcomes were favorable overall. But low-access countries, particularly in Sub- Saharan Africa, were not particularly well served. Low-access countries received only 22 percent of World Bank, and 6 percent of IFC lending for the sector. The report calls for a new and transformative Bank Group strategy to help country clients—one that moves from a predominantly project-by-project approach to a greater use of a sectorwide organizing framework and process for mainstreaming the sustained engagement needed to implement rapid access scale-up. To crowd in much-needed financial resources from both public and private sources, the report calls for a new investment financing platform that goes beyond the Bank Group’s own conventional project and transaction modes of operation. Target 8.2: Innovation IEG’s 2013 evaluation of World Bank Group Support to Innovation and Entrepreneurship noted the critical role of innovation in economic growth and the substantial World Bank Group engagement in the area. Although market and government failures and other bottlenecks impede innovation and entrepreneurship, developing countries need to build the capacity to find, absorb, and use new technologies and processes as well as foster entrepreneurs who can take risks, look for finance, and bring new products and processes to market. The evaluation states that “countries at different stages of development identified five common principles that can be useful in promoting innovation and entrepreneurship:  Support public investment in research and development (R&D) that focuses on improving efficiency and relevance to end users as well as on strengthening the use of research results in public policy decisions;  Build domestic science, technology, and innovation capabilities to make effective use of global knowledge;  Strengthen linkages between public R&D and private sector users of technology and knowledge;  Build a strong enabling environment, including effective use of information and communications technology; and  Provide flexible financing arrangements to encourage innovative firms to develop new products, processes, and services.” Although the World Bank Group’s portfolio contains interventions in these areas, a number of client countries, especially upper middle-income countries like China, Chile, and Brazil have given high priority to 19 | T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t innovation in their development plans, which then is reflected in countries strategies of the World Bank Group and its lending portfolio. The evaluation found that interventions may individually be successful, but they do not automatically foster synergies or an integrated system. Instead, it is essential to—and the Bank Group has done so—invest in fostering linkages between actors in the innovation system. The examples given in the evaluation suggest that successful projects linked actors from various domains (research, private, sector, service providers, government, development partners, etc.) and used a variety of mechanisms, including competitive grants and business incubators to foster the development of innovation and entrepreneurship. Being successful in adopting policies that stimulate private investments in innovation, and making public investment choices that further R&D will require the understanding of the dynamics among agents to make strategic choices about where and how to intervene. Target 8.10: Financial Inclusion Access to financial services has long been believed to lift people out of poverty by allowing them to seize economic opportunities. Strong progress has been made in the last three years: the number of people reached by formal financial services increased by 700 million. Yet a substantial credit gap remains, and 2 billion adults do not have access to formal financial services. IEG’s forthcoming evaluation, Financial Inclusion: A Foothold on the Ladder toward Prosperity?, seeks to inform both the short-term drive and the longer-term activities concerned with connecting the poor to financial services that improve their well-being. The evaluation found that:  The World Bank Group contributed significantly to progress in financial inclusion. It has reached a significant share of the microfinance industry and helped make it more sustainable. Yet it is unclear how much of this support reaches actual microenterprises and the poor—as opposed to substantially larger businesses, like small- and medium-sized enterprises (SMEs).  World Bank Group support to create policy frameworks that allow financial intermediaries to better serve the needs of the poor was effective. Yet its approach to identifying and tackling constraints to financial inclusion is neither systematic nor comprehensive—of particular concern is areas that affect the poor, like mobile banking or rural savings and credit cooperatives.  Through its global partnerships, the Bank Group has been able to substantially increase its impact in advancing financial inclusion; however, partnerships also carry some risks and costs. Most importantly, the World Bank Group’s strategy on financial inclusion can be strengthened. The Bank Group’s public commitment to Universal Financial Access by 2020 is likely to yield a push for transaction accounts. However, high dormancy rates of newly opened accounts raise doubts about whether providing access through such accounts results in universal inclusion. Instead of focusing on “headline numbers” the relevant goal may be to provide service to everyone with a productive and beneficial use. Finally, IEG’s findings point to a need for a more sequenced and evidenced approach as evidence has been growing showing that financial inclusion has been less effective in fighting poverty than was previously hoped. F i n a n c i n g f o r D e v e l o p m e n t | 20 Target 9.3: Small and Medium Enterprise Development IEG’s 2014 evaluation The Big Business of Small Enterprises observed that the World Bank Group promotes SME growth through both systemic and targeted interventions. A critical challenge is to root the many activities now undertaken in this broad space in a clear understanding of the characteristics and dynamics of SMEs, their role in the broader economy, and their actual and potential contribution to jobs, growth, and shared prosperity. A closely related challenge is to formulate clear strategies that connect interventions to intended outcomes and are accompanied by solid measurement systems that provide evidence of results and allow learning. Targeting means focusing benefits on one size class of firms to the exclusion of others. Targeted support for SMEs is a big business for the World Bank Group, averaging around $3 billion a year in commitments, expenditures, and gross exposure over the 2006–12 period. In the context of broader reforms, targeted SME support can be a powerful tool and, given the size of the recent program, it is vital for the World Bank Group to use it effectively. Targeting SMEs is not an end in itself, but a means to create economies that can employ more people and create more opportunity for citizens to achieve prosperity. A thriving and growing SME sector is associated with rapidly growing economies. As the World Bank Group continues to support SMEs, to help them realize their potential contribution to developing economies, the IEG evaluation concludes that, to make targeted SME support more effective, the Bank Group needs to do several things. Targeted support for SMEs needs to be firmly rooted in a clear, evidence-based understanding of how the proposed support will sustainably remove the problems that constrain SMEs’ ability to contribute to employment, growth and economic opportunity. This requires a tailored research agenda to support and assist these clarifications and refinements of its SME support approach, using the best qualified researchers. Likewise, interventions that make it their objective to assist SMEs should be designed in ways that they define and justify the specification of the beneficiary group, provide specific targeting mechanisms, and include impact indicators in their results and monitoring and evaluation frameworks. Finally, the Multilateral Investment Guaranty Agency’s (MIGA) Small Investment Program should undergo a radical change in approach to providing guarantees for investments in SMEs, considering either a merger with its regular program or a fundamental redesign to improve performance. Learning from the Previous for the Next 15 Years Translating global goals to sustainable national programs remains a key challenge in the post-2015 era. In particular, it will require ensuring country ownership and leadership that is combined with the best advice and solutions from a global perspective so that countries can learn from each other and together achieve the more challenging SDGs. For development finance institutions like the World Bank Group it requires balancing its global mandate with its country engagement, both of which the institutions aims to strengthen through the recent reform process. The experience with the MDGs highlights the importance of having a country instrument such as the PRSP to help identify realistic national medium-term targets and programs toward achieving long-term goals and securing ownership among stakeholders. In this process, there is a need to balance “quick wins” and long- term sustainability, manage the constructive tension between country focus and issue focus, address domestic resource mobilization and institutional capacity issues, and build strong partnerships at all levels. 21 | T r a n s f o r m i n g O u r W o r l d – A i m i n g f o r S u s t a i n a b l e D e v e l o p m e n t To deliver on the twin goals and the post-2015 agenda, the Bank Group would benefit from a clearly articulated role, approach, and expected contribution to the SDGs, both externally for enhancing partnerships and internally to facilitate prioritization and synergies. As this paper has shown, the World Bank Group works actively in many areas relevant to the SDGs—actually many more than covered here— but various evaluations have pointed to the importance of multi-sector integrated approaches that challenge countries and their partners to find new ways of working. The challenges that the SDGs aim to address, and the SDGs themselves, are complex, and solutions will have to be tailored to context, bring together multiple actors, and benefit from dynamic, constantly adjusted planning and execution that is informed by ongoing monitoring and evaluation. F i n a n c i n g f o r D e v e l o p m e n t | 22 www.ieg.worldbank.org