Report No. 27945 An OED Review of Social Development in Bank Activities February 17, 2004 Operations Evaluation Department Document of the World Bank Abbreviations and Acronyms AFR Sub-Saharan Africa ARDE Annual Review of Development Effectiveness CAS Country Assistance Strategy CBO community-based organization CDD community-driven development EAP East Asia and Pacific ECA Europe and Central Asia ES Evaluation Summary ESSD Environmentally and Socially Sustainable Development FY fiscal year HD Human Development HR Human Resources ICR Implementation Completion Report IP indigenous peoples LCR Latin America and the Caribbean MDG Millennium Development Goal MNA Middle East and North Africa NEAP National Environmental Action Plan NGO nongovernmental organization OED Operations Evaluation Department OPCS Operations Policy and Country Services PAR Project Performance Assessment Report PMU Project Management Unit PREM Poverty Reduction and Economic Management PRSP Poverty Reduction Strategy Paper QACU Quality Assurance Compliance Unit QAG Quality Assurance Group QEA Quality at Entry Assessment SAR South Asia SDV Social Development Department SME small and medium enterprises SSP Sector Strategy Paper Director-General, Operations Evaluation : Mr. Gregory K. Ingram Director, Operations Evaluation Department : Mr. Ajay Chhibber Manager, Sector and Thematic Evaluation : Mr. Alain Barbu Task Manager : Mr. Ronald S . Parker i Contents ....................................................................................................................................... Preface v 1. The Analytic Context for the Review ............................................................................... 1 Evolving Staff and Shareholder Values Led to Changes in Focus ................................ 1 Social Development I s a Fuzzy Concept ....................................................................... 1 Differing Perspectives Have Bureaucratic Implications ............................................... 2 The Bank’s Social Development Objectives .................................................................. 3 Study Design .................................................................................................................. 5 2 . Where I s Social Development Being Conducted? ........................................................... 7 I s There a Social Development “Portfolio? ” ................................................................ 7 It’s Done Everywhere, But the Portfolio Varies By Region and Sector ........................ 8 Regional Distribution........................................................................................ 8 Sectoral Distribution ......................................................................................... 9 Portfolio Overlaps ............................................................................................. 9 3 . I s Social Development Relevant? ................................................................................... 11 I t I s Relevant to the Bank’s Broader Development Objectives ................................... 11 I t I s a High Priority at the Country Level ................................................................... 11 I t I s Relevant at the Project Level ............................................................................... 15 4 . Have Social Development Interventions Been Effective? ............................................ 16 Effectiveness o f Projects with Social Development Components Compares Well with Others .................................................................................................................... 16 Outcome .......................................................................................................... 16 Sustainability .................................................................................................. 17 Institutional Development Impact................................................................... 18 Projects with Overlapping Social Development Themes Have Better Ratings ...........18 5 . W h a t Challenges Affect Implementation of Social Development? ............................. 20 Consistency: Treatment the Various Social Themes Receive I s Highly Variable .......20 Maintaining the Flow of Socially Relevant Information ............................................. 21 Borrower Institutional Development in the Broadest Sense ....................................... 24 f Social Themes....................................................................... The Interdependence o 27 6 . W h a t A r e the Implications for Bank Organization and Procedures? ............ 28 The Bank’s Focus on. and Support for. Social Development Could Be Improved .....28 .. 11 Working Across Internal Units M o r e Would Help ..................................................... 30 Regional Groups Are a Key Link ................................................................................ 31 Bringing Social Development Staff Closer to Task and Country Teams Would Help .. 32 Bank and Borrower Social Development Inputs Could B e Fine-Tuned ..................... 33 Good Social Assessments Focus on Vital Issues ........................................................ 34 7 . Conclusions and Recommendations f o r the Bank’s Sector Strategy ............. 36 Recommendations ....................................................................................................... 41 Notes ....................................................................................................................................... 43 Attachment: Supplemental Charts and Tables .................................................................. 49 Annex A . Methodological Note ............................................................................................ 53 Annex B.Definitions o f Social Development ...................................................................... 65 Annex C .W o r l d Bank Operational Policies Relevant to Social Development ................67 Annex D. Key Characteristics o f Highly Satisfactory Projects......................................... 68 Annex E. Selected Comments o f W o r l d Bank Staff on Social Development Issues ........72 Annex F. Survey Instruments.............................................................................................. 81 Annex G.Management Response ........................................................................................ 88 Annex H. Chairman’s Summary: Committee on Development Effectiveness (CODE) .97 Works Cited ......................................................................................................................... 103 Bibliography ........................................................................................................................ 104 Boxes B o x 2.1: H o w M a n y Social Development Professionals Work for the Bank? ............................... 8 B o x 3.1 : Latin American CAS Success for Indigenous Peoples .................................................... 11 B o x 3.2: C A S Priority Issues (Those Covered Primarily by the Social Development Department and i t s Regional Specialists in Bold) ..................................................................................... 13 B o x 4.1: Key Characteristics o f Highly Satisfactory Projects ...................................................... 19 B o x 6.1 : Modifications Suggested by Task Managers .................................................................. 29 B o x 6.2: Country Directors’ Recommendations ........................................................................... 32 B o x 6.3: Social Analysis Versus Social Assessment .................................................................... 33 B o x 7.1 : Success Factors in Highly Satisfactory Projects with Social Development Components. 36 ... 111 Tables Table 1.1: Coverage o f Social Development Priorities by Selected Documents ............................. 4 Table 2.1 : Social Development Portfolios ....................................................................................... 7 Table 2.2: Regional Distribution o f Portfolios (percent) ................................................................. 9 Table 2.3: Portfolio Overlaps (percent) ......................................................................................... 10 Table 4.1 : Ratings for Projects with Two, Three. or Four Overlapping Social Development Themes ................................................................................................................................... 19 Table 6.1 : Staff Groups Generally Oppose Merging Social Development Groups ....................... 31 Table 6.2: Quality o f Social Risk Assessment (Percent Satisfactory or Better) ............................ 34 Figures Figure 3.1: Percent o f CASs that Mentioned Various Topics ....................................................... 12 Figure 3.2: Relationship o f Social Development Themes in CASs to Portfolios .......................... 14 Figure 4.1: Outcome Ratings o f Social Development Portfolios Compared to All Bank Projects from the Same Period............................................................................................................. 17 Figure 4.2: Sustainability Ratings of Social Development Portfolios Compared to All Bank Projects from the Same Period ............................................................................................... 17 Figure 4.3: Institutional Development Impact Ratings o f Social Development Portfolios Compared to All Bank Projects from the Same Period .......................................................... 18 V Preface W o r l d Bank staff were first directed to address the social aspects o f operations in 1984, when social scientists were brought in to deal with rural development issues, but only in 1995, soon after President Wolfensohn’s f i r s t annual meetings address, “New Directions and New Partner- ships,” did the Bank make rapid progress in mainstreaming attention to social concerns. The culmination o f this effort came in 1996, when President Wolfensohn convened a Social Devel- opment Task Force to address the role o f social assessment in Bank lending instruments and Country Assistance Strategies. Then, through the 1997 Strategic Compact, the Bank provided additional resources for social development capacity building, as well as funds to help execute the regional social development action plans. As a result o f these initiatives, Bank social scien- tists began to provide social analysis and take steps to understand the social impacts o f both tradi- tional Bank projects and the Bank’s growing portfolio o f social service projects. The increased attention to the social impacts o f i t s operations led the Bank to build up i t s staff resources in this area. The f i r s t social development unit was created in 1993 with six staff. By 1997 there was a stand-alone Social Development Department (SDV), created as part o f ESSD. In 2002, the scale and importance of social development work in and around Bank projects led management to commission a Sector Strategy Paper (SSP) for social development to guide Bank work in this thematic area. An analysis o f past performance i s an essential building block in the strategy formulation proc- ess. M a n y OED evaluations touch upon social development issues; for this study w e reviewed: Gender (1995, 1997,200 l), Post-Conflict Reconstruction (1998), Nongovernmental Organiza- tions (NGOs) (1999), Participation (2000), Rural Water (2000, 2002), Resettlement (2001), Cul- tural Heritage (200 l),Community-Driven Development in the Sahel(2003), and Forestry (199 l , 2000). In addition, the Social Funds study (2002) and the 2002 Annual Review o f Development Effectiveness (ARDE) deal with the subject. This study i s in large part a meta-evaluation that draws on the work listed above. The study has used the standard OED methodology to analyze which aspects o f the Bank’s social development agenda have attained satisfactory outcomes, significant institutional development impact, and likely sustainability. I t analyzed the degree to which they build upon the Bank’s strengths or strain institutional capacities, and to which they attain their objectives in project preparation or on the ground. Very recent aspects o f Bank social development w o r k that have not yet been evaluated are not covered in this report. A description of the study methodology can be found in Annex A to this report. This report presents the collected findings o f several evaluative exercises: a literature review, an analysis o f the organization o f social development in aid agencies, surveys o f Bank staff, and a meta-analysis of the previous and ongoing OED studies related to social development. Separate reports on each of these exercises are available o n request. The report was prepared by a team under the leadership o f Ronald S. Parker and consisting o f Kristin Little, Anna Amato, and Helen Phillip. Alf Morten Jerve and Gunn Helen Softing of the Chr. Michelsen Institute (Bergen, Norway) prepared the analysis o f the organization o f social development in aid agencies. Kavita Mathur performed the CAS review. Maria M a r provided survey support. William Hurlbut edited the report and provided document production support. vi The authors gratefully acknowledge M a r y Anderson, Samuel Paul, and Alison Scott for their pa- tient and detailed review o f previous drafts o f this report. We also thank Susan Jacobs Matzen for coordinating the efforts o f this study with scheduled events leading to the Sector Strategy Paper and Lawrence Salmen for general coordination with the Social Development Department. We are also indebted to Michael Cernea, Anis Dani, Gloria Davis, Judith Edstrom, and Ashraf Ghani for sharing their insights. The study conducted a mission to learn more about how other aid agencies handle social devel- opment. In connection with this mission, the authors thank Claudia Fumo, Lucia Hanmer, Colin Kirk, Michael Schultz, and Rachel Turner o f DFID; Martin Pallman and Marco Rossi o f the Swiss Agency for Development and Cooperation; Andrew Norton o f the Overseas Development Institute; David Archer and Antonella Mancini o f ActionAid; Cynthia Hewitt o f UNRISD; Helmut Asche, Barbara Hamming, Martina Kampmann, Bianca Schimmel, and Sabine Trom- mershauser o f GTZ; Analie John-Hubach, Pieter Lammers, Margreet Moolhuijzen, Karin Roelofs, Hans Slot, Marijke Stegeman, Rob D. van den Berg, Rob Visser, and Sjoukje Sandra Volbeda o f the Netherlands Ministry o f Foreign Affairs; M a x Spoor o f the Institute o f Social Studies; Poul Engberg Pedersen o f the Centre for Development Studies (Denmark); Lars Elle and Anders Baltzer Joergensen o f DANIDA; Johan Helland, Arve Ofstad, and Gunnar M. Sorb0 o f the Chr. Michelsen Institute; Bergljot Baklien and Jon Naustdalslid o f the Norwegian Institute for Urban and Regional Research; Jan Dybfest and Sigurd Endresen o f the Norwegian Ministry o f Foreign Affairs; Einar Vetvik o f Diakonhj emmet College; Stefan Molund, Alexandra Silfver- stolpe, and Torsten Wetterblad o f the Swedish International Development Cooperation Agency; and Eva Tobisson o f Hultcrantz and Tobisson AB. 1 1. The Analytic Context for the Review 1.1 In 2002, the scale and importance o f social development work in and around Bank pro- jects led management to commission a Sector Strategy Paper (SSP) for social development to guide Bank w o r k in this thematic area. An essential building block in the strategy formulation process i s analysis o f past performance. O E D has carried out evaluations o f many o f the constitu- ent activities o f social development over the past few years, and this review (hereinafter referred to as “the study”) systematically reviewed their findings (a process k n o w n as meta-evaluation), together with the findings o f a literature review, portfolio analysis, and various survey instru- ments. Working papers have been available for staff who are preparing the SSP. Several back- ground reports more fully document the findings presented in this summary report. STAFF AND SHAREHOLDER VALUES LED EVOLVING TO CHANGES IN FOCUS 1.2 The Bank’s original emphasis o n investment gaps and infrastructure deficits has gradu- ally broadened over time into a more holistic vision encompassing knowledge, participation, in- stitutions, and the enabling environment for development because less complex conceptualiza- tions o f development did not attain the expected results. Concomitant with this broader develop- ment vision in the Bank has been both a change in the way projects are analyzed and a change in the m i x o f operations that the Bank supports. Analysis in support o f Bank operations has widened from economic and technical impacts to include indirect institutional and social impacts. 1.3 The focus o f the international development community also has shifted strongly toward the social sectors. This i s particularly evident in the Millennium Development Goals (MDGs)- seven o f the nine are human development related-and sustaining human development gains means p a y n g attention to the social development issues that are at their root (such as inequality, conflicts and social instability, and exclusion). Moreover, the diverse community that constitutes the Bank’s authorizing environment had unambiguously committed i t s e l f to a number o f social development goals before the March 2002 International Conference o n Financing for Develop- ment in Monterrey. The 1995 W o r l d Summit for Social Development in Copenhagen, Denmark, was the first major UN conference dealing specifically with issues o f social development. At that time it was the largest gathering o f world leaders ever-1 17 heads o f state or government. Also in attendance were 14,000 participants, 2,300 o f them representatives from 81 1 NGOs, and more than 2,800 journalists. The m a i n conclusion o f the conference acknowledged that alleviating poverty i s not just a matter o f providing more services-political, economic, institutional, and cultural factors play a critical role. The event culminated in the signing o f the Copenhagen Decla- ration and Program o f Action, putting people at the center o f development. The signatories pledged to “make the conquest o f poverty, the goal o f full employment, and the fostering o f sta- ble, safe and just societies their overriding objectives.” I S A Fuzzy CONCEPT SOCIAL DEVELOPMENT 1.4 The Copenhagen Summit also attempted to define social development, but was unable to arrive at a definition satisfactory to the assembled group. The lack o f broadly accepted definitions for the t e r m social development and a number o f related concepts continues to be a problem, not only for this study but also for social development practitioners everywhere. Development theory i s enriched with many comprehensive concepts l i k e social development, but social development i s more comprehensive than most. In addition, certain new concepts are taken as symbols o f the need for a major reorientation o f operations. Social development i s o f that kind. It i s presented by 2 many as the “missing link” in development aid. I t represents, it i s argued, a perspective o n devel- opment goals, analysis, and working methods that holds the prospect o f ensuring that the key ob- jective o f aid-to eradicate poverty-will be met more effectively. Reflecting this realization, most development organizations n o w address social development in some way. 1.5 Issues covered under the rubric o f social development vary from organization to organi- zation, but even those that do not have a particular policy or strategy for social development at least recognize that their work has social dimensions.’ The report o n the literature review con- ducted by the study traces the evolution o f the notion o f social development over the past few decades. 1.6 N o t only are there countless definitions o f social development (see Annex B for a collec- tion o f suggested definitions for social development), but many o f the terms related to the con- cept m a y have a different meaning from one organization to another or even within the same or- ganization. Moreover, the field o f social development i s notorious for i t s lack o f indicators pre- cisely because o f inherent measurement difficulties that exacerbate the monitoring and evaluation deficiencies o f development projects generally. 1.7 Some attempts have been made in the Bank to define social development. When this study began, the accepted definition was: “Social development is equitable, socially inclusive and therefore sustainable. I t promotes local, national and global institutions that are responsive, accountable and inclusive and it empowers poor and vulnerable people to participate effectively in development processes. ’” The “Issues Paper for a W o r l d Bank Social Development Strategy,” prepared by the Bank’s So- cial Development Department (SDV) commits SDV to developing a more pragmatic and opera- tional definition as part o f the SSP process. A f i r s t attempt3 has been made by the team develop- ing the SSP: “Social development begins with the perspectives o f poor and marginalized people and works towards positive and sustainable changes to make societies more equitable, inclusive and just.” While this i s s t i l l more a statement o f the goal o f social development than a true defini- tion o f what it is, and there i s a clear connection between fuzzy definitions and a lack o f indica- tors, it i s at least somewhat less all-inclusive than the previous definition. DIFFERINGPERSPECTIVES BUREAUCRATIC HAVE IMPLICATIONS 1.8 All definitions o f social development are made up o f building blocks that exhibit a ten- dency to fall into four schools o f thought. Each o f these gives a very different perspective o n which activities to undertake. Development organizations can adopt just one perspective or mul- tiple perspectives. In this they are guided to some degree by the history and traditions o f the country where they are located. Or, for the multilateral banks, the changes in their authorizing environment. 1.9 In a paper commissioned for this study, CMIINonvay develops these four perspectives. Within a single development organization, any combination o f these classifications can b e at play, thus making the internal dialogue quite challenging. People speaking convincingly and pas- sionately about “social development” may not be talking about the same thing. The relative em- phasis given to each varies among agencies and it i s often reflected in the manner in which they structure themselves to deal with social issues. 3 1.10 The first perspective relates to the concept o f “social sectors” and embraces the corner- stones o f Western welfare states, namely public health and education systems, and social insur- ance. I t i s a perspective that hnctions well administratively, being linked t o specific government sectors. 1.11 The second perspective i s similar, but focuses more narrowly o n safety nets and social exclusion. It denotes those areas o f public policy that explicitly attempt to target particular vul- nerable groups. Typically, the activities o f a number o f countries’ ministries o f social affairs cen- ter essentially o n this mandate. 1.12 The third perspective i s more political and influenced by development theory. I t i s about inclusion and empowerment, and to some extent about equity. I t holds that people have certain rights to participate in and benefit from planned development. This perspective has inspired both a far-reaching discourse o n rights-based approaches to development and a more restricted con- cern that, as a bare minimum, people shall not suffer as a consequence o f induced development efforts, bringing about measures such as social safeguards. 1.13 Fourth, social development i s being used as an even more encompassing concept, linking it to efforts and processes induced by development agencies that essentially influence social rela- tions and institutions in a ~ o c i e t yThis . ~ w a y o f perceiving social development has been rein- forced with the growing popularity o f the “social capital” concept and the recent accentuation o f the need for a holistic and multi-dimensional approach to poverty reduction, (the Comprehensive Development Framework i s an example). For partisans o f this approach, social development i s defined primarily in terms o f what you achieve (the impact or end result) and not what you do (the input), much l i k e the concept o f “economic growth” or development. T h i s perspective does not lend itself to bureaucratic departmentalization. 1.14 The phenomenon o f people talking about the same concept but meaning slightly different things has led to a blending o f perspectives that culminated in the definitions o f social develop- ment used by the Social Development Department o f the W o r l d Bank. B o t h the o l d and new definitions cover all four perspectives. The breadth o f the o l d definition seriously limits i t s utility. In addition to reviewing definitions, this study sought to determine what constitutes social devel- opment through an analysis o f the Bank’s business practice in social development. SOCIAL DEVELOPMENT THEBANK’S OBJECTNES 1.15 The Bank’s Social Development Department identifies five “business lines” in social de- velopment: participation and civic engagement; social analysis; community-driven development; conflict prevention and reconstruction; and social safeguards. These roughly correspond to the aggregated social development “portfolio” identified by this study (discussed in Chapter 2). The portfolio includes other themes, such as gender, an important area o f focus that i s handled by several groups within the Bank and the subject o f a “sector and thematic strategy” Operational Policy. Most other organizations consider gender part o f social development. Using the business lines as a guide, the study team dissected their components into objectives t o arrive at a charac- terization o f what social development i s in the Bank. M a k i n g them discrete i s essential to the ana- lytic process, although it tends to obscure their connectedness and interactivity. 1.16 T o develop this characterization, an objectives matrix was created that synthesizes Bank objectives for social development. Internal and external documents and statements relevant to the Bank’s work with social development are listed in Table 1.1 (see also Annex A, B o x Al). 4 1.17 The objectives outlined by the documents are ~ o h e r e n tF o r example, gender objectives .~ in the Concept Paper, the Task Group Report, the Annual Meetings Speeches, the W o r l d Devel- opment Report, the safeguards, and the Strategic Compact are a l l consistent with one another. Furthermore, this B a n k consensus o n gender objectives resonates with the objectives in the statements agreed o n by the broader development community (the Copenhagen Declaration and the MDGs), and thus reflect the views o f the Bank’s authorizing environment w h i c h includes the broader development community. Aside f r o m the consistency in message across the board, the sheer number o f times objectives were mentioned in such a w i d e variety o f documents i s testa- ment t o their importance to the Bank. The matrix shows that, in the absence o f explicitly adopted objectives and a strategy to achieve them, the Bank i s nevertheless sending out a consistent and cohesive message in i t s broad social development goals. Table 1.1: Coverage o f Social Development Priorities b y Selected Documents Poverty Reduction 9/9 X X X X X X X X X Gender 8 X X X X X X X X Sustainability 7 X X X X X X X Partnership 7 X X X X X X X Participation 7 X X X X X X X Human Development 7 X X X X X X X Indigenous Peoples 7 X X X X X X X Conflict Prevention and 6 X X X X X X Reconstruction NGOs/Civil Society 6 X X X X X X Empowerment 6 X X X X X X Culture 6 X X X X X X Security 5 X X X X X Inclusion 5 X X X X X Resettlement 4 X X X X Governance 4 X X X X Social Capital 4 X X X X Livelihoods 2 X X Community-Driven 1 X ~Development _ _ _ _ -~ No. themes covered - . 16/18 15- - 13 12 13 _________________ 11 10 7 4 1.18 One very clear observation that emerged from the exercise is that there is a remarkable interconnectedness between the various documents on nearly every subject. M a n y o f the docu- ments had a multidimensional perspective, and showed the interconnectedness o f most o f the ob- jectives. The study observed that m a n y o f the individual objectives were closely related t o each other. F o r instance, the WDR objective “make progress toward gender equality and empowering women by eliminating gender disparities in primary and secondary education by 2005,” could b e considered part o f not only gender, but empowerment, and human development as well. 5 STUDY DESIGN 1.19 The study was largely a desk review, drawing i t s conclusions from a number o f recent evaluative exercises. T o map out the methodology and related research process the study team prepared a detailed D e s i g n Paper. The Design Paper establishes a pattern o f work and analysis (including proposed survey instruments, a framework for analysis, and a description o f the meth- ods that were ultimately used). A portfolio review was conducted using available Bank documen- tation and databases. The other components are described briefly below and in more detail in Annex A. 1.20 Literature Review. T o capture a broad understanding o f the field, the study prepared a detailed literature review. I t focuses on k e y policy issues such as social development tools and indicators, current debates in the field, programs led by other institutions, and cutting-edge issues in social development. 1.2 1 Individual Interviews and Surveys. Within the Bank and other development agencies, open-ended interviews were conducted before and after the preparation o f the Design Paper. Ini- tial consultations were carried out with the Utstein group,6 Switzerland, Sweden, and the UN. Early interviews helped inform the development o f survey instruments.An Internet-based system (WebIQ) was used with Bank staff and outside social development experts to field-test draft in- struments before their widespread use and to evaluate and “reality check” preliminary results. Three survey instruments (described inpara. 6.1) were administered to selected Bank staff through Lotus Notes (the survey instruments are attached as Annex F). 1.22 Meta-evaluation. Ten topics have been examined by recent O E D evaluations, enabling the extraction o f further, broader lessons from the existing data. The evaluations examined were: Gender (1995, 1997,2001), Post-Conflict Reconstruction (1998), NGOs (1999), Participation, (2000), Rural Water (2000,2002), Resettlement (2001), Cultural Heritage (200 l), Community- Driven Development in the Sahel(2002), and Forestry (199 1 2000). The team also drew o n the ¶ Annual Review o f Development Effectiveness (ARDE) (2002) and the Social Investment Funds review (2001) for additional information. 1.23 Country Assistance Strategy (CAS) Reviews. The study reviewed the role o f social de- velopment in CASs and Country Assistance Evaluations (CAEs), and reviewed Quality Assur- ance Group (QAG) assessments o f quality at entry and economic and sector work. 1.24 Review Panel. A group o f external reviewers was established, consisting o f three social development experts, to review the steps taken by the study team and to comment o n the final study report (see Preface). 1.25 This report presents the major findings o f the meta-evaluation supplemented with the analysis o f data from the other sources (triangulation). Work with Poverty Reduction Strategy Papers and community-driven development are the subject o f ongoing OED evaluations and were not covered by the study. SDV’s recent and ongoing work o n social accountability, citizen report cards, public expenditure analysis, and the social impact o f adjustment lending and policy re- forms, i s not yet ready for evaluation. 1.26 The several components o f the study addressed a set o f broad evaluative questions that were set out in the Approach Paper and elaborated in the Design Paper for the study. While many o f these questions are covered in several places in this report, they generally map to the organiza- tion o f this report as indicated in the following table. 6 Chaoter Evaluative Questions 2. Where is social development What constitutes the Bank’s social development “portfolio”? being conducted? To what degree have social development issues been incorporated into Bank lending activities? 3. Is social development relevant? To what degree have social development issues been incorporated into Bank policy formulation? To what degree has attention to social development issues influenced Bank activities in the field throughout the project cycle? To what extent does social development show up within important Bank macro processes such as the CAS? Is there a rhetoridreality gap between strategy and project programming? 4. Have social development How do social development interventions compare with other Bank interventions? interventions been effective? What light does (already-completed and ongoing) self and independent evaluation shed on the impact of social dimensions? Have some methods of introducing social concerns into Bank work been more effective than others? 5. What challenges affect What factors identified in completed and ongoing self and independent evaluations implementation of social enhance and constrain the general pursuit of the Bank‘s social development develoDment? objectives? 6. What are the implications for What organizational and procedural factors do Bank staff identify as enhancing or Bank organization and constraining the general pursuit of the Bank‘s social development objectives? Drocedures? 7 2. W h e r e I s Social Development Being Conducted? 2.1 This chapter reviews the nature o f the Bank’s social development portfolio and analyzes the regional and sectoral distribution o f completed and ongoing social development work. I t also identifies overlaps in practice. That is, certain themes have received complementary treatment in the past, and the study identified the thematic combinations that have tended to occur. I S THERE A SOCIAL DEVELOPMENT “PORTFOLIO?” 2.2 Portfolio reviews were undertaken for evaluations OED completed in eight social devel- opment areas (plus CDD where sufficient data was available). This data-gathering and analysis exercise i s not a portfolio review in the traditional sense because very few stand-alone social de- velopment projects exist. Nonetheless, the completed OED evaluations identified clusters o f pro- jects that can be conceptualized as being aligned with four out o f the five “business lines” o f so- cial development (safeguards, participatiodcivic engagement, CDD, conflict prevention). The fifth business line, social analysis, does not meaningfully generate a portfolio, and Bank experi- ence with this process i s explored in Chapter 6 . 2.3 The nature o f the various “portfolios” and their contents varied widely. For instance, the Participation portfolio includes projects where there might have been one consultation ac- tivity, as well as projects in which every phase o f the project had participatory aspects. To the extent practicable, these “portfolios” o f Bank-supported projects with social development proc- esses or targets were updated by this study to include projects through fiscal 2002, where pos- sible, including a l l ongoing and completed projects so that they take recent experience into ac- count. Following the update process, the study database documented those projects that ad- dressed the social themes in Table 2.1. Follow-up research t o ascertain the efficacy o f social activities was undertaken with the OED database o f I C R and P P A R findings. The results o f that analysis are in Chapter 4. Table 2.1: Social Development Portfolios No. of Approval Years Portfolio Projects Covered Countries Community-Driven Development 285 2000-2002 All Conflict 264 1978-2002 All countries with conflict Culture 150 1972-2002 All Gender 738 1994-2001 All Indigenous Peoples 462 1987-2002 All NGO/Civil Society 1767 1972-2002 All Participation 671 1994-1 998* All Resettlement 308 1985-2002 All Social Funds 84 1987-2001 All countries with Social Funds *Portfolio from 1999 to 2002 not available. 2.4 W h i l e it i s only recently that the evolving development focus led to Bank lending for so- cial development, the Bank has contributed to the economic and social progress o f the past 56 years by supporting more than 8,000 operations in 140 countries with almost $500 b i l l i o n in fi- nancing (commitments). In the early 1980s, investment with at least some support for social de- velopment amounted to 5 percent of Bank lending. By 1994, it accounted for 15 percent. Since 8 then it has grown steadily. The Bank i s cur- Box 2.1: How M a n y Social Development rently “the world’s largest financing source Professionals W o r k for the Bank? for investments in p e ~ p l e . The ” ~ study port- folio review identified 2,577 projects ap- Increased attention to the social impacts o f i t s op- proved between 1972 and 2002 that ad- erations l e d the Bank t o build up i t s staff resources dressed social development themes. Most in this area. The first social development unit was created in 1993 w i t h six staff. By 1997 there was a projects took place in the later years, with a stand-alone Social Development Department (SDV) dramatic increase in approvals beginning in within the Environmental and Socially Sustainable 1989 (see Attachment Figure SI). Between Development vice presidency. Taking into account 1985 and 2002 more than h a l f o f all Bank b o t h staff and short-term consultants, OED esti- operations (56 percent o f all approved pro- mates that as o f January 2002, the B a n k employed jects [2,291 out o f 4,2661) addressed social 175 social development professionals and 22 gender development themes.8 The number o f social specialists. development professionals in the Bank mir- M a n y Bank staff members not working o n social i s - rors the growth in lending (see B o x 2.1). sues are educated in the social sciences, and they constitute a resource upon which the Bank could 2.5 M a n y projects are identified as hav- draw. Information from the Human Resource Service ing social development activities in the data- Center shows that 249 additional Bank staff members base without having a specific dollar amount h o l d Master’s or Doctorate Degrees in the non- tied to the social development aspect(s) in economic social sciences.* I t i s not k n o w n h o w they the project documents and budget. In some have incorporated their academic background in their instances, the social development component work. could be a small amount directed to the cul- *This i s based o n voluntary information p r o v i d e d by staff tural heritage aspect o f a large infrastructure through the HR K i o s k as o f M a r c h 2002, and should project. Conversely, the entire loan amount therefore n o t b e considered 100 percent inclusive o f a l l graduates. of a Social Fund project could be directed at social development. 2.6 T o get a sense o f the resources applied to social development efforts, the total o f the loan amounts committed to the 2,577 projects in the OED-identified portfolios i s $188.89 billion dol- lars. O f course, only a fraction o f the proceeds o f many loans went to social development. As par of last year’s re-categorization o f projects into multiple sectors and themes, Operations Policy and Country Services (OPCS) estimated the proportion and amounts o f each project approved since 1990 for social development plus gender. Between 1990 and 2002, the amount allocated to social development (but including gender in that category) in the OPCS database i s $13.3 billion DONE IT’S BUTTHE PORTFOLIO VARIES B Y REGION AND SECTOR EVERYWHERE, Regional Distribution 2.7 Table 2.2 shows the percentage o f each thematic portfolio represented by work in the vari- ous World Bank geographic regions. The East Asia-Pacific (EAP) Region has almost half o f the Resettlement projects (48 percent) and the Latin America and Caribbean (LCR) Region has the vast majority o f the Indigenous People portfolio with 46 percent. Work in and around physical cultural resources figured most prominently in the Europe and Central Asia (ECA) Region, although with the exception o f South Asia (SAR) it i s quite evenly distributed. Actual figures can be compared with the “all Bank projects” column to see the extent to which the observed percentage i s different from what might otherwise be expected. (Attachment Figure S 1 shows the growth in the actual 9 number o f social development projects by year by region up to FY 2001, after which changes in thematic recording and classification makes graphical representation difficult.) Table 2.2: Regional Distribution of Portfolios (percent) ALL BANK PROJECTS Community- (1992- Driven Indigenous Social Region 2002) Development Culture Gender People NGOs Participation Conflict Resettlement Funds AFR 28 34 18 31 5 33 25 45 9 35 EAP 15 8 16 15 22 14 19 12 48 5 ECA 20 19 21 12 6 13 16 19 8 20 LCR 21 22 20 21 46 21 23 14 11 30 MNA 7 6 18 8 4 6 6 7 4 10 SAR 9 11 7 14 17 13 11 3 20 1 Note: Columns may add to more than 100 percent due to rounding. The largest percentage in each cate- gory i s in boldface type). 2.8 The regional distribution table clearly shows that the Africa (AFR) Region has been the beneficiary o f the bulk o f social development work, having the largest percentage o f projects in six o f the nine social development thematic portfolios. This i s partially because AFR receives the largest number o f Bank projects generally, but five out o f those six portfolios exceed their actual percentage share. The three thematic areas in which Africa does not predominate are Indigenous Peoples (5 percent o f projects), Resettlement (9 percent), and Culture (1 8 percent). Indigenous people have tended not to be identified as an issue in AFR (and MENA) because o f the legal and technical differences between indigenous and ethnic minority, even though there are groups in these Regions that would qualify under the terms o f Operational Directive 4.20.9 Africa’s fourth rank in Resettlement i s probably a function o f the Bank not financing many large dams and large- scale transportation projects in the Region. The OED evaluation o f physical cultural property found that it i s a common perception that Afnca’s physical cultural heritage i s meager, biode- gradable, and excessively vulnerable to i t s climatic conditions. But the data are poor and there has been insufficient research and, allowing for a high degree o f country variability, cultural property actually m a y be substantial and widely distributed. Sectoral Distribution 2.9 The Agriculture sector had the highest proportion o f projects in five o f the nine portfolios (see Attachment Table S 1). M a n y o f the portfolios have 20 percent or more o f their projects in Agriculture, while that sector made up 14 percent o f projects approved by the Bank between 1992 and 2002, which s t i l l makes i t the sector with largest number o f projects. N o t surprisingly, almost a third o f the Resettlement portfolio (30 percent) occurred in the transportation sector giv- ing that sector first rank for that portfolio. The sector with the most Culture projects was Urban. Compared to the whole Bank portfolio, social development themes were seen least frequently in the finance, private sector development and economic policy sectors. Portfolio Overlaps 2.10 The portfolio databases were compared t o see h o w many projects appeared in more than one portfolio. (The portfolios were aligned to make sure the years covered were the same, see 10 Table 2.3.) This analysis was done because most social development experts believe that there are synergies that occur when projects finance complementary activities. In fact, there i s good reason to expect that the more social development aspects are covered under one project, the more they will interact with each other in a manner that increases their development impact.” The impact o f overlaps o n project ratings i s explored in Chapter 4. 2.1 1 As might be expected, the Participation portfolio had large overlaps with almost all the other portfolios. Other portfolios with large overlaps were N G O / C i v i l Society and Social Funds. To give an idea o f the scale o f this phenomenon: when the number o f projects in each portfolio i s added together, the sum i s 4,729. Yet the actual number o f discrete projects i s only 2,577. Table 2.3: Portfolio Overlaps (percent) 3 *I C .P .*, 5 I .. 0 m 3 -8 -2 t s 0 * u I v) z .$ 0 Portfolio 2 G 6 Community-Driven Development 37 32 4 74 * 30 10 76 Culture 28 19 29 51 5 14 3 Gender 41 77 52 25 22 75 Indigenous Peoples 46 57 6 21 23 NGOs 73 37 50 94 Participation 41 77 90 Conflict 2 13 Resettlement 1 * There is no overlap between the CDD and Participation portfolios as the data for these two themes are from different years. Note: Portfolios which overlap 50% or more are in boldface type. 11 3. I s Social Development Relevant? 3.1 Having examined the composition o f the array o f projects in the Bank’s social develop- ment portfolio, it i s important to consider their relevance. Normally, the study w o u l d have turned f i r s t t o the Sector Strategy. In this case, since there i s as yet n o social development Sector Strat- egy Paper (SSP), the examination o f relevance had to b e directed elsewhere. 3.2 The Bank’s commitment to social development i s reflected first and foremost in i t s Op- erational Policies, especially those k n o w n as safeguards (see Annex C for a l i s t o f relevant OPs). The application o f the Bank’s safeguard policies often involve social development staff and they require an in-depthknowledge o f country realities and social development processes. For safeguards to be applied effectively, there needs to be government ownership o f their princi- ples, and institutional capacity in client countries t o put them into effect. Strengthening existing institutions, and assisting in the creation o f new ones when necessary, are tasks the Bank’s social development staff i s qualified to carry out. 3.3 OED’s benchmarks for rating relevance involve two key factors: First, the benchmarks cover both borrower development priorities and W o r l d Bank strategies, goals, and policies:Thus, the CAS, as a repository o f agreed Bank-borrower priorities, i s a key source for rating relevance in the absence o f a strategy. Second, the benchmarks must be current. In many cases, priorities, strategies, and goals have not changed. I T I S RELEVANT TO THE BANK’S DEVELOPMENT BROADER OBJECTIVES 3.4 To b e able to examine the relevance o f social development to the Bank’s broadest de- velopment goals, and t o compensate for the Bank’s lack o f a formal strategy o r social objec- tives, the study created a matrix that groups the social development objectives contained in in- ternal and external documents and statements relevant t o the Bank’s work-see the discussion o n the objectives matrix in Chapter 1, and Annex A. This exercise highlights the nature o f the Bank’s tacit objectives (or what they were before the Sector Strategy development process be- gan) and i t shows the strong linkages from social development objectives to sustainability and poverty reduction.” I T I S A HIGHPRIORITY AT THE COUNTRY LEVEL establish what they said about thematic Box 3.1: Latin American CAS Success for Indigenous inclusion in the CAS. on thewhole, the Peoples OED evaluations had found that, al- Recent Latin American CASs attend to indigenous peo- though treatment Of n CASs issues i ples’ (IP) issues. The latest CAS o f every sample country had expanded in recent Years, the quality in LCR integrated issues related to IP into their strate- of the treatment remained variable. gies. This i s true, even in counties where the percentage Treatment o f five themes (NGO, Partici- o f IP populations i s low such as Argentina, Brazil, Chile, pation, Indigenous Peoples, Cultural and Colombia. The Chile CAS, for example, takes the Heritage, and Gender) in the CAS had Inter-American Development Bank (IDB) strategy into improvedsignificantly inthepast few consideration. years, according to those OED evalua- Source: Indigenous Peoples study, 9, paragraph 2.8. tions. For example, the Indigenous Peo- 12 ples (IP) study highlights progress in L a t i n America (Box 3.1). The Participation evaluation found that despite increased attention, social concerns were s t i l l not adequately reflected in CASs and National Environmental Action Plans (NEAPS), and when they were reflected, they were often not reflected in the country portfolios in a manner befitting the priority accorded to them. The evaluations determined that the development potential o f social themes should also be rec- ognized more explicitly. In particular, more attention t o the appropriate role o f the social themes within the country’s overall institutional framework i s needed. 3.6 A review of the CASs revealed that social development is highly relevant to the lending priorities agreed upon by the Bank and borrower. Given that several OED evaluations concluded that social development themes need to be explicitly recognized in the Bank’s corporate priorities as they are expressed in country strategies, the study team reviewed all o f the most recent CASs to see if there had been a change since completion o f the various OED evaluations o f social de- velopment themes. 3.7 The review centered o n the question o f h o w social development had been addressed in the CAS. This document, in which the Bank delineates i t s strategy for a country based o n a Bank assessment o f country priorities and input from the borrower, indicates the level and composition of assistance to be provided based o n the strategy and the country’s portfolio ~ e r f o n n a n c e . To ’~ approach the question, the study team examined the most recent CASs o f 109 countries for in- formation o n the Bank and borrower’s assessment o f social development needs. The documents were analyzed to see which social development areas the Bank works in were mentioned in the text o f the CAS. In addition, issues highlightedby survey respondents were also i n ~ 1 u d e d . l ~ 3.8 The results of the search revealed that the issues appear in the most recent CASs quite often (Figure 3.1). The average number o f these topics mentioned per C A S was 2.7. The social development topics that were mentioned most often were participation (74 percent o f the CASs), gender (67 percent), and NGOs/civil society (55 percent). Figure 3.1: Percent of CASs that Mentioned Various Topics indigenous people Community-driven development improve public services for the poor Anti-COrNptiOn measures NGO/civii society 74 0 10 20 30 40 50 60 70 80 90 Percent of CASs mentioning SD topics Note: The Resettlement figure is probably low because there is a safeguard that needs to be followed, and this topic is typically dealt with project by project rather than in the CAS. 13 3.9 In addition to these topics, the analysis Box 3.2: C A S Priority Issues also noted h o w often institutional reform (trans- (Those Covered Primarily b y the Social parency/ accountability and decentralization), Development Department and its anti-corruption, and public services for the poor Regional Specialists in Bold) were mentioned. Institutional reform was men- 1. Vulnerability/exclusion tioned more often than any o f the topics currently (empowering vulnerable groups) (15) defined by the Bank as being social development 2. Participation (15) (in 83 percent of the documents). 3. W e a k civil society (13) 4. Indigenous peoples (6) 3.10 In addition to the text searches, the 5. Decentralization (5) “Summary o f Development Priorities” section o f 6. L a c k o f social capital each CAS was also examined. In this section o f (capacity o f local institutions/ the CAS, the borrower lays out and rates i t s pri- community development) (4) orities. The Bank rating o f these priorities i s also 7. Inequality (4) noted in this section. Three segments o f the 8. Implementation capacity (4) 9. Violence reduction (4) summary were examined: public sector, gender, 10. Poverty (4) and social development. 11. NGO presence (3) 12. Social services-high cost and delivery (3) 3.1 1 The study team analyzed the contents o f 13. Internally displaced people (2) the social development segment o f the summary. 14. Food aid to support social sectors (2) In this section, the borrower must define one 15. Education and health (2) “major social development issue”’5 in the coun- 16. Transparency (2) try. Looking across a l l o f the CASs, these issues 17. Lack o f focus in public programs (1) fell into 23 categories, listed in B o x 3.2. Notably, 18. Urbadrural disparities (1) 14 o f the 23 groupings fall outside o f the purview 19. Youth (1) o f the Bank’s Social Development Department 20. Safety nets (1) 21. Social cohesion (1) and specialists in the Regions who are attached to 22. Consolidate the democratic process (1) SDV (those in plain-face type). 23. Income-generating opportunities (1) 3.12 Social development issues: a high prior- Note: Numbers in parentheses refer to the number o f CASs that mentioned the listed topic as a priority. ity for both the Bank and borrower. Regardless o f whether the “major social development issue” stated by the borrower was covered by S D V and i t s Regional specialists, analysis revealed that social issues are a high priority for both the Bank and the borrower.16 O f those with a stated “ma- j o r social development issue” (76 CASs), 91 percent were rated moderate or high priorities by the borrower and 97 percent were rated such by the Bank. 3.13 Twenty-two CASs were rated differently by the Bank and borrower. In 16 o f the 22, the Bank wanted to give the issue a higher priority than the borrower did. In o n l y six cases did the borrower want to give the issue a higher priority than the Bank did. This pattern was confirmed by the surveys. When country directors were asked whether the governments they work with are interested in confronting social problems in Bank-financed projects less or more intensively than i s normal under current Bank practice, almost two-thirds said less. 3.14 Strong consistency in Bank and borrower rating o f priorities was apparent. In the in- stances that the Bank wanted to upgrade the priority, the “major issue” generally centered around institutions, transparency, or c i v i l society exclusion. Where i t wanted to downgrade the priority, the “major issue” was mainly participation, indigenous peoples, or refugees. Conflict prevention 14 does not appear in the table because borrowers chose to emphasize components o f this work (such as violence reduction, social cohesion, and vulnerability). 3.15 Table A.3 in Annex A shows the percent o f CASs for each Region that mentioned a par- ticular social development topic. The social development subjects appear in order o f their impor- tance in the CASs, overall-participation being mentioned the most, resettlement the least. 3.16 The study also compared what was said in the CASs with what has been dealt with at the project level. (See Figure 3.2, which shows a frequency scatterplot comparing the two; a tabular description o f the data points i s in Annex A.) In most instances, statements in the C A S do not directly correspond with the Bank’s actual work portfolio. I t would be more accurate to say that they indicate the w a y forward (they are “stretch” goals). In Figure 3.2, the points above the 45 degree line show that the Regions are working more intensively o n the issues than was called for in the CAS. Conversely, points below the line show the opposite. The majority o f the points (25 out of 40) fall below the 45 degree line indicating that rhetoric may be a bit ahead o f reality. Each point o n the scatterplot compares the percentage o f projects that deal with a particular social de- velopment theme in a Region with the percentage o f the most recent CASs in the Region that mentions that theme. Points that are on the l i n e represent themes have the same likelihood that they will appear in the CASs as the portfolios. Figure 3.2: Relationship o f Social Development Themes in CASs to Portfolios 100 A b o v e the line: P o r t f o l i o s go ._have higher frequency o f S D themes than C A S s 80 -. - .E! 70 , 60- n e - 0 2 50 - 0 e .- 0 0 5 0 B 40; -0 1c 0 e 0 0 30 s e 20 - B e l o w t h e line: C A S s h a v e hiaher 10 frequencyof S D themes than regional portfolio 0 0 10 20 30 40 50 60 70 80 90 100 %of CASs in Regions 3.17 The two most outlying examples are the Gender portfolio in the South Asia region, where 100 percent o f the CASs mention this theme, and 58 percent o f the projects have some gender aspect to them, and the Resettlement portfolio in the East Asia region, where none o f the CASs mention it, but 37 percent o f the portfolio has resettlement issues. (Resettlement tends to fall near zero o n the x axis because it i s usually not mentioned in the CASs but it comes up in the context of the safeguards, and it therefore tends to be well represented in the portfolio). 3.18 Data points with fairly similar percentages are: CDD projects in the MNA Region, where 3 8 percent of CASs mention C D D and 39 percent o f projects do some f o r m o f CDD; and NGO involvement in the ECA Region, where 46 percent o f CASs mention it, and 43 percent o f 15 projects address the theme. Also notable i s the Participation portfolio in the South Asia Region, where 80 percent o f CASs mention it, and 76 percent o f projects claim to do some f o r m o f par- ticipation. 3.19 OED’s Country Assistance Evaluations (CAEs) evaluate progress toward CAS-stated goals. The study team reviewed 2 1 recent CAEs (those produced in the past three years) to iden- . tify conclusions relevant to social development. The most frequently mentioned social develop- ment topic in the CAEs was gender, mentioned in 10 out o f the 21 evaluations. Several CAEs cited the Bank as the initiator o f discussion and action o n gender in the studied countries. Also noted in some CAEs i s the unsuccessful mainstreaming o f gender, which one CAE referred to as i t s “ghettoization” in the education and women’s health sectors. CAEs in several conflict-prone countries noted that the Bank’s social development efforts had made significant contributions to physical reconstruction and social cohesion. A lack o f knowledge by Bank operational staff re- garding relevant NGOs was frequently mentioned. A number o f CAEs noted the increasing ten- dency o f projects to devote resources to participation, although two CAEs cited instances where Bank-borrower tensions were provoked when the project staff included certain beneficiary groups in participatory activities without clearing their participation with concerned government entities. T w o countries see indigenous peoples issues as poverty issues and deal with them in an overall poverty reduction framework, rather than separately. Resettlement issues figured largely only in the India CAE, where the Bank has been a significant shaper o f that country’s resettle- ment policies commencing with the Sardar Sarovar dam project. I T I S RELEVANT AT THE PROJECT LEVEL 3.20 An analysis o f relevance was done o n the completed projects o f the social development portfolios based o n OED evaluations. O f the 713 projects rated by OED either in Evaluation Summaries (ESs) or Project Performance Assessments (PPARs), over 89 percent o f the overall objectives o f the projects were found to be highly or substantially relevant to the overarching country and sector developmental priorities-the nature o f which was explained in the preceding section o n the CAS. The comparator group, which would be a l l Bank projects rated in the same period, has about 83 percent o f the projects rated as highly or substantially relevant. The social development portfolio fares better than the Bank’s portfolio as a whole. I t should be noted, how- ever, that this rating pertains to the whole project, not just the social development component. For example, some projects could be highly relevant transportation projects that contain a social development component. Thus, the relevance rating would pertain to not only the social devel- opment piece but also to the construction o f roads and other infrastructure. 3.21 Some projects were rated for the relevance o f their social development objectives as well. That rating was analyzed separately when available, and 508 projects were thus rated. These were highly or substantially relevant in 79 percent o f these operations. The relatively lower rele- vance o f this subgroup (and i t s below-comparator status) m a y just reflect the late arrival o f social development themes into the Bank-borrower dialogue, or the rhetoricheality gap revealed by the CAS analysis. 16 4. H a v e Social Development Interventions Been Effective? 4.1 The study combined a l l the completed and evaluated projects in the various social devel- opment portfolios to arrive at an overview o f h o w projects with social development themes in the aggregate have been performing over their varying l i f e cycles. Some themes began to receive attention as early as the mid-l970s, while others only began a decade or more later. EFFECTIVENESS OF PROJECTS W I T H SOCIAL DEVELOPMENT COMPONENTS COMPARES W E L L W I T H OTHERS 4.2 The combined social development portfolios have a satisfactory outcome rating 72 per- cent o f the time, attain likely sustainability 53 percent o f the time, and achieve substantial institu- tional impact 37 percent o f the time. This pattern, higher outcome than sustainability, higher sus- tainability than institutional development impact, i s typical o f all Bank projects. Still, in order to have an idea o f h o w well projects that address social development themes are doing, it would be helpful to have a comparator group. H o w does this aggregate rating compare with other projects implemented during the same period? Quite well: All Bank projects over the same period have a satisfactory outcome rating 68 percent o f the time, likely sustainability 50 percent o f the time, and substantial institutional development impact 34 percent o f the time. Thus, over a period o f nearly 30 years, projects that address social development themes are consistently rated 3 to 4 per- cent better, o n average, than projects that do not address these themes. The overall percentage o f Bank-financed projects rated highly satisfactory i s 4.4 percent, while the percentage o f Bank- financed projects with social development themes rated highly satisfactory i s 5.8 percent. Outcome 4.3 The study also compared the outcome ratings o f each social development portfolio to a cohort o f all Bank projects that were approved during the same period. I s the comparatively bet- ter performance o f the various disaggregated portfolios consistent across thematic areas? T o a large degree. The results showed that the social development project outcomes were rated more highly in a l l portfolios but one (resettlement projects). Figure 4.1 shows the portfolios and the relevant comparator for the period in ascending order o f performance. The comparator i s differ- ent for several o f the themes because their implementation period covers distinct time spans. W h i l e the Social Funds portfolio i s the highest o f the rated portfolios, the recent in-depthOED evaluation of Social Funds raised a number o f caveats that warrant consideration in weighing relative performance. 4.4 The study also examined the combined outcome ratings o f a l l social development portfo- lios over the period 1992-2002. The data show that completed projects in this portfolio exhibited an improving trend (Figure S2 in Attachment). 17 Figure 4.1: Outcome Ratings of Social Development Portfolios Compared to All Bank Projects from the Same Period SD Portfolio UAll Bank NGOslCivil Societ 77 Culture 79 Participation Indigenous People 79 Social Funds 92 I 0 10 20 30 40 50 60 70 80 90 100 Sustainability 4.5 The disaggregated social development portfolios did not fare as well against their com- parators o n sustainability, however (Figure 4.2). Only three o f the eight portfolios (Indigenous Peoples, Culture, and Resettlement) have higher percentages o f likely sustainability rating than the comparable cohort of Bank projects from the same time period. Indeed, the highest perform- ing portfolio o n outcome, Social Funds, had the lowest likelihood o f sustainability. And the worst performer o n outcome, Resettlement, attained sustainable results more often than the average Bank project during the time period that the Resettlement portfolio covers. The difference be- comes even more striking when one looks at the sustainability ratings for only those resettlement projects that were rated Satisfactory or Highly Satisfactory in outcome (67 percent o f them)-85 percent o f that group o f projects i s likely t o be sustainable. Figure 4.2: Sustainability Ratings of Social Development Portfolios Compared to All Bank 67 Indigenous Peoples Gender co Culture Participation Resettlement NGOlCivil Society Conflict HSD Portfolio Social Funds 0 10 20 30 40 50 60 70 80 Percent Likely 18 Projects f r o m the Same Period 4.6 A recent OED Lessons and Practices publication o n improving communities’ ability to manage their o w n development examined a cohort o f 48 social development projects. The pro- jects studied-including Social Funds and Slum Upgrading*’-while they did well in the other ratings scales, also compared poorly to the average Bank project with respect to sustainability. The observed performance weakness in terms o f sustainability i s largely a function o f projects’ tendency to leave infrastructure behind in poor communities without leaving a functioning social structure to support it, or to create new formal organizations and then to leave them o n their o w n prematurely. This explains the comparatively better sustainability performance o f successful re- settlement projects (which have less to do in this regard once beneficiaries are successfully reset- tled). This point i s discussed further below. Institutional Development I m p a c t 4.7 Bearing in mind that even in the aggregate institutional development impact was rated quite l o w (37 percent substantial impact), the disaggregated portfolios fared better than their comparators for a majority o f the portfolios (Figure 4.3). W h e n compared with their contempora- neous cohort o f Bank-wide projects, the institutional development impact (substantial) ratings were higher for five social development portfolios: Indigenous Peoples, Resettlement, Social Funds, Participation, and Culture. Figure 4.3: Institutional Development I m p a c t Ratings o f Social Development Portfolios Compared to All Bank Projects f r o m the Same Period Indigenous Peoples 50 Resettlement Social Funds Participation 5 Gender A6 Culture NGOlCivil Society 3 $SD Pon‘r, o 34 U A Bai< 3 Conflict 34 -- - . 0 10 20 30 40 50 60 Percent Substantial PROJECTS WITH OVERLAPPING SOCIAL DEVELOPMENT HAVE THEMES BETTER RATINGS 4.8 Portfolio overlaps result when projects address multiple social development themes. As Chapter 2 notes, the portfolio review process uncovered considerable overlap between the pro- jects mapped to each thematic area. In light o f the meta-evaluation finding that there are syner- gies that result from the interdependentnature o f social development themes, the study subjected the portfolio to further analysis to learn more about those synergies. 4.9 The study reviewed completed projects for those portfolio pairings that had an overlap o f 50 percent or more to search for thematic complementarity. In order to see whether more com- 19 prehensive treatment of social themes Table 4.1: Ratings for Projects with Two, Three, had an impact o n project performance, o r Four Overlapping Social Development Themes the study reviewed the OED ratings o f No. of Outcome Sustainability lnstitutional projectsfor those portfolio overlapping (%Satisfactory) (% Likely) development impact pairings. The average outcome rating themes (% Substantial) for all Bank projects from 1972 to 2001 2 81 62 45 i s 68 percent satisfactory. One grouping 3 84 63 47 attained a 100 percent satisfactory rating 4 90 64 49 and eight out o f nine bested the aggre- gated rating for the comparator group. Table 4.1 summarizes the ratings for projects with over- lapping themes. Further detail o n each overlapping portfolio i s in Attachment Table S2. 4.10 Sustainability and institutional development impact ratings were also substantially above Bank averages. This finding, together with the parallel meta-evaluation results, leads to the ines- capable conclusion that many aspects o f proj ect-level social development work are highly com- plementary, and they (probably) need to be designed and implementedin close coordination. 4.1 1 The study took this analysis a few steps further to ascertain whether increasing social the- matic coverage led to improved project outcomes. I t did. Taking the pairings with the greatest overlap and adding a third portfolio, the study found that 11 o f 13 o f these portfolio combinations had satisfactory outcomes o f 78 percent or better. Four groupings attained a 100 percent satisfac- tory rating (Attachment Table S3). 4.12 The more themes overlapped the better outcome, sustainability, and institutional devel- opment impact as shown in Table 4.1. In particular, the outcome ratings increase from 8 1 percent in projects with two overlapping themes to 90 percent for projects with four overlapping themes. (Also see Attachment Table S4). There were considerably fewer examples o f a five project over- lap, but three out o f four permutations also had a 100 percent satisfactory rating. 4.13 O f those projects that overlapped at least four themes, ten were rated highly satisfactory in outcome, and/or highly likely in sustainability (four o f them were both). The study team per- formed a desk review o f these ten projects t o determine what success factors they might have in common. (The full report o f the findings o f this exercise can be found in Annex D.) As antici- pated, these factors mirror findings o f the meta-evaluation quite closely (see B o x 4.1). I Box 4.1: K e y Characteristics o f Highly Satisfactory Projects Ten highly successful projects that each dealt with at least four social development themes, had several o f the following factors in common: Success stems from quality participation o f all stakeholders. 0 Giving the participants the responsibility for structuring their involvement in the project increases the likelihood o f success and sustainability. Stakeholders-beneficiaries in particular-participating in project design led to project success. 0 Project team composition and team continuity was critical to the success o f the projects. Providing integrated attention to social development themes led to better implementation and more socially sustainable benefits. The analysis of socially relevant aspects o f projects was necessary for success. Reproducing many o f these success factors in new projects requires sustained social development support at the project level. 20 5. What Challenges Affect Implementation of Social Development? CONSISTENCY: THE VARIOUS SOCIAL THEMES TREATMENT RECEIVE I S HIGHLY VARIABLE 5.1 Although there i s an improving trend (the Bank has a long history and extensive imple- mentation experience in several social development areas, and performance has been improving through the years) the way forward has not always been smooth. Every OED evaluation reviewed for the meta-evaluation reported highly variable treatment o f social issues during implementa- tion. Descriptions ranged from “uneven” to “unsatisfactory,” a l l the way to “destructive and so- cially inequitable.” Integration o f social development good practice into Bank processes has been slow, and good quality assurance mechanisms are scarce. Implementation difficulties come in part from a fragmentation o f responsibility, the consequence o f the cross-sectoral nature o f many social concerns. 5.2 Regional variation. Some Regions address social issues in projects more often than oth- ers (AFR and L C R most often, SAR and MNA least often-see Table 2.2). But there are excep- tions to these patterns-the treatment o f ethnic minorities and/or indigenous people in AFR and MENA (para. 2.8), for example. 5.3 Variation due to staffexpertise. Treatment o f these issues i s highly variable for a number o f reasons. Often it i s dependent o n the level o f staff interest and expertise. For instance, staff interested in cultural heritage, and those who realize i t s organizing power, take advantage o f op- portunities when they find significant physical cultural resources in their path. l8 5.4 Hesitation to deaZ w i t h safeguard issues. Several o f the OED evaluations reported a disappointing or uneven past record o f safeguard compliance,” although projects that address social development themes m a y do better in this respect than projects that do not.20M a n y o f the O E D evaluations highlighted recent improvements or identified an improving compliance trend. (The creation o f the Quality Assurance Compliance Unit, QACU, and the development o f the Integrated Safeguards Data Sheet are contributing to this improvement, and they were partly organizational responses to the findings o f the stand-alone OED thematic evaluations that this meta-evaluation draws upon.) Some social issues are less likely t o be checked or in- cluded in projects than others, according to the evaluations. F o r instance, the OED Social Funds Study concluded, “environmental assessment, natural habitats, indigenous peoples, and cultural property a l l appear l i k e l y t o be relevant in a larger number o f cases than have included them thus far.”2’ The Water Resources Study pointed out that some social issues are checked more, relative to others, during project appraisal, namely, poverty, participation, and gender. Similarly, o f the safeguard policies, resettlement receives a more routine consideration than indigenous populations and cultural heritage.22 5.5 The OED evaluations reported that dealing with safeguard issues presents many difficul- ties-high transaction costs, risk, and a lack o f capacity and ownership. These difficulties, com- bined with weak incentives within the Bank to implement safeguard policies, result in shortcom- ings. But despite the difficulties, performance has been improving in five o f the evaluation ar- e a ~However, . ~ ~ these improvements may only be in project design. According to QAG, the Bank’s performance in monitoring actual compliance does not meet the 100 percent quality stan- 21 dard expected for safeguard policies. (See the “Policy and Safeguards” section o f the meta- evaluation report for specific supervision improvement recommendations.) 5.6 Recommendationsfor improvement. The OED evaluations had several strong recommen- dations for improved implementation performance in common. First, beflexible/agile. O E D evaluations in seven o f the ten areas stressed the importance o f flexibility-in processes, imple- mentation, procedures, programming, and design. This strong message coming out o f the OED evaluations resonates with general development thought in that the ability to adapt to events and changes in stakeholders’ priorities i s critical to effective projects. 24 Second, supportive manage- ment is key. Several o f the evaluations noted the need to lend greater support to staff to imple- ment the policies, as well as the need to give high priority to appropriate staffing and structur- ing.25Third, clearprocedures are imperative. Most o f the evaluations flagged the need to de- velop or clarify procedures and guidelines. Things t o consider included: determining responsibili- ties and designating leaders, refining procedures to be used, developing guidelines for planners, determining benefits and costs (for better planning), and outlining recommendations for imple- mentation. Fourth, cultivate an enabling environment. An enabling environment i s critical for a satisfactory outcome. Developing such an environment calls for support by government leaders and, more important, the willingness o f public sector managers in line agencies to commit them- selves and their staffs. Further, the evaluations found that a supportive legal framework was nec- essary for smooth implementation. 5.7 On the Bank side, incentives to work with social development themes are important. Some OED evaluations suggested that task managers need incentives to accommodate and ex- periment with social development approaches, especially their f i r s t time. Suggestions for maxi- mizing effectiveness from the various evaluations include: 0 Senior management must give high priority to staffing and structuring country teams. 0 Country offices need to be adequately staffed.26 0 Country directors need sufficient authority to make a wide range o f programming and implementation decisions in the field.27 0 Good practices should be developed that encourage staff to be sensitive to predatory and exclusionary behavior that adversely affects projects and shared development objectives.28 0 Completion reporting needs to deal with appraisal commitments o n social and environ- mental matters, especially those dealt with in the project EA.29 5.8 Social Assessments need to be done consistently where needed, but sector and regional assessments m a y be able to make the process within the project context less resource-intensive in some cases, and depending upon the size and ethnic diversity o f the country in question, there m a y be a role for national-level exercises. Social development plans stated in Project Concept Documents need to be consistently implemented and monitored. MAINTAINING THE FLOW OF SOCIALLY RELEVANT INFORMATION 5.9 The O E D evaluations found that learning about peoples’ behavior and opinions in a con- tinuous fashion i s essential to ensuring that project objectives are relevant to people’s needs, and that project implementation must take account of local realities. Evaluations in nine o f the ten areas agreed that awareness o f the socio-political realities and/or the performance o f social analy- sis within borrowing countries i s crucial t o effective social development assistance. The Post- 22 Conflict Study noted that the Bank must thoroughly grasp a country’s political framework to be relevant and effective. The Bank must understand that framework, since i t i s a major determinant o f the prospects for sustained peace.3oLooking at social analysis in particular, the evaluations underlined i t s importance, noted that it i s too often lacking when it was needed, and recom- mended it be performed in a more timely manner and then used. 5.10 Seven o f the ten OED social development evaluations reviewed by the meta-evaluation found that Bank operational staffdo not have easy access to the technical social development knowledge they require. Knowledge i s often either not organized in a manner that facilitates re- trieval, or has just not been gathered, at least not by the Bank. The lack o f knowledge and capac- ity handicaps subsequent projects. The Culture Study noted that task managers preparing projects lack information o n whether physical manifestations o f heritage3’ are found in the project area, and reports, “even the most qualified staff are hampered by lack o f sector analysis.”32 Even when the mechanisms for keeping track o f experiences and building capacity are in place, it i s a general fact that capacity building takes time and training and ongoing technical assistance i s often nec- essary for benefits to materialize. The OED evaluations called for agreements with specialized partners for knowledge support, more intensive practical guidance, increased communication (especially regarding funding) with NGOs/CBOs, and a greater commitment to learning by do- ing, and exchange o f experiences across countries. 5.1 1 Awareness o f the socio-political realities within borrowing countries is crucial to effec- tive social development assistance. Awareness o f and responsiveness to a country’s agenda for a given social topic in Bank assistance proves to be a very important factor for successful results. Eight OED evaluations reviewed by the meta-evaluation reported that a lack o f specific country knowledge hindered project success.33The evaluation reports recommend that the Bank become more knowledgeable in country issues by conducting inventories, performing local institutional analyses, and developing good practices, among other things, in order to avoid blunders and im- prove ~ e r f o r m a n c e . ~ ~ 5.12 Though social analysis has been improving, OED evaluations in seven o f the ten areas reported that often, the focus is off the mark, the timing is problematic, and the findings are not used.35 The evaluations found that there i s considerable scope for improving the comprehensive- ness and timeliness o f assessment diagnosis, for expanding the analysis to include more than just traditional sectors, and for better mainstreaming o f social themes into the Bank’s analytical work in In addition, the focus o f analysis that does take place was found to b e relatively nar- row. 37 M o s t projects incorporated a focus o n one or another social issue, and a holistic social as- sessment was missing from the majority o f projects. The projects had not only a narrow treatment of the behavioral and social structural elements, but provided little evidence that an in-depth knowledge had been acquired, as found by the 200 1 O E D Gender Study. Similarly, assessment o f women’dgender issues was often limited and tended to ignore differential access to and control over assets and resources. Where i t did pay attention to these issues, the analysis did not result in appropriate recommendations. 5.13 Even where assessments have been satisfactory, it i s unclear from available documenta- tion whether recommendations have been incorporated into project design and implementation, according to the Forestry Study.38The Water Resources Study found that project documents sug- gest that social assessment i s deferred to the implementation stage without specific budget alloca- tion. Similarly, analysis o f gender issues, often formulated within the framework o f poverty 23 and/or exclusionary policies for service delivery, has been integrated into the design o f many pro- jects; but attention to gender i s seldom translated into action or monitored.39 5.14 Several o f the O E D evaluations also pointed out that, given the Bank’s current strategic thinking, little can be done until the link of a given social topic with poverty reduction is more fully explored. Numerous approaches to the cultivation and retention o f technical knowledge were suggested. (For more information and examples, see the meta-evaluation report, “Technical Knowledge” section.) 5.15 N o t surprisingly, several o f the OED evaluations recommended that the poverty- reduction effects o f activities and the economic growth and poverty reduction linkages need to be documented and disseminated. The Bank needs to clarify h o w i t s gender, cultural heritage, and forestry policies are linked with i t s poverty reduction mandate, and explain the operational impli- cations for Bank processes and practices-this would be the responsibility o f the relevant sector or thematic group. The evaluations also recommended that the Bank recognize the importance o f specialized knowledge and cultivate such knowledge. 5.16 Part o f maintaining a strong knowledge and resource base involves monitoring and evaluation in order to feed lessons learned back into the process and improve projects in action Unfortunately, in a l l but one o f the areas there i s a critical lack o f monitoring and e ~ a l u a t i o n . ~ ’ 5.17 Resources are needed to maintain theflow o f socially relevant information. The human and financial resources in the Bank for social development are not adequate for the number and scope o f the mandates received. Though human resources are undoubtedly critical to the suc- cess o f social development aims, all but one o f the OED evaluations claimed that the human and/or financial resources for the task at hand were not adequate.41This finding was confirmed by the survey o f country directors. When asked whether there are areas where increasing the investment in social development support w o u l d yield major benefits, 87 percent said yes. In order to address risky and controversial social areas, the Bank’s internal incentives and s k i l l mix need to b e enhanced so that operational staff feel they have the support and confidence o f Bank management and country borrowers and access t o the human and financial resources needed, as found by the Forestry Yet other evaluations mentioned the need to strengthen program management. 5.18 Each OED evaluation offered a slightly different suggestion for the most cost-effective way to deal with the lack o f staff knowledgeable in social development areas. Suggestions in- cluded: rely o n consultants and partnerships to do specialized work, maintain a minimum core o f competency to deal with the “do n o harm” aspects o f the work, and employ a few strategically placed staff, Also noted was that these areas are generally risky and controversial, thus demand- ing not only operational staff support but also management support. Suggestions along these lines included: give high priority to staffing and structuring country teams, and make sure there i s up- per management support for the ideas.43 5.19 Budget allocations affect project performance. The OED evaluations found that even the most qualified staff are hampered by inadequate budgets for strategy implementation and project preparation (Culture, Forestry, and Gender). Several evaluations noted difficulties complying with policy because the control o f the budget was in the hands o f country managers who had t o deal with a tight budget constraint and whose highest priority was not always social development. 24 5.20 The evaluations noted the importance o f aligning resources with strategies and objec- tives. T o ensure safeguard compliance, expertise and funds need to be available to facilitate the process-in community-driven development, in culture, in forestry, in post-conflict situations, among others, “Adequate funding, institutional capacity, and time are.. .needed within the Bank to ensure safeguard compliance and responsiveness to high-priority country demands.”44 The in- vestment pays o f f when country offices are adequately staffed, the Post-Conflict Study report explains. I t describes those investments as “a precondition for successful Bank intervention in post-conflict situation^."^' 5.2 1 Continuity in stafing is key. This general OED lesson also applies to social development themes. The China Case Study o f the Forestry Study noted that continuity among the staff man- aging Bank projects has been high at all levels in Bank forestry projects in that country, generally ensuring accumulated learning and greater impact. With this continuity, China has been able to create successful models that can be replicated within the forest a d m i n i ~ t r a t i o nThis . ~ ~ lesson i s underlined by the review o f highly satisfactory projects in the previous chapter. INSTITUTIONAL DEVELOPMENT BORROWER IN THE BROADEST SENSE 5.22 OED evaluations have long highlighted the lack o f institutional capacity in borrower countries, especially in less powerful ministries such as those that generally deal with social de- velopment themes. N o t surprisingly, a l l o f the OED evaluations examined for the meta- evaluation agreed that the Bank’s impact o n institutional development had been limited. This as- pect o f project context i s an important factor in project success and can have broad effects when neglected. For instance, according to the OED Forestry Study, the Bank’s Forest Strategy over- looked fundamental governance issues that affect forest sector development, thereby hindering Bank the entire ~ r o g r a m . ~ ’ impact o n institutional development has been limited-“modest at best,” according to one evaluation. Another evaluation was unable to link provision for i t s highly pertinent theme (NGOsKBOs) to institutional development at all. 5.23 T o give an example o f the sort o f skills deficits that social development staff (and the SSP) need to be prepared to routinely address, Bank-financed rural water projects regularly build water systems and expect farmers to be able maintain the pump and the piped system, keep books, collect user fees, run meetings, answer correspondence, budget for equipment replace- ment, and so on. Without sustained help, they almost never can. Skills differences are not con- fined t o rural water projects. Successful forestry, housing, disaster, sewerage, and other projects that leave behind formal organizations for communities to manage require beneficiaries to per- form similarly complex tasks. Such projects often r e l y o n public employees (field staff) to super- vise the sort o f participatory activities they have precious little experience with, often in difficult- to-find and hard-to-reach areas. 5.24 The Bank’s focus i s currently o n building national-level government capacity. For lasting and broad impact, however, the OED evaluations found that it i s important to l o o k at a l l levels o f institutional development-national, local, and NGOKBO-and w o r k to improve the existing system by clarifying (or creating where necessary) procedures, roles, and responsibilities. 5.25 The Bank recognizes that creating effective borrower capacity for dealing with social themes i s more useful than hiring expensive foreign experts and that it has a lasting and broader impact, and Bank activities have explored some successful methods o f capacity building. How- ever, as noted, projects tend to focus only o n institutional capacity building at the national gov- 25 emment level, ignoring the local institutions, NGOs, and CBOs. Recommendations from the OED evaluations had two broad messages: 0 L o o k at all levels o f institutional development-national, local, and NGOKBO-and work to improve capacity within the existing system. 0 Make a roadmap by clarifying (or creating where necessary) procedures, roles, and responsibilities. 5.26 Develop local capacity. Improvements in sustainability were found when efforts were made to create effective local capacity for management. The Culture Study found that reliance o n local knowledge and experience m a y result in higher upstream costs (building in time for input, consultation, education, and conflict resolution), but determined that the improved sustainability o f projects where stakeholders are sufficiently involved made up for this. This i s likely to b e true in other thematic areas, and i s probably reflected in the higher sustainability ratings (relative to their comparators) for Indigenous Peoples, Culture, and Resettlement (see para. 4.5). Developing partnerships and alliances: NGO participation in Bank projects i s increasing, but actions vary greatly in depth and quality 5.27 Often, the best way to adequately plan and supervise projects may be through partnerships. T h e Bank has had some valuable positive experiences with partnerships and with innovative ap- proaches to participation and the N G O Study reported that Bank NGO guidelines are sound. 5.28 The Bank has been encouraging the participation o f partners (at the institutional level) and alliances (at the project level, “coalitions for change” in current parlance) more and more since the early 1990s. For instance, the Water Resources Study reported that the number o f pro- jects that involved an NGO specializing in gender at the appraisal more than doubled (from 8 percent to 19 percent) from one period to the next.48This participation had not been o f the neces- sary quality, however. The NGO Study showed that, in fact, N G O K B O involvement varies greatly in depth and quality, and that skepticism o f NGOs leads to easy failure o f partnerships o n many levels.49 5.29 Partnerships have generated enhanced results in many Bank activities, but the OED evaluations recommended that the Bank not rush the process, and: (1) make deliberate and in- formed choices about who to partner with (consider all levels o f government and NGOKBOs); (2) delineate a rationale, rules, and procedures; and (3) develop capacity. 5.30 NGOs and CBOs should be involved in the right sectors and “owned” by the borrowing country. OED evaluations in nine o f the areas5’ found that, overall, partnerships are beneficial. However, they strongly urged more selectivity when choosing Bank partners. The NGO Study reported that the Bank’s partnership approach i s “ad hoc.”j’ The rhetoric o f partnership and par- ticipation i s “moving faster than the reality,”j2 according to NGOs in Uganda quoted in the Post- Conflict Study.j3Bank support for NGO and CBO involvement should b e provided to projects in those sectors, and for those activities, where their involvement i s believed t o have a comparative advantagej4-such as in cultural heritage.j’ Additionally, partnerships in support o f country pro- grams work best when there i s an agreed policy framework “owned” by the borrowing country, when sector professionals clarify the Bank’s core competencies and comparative advantage rela- tive to partners, and when i t establishes businesslike alliances (that benefit b o t h parties) with ap- propriate sharing o f responsibility, according to the NGO Study.j6 There i s n o one right partner in 26 most cases, however, and effective partnerships are also the result o f hard work, mutual learning, and common interests. Participation i s up, but the quality i s uneven 5.3 1 Participation is up. In six o f the O E D evaluations, plus the Social Funds Study, participa- tion was reported to be high or increasing.” However the quality and impact ofparticipation was found to be uneven. The O E D Participation Study found that participation was often limited to only small parts o f projects, too rushed or superficial, or had been too ineffective to make much difference. Similar to other social strategies, plans for participation have become more ambitious during appraisal and design, while implementation has lagged. Participation was uneven over the project cycle and gains varied across Regions, often reflecting the extent to which borrowing countries pursue participatory approaches in their o w n programs, as found by the Forestry Study.’* Within the Bank, the most significant constraints were scarce time and money, rigid pro- ject cycles, inadequate incentives, and inconsistent management support for participation.” 5.32 Participation may be increasing, but key stakeholders are often not consulted. As a result o f safeguards, vulnerable stakeholders have been incorporated in project design, but other key stakeholders, including those most likely to cause harm, are often not consulted. The ways, for instance, that safeguards have been applied can thus contribute to conflicts among interests com- peting for resources-including conflicts between the indigenous poor and the non-indigenous poor, between the powerful and the poor, and others-without helping to establish transparent r u l e s to hold all accountable and to monitor performance.60Often, participation captures the input of an enclave within a community, partly because in Bank-assisted projects it often bypasses ex- isting processes or organizations.6’ 5.33 Often, the poorest-landless farmers, pastoralists, smallholders, and minority communi- ties-have little voice in planning and tend to get further marginalized in implementation.62 Even when community participation mechanisms are in place, as in social funds projects, i t can be dif- ficult to reach the poorest communities because they are often the least competitive in preparing proposals. The Social Funds Study also noted that this kind o f decision making m a y not be ideal for a l l types o f projects, especially those that require higher-level decisions to benefit from economies o f scale or deal effectively with e ~ t e m a l i t i e s . ~ ~ 5.34 Concentrating on participation at the micro level can lead to the loss o f the macro plan- ningperspective. W h i l e participation points in the right direction, according to the OED evalua- tions-enabling the Bank to play a catalytic role in operationalizing and scaling up domestic par- ticipatory approaches in several countries that were committed-it lacks financial resources for the training and capacity building needed to put the approaches into practice. Without an equita- ble country institutional framework overall, the impact o f Bank attempts to focus o n social issues at the community level will be marginal. This i s the case with respect to women’s participation, according to the Gender Study, among others. A better understanding of participatory techniques i s needed 5.35 Increased stakeholder participation has improved outcomes in areas related t o social de- velopment, but a better understanding o f participatory techniques could lead to more consistent outcomes and impact. There i s considerable room for quality improvement according to the OED 27 evaluations. L o c a l community members can be agents o f transformation, but they must be ade- quately mobilized and organized; they must participate. Participation results in: increased owner- ship; improved transparency and accountability in contracting and procurement; and better rela- tions between men and women, between villages and government agency staff, and between groups that have not traditionally c ~ o p e r a t e dIn ~ case o f cultural heritage, bringing together . ~the those who l i v e near an asset and those whose actions are destroying i t to participate in the preser- vation process m a y be the best way to preserve physical and living culture, and to ensure ongoing use by the surrounding community.65However, several evaluations noted that care must b e taken that intensive community involvement in planning and project design does not generate expecta- tions that are impossible (or inadvisable) to 5.36 Seek solutions with a broad support base. Three OED evaluations recommended that within and beyond the project context, solutions should be sought that have broad support and widespread public a~ceptance.~’ Broad-based approaches that include the views o f all stake- holders offer more sustainable results. In post-conflict countries, sustained peace i s essential to sustained development. According the OED Post-Conflict Study, broad-based development, im- portant in i t s o w n right, also contributes to sustainable peace.68 THEINTERDEPENDENCE OF SOCIAL THEMES 5.37 M a n y social themes are interdependent. Four OED evaluations (Participation, Forestry, NGOs, and Culture) emphasized the value o f integrating w o r k o n these topics with other social topics and with other areas in development to create synergies. Individual attempts to address single issues often suffer, while integrated approaches can create synergistic effects, and lead to better implementation and more socially sustainable benefits and institutions. The push for an integrated approach should not lead to (or be an excuse for) overly complex design. For example, the Post-Conflict Study underlined this point, warning that challenging circumstances coupled with good intentions often lead project designers to attempt to implement more components than can be successfully managed. 5.38 While integrating social development themes in the design and implementation o f pro- jects can be advantageous, maintaining individual focal points within the Bank that center o n a single theme, increases the “visibility” o f an issue. Various social development topics receive targeted support and individual treatment across the Bank structure. The current individual focal points m a y only be a temporary or interim measure (as social development efforts seem l i k e l y to become more integrated), but they help gain visibility through stand-alone units, leading to even- tual mainstreaming. 28 6. What Are the Implications for Bank Organization and Procedures? 6.1 Three survey instruments were administered to selected Bank staff through Lotus Notes (see Annexes E and F). Each instrument had about 20 questions in four broad categories: 0 Improving the Bank’s practice in social development 0 Improving the skills balance o f social development staff 0 Bank organization and social development 0 Contributions o f social development staff to the work o f task teams and country teams. 6.2 Given that the number o f task managers i s very large (more than 800 were identified by the study), a randomly selected sample o f 263 task managers o f ongoing projects with social de- velopment themes was identified, stratified according to the degree that sectors were present in the overall portfolio. The response rate was 17 percent (44 task managers), which i s fairly typical o f Bank staff response rates to surveys. The social development staff survey did not select a sam- ple, opting instead for the complete universe, based o n the SDV roster. This consisted o f 164 permanent staff and long-term consultants. The response rate was 44 percent, and those who re- sponded tended to be experienced. The average time respondents had spent working in social de- velopment in the Bank was 5.7 years (median o f 5). All 49 country directors were sent the sur- vey, and were asked to complete the survey themselves or pass it o n to knowledgeable staff o n their country team to fill out. The response rate was 53 percent. This i s higher than normal and reflects the support given by the Country Directors network.69 THEBANK’S COULD BEIMPROVED FOCUS ON, AND SUPPORT FOR, SOCIAL DEVELOPMENT 6.3 A large majority o f task managers (93 percent) said that the nature o f their work routinely required attention to social development themes. Country directors rated the importance o f social development quite high: 96 percent o f respondents said that, based o n their experience, attention to social development improves the development outcome o f Bank operations. Randomly sam- pled task managers were also very positive: 83 percent o f them said the same thing. Over 94 per- cent o f the social development (professional) staff members that responded cited specific and unique contributions that social development has made to the work o f the Bank (a selection o f survey responses i s in Annex E). The most common responses were (in descending order): 0 M o r e realism has been brought into all types o f projects and instruments and it has opened the eyes o f the Bank and i t s borrowers to the stakeholders’ perspectives. 0 It has married the economic and social development agenda for sustainable development and poverty reduction. 0 It has focused attention o n indigenous peoples, resettlement, rehabilitation, post-conflict, and culture. 0 The work o f social development staff has fostered greater inclusion in Bank projects. 6.4 The social development contributions to the w o r k o f the Bank identified most often by the task managers were: 0 I t improves project design through better understanding o f the social context. 29 0 I t leads to a clearer understanding o f project impact. I t contributes to sustainability. 0 I t improves relations with clients. 6.5 The meta-evaluation found that the OED evaluations generally reported an improving trend. The social development work that i s being done in the various thematic areas i s more likely to build o n best practice, and attention i s being given where it i s needed more consistently than in the past. Social development staff perceive this trend as well. A very large majority (91 percent) noted that attention to social development i s currently being handled more appropriately than in the past-9 percent said that there has been n o improvement. When asked about the im- pact o f their o w n work, 89 percent said that it has had a significant impact o n Bank operations. 6.6 Staff were also asked about the project cycle-specifically, which activities required more attention from social development Monitoring and evaluation (68%) staff than they were currently receiving. Nearly everyone thought Supervision (61%) that at least one aspect needed more attention. O f those that speci- Project identification (49%) fied further, 17 percent said that all o f them requiredmore. The Design and appraisal (46%) spheres are ranked in the table (right) in descending order (of need- ing attention). Completion reporting, a subset o f M&E, was named by 32 percent o f respondents as the evaluative aspect most in need o f attention. 6.7 For the most part, task managers found the social development staff with w h o m they had worked o n a project to be knowledgeable (90 percent) and found their inputs useful to the task they were addressing (92 percent). W h e n task managers were asked whether they had recom- mendations for increasing the impact o f the work done by social development staff o n poverty reduction, their responses f e l l into four categories. O f those w h o responded, 23 percent thought the way the Bank approached social development needed to be modified (see B o x 6.1). Box 6.1: Modifications Suggested by Task Managers 0 Social development staff should w o r k with their colleagues as part o f interdisciplinary teams. 0 Social development staff should be a n integral part o f country and task teams and should participate in development o f CASs and Poverty Reduction Strategy Papers (PRSPs),” as w e l l as be part o f the qual- i t y enhancement team for projects. 0 Social development staffs contributions should be less “audit-like.” 0 There needs to be an increase in staff with social development skills and they need a budget adequate to complete their tasks. *One surprising survey result was that about half o f the task managers (50 percent) and social development staff (53 percent) opine that their work will not change and does not need to change as a result o f the PRSP process. 6.8 An equal number o f task managers (23 percent o f respondents) were o f the opinion that social development staff need to change their way(s) o f thinking. Their comments f e l l into sev- eral groups-they suggested S D staff integrate better with their Bank colleagues and be more readily available when their help i s sought; be open to new initiatives that might have social im- pact, even when they are not traditional social foci; develop some new techniques to w o r k with the clients; listen more closely to the beneficiaries; and focus o n the most pressing problems o f the poor, and their limitations to address them. 30 6.9 About 18 percent o f the task managers were o f the opinion that social development staff needed to take a more practical approach. They urged social development staff to be more en- gaged in the monitoring and evaluation o f project activities, with a more quantitative approach to situation analysis. They urged that social assessments and social development issues in projects be more sharply focused o n just the pertinent issues. Survey respondents suggested that social development staff should avoid working in isolation, interacting more with task teams to make sure the approaches they suggest are relevant. 6.10 Task managers also had some ideas when asked the open question o f h o w the role o f the central Social Development Department (SDV) should be changed. They requested that social development staff stop “policing,” coordinate more, and provide more support to operations. One respondent suggested that people be flexibly grouped, by team or taskforce as needs arise and change. Respondents noted that PRSPs are often supervised by people who have limited opera- tional experience and suggested that social development staff could add operational “know-how,” to increase the likelihood that PRSPs would lead to action. 6.1 1 Some task managers thought that social development staff needed earlier involvement with projects so that the project design and project area could s t i l l be easily adjusted if that was required. The suggestion was made that social development staff become involved in all poverty reduction interventions from the very beginning. 6.12 B o t h country directors and task managers were o f the opinion that social development staff could do more to train task managers so that they could handle routine matters without fur- ther support. Other related suggestions included a request for training in specific approaches that allow infrastructure projects to do a better j o b o f targeting the poor. 6.13 A small number o f task managers suggested that social development staff be recruited and selected differently. They thought that it was important that social development staff bring real grassroots experience and solid analytical skills to the Bank. They also suggested that re- cruiters l o o k particularly for practical people who are able to communicate in plain language, as well as those w h o have the patience to build capacity and the interpersonal skills to network with a wide range o f groups. 6.14 One area where country and task teams appear to be well supported i s for participatory processes. W h e n country directors were asked if they get the support they need to l a y out partici- patory processes, 92 percent said they did. One cause for concern, however, i s that a majority o f country directors (58 percent) said that task managers are not adequately trained to recognize so- cial development issues that arise during the project cycle and that required specialized interven- tion. T h i s perception i s in line with the task manager’s request (immediately above) for more practical training. WORKING INTERNAL ACROSS UNITS WOULD MORE HELP 6.15 One consequence o f the Bank having moved into social development incrementally (and at least partially as a response to outside criticism) has been the creation o f internal divisions, which some in the Bank refer to disparagingly as “silos.”70 Dealing with each topic as it came up led in- evitably to a bureaucratic separation o f some social science practitioners from others with whom it might have made sense to work together or the separation o f some policy aspects from others. For example, a gender unit was created in PREM, where it functions without day-to-day interactions 31 with SDV. PREM staff often supervise the preparation o f PRSPs, even if they lack the social sci- ence background (which S D V staff have) that would facilitate such a task. Similarly, crisis re- sponse was divided when a post-conflict unit was created in ESSD and a Disaster Management Fa- cility created in the Private Sector and Infrastructure Network. Further reinforcing these bureau- cratic separations, the creation o f separate safeguard policies directed staff focus to single issues, spurring the creation o f additional separate units to deal with the various topics. T h i s has led to the treatment o f a number o f social issues as individual entities in Bank operations. 6.16 Staff opinions o n the advisability o f changing the organizational structure to bring practi- tioners closer together are negative (Table 6. l), in part because they have change fatigue but also because they do not believe that the organization will get the structure right n o matter h o w many times i t tries. One person wrote, “ Ithink it i s less important whether or not [social development] i s in SDV/PREM or HD [the Human Development Department] than whether w e can break down a l l sectoral boundaries and get o n with the business o f multidis- Table 6.1: Staff Groups ciplinary poverty reduction. Social development should be part Generally Oppose Merging o f this agenda and a l l the rest about where i t sits and what i t i s Social Development Groups called i s pure bureaucratic infighting.” Another person com- mented, “Because I a m a realist and Ik n o w the Bank will al- Percent opposed ways be dominated by economists, I would not recommend that Social Development Staff: 60% we try to make gender and poverty ‘part o f S D V ’ (though there Task Managers: 73% i s a great deal o f justification for making gender part o f SDV). I Country Units: 65% fear also that if there were to be a ‘new stand-alone unit,’ it w o u l d be dominated by P R E M economists.” 6.17 O f course, n o unit can possibly contain everyone and everything it might need, and any w a y staff are organized inevitably will lead to new issues. Part o f the solution i s to work effec- tively across internal divisions. A commendable example o f this i s the Poverty and Social Impact Analysis web site (created in January 2003 with inputs from across the Bank) that i s managed j o i n t l y by P R E M and SDV. A coordinated plan o f w o r k building o n the site’s information i s planned for FY04 and beyond. REGIONAL ARE A K E Y LINK GROUPS 6.18 Most o f the Bank’s Networks have representative groups in the Regions-ESSD, for ex- ample, has six regional units. The regional units are the link between the Bank and the various project stakeholders. For example, because o f the controversy surrounding Brazil’s POLONOROESTE projects, L C R was one o f the f i r s t Regions to deal with the rights o f indige- nous peoples. The approach o f regional social development groups consists o f several tactics: 0 Safeguard complianceensuring compliance with Bank social safeguard policies 0 Operational support-providing support in social assessment, involuntary resettlement, indigenous peoples, and cultural property 0 Participationfacilitators-assisting the Region in promoting poverty reduction and par- ticipation in the development process by helping the Region to connect with c i v i l society organizations and local communities 0 Social development promotion-promoting social development considerations in a l l Bank-financed operations 32 0 Project preparation-preparing a n e w generation o f Bank-financed social development projects. ” 6.19 Even though they a l l consider similar tactics, it i s important t o note that the regional units focus to varying degrees o n each o f them. Additionally, some regions focus m o r e o n task man- agement, w h i l e others perform more o f a support role t o task teams. One o f the m a i n challenges faced by these regional units i s coordination-between their regional and headquarters staff, with the rural and environmental teams they are linked to, and with other B a n k task groups. They are also concerned with monitoring and evaluation o f the social dimensions o f development. 6.20 Country directors were asked to recommend ways to increase the utility o f the w o r k o f so- cial development staff in the Regions. Some o f the suggested modifications are listed in B o x 6.2. Box 6.2: Country Directors’ Recommendations Involve social development staff more in project preparation and supervision, and in helping field- based task teams to improve projects. Social development staff in the Regions need to develop more o f a sense o f ownership for development outcomes. Social development staff need to strengthen their analytic and presentation skills. Standards for ana- lytical contributions need to be raised. Social development staffs knowledge o f country-specific issues must be first-rate. Get rid o f regional social development staffs safeguard clearance function and make it advisory. A closer examination o f staffing levels needs to be undertaken in the field and in headquarters to en- sure that the proper mix i s available to satisfactorily support social development interventions both at policy and project levels. Social development staff need to do much more “hands-on” work and be exposed to field conditions. More social development staff could be based in the country offices. Encourage more exchange o f experiences between social development staff from different Regions and among those from the same Region. Improve knowledge sharing with country departments and country offices. 6.21 Regarding the role o f the regional units, task managers were in agreement with the coun- try directors. They w o u l d particularly l i k e t o see the regional social development units b e more team-oriented, and they see m a n y advantages t o having them merge with sector units. One task manager expressing a point o f v i e w shared by many said: “Social development staff sometimes act l i k e police constables rather than broadly trained development professionals.. . [They are like] doctors dealing with malpractice problems-no amount o f diagnosis i s too much, especially when spending other people’s money.” Bringing Social Development Staff Closer to Task and Country Teams W o u l d Help 6.22 Social development staff themselves stated that their effectiveness w o u l d increase if they could be out in the field. W h e n asked whether increased decentralization o f social development staff w o u l d b e an improvement, 56 percent said i t would. Task managers felt even more strongly about this: 69 percent o f them answered this question affirmatively. W h e n they were asked about development outcomes the pattern was m u c h the same (69 percent o f task managers and 56 per- cent o f social development staff answered “yes”). A majority in each o f the three surveyed groups also stated that relocating some headquarters staff t o country or regional offices w o u l d have a positive impact o n development outcomes. Country directors were most l i k e l y t o say that 33 it w o u l d have a positive impact (82 percent affirmative), followed by task managers (61 percent) and social development staff (60 percent). The w a y in w h i c h this could be made t o happen i s a matter f o r the upcoming sector strategy t o address. 6.23 An area o f intense conflict explored in the background reports centers o n the administra- t i o n o f safeguards. Although staff agree that significant benefits accrue to projects f r o m the ap- plication o f the social safeguards, the manner in w h i c h this should b e done i s the subject o f acri- monious debate. A b o u t 48 percent o f social development staff see the transaction costs o f the current patterns o f practice as being so high as t o discourage colleagues in Operations f r o m pur- suing other social development goals, and 83 percent o f task managers thought that better ways could b e found. BANKAND BORROWER INPUTS COULD BEFINE-TUNED SOCIAL DEVELOPMENT 6.24 The importance donor agencies place o n social assessment i s growing (see B o x 6.3 for relevant definitions). SDV recommends social assessment in projects that “depend o n behavioral change among individuals or within organizations, involve community implementation or benefi- ciary participation, o r are targeted t o the needs o f the vulnerable and the poor.” They note that it i s “useful f o r bridging ‘ownership gaps’ o n the part o f the borrower, by involving formal and in- formal stakeholders in design and i m p l e m e n t a t i ~ n . ” ~~ Taking it a step further, the Second Social Summit placed significant emphasis o n social assessments, noting that their future scope and purpose will extend beyond use as simply a tool in project preparation. Summit participants called f o r it to include the implementation o f systems t o assess and monitor the social impact o f macro-economic Box 6.3: Social Analysis Versus Social Assessment Social analysis refers generally to the Bank’s expert social development input. According to ESSD, social analysis i s “the systematic investigation o f relevant demographic factors, socioeconomic determinants, social organization, sociopolitical context, and stakeholders’ needs and values in order to account for the social differences, assess impact and risks, mitigate adverse impacts, and build capacity o f institutions and individuals.” I t includes many types o f analysis: stakeholder analysis, institutional and organizational analysis, participation analysis, gender analysis, beneficiary assessment, the social capital assessment tool, participatory poverty assessment, social assessment, conflict analysis, transitional support strategies (a short-to-medium-term plan for Bank involvement in a country in conflict), social impact analysis, safe- guard assessment, and macro-social analysis. Social assessment is the main analytical instrument used by aid agencies and their clients to examine the social context in which a given project w i l l take place and integrate the findings o f social analysis and par- ticipation into project design.74 Assessments take into account social diversity, social cohesion, security, livelihood systems, gender equity, social opportunities and constraints, stakeholder interests, participation, and social risks, associated with a certain project. The basic components o f the social assessment process, according to (draft) OP 10.05, Social Analysis, are (a) identification o f key social development and par- ticipation issues; (b) evaluation o f institutional and organizational issues; (c) definition o f action plans and participation framework; and (d) establishment o f mechanisms for monitoring and evaluation. For Bank- financed projects, the borrower performs such an assessment in order to weigh the potential benefits and costs (sometimes they hire external consultants). The assessment provides a “framework for reaching agreement about intended social development outcomes and the indicators to monitor those outcomes;” and it “promotes dialogue and understanding among stakeholders; in tum fostering the organization o f alliances for action.”’’ I t also helps build the capacity o f local social scientists. 34 6.25 Social assessments are performed (by the borrower) o n about a third o f World Bank- financed projects. In exceptional circumstances, such as a controversial large dam project, the cost o f a social assessment can exceed $500,000. This has only happened in a handful o f cases, how- ever, and several years o f conducting social assessments has led to cost efficiencies across the board. SDV data shows that the costs for social assessments has declined slightly from an average o f $75,000 in 1996, to $37,500 (plus four staff weeks, or a total o f $57,500) per social assessment in FYO1-02, with a range o f $15,000 - $120,000. Two OED evaluations found that the average time from terms of reference to draft for a social assessment at the Bank was six months. FOCUS ON VITAL ISSUES SOCIAL ASSESSMENTS GOOD 6.26 W h e n social assessments are performed, they can strengthen projects. Successful social assessments are tightly focused o n vital issues, giving them the minimum treatment necessary for full coverage o f pertinent social aspects in their specific project context. W h e n they concentrate o n just a single component, social assessments can s t i l l be more u s e h l than if they are used to make general statements about overall social and economic policy. Often, however, assessments are not performed due to shortage o f hnds, time, or skills. Worse still, they are often done super- ficially, leading to “comfortable” findings, rather than high-quality assessments, or they overlook key and interrelated issues. One Bank study found a lack o f methodological rigor, centering o n a question o f the representativeness o f the samples chosen; a failure to limit their focus to only those issues critical to the project; and unevenness in coverage across projects, sectors, and Re- gions. Another difficulty with social assessments i s that even when they are done well, the find- ings m a y not be h l l y used, or even used at all. Several OED evaluations o f topics related to so- cial development found instances where analyses/assessments (environmental and social) were h l l y satisfactory, but i t was unclear from available documentation whether recommendations had been incorporated into project design and implementation. 6.27 According to the Q A G Quality at Entry Assessments Table 6.2: Quality of Social (QEAs), which examine a random sample o f 100 new lending op- Risk Assessment (Percent erations each year, the quality o f social risk assessment has Satisfactory o r Better) dropped from CY98 to the present, from 77 percent satisfactory or better in QEA1, to 60 percent in QEAS. The Q A G social risk as- Social Risk sessment ratings are shown in Table 6.2. Assessment QEAI (CY98) 77 QEA2 81 6.28 An unpublished SDV study noted that the portion o f the QEA3 81 Bank projects that underwent social assessments in fiscal 2002 QEA4 79 was 46 percent. The study examined 44 social assessments in de- QEA~ ( ~ ~ 0 2 ) 60 tail. Within that sub-group i t found: Source: QAG Fifth Quality at Entry Assessment, 53. 0 About 75 percent o f the operations incorporated the recommendations in project design. 0 Eighty-four percent informed the design and implementation o f the M&E system. 0 Ninety-five percent resulted in specific action plans for social issues a n d o r impacts. 6.29 The study noted that about h a l f o f the assessments were concerned with adverse impacts o f Bank-hnded projects. Issues most frequently reported were equity and distribution concerns and, to a lesser extent, social exclusion, vulnerability, and gender discrimination. I t also found that where social benefits and adverse impacts were identified, less than h a l f o f the assessments defined monitoring indicators fully. 35 6.30 Assessment quality i s only likely to improve if client commitment to social development goals increases. One o f the country directors surveyed noted a “disconnect o f Government and Bank policies and an unclear understanding and commitment by clients to social assessment proc- esses and application o f safeguard policies.” Staff interviewedfor an SDV study o f social analysis strongly believed that if the Bank wishes to require such assessments, it should fbnd them. 6.3 1 The OED evaluations suggest that clear guidance and rules concerning when and h o w social assessment should be performed are needed. Overarching social analysis should take place, covering issues in a coordinated way, considering regional and sectoral issues, and then supple- menting that knowledge with only necessary, targeted, and project-related assessment. Because social assessment i s an iterative process, this information should be promptly integrated into pro- ject preparation and/or implementation, and the assessment process continued as needed. T o co- ordinate such a process, the Bank m a y consider having each country team designate one person for each country to oversee the collection and distribution o f knowledge generated by social sci- entists working as c i v i l society specialists and those working o n social assessments and economic and sector work. The system o f responsibilities should be reworked to make Bank social scien- tists assigned to task teams into overseers, affording them time to make sure that the whole i s performing well. T h i s i s but one o f several reasons why social development staff need to recon- ceptualize their work into a role that makes them function more as managers and coordinators o f larger groups o f less expensive consultants. 36 7. Conclusions and Recommendations for the Bank’s Sector Strategy 7.1 The surveys conducted for this study show that Bank operational staff generally n o w consider social development integral to what they are doing, and they state that it has a signifi- cant impact o n development outcome. The OED performance data that supports their view seems compelling: the inclusion o f social development themes makes a difference (para. 6.3). Projects that addressed o n l y one social development theme were rated 3 to 4 percent higher in three rat- ings categories (outcome, sustainability, and institutional development impact) than the overall average o f Bank projects for a 30-year period.76Integrated attention to social themes (covering more than one theme, as appropriate) i s apparently a recipe for success according to the broader ratings analysis: projects that address multiple social themes concurrently perform even better o n the same three ratings. Two-theme combinations beat the one-theme average, and larger overlaps did better yet (paras. 4.8-4.12). Even more encouraging i s the likelihood that this picture will improve considerably: there i s an improving project performance trend, there are highly satisfac- tory projects to study, and there i s likely to be more effective quality control and a clearer vision o f priority areas as a result o f the elaboration o f the social development sector strategy. This i s not to say that improvements cannot be made. 7.2 Meta-evaluation findings (paras. 5.3-5.38) and the results o f the staff surveys (paras. 6.3-6.3 1) highlighted a number o f major challenges that will need to be overcome if social de- velopment i s to filfill i t s potential to improve projects and the lives o f the poor. 0 There i s a lack o f clarity about what social development i s and about when to use i t s dif- ferent instruments (social analysis, social assessment, beneficiary assessment, social ap- praisal). 0 There i s a lack o f consistency/quality in the application o f these instruments. 0 Recommended actions are often not implemented or monitored. 0 Social issues identified in the CASs are not fully reflected in the Bank portfolio. 7.3 One challenge (which shows up more Box 7.1: Success Factors in Highly Satisfactory Projects with Social Development Components clearly in the survey summary background re- port) i s that staff are evenly divided and disagree 0 Stakeholders, especially beneficiaries, strongly about a number o f issues. Perhaps most participated in project design. contentious i s the safeguard clearance issue dis- 0 The project made participation inclusive cussed in the previous chapter. and brought in participants early on. 0 Implementers viewed participation as a 7.4 T o give another example o f contention, continued process. 0 The project built institutional capacity at operational staff and task managers want those a l l levels. social development staff who provide operational 0 I t involved beneficiaries in project support to spend more time out in the field, and implementation. the majority o f social development staff believe I t secured a community contribution to the that they would function better with a closer con- project and its future maintenance. nection to Operations. Nevertheless, the 44 per- 0 I t drew o n community traditions. cent who do not agree represent significant resis- 0 I t built on local NGO capacity. tance to any initiative in that area (assuming that 0 I t made sure participation was culturally many o f the staff who do not want a closer con- appropriate. nection are in the business o f providing opera- 37 tional support) and relocation does not always lead to integration in any event. The social develop- ment leadership in the Bank needs to work o n developing a consensus around some o f these issues. T h e Closer You G e t to the Poor, the M o r e You Need Social Development 7.5 In the past the Bank built things, n o w projects involve people. This can b e seen in the success factors listed in B o x 7.1. The nature o f these factors (detailed in Annex D) shows that there i s more interaction with communities than ever before. The Bank has gotten into this situa- tion because o f i t s renewed focus o n poverty. Projects deal increasingly with community-based organizations because they have the best record at bringing services to the target groups, and they can do this at a relatively l o w cost. With this change, however, the Bank, w h i c h i s tradition- ally good at “wholesale” projects, n o w finds i t s e l f doing “retail” projects quite often in some sec- tors. All three surveys show that social development staff need (and many o f them want) to spend more time in the field to support this effort: they recognize that within many projects there i s a situation in need o f attention. 7.6 Almost by definition, the poorer the members o f a community organization, the lower the levels o f i t s leaders’ technical skills and formal education. Consequently, Bank operational staff (and anyone who purports to work at poverty alleviation) need to be ready to address skills dif- ferences and deficits (whatever they m a y be) when they work with local institutions. Insofar as formal organizations are required to achieve project aims, while there have been some isolated (and successful) attempts to build o n local organizations’ capacities, OED evaluations show that the need o f those organizations for systematic support has not been adequately addressed (paras. 5.22-26). Bank projects that leave infrastructure behind in poor communities cannot assume that putting hardware into place i s a contribution to sustainable development in the absence o f a func- tioning social structure to support it. (This i s the only problem for social development reflected in the ratings data: five out o f eight disaggregated social development portfolios were rated lower than their comparators o n sustainability, para. 4.5). 7.7 The institution’s renewed emphasis o n poverty reduction will depend heavily upon the tools that the Bank’s social development specialists bring to the job. Bank operational staff need to be able to work effectively with poor communities so that they become involved in effective change. The success factors (Annex D) and the problem discussed above are worth looking at again in light o f the following question: Can the necessary activities be managed by Bank staff! The answer is: N o t very easily. I t i s too expensive to have headquarters-based staff spend nearly a l l their time in the field, and, even if they were relocated, there are so many communities per project that need support that the size o f the Bank would have to be multiplied. Instead, dedicated project staff, local networks, NGOs, and even project champions, can w o r k with communities until they develop the necessary new skills and can go it alone (paras. 5.26-5.30). If such a course o f action i s chosen, the resource implications must b e recognized. Social development staff working with project teams (especially those from the country office) can b e used as man- agerslcoordinators o f those collaborators that will work closely with project beneficiaries. The W o r l d Bank simply must develop the skills i t needs (and to some degree already has) to work with poor communities o n poverty alleviation. This cannot be l e f t to others. There i s n o evidence that smaller and more agile organizations do poverty alleviation consistently w e l l either. The highly successful projects show clearly that a few task teams have learned what needs to be done. A major challenge i s to identify champions within the Bank and learn more f r o m the inside. 38 Social development efforts need to move upstream and downstream while recognizing or- ganizational limitations and borrower reluctance 7.8 The meta-evaluation argues that (a very broad range o f ) projects require more attention to field conditions than i s the case at present. One mistake the institution has made has been to load onto social development what economic development should do. Consequently, social develop- ment has historically been focused o n the negative “policing” side-ensuring compliance with policy-at the expense o f the positive “results-oriented” side-helping to ensure that project benefits have an impact on poverty. I t should not be the role o f social development specialists to make a socially negative project work out. W h i l e the meta-evaluation and staff opinions show that social due diligence i s taking place increasingly often at the project level, they also show that too often i t i s s t i l l just an add-on. 7.9 What makes the challenge facing social development so difficult i s that more needs to be done both upstream and downstream. Preparing projects that attain sustainable outcomes and strengthen local institutions requires (as prerequisites) a supportive policy environment, the availability o f social knowledge, and putting timely, socially relevant information into the right hands (paras 5.9-5.2 1). Trouble-free implementation often depends o n the provision o f medium- t e r m implementation support, often up to loan closing. Satisfactory project outcomes also fre- quently depend o n these things. The study has identified a number o f problem areas that need to be addressed: Social development advice i s often inappropriate for operational needs or i s related to policy compliance. The most pressing social issues identified by country directors are not addressed by the Bank’s Social Development Department. Working with borrowers at the policy level should be a higher priority for social devel- opment staff. Social development specialists need to have a greater presence in the field. When social development staff are providing operational support they need to be more closely integrated with the task team and country office and to share the team’s agenda and timetable. Most poverty-focused projects need more institutional support f r o m social development specialists than other interventions, and far more than they are currently receiving. The only social themes that currently have any operational guidance are those covered by social safeguards, including involuntary resettlement and indigenous peoples. Clearly these are important issues that should not be neglected; however, rather than being the entry point for good social analysis, the special emphasis o n existing safeguards seems to be crowding other important social issues out. The study can only speculate whether this i s due to lack o f resources, adverse reactions to overzealous enforcement, or fragmentation o f the social issues across structural boundaries, especially across Networks and Departments. But historically, within the Bank, the most significant constraints have been scarce time and money, rigid project cycles, inadequate incentives, and inconsistent management support (paras. 5.7-5.8, 5.17-5.21). Even within SDV, both at the level o f country analytical work and at the operational level, more effort should be made to make this work more systematic. 7.1 1 When country directors were asked whether the governments they w o r k with are inter- ested in confronting social problems in Bank-financed projects less or more intensively, 63 per- 39 cent said less intensively (para. 3.13). This challenge needs to be confronted. The lack o f institu- tional capacity in borrower countries, especially in less powerful ministries such as those that generally deal with social development themes, i s problematic. A supportive enabling environ- ment i s critical for a satisfactory social development outcome. Developing such an environment calls for sustained outreach to government leaders and public sector managers. Further, a suppor- tive legal fiamework and policy framework will help to improve implementation and the attain- ment o f the MDGs. OED will need to evaluate ongoing policy and macro-level w o r k with c i v i l engagement, social accountability, social impact o f public expenditures and adjustment lending, when enough time has passed to allow i t s impact t o b e seen. 7.12 To b e quite succinct, the Bank needs to do more than it has to date to convince i t s o w n staff as well as i t s borrowers o f the potential o f social development. Even though the majority o f staff strongly support the Bank’s social development aims, an important minority (17 percent o f surveyed task managers, for example, seepara. 6.3) do not share them and consequently are less likely to design projects that take social development into account. One task manager had the fol- l o w i n g suggestion for social development colleagues, “[You] have to demonstrate win-wins to economists and country directors, reduce transaction costs, and learn to communicate in Bank jargon.” M o r e visible and widespread project success will undoubtedly help. What i s currently being done overall i s not commensurate in scale with what i s needed and it often does not fall in high-priority areas 7.13 The Bank cannot achieve i t s development objectives by providing financing and then walking away. Ignoring the risks o f adverse social impacts and evolving beneficiary priorities can lead to highly visible and costly errors that damage the Bank’s reputation. Under the current organizational structure, established resource allocation patterns, and existingj o b descriptions, social development staff have not been able to provide country and project teams with the social development support they believe i s needed if project-level interventions are to fully and sustain- ably achieve their objectives. Staff, and especially task managers and country directors, state this unambiguously. Achieving all o f the above will require more effective work with NGOs, con- tracting out, greater flexibility in processes, and supportive management. More care will also have to be taken in determining responsibilities and designating leaders, refining procedures to be used, developing guidelines, determining benefits and costs for better planning, and outlining recommendations for implementation. Obviously, this w o u l d have significant resource allocation implications which w o u l d have to be carefully considered. 7.14 Institutional reform was mentioned as a priority more often in the CASs than any topic that the Bank currently defines as social development. When country directors l i s t the highest- priority social concerns facing their countries (see the background report summarizing the survey findings), it t u r n s out that only a few of them are areas currently supported by SDV. Similarly, a review o f Bank and borrower social development priorities as expressed in the CASs highlights many areas not currently in i t s purview. In addition, the C A S analysis also noted h o w often transparency, accountability, decentralization, anti-corruption, and public services for the poor were mentioned. 40 Achieving the Bank’s poverty goals requires more intensive and prolonged interactions with the poor, and the resolution o f lingering M&E and knowledge management deficiencies 7.15 Project designs and related participation activities need to take more into account those stakeholders most likely to undermine or harm development efforts (Annex D). Results that alter negative power relationships, or that give communities new control over their destinies through new organizations and infrastructure have a long gestation period, and solutions are almost al- ways highly context-dependent. Meanwhile, the institution faces a serious incentives problem due to the conflict between activities that lead to rapid disbursement and activities that bring stakeholders t o the point where they can go i t alone. Exacerbating this situation are the well- k n o w n monitoring and evaluation deficiencies: without being able to demonstrate that local groups need more attention than they are currently getting, i t has often transpired that field needs have been overshadowed by procedural requirements and the need to work with government staff in the implementing ministries. 7.16 The Bank works in the same countries, regions, and sectors over and over again. The les- sons o f experience need to be incorporated in projects, but doing that requires appropriate inte- gration. SDV has prepared a draft operational policy that provides a framework for a more sys- tematic approach. Chapter 1 suggested a four-part conceptual typology for social development, and argued that the inadvertent blending o f these conceptions had led to some o f the fizziness that typified discussions in the Bank about this topic (paras. 1.9-1.14). Even in the same project, i t i s not unusual to find multiple conceptions o f social development in use. Social issues are cur- rently addressed systematically to the degree to which individual task managers buy into these issues and are able to allocate resources for them from project preparation budgets. Assigning a member o f the country team to coordinate the f l o w o f socially relevant information, funded di- rectly by country budgets would lead to greater consistency. The Sector Strategy Paper should address this knowledge management challenge. 7.17 If the demands o f country directors and task managers (as reflected in the surveys) are to be met, it will mean more resources for social development without less resources for something else. I t i s also likely to have a bearing o n staff s k i l l s mix and j o b descriptions, staff deployment, and coordination. 7.18 Country directors and task managers call for more support and a separation o f staff with enforcement and project preparation responsibilities. I t i s too soon to determine whether the re- cent creation o f a Regional Safeguard Coordinator” will m o l l i f y the critics, but it i s clearly an important step in the right direction. 7.19 Other steps being taken include a number o f recent initiatives developed by the Bank’s social development specialists o n policy and macro-level w o r k with c i v i l engagement, social ac- countability, and analysis o f the social impact o f public expenditures and adjustment lending. These initiatives have the potential to raise client awareness o f social development’s potential to enhance project outcomes, and to bridge the organizational divisions within the Bank. W h i l e not a part o f this study, they will likely be covered in the context o f forthcoming OED evaluations. 41 RECOMMENDATIONS 7.20 Drawing on these conclusions, the study makes four recommendations to be followed up by the Bank as part o f i t s forthcoming social development strategy. Since the study shows the strong positive interaction o f some social themes, the Bank (through the Social Development Sector Board) should identify, and promote integration within, the thematic combinations that improve outcomes. 9 Even though the most effective thematic combination for any given project i s generally context-dependent, the summary o f recent experience in this paper can i n f o r m task man- agers designing new projects. Social development experts should help task managers to maximize the use o f integrated approaches to social issues. 9 Social development specialists should undertake further research o n thematic interaction to discover h o w to maximize the benefits o f this finding, including h o w t o best monitor i t s impact. 9 As projects that integrate multiple social development themes close and prepare comple- tion reports, SDV tracking systems should record the results o f thematic interaction, and responsible staff should share findings through the appropriate networks and thematic groups when their analysis shows that improvements in project design are possible. The Bank’s human capital and the borrowers’ existing institutional capacity need to be employed to provide task and country teams with t h e relevant social development expertise throughout the project cycle. 9 Social development specialists should be engaged early in the project design phase and as required thereafter. 9 Project teams need to make use o f and strengthen institutions within the project area and those already serving it to some degree, especially those national organizations that have training institutions o f their own. 9 National and local institutions should remain engaged long enough to ensure that local groups can perform all o f their critical functions without further assistance. Country teams need to have the capacity to identify critically needed social knowledge and to facilitate i t s flow. P Procedures, accompanied by adequate resources, need to b e put in place to ensure that country and task teams have access to the social information they require. 9 Country and task teams should be assigned clear responsibility for ensuring that social assessments are carried out at the right level (project, country, sector, or regional) where social information can be most usefully aggregated to address project issues. 9 Social development staff within the country team should compile relevant existing social information, such as inventories o f physical cultural heritage, analysis o f potential con- flicts, institutional analysis, and culturally relevant good practice. Technical knowledge o n some social topics (notably h o w to preserve cultural heritage and effectively promote participation) needs to be developed a n d o r better organized. Sector staff need to ensure that stated Bank o r policy priorities receive adequate treatment across Regions and countries, and B a n k strategic planning needs to address current s k i l l s and monitoring and evaluation gaps. 42 9 Country strategies should more consistently incorporate social themes to strengthen the overall institutional framework in the country. 9 Unless the Bank i s prepared to relax existing requirements for social due diligence, i t must maintain a sufficient core o f staff who are adequately trained in the specialized the- matic areas o f social development. But all Bank operational staff also need to be suffi- ciently skilled to routinely take into account in their work the themes o f gender division o f labor, cultural traditions, and conflict-reducing ethnic interactions. 9 The Bank's social development strategy should give particular attention to monitoring and evaluation. Social development plans stated in Project Concept Documents should be more consistently implemented and monitored. Completion reporting should more sys- tematically deal with appraisal commitments o n social matters, especially those included in the project's socially relevant assessment reports. 43 Notes 1. Thin, Good, and Hodgson 1997, 15. 2. As posted o n the Social Development Department’s intranet homepage, consulted 13 September 2001. 3. The SSP team notes that the definition below i s s t i l l a work in progress. 4. Thin 1998. 5. All o f the documents mention poverty reduction objectives (see Table 1.l), and all but one (the Stra- tegic Compact) speak o f gender. After that, sustainability, partnership, participation, human develop- ment, and indigenous peoples are the most popular categories (mentioned in 7 o f the 9 documents). The Issues Paper was the most comprehensive o f the documents, covering almost all the categories (16 out o f 18), while the Strategc Compact and the MDGs had the least to say about the various areas o f social development (4 and 7 categories, respectively). Some o f the documents contained concrete objectives, (safeguards, and the MDGs). Other documents were less about concrete, measurable objectives, and more about direction and theory-loose objectives (the annual speeches, the WDR, and the Copenhagen Commitments). Still others provided more on how to reach the objectives and theories expounded in the other documents. (The objectives were implicit in these “how-to” statements.) These included the Par- ticipation Sourcebook (how-to), the Task Group Report (recommendations), the Issues paper (strategy), and the Strategic Compact (a plan to improve Bank effectiveness). 6. Germany, Netherlands, Norway, and the United Kingdom. 7. From “Advancing Social Development, a W o r l d Bank contribution to the Social Summit” W o r l d Bank 1992,48-9. 8. This total i s conservative: projects that involve participation have become so common that since 1999 they are n o longer tracked. Also, C D D projects that predate 2000 are not identifiable in the Bank’s databases. A forthcoming OED review o f C D D will identify this portfolio, however. 9. The OED study, Implementation o f Operational Directive 4.20 o n Indigenous Peoples (2003) notes that ‘‘.. .countries adopt different approaches to the protection o f E’ in their countries, not always con- sistent with the approach adopted by OD 4.20.” 10. See discussion on Nancy Birdsall’s findings o n synergies between health, education, and commu- nity organization activities in the study literature review. 11. The 10 safeguard policies are Environmental Assessment (OP 4.01), Natural Habitats (OP 4.04), Pest Management (OP 4.09), Forestry (OP 4.36), Safety o f Dams (OP 4.37), Cultural Property (OPN 11.03), Indigenous Peoples (OD 4.20), Involuntary Resettlement (OD 4.30), Projects in International Waterways (OP 7.50), and Projects in Disputed Areas (OP 7.60). 12. Documents examined include: World Bank Safeguards, 19842002; The Copenhagen Declaration and Program o f Action, 1995; The Participation Sourcebook, 1995; The Social Development Task Group Report, 1997; The Strategc Compact, 1997; The W o r l d Development Report 2000/2001; The Millennium Development Goals (MDGs), 200 1; The Social Development Strategy Issues Paper, 200 1; and Relevant speeches given by (Bank President) James Wolfensohn. These documents were chosen because they represent prominent statements o f the Bank’s position with respect to social develop- ment and/or goal statements made by the wider development community. More > 44 Notes (continued) 13. Though the C A S i s prepared with borrower participation, i t i s not a negotiated document. For more information o n the CAS, see “Country Assistance Strategies” The W o r l d Bank Operational Manual, B P 2.11, January 1995. 14. CASs were not reviewed for mention o f Social Funds, as it i s to be expected that countries with- out Social Funds would not mention them in their CASs. 15. Sometimes two were stated. 16. Each CAS n o w includes an annex that provides separate ratings o f Bank and borrower priorities by sector and subsector. 17. T o explore this in greater depth, OED i s considering a major review o f the Bank’s approach to CDD projects. Lessons and Practices. Number 12. Washington, D.C.: OED, August 1, 2000. Within that document the OED ratings for the most recent 10-year period were reviewed. The projects were rated 12 percent lower than the all-project average in terms o f likely sustainability. 18. Culture Study, 37 (finding). 19. Social Funds, Water Resources, Resettlement, Forestry, Cultural Heritage, Indigenous Peoples, ARDE, C D D in the Sahel. 20. The study was not able to do a systematic comparison due to the lack o f an adequate comparator. 2 1. Social Funds Study: Social Funds: Assessing Effectiveness 2002, 97. “For instance, analysis o f the project status reports for a random sample o f 25 social fund projects found that, for most, safeguards are not rated, not applicable, or l e f t blank. Only 15 (6 percent) ratings were given in the 25 reports. M o r e than half (9 ratings) o f these were for environmental assessment, with two each for indigenous people and involuntary resettlement, and one each for forestry and in- ternational waters. This overall coverage i s comparable to that found in a random sample o f other PTIs (drawn from the same countries), for which 12 (5 percent) o f the possible 250 ratings were given. This does not mean that safeguards are being adequately applied, however.’’ “Reporting on safeguard compliance in project status reports (PSRs) for social fund projects appears similar to that in PSRs for P T I projects, but seems l o w overall, especially since a number o f safeguard issues are likely to be relevant for social fund activities.” Social Funds: Assessing Effectiveness 2002, 19. 22. Water Resources Study, 83. Additional note: The Resettlement Study concludes that the Bank’s record with relocation-as it refers t o the individual or collective movement o f people up and out o f reservoir boundaries-was also uneven but on the whole quite satisfactory across the study’s sample o f projects. 23. Cultural Heritage, ARDE, Water Resources, Resettlement, Forestry. 24. NGO, Participation, Culture, Forestry, Indigenous Peoples, Post Conflict, Water Resources. 25, Post-Conflict Study, xvi (executive summary), for example. 26. Post-Conflict Study, xvi. 27. Gender Study (1995) Precis, “Gender Issues in W o r l d Bank Lending,” n o 79,4. 28. Post-Conflict Study, xiii. More > 45 Notes (continued) 29. Interestingly, when social development staff were asked to name the one topic that should receive more attention from social development specialists, the most frequent response was monitoring and evaluation. Furthermore, when they were asked what topic related to social development should re- ceive more attention from task managers, they gave the same answer. 30. The Post-Conflict Study, 21 (text). 3 1. Culture Study, 10 (text). 32. Culture Study, 37 (finding). 33. Cultural Heritage, Water Resources, NGOs/CBOs, Gender, Post-Conflict, Participation, Forestry, and Indigenous Peoples. 34. For specific examples, see the meta-evaluation section on Country Knowledge. 35. Client stakeholders, including women, are more visible in the preparation o f ESW; stand-alone gender assessments have been rated highly satisfactory; C D D projects are noted for being more sys- tematically built on organized social assessment than projects with a more centralized design, having a visibly broader treatment o f social development issues and tending to monitor progress with respect to these more often than other projects; and the Gender Study found, not surprisingly, that an increase in conducting poverty analysis resulted in an enhanced poverty focus. Gender Study (36 percent pre- to 53 percent post-1993), 80 (finding). 36. QAG’s annual Quality at Entry Assessments ratings for Poverty and Social aspects o f projects have risen to 80 percent satisfactory in QEA5 (FY02) (up from 70 percent in Q E A l (CY98), but down from 86 percent in QEA4 [Jan 2000-June 20011). 37. Gender Study (2002), 5. Additionally, the water sector study found that those projects have not only a narrow treatment o f the behavioral and social structural elements, but provide little evidence that an in-depthknowledge has been acquired. Indeed, only a small percentage o f the analyzed pro- jects include social analysis, participation, institutional analysis, and impact monitoring. Rather, pro- jects documents suggest that social assessment i s deferred t o the implementation stage without spe- cific budget allocation. The findings o f the Quality Assurance Group also show that a holistic integra- tion o f social development concems lags behind the performance o f other quality concems (eco- nomic, financial, and technical), despite the indication that the water sector performs better than oth- ers in i t s treatment o f poverty and social issues.(83) 3 8. Forestry Study. The Bank has recently recognized these weaknesses, however, and has improved the design o f the supervision form so that implementation can be tracked and monitored. 39. Water Resources Study, 82 (lesson). 40. See the meta-evaluation section o n M&E for more information and examples. 41. Resettlement did not. 42. F r o m Forestry Study, xxv (executive summaryirecommendation). 43. See the meta-evaluation Staffing section for a discussion o f several staffing roadblocks. 44. Culture Study (Precis). 45. Post-Conflict Study, xvi. More > 46 Notes (continued) 46. China Case Study, OED Forestry Study, 94. 47. Forestry Study, 45 (text). 48. Water Resources Study, 80 (finding). 49. NGO Study, (Precis). For more information o n this point, see the N G O study, p. xvii. (executive summary). 50. Culture, NGOs, Forestry, Gender, Participation, Water Resources, Post-Conflict Reconstruction, Social Funds, ARDE. 5 1. NGO Study, xiv (executive summary). 52. Post-Conflict Study, 26. 53. OED’s Annual Review of Development Effectiveness (ARDE 2000) reports that the Bank has initi- ated a wide range o f productive partnerships, but it can only realize their potential by being more busi- nesslike and selective about what they are intended to achieve, what risks they entail, and h o w they can yield operating economies for the Bank through sharper delineation o f i t s comparative advantage. 54. N G O Study, 3 1 (conclusion). 55. Culture Study, 39 (finding). 56. NGO Study, 41 (conclusion). 57. Participation, Forestry, Water Resources, Gender, and Resettlement. 58. Forestry Study, 15. 59. Participation Precis, 3. 60. Forestry Study, 16. 61. Participation Precis, 2. 62. Water Resources Study, 83. 63. Social Funds Study, xxvi. 64. Participation Precis, 3. 65. Culture Study, 27. 66. For more on this issue, see the Culture Study, 80. 67. Post-Conflict, Forestry, and Water Resources. 68. Post-Conflict Study, 21. 69. A borrower survey envisioned in the D e s i g n Paper was dropped in order to avoid survey fatigue (the teams working on the S D V Regional strategies will be surveying borrowing country project part- ners). Additionally, Operational staff are better placed to identify which borrower agency i s the proper interlocutor with regard to social development. 70. Some have argued that these discrete units have been created as the Bank’s response to various UN conferences dealing with social development topics. 7 1. Social Team web site, Consulted 18 September 2001. More> 47 Notes (continued) 72.The Social Analysis Sourcebook, http://www.worldbank.org/socialanalysissourcebook/ socialassess3 .htm 73. “The Geneva Special Session and Beyond,” The Social Development Review, Sept. 2000, v 4, n 3, p 6. g LCR 2 150 ECA z 0 EAP 50 AF R 0 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 Note: Actual numbers were not available for the participation portfolio for the years 1998-2001. The numbers for this portfolio were projected for 1998-2001 using the rate of growth in the portfolio between 1994 and 1998. Table S1: The Top Three Sectors for each Social Development Portfolio Ag ricuIture I 1 1 1 1 1 3 3 Economic Policy 3 Education 3 3 3 2 Energy/Electricity 2 2 Environment 3 2 Finance Health, Nutrition, Pop. 2 2 2 3 3 Industry Mining, Oil, & Gas Multi-sector 2 Private Sector Dev. Public Sector 3 Social Protection 3 1 Social Development Telecommunications Transportation 2 3 1 Urban 1 3 Water Attachment 50 Figure S2: The Outcome Ratings of Social Development Portfolios Show an Improving Trend 5 8o 3 70 2 rc 60 v) 50 40 5 U 30 g 20 10 0 1992 1994 1996 1998 2000 2002 Completion Year Table S2: OED Ratings o f Completed Projects with Highest Overlap of Two Social Development Themes (In descending order of satisfactory outcomes) No. Rafed % Satisfactory % Likely % Substantial Projects Outcomes Sustainability lnsfitutional Dev. lmpact Participation & Social Funds 19 100 50 39 NGOs & Social Funds 39 92 45 45 Gender & Social Funds 19 89 53 44 Indigenous Peoples & Participation 80 81 70 55 Gender & Participation 122 80 65 47 Gender & NGOs 145 79 61 40 NGOs & Participation 175 79 63 43 Resettlement & Participation 28 70 63 44 Culture & Participation 3 67 67 33 Table S3: Comparing Outcome Ratings: Projects Dealing with Three Social Development Themes No. No. Projects % Sat. Projects Rated NGOs, Participation, Social Funds 28 19 100 Gender, Participation, Social Funds 24 16 100 NGOs, Social Funds, Indigenous Peoples 17 11 100 Gender, NGOs, Conflict 32 11 100 NGOs, Participation, Conflict 28 14 93 Gender, NGOs, Social Funds 48 19 89 Gender, Indigenous Peoples, NGOs 118 51 84 Gender, Indigenous Peoples, Participation 81 44 84 Gender, NGOs, Participation 302 110 81 Gender, NGOs, Resettlement 48 5 80 Indigenous Peoples, NGOs, Participation 109 64 78 Resettlement, participation, Gender 32 4 75 NGOs, Participation, Resettlement 63 16 63 51 Attachment Table S4: Comparing Outcome Ratings: Projects Dealing with Four Social Development Themes No. #Projects % Projects Rated Satisfactory NGOs, Participation, Social Funds, Conflict 5 4 100 NGOs, Participation, Social Funds, Gender 24 16 100 NGOs, Participation, Social Funds, Indigenous 8 5 100 Peoples Gender, Participation, Social Funds, Conflict 4 4 100 Gender, Participation, Social Funds, Indigenous 8 6 100 Peoples Gender, NGOs, Social Funds, Conflict 5 4 100 Gender, NGOs, Social Funds, Indigenous Peoples 12 6 100 Gender, NGOs, Participation, Conflict 17 9 100 Indigenous Peoples, NGOs, Participation, Gender 75 41 85 Indigenous Peoples, NGOs, Participation, Conflict 6 4 80 Gender, NGOs, Participation, Resettlement 28 4 75 Indigenous Peoples, NGOs, Participation, 19 8 63 Resettlement 53 Annex A A n n e x A. Methodological N o t e 1. This study i s essentially a meta-evaluation that draws on recent and ongoing OED evalua- tions that address specific components o f social development or stakeholder social processes. I t nevertheless supplements this process with the analysis o f data from several other sources (triangu- lation). Key aspects o f the research process are described in this annex. A fidler and more detailed description o f the study methodology i s contained in a Design Paper that i s available o n request. 2. The study addresses the following broad evaluative questions: 0 What factors identified in completed and ongoing s e l f and independent evaluations en- hance and constrain the general pursuit o f the Bank’s social development objectives? 0 Have some methods o f introducing social concerns into Bank work been more effective than others? What has been the experience o f other donors in terms o f effectiveness? 0 To what degree have social development issues been incorporated into Bank lending activi- ties and policy formulation? What constitutes the Bank’s social development portfolio? 0 T o what degree has attention to social development issues influenced Bank activities in the field throughout the project cycle? I s there a rhetoricheality gap? 0 What light does (already-completed and ongoing) s e l f and independent evaluation shed o n the impact o f social dimensions? T o what extent does social development show up within important Bank macro processes such as the CAS? H o w do social development in- terventions compare with other Bank interventions? Design Paper and Consultations 3. The study team prepared a detailed Design Paper which i s also available o n the study web site.’ Initial outside consultations carried out with the Utstein group,2 Switzerland, Sweden, and the UN informed the Design Paper process. Input from within the Bank and other develop- ment partners was used to i n f o r m the development o f survey instruments. Study team members approached key internal stakeholders inviting suggestions for questions, themes and topics that should be explored within the context o f the study. Responses were received f r o m the Country Directors network, the Social Development Department, and the Safeguards team. Three prelimi- nary instruments were put together to survey social development staff, operational staff and country directors. These f i r s t drafts o f these instruments were shared with selected evaluation partners, development institutes, and members o f the Social Development Advisors network. Subsequently, Bank staff f r o m all around the organization are being given the chance to ensure that the questions are relevant and sufficiently comprehensive. 4. Web I Q decision-making software, a new technology for OED evaluations, was used to gather staff feedback. WebIQ interface offers a unique opportunity to see what informed colleagues have to say about the surveys while there i s s t i l l time to make them better. Clean copies o f the three preliminary surveys were put into the WebIQ format, online. Staff were notified o f the session and could begin by loggmg o n at their convenience and reviewing the proposed survey questions. Given a set period o f time in which to respond (in this case, one week), they were free to comment o n individual questions, and/or respond to comments already posted at their leisure. The informa- tion and suggestions gathered from the reviewers were taken into account by the study team, and the questionnaires were reworked accordingly. Subsequently, each pilot participant was sent a Clo- sure Report, summarizing the findings o f the session, and the new version o f the questionnaire. Even when the suggestions proved to be contradictory, contributors appreciated the chance to be Annex A 54 heard, and were able to see why their particular suggestion m a y not have been taken o n board by the study team. A f t e r this process, the surveys went out to the respondents Surveys 5. Staff f r o m the B a n k were surveyed with the three instruments described above. These are included in A n n e x F. Special questions that reflect the reality o f staff sitting in different places in the bureaucracy were formulated. The Goal M a t r i x 6. The Study Design document developed benchmarks for the review o f social development in the Bank, given the absence o f organizational goals and strategies. OED evaluations normally compare the Bank’s actions with i t s objectives, but social development lacks clear objectives o r a formal strategy. T o compensate for this, a goal matrix that synthesizes B a n k objectives for social development was developed (see B o x Al). Internal and external documents and statements rele- vant to the W o r l d Bank’s w o r k with social development were reviewed. These included: the So- cial Development Strategy Issues Paper, 2001; W o r l d B a n k Safeguards, 1984-2002; the W o r l d Development Report 2000/200 1; the Social Development Task Group Report, 1997; the Strategic Compact, 1997; the Participation Sourcebook, 1995; The M i l l e n n i u m Development Goals (MDGs), 2001; the Copenhagen Declaration and Program o f Action, 1995; and relevant speeches given by (Bank President) James Wolfensohn. The goal matrix summarizes the social develop- ment objectives contained in the previously mentioned documents. I t assesses their priority (based o n frequency o f appearance), their clarity, and their applicability t o social development (as opposed t o human development). The goal matrix facilitates the assessment o f relevance ( o f the p o l i c y yardsticus used t o measure Bank progress to date) in the various research activities. Box A l : How the Objectives M a t r i x Was M a d e First, the Bank documents were examined and the objectives that emerged were placed in the matrix. The objectives were found to cluster around several subjects-poverty reduction, gender, sustainability, part- nership, participation, human development, indigenous peoples, conflict prevention and reconstruction, NGOs/civil society, empowerment, culture, security, inclusion, resettlement, governance, social capital, livelihoods, and community-driven development. W h i l e there i s no consensus within the development community regarding which subjects should be included in a definition o f social development and which should not, these subjects cover the middle o f the spectrum. They are listed in order o f their importance as judged by the number o f documents mentioning objectives in these categories. Next, the Bank-adopted international documents were examined and the objectives placed in the matrix. Be- cause parts o f those documents relate to the Bank system and others do not, the identification o f relevant ob- jectives within these statements was limited to the categories derived from the examination o f the Bank documents to ensure that the matrix o f Bank objectives did indeed only include Bank social development objectives. For the most part, the objectives in the two sets o f documents were parallel, however, indicating that the Bank objectives are in fact hghly relevant to internationally accepted social development objectives. Literature Review 7. The literature review (available upon request) was prepared in parallel with the Design Pa- per. It i s not intended to be exhaustive or to cover the m a n y thematic areas o f social development in depth. Rather, the intention i s for the exercise to give the study an overview o f what these areas are, and, more importantly, a broad understanding o f social development as a field in and o f itself. The 55 Annex A literature review draws from books, journal articles, and bilateral and multilateral aid agency publi- cations, evaluations, and studies. It focuses o n key policy issues, social development tools and indi- cators, and current debates and (anticipated) cutting-edge issues in social development. “Portfolio” Review 8. The study did not undertake a traditional portfolio review because very few stand-alone social development projects exist. Nevertheless, several completed OED evaluations identified clusters o f projects that capitalized o n social processes (participation, empowerment, NGOs, etc.) or addressed specially defined groups (indigenous peoples, gender). T o the extent practicable, these “portfolios” o f Bank-supported projects with social development processes or targets were updated so that they take recent experience into account. “Pipeline” projects that were included in earlier OED evaluations were dropped from the analysis in this study, however; if they had not been ap- proved in the intervening years, there was little reason to expect that to happen in the near future. Social aspects o f projects were identified through documentary analysis o f a sample o f completed, ongoing, and proposed projects. Additional sources o f data include Initial Executive Project Sum- maries, Project Concept Documents, and Staff Appraisal Reports. Follow-up research to ascertain the effectiveness o f social activities began with the OED database o f I C R and P A R findings. The study also reviewed the role o f social development in Country Assistance Strategies, and drew o n Quality Assurance Group assessments o f quality at entry and Economic and Sector Work. Meta-Evaluation 9. The t e r m meta-evaluation refers to any type o f evaluation designed to aggregate findings from a series o f evaluations. A new analysis o f a series o f completed evaluations often can iden- tify more definitive evidence ( o f effectiveness or ineffectiveness) than can be obtained by under- taking a stand-alone evaluation, that is, one that does not build o n what has already been done. Meta-evaluations can put project and program achievements in perspective and provide summa- tive findings about impact that are missed unless the results o f individual evaluations, which on the surface m a y appear contradictory, are examined systematically as a single b o d y o f evidence. 10. The task facing the meta-evaluative component o f the social development study was to find ways in which the findings o f many evaluations could be compared and contrasted, while letting the various completed OED evaluations speak for themselves as much as possible. Using the goal matrix as the study analytic framework could obscure some clear and consistent mes- sages that come out o f this important body o f work and constrain an in-depth analysis that has the possibility o f taking many interesting and important factors into account. Instead, the analytic framework below (see Table A l ) has been tailored to gather and organize the less quantitative information from the O E D evaluations. Pilot testing showed that i t seems to highlight the most operationally relevant conclusions, findings, and recommendations o f the 15 relevant evaluations. 11. The information gathered from the various OED evaluations fell into several categories o f findings. Below, the findings are listed and those evaluations that agreed with a finding are noted with an X in the table. Annex A 56 Table A l : The meta-evaluation findings on the degree of agreement among OED evaluations of social development-related topics c 5 C % al s 0 c n 2 4 C n 8 - u al Q - I -- - M&E i s lacking X X X X X X 30 -- - - - - Increased stakeholder par- X X X Social 50 Funds ticipation has improved wr outcomes in areas related t o social development, but a better understanding o f participatory techniques could lead to more consis- tent outcomes and impact. There i s considerable r o o m for qualit! impro\.ement I The actit'e participation of X X X X X X X X 80 NGOs in B a n k projects i s increasing, but actions vary greatly in depth and qual- - ---- - - - - - X X 100 ~ X X X X X X X X tory and extensive imple- mentation experience in several social development areas, and has been improv- ing through the years but there i s highly variable treatment o f social issues within countries and across Regions. T h e human resources in X X X X X X X X X 90 the B a n k for social devel- opment are n o t adequate for the number and scope o f the mandates received. - - - - __ - - - - __ X X X X X 50 though policies have been made, a lack o f funding hampers their effectiveness. - ---- - - - - KM: Awareness of the X X X X X X X X 80 socio-political realities within borrowing countries i s crucial to effective SD assistance. The B a n k needs to improve its project con- text knowledge. - --- - - - - KM: B a n k staff d o n o t X X X X X X X 70 have easy access to the technical knowledge they require (dealing with social topics). - 57 Annex A s 0 - Poverty reduction knowl- X X X X edge: There i s little knowl- edge within the Bank con- cerning the link between project activities and pov- erty reduction. - - - - - - - Treatment o f social issues X X X X X X Social Funds in CASs has improved, however, the quality o f the treatment i s variable. Leaming about peoples' X X X X X X X X 80 actual behavior and opin- ions in a continuous fashion i s essential to ensuring that project objectives are rele- vant to people's needs, and that project implementation takes account o f local reali- ties. Six evaluations under- lined the importance o f social analysis in the areas being studied, noted that it i s too often lacking when it was needed, recommended i t be performed in a more timely manner, and then used. - Bank impact on institu- X X X X 100 tional development has been limited. For lasting and broad impact, it i s im- portant to look at all levels o f institutional develop- ment-national, local, and NGOiCBO-and work to improve the existing sys- tem b y clarifying (or creat- ing where necessary) pro- cedures, roles, and respon- sibilities - - - - - Many social themes are X X X 40 interdependent and need to be addressed from a holistic perspective. Individual attempts to address single issues often suffer, while integrated approaches can create synergistic effects. - - - - Need to improve sustain- X X X 90 ability. Suggestions given. - - - - Annex A 58 .- I ‘CI S C U m v) C .- C k 0 8 ‘CI E ii Partnerships have gener- X ARDE Social 70 ated enhanced results in Funds m a n y B a n k activities, but choosing partners wisely i s key. - - There i s a lack of bor- X X 70 rower ownership of social development activities. Bank-funded activities can o n l y achieve sustainability t o the extent that borrower “owns” a given project. A broad base o f support f r o m stakeholders i s k e y to suc- cessful outcomes. - B a n k assistance that is X 30 responsive to o r framed within the country’s agenda for that issue has been proven t o b e a very important factor for suc- cessful results. Some issues m a y b e better addressed at a country level rather than at a project level. - There has been p o l i c y suc- X X X X X 50 cess, but too often, a lack of clear policy hinders the Bank’s efforts in social development. - - - Within the project context, X X X X ARDE Social 60 attention to social issues Funds and safeguards i s uneven. M a n y evaluations identified an improving trend, h o w - ever Foculpoints: critically X X X X X 50 needed (social) information needs t o b e channeled through a stand-alone unit, in some cases with the a i m o f eventual mainstreaming, but in others permanent 59 .- n v) 0 3 i? .cI u) 4- E z U v) E 0 IA $ P - (combines two above) X X X X Social 70 There has been policy suc- Funds cess, but too often, a lack o f clear policy hinders the Bank’s efforts in social development. An improvedpolicy envi- ronment leads to more socially sustainable and “pro-poor’’ projects. Social capital i s discussed X X 40 in only a few o f the evalua- tions. Those evaluations that did mention i t stressed i t s importance and noted that the Bank was not ade- quately pursuing measures effective in building it. Designing the Framework 12. The process that was used to develop the framework i s straightforward. First, the text of six o f the evaluations was reviewed for conclusions relating to social development generally. These were combined into one master l i s t . Even this l i s t included findings pertaining to a very broad range o f issues. The challenge facing the study team was to resist the temptation to impose categories on the various findings based o n collective preconceptions. Consequently, the decision was made to use emic categories, that is, those that were suggested by the findings themselves. T o the degree possible, the preliminary framework presented below used the wording and taxon- omy o f the documents being drawn upon. The Use o f Emic versus Etic Categories in the Meta-Evaluation 13. Although emic and etic categories are most commonly used in anthropology, and re- search that tries to capture the reality o f studied groups, it i s a powerfbl tool for use in meta- evaluation, keeping the research team faithful to the words and concepts used by the authors of the various OED evaluations which constitute the primary data for this undertaking. Emic catego- rization makes sense o f things according to the way participants classify their o w n world. Dis- covering, recording, and analyzing emic categories i s the basis o f m u c h recent research in a num- ber o f social science disciplines. Etic categories classify research findings according to some ex- ternal system o f analysis, often brought in by someone who i s an outsider t o the process or proc- esses being studied. The use o f etic categorization has been discredited in that i t assumes that the current researcher i s better able to see an objective reality than the participants in a process and those researchers who have preceded him. While it i s often not possible to find usable emic tools for analysis, where i t can b e done it often yield better results. N o t to mention that it i s more re- spectful o f the work which has already been completed. Annex A 60 14. Comparisons are greatly facilitated by a systematic consideration o f the factors that are integral to the p r i m a r y data. In a meta-evaluation, the findings o f the various evaluations are the primary data. Grounded theoq? i s a general method for developing theory that i s grounded in data systematically gathered and analyzed. The method i s applicable t o data collected f r o m inter- views, observation, or documents, or f r o m a combination o f these sources. The grounded theory i s a function o f the study’s research questions. The information i s analyzed using open coding. Open coding i s a n analytic device that identifies patterns in the data using emic categories. The development o f open codes allows the study to summarize and synthesize the various evaluation findings, without foreclosing the richness o f the data in the rush to make sense out o f it. There- fore, the process i s o n l y valid to the degree that the codes fit the data. Forcing the data to fit the codes w o u l d be a serious analytic error. 15. In the framework below, bullet points w o u l d b e the emic categories, in that they come f r o m the findings o f the various evaluations. As a point o f departure, the b o l d heads in the shaded bars represent a summarized taxonomy f r o m the source documents. But they could also be de- r i v e d by a process Glaser called focused ~ o d i n gF .o~ r the final report, broader analytic categories will b e developed f r o m the coded findings (that i s those placed in the analytic framework). In focused coding, the preliminary categories (codes) that were developed in the initial analysis are applied to larger amounts o f data-in this case, the full universe o f OED’s socially relevant evaluations described in a separate annex. The coded data in the analytic framework was com- pared with data f r o m self-evaluation and data f r o m other non-Bank studies. T h i s process i s some- times referred to as the constant comparative method, whereby analyzed and coded data i s con- stantly confronted with new data for verification purposes. The resulting broader categories are the result o f the w a y coded data clusters together. The strength o f the approach i s that it provides a strongly supported analytic insight rather than a simple summary o f discrete works. This i s an iterative process: m a n y versions were required before i t was possible t o present the vast body o f information in a w a y that i s true t o the source documents w h i l e responsive to the study’s evalua- tive questions. Each n e w version sheds light o n the data, however. A study o f the material that does n o t yet fit the framework (labeled in the current version as “Extras,”) leads inexorably to better framework, w h i c h deals more simply and transparently with the primary data. 16. The columns, o f course, represent the different OED evaluations. It i s easy to imagine more categories than those presented here. The meta-evaluation (available upon request) i s based o n a more evolved version o f this framework. As findings required additional categories during the meta-analysis, they were created. Conversely, however, categories are n o t created just to unleash a search f o r data w h i c h i s n o t there, o r w h i c h (in i t s o w n terms) already f i t s elsewhere. 61 Annex A CAS Review 17. The study analyzed the manner in which social development i s addressed in the Country Assistance Strategies (CASs). This document, in which the Bank delineates i t s strategy for a country, based o n a Bank assessment o f country priorities and input from the borrower, indicates the level and composition o f assistance t o b e provided based o n the strategy and the country’s portfolio performance. 18. The study team examined the most recent CASs o f 109 countries for information o n the Bank and borrower’s assessment o f social development needs. First, the documents were ana- lyzed to see which of eight social development areas the Bank works in were mentioned in the text o f the CAS, to see if in fact those issues were coming up in the documents. The categories searched for include: gender, indigenous peoples, participation, community-driven development, NGOs/civil society, resettlement, culture, and conflict. In addition to these topics, the analysis Annex A 62 also noted h o w often institutional reform (transparency/accountability and decentralization), anti corruption, and public services for the poor were mentioned (see Table A2). T a b l e A2: Percent of CASs for E a c h Region that Mentioned Various Social Development Topics (82%) (73%) (63%) (13%) (0%) (16%) (5%) (3%) 97 2,5 AFR 38 31 28 24 5 0 6 2 1 (73%) (82%) (45%) (18%) (9%) (9%) (18%) (0%) 2.6 EAP 11 28 8 9 5 2 1 1 2 0 (73%) (54%) (46%) (15%) (12%) (19%) (15%) (0%) 61 2,,3 ECA 26 19 14 12 4 3 5 4 0 (67%) (57%) (48%) (14%) (52%) (8%) (9%) (0%) 53 2.5 LCR 21 10 3 11 1 2 0 14 12 (63%) (75%) (63%) (38%) (0%) (13%) (50%) (0%) 24 3.0 MNA 8 5 3 1 4 0 5 6 0 (80%) (100%) (60%) (80%) (40%) (20%) (0%) (40%) 21 4.2 SAR 5 3 4 2 1 0 2 4 5 Total 109 81 74 59 21 17 15 14 3 284 2.6 %Of 100% 74% 67% 55% 19% 16% 14% 13% 3% total Note: The number below the percent is the actual number of CASs in which a topic was mentioned. The highest occurrences for each topic are in bold type, while the second highest occurrences are in italicized type. *Number of CASs for that Region to mentions of topics 19. T o check if Bank priorities resonate with borrower priorities and with what the Bank can operationally manage given i t s structure, the analysis included an examination o f the “Summary of Development Priorities” section o f each CAS. In this section, the Bank lays out and rates the priorities it has determined for i t s work in that country. The borrower rating o f these priorities i s also noted in this section. The team examined three parts o f the summary: public sector, gender, and social development. 20. Whether or not a given country’s portfolio reflects what was said about social develop- ment in the CAS i s an issue covered by the scatterplot in Chapter 3. The data behind the plot i s the following: 63 Annex A Table A3: Social Development Themes in CASs and Regional Portfolios: Percentage of Occurrence Social Devel- LATIN opment EUROPE & MIDDLE EAST AFRICA EAST ASIA AMERICA & SOUTH ASIA Theme >ENTRAL ASIA & N.AFRICA CARIBBEAN CAS Portfolio CAS Portfolio CAS Portfolio CAS Portfolio CAS Portfolio CAS Portfolio Community-Drive Development 13 43 18 27 15 35 14 42 38 39 80 47 Conflict 16 9 9 9 19 10 8 9 13 10 20 3 Culture 5 2.5 18 4 15 4 9 2.5 5 4 0 2 Gender 73 40 82 34 54 20 57 34.5 75 38 100 58 Indigenous Peopl 0 1 9 12 12 2 52 32 0 4 40 21 NGOs 63 55 45 51 46 43 48 57 63 57 60 74 Participation 82 55 73 61 73 39 67 53 63 43 80 76 Resettlement 3 3 0 37 0 5 0 7 0 5 40 25 Desk Study o f Highly Satisfactory Projects 2 1. O f those that overlapped four themes, only ten projects were rated highly satisfactory, in outcome, and/or highly likely in sustainability (four o f them were both). These were: Peru Rural Roads and Rehabilitation Project (Loan 3962-PE) Morocco Emergency Drought Project (Loan 3935-0, 3935-1 MOR) Bosnia-Herzegovina Local Initiatives ("002-0 BOS) India Maharashtra Earthquake Recovery (Credit 2594-IN) Kazakhstan Technical Assistance (CPL-36420) Indonesia Village Infrastructure Project (Loan 3888-IN.) Indonesia Second Village Infrastructure Project (Loan 41 00-NIN) China Iodine Deficiency Disorder (Project 39140; 3914A; 27560) K y r g y z Republic Health Project (IDA-28600) Chile CL Secondary Education (CPL-38830; SCL-38836) 22. The study team examined these projects to determine what success factors they had in common, if any (see Annex D). Project Performance Audit Reports (PPARs), if available, and Implementation Completion Reports (ICRs) for the projects were examined for factors and find- ings pertinent to social development success. The conclusions were developed by an iterative process similar to that described for the analytic framework. A separate analytic framework was generated early in the process (derived from a case study). This was then repeatedly confronted with new data from subsequent cases-the constant comparative method described above. Suc- cess was attributed to several common project characteristics which were also emic categories. These characteristics were noted and then separated into two groups: social development related factors, (e.g., stakeholders, especially beneficiaries, participated in project design) and general factors o f success (e.g., performed consistent monitoring and took corrective action when neces- sary). The l i s t o f general factors o f success contained in this volume i s incomplete, as there were many more that were not at all related to social development, yet s t i l l important to project suc- cess. I t will be recalled that the focus o f the exercise was o n the social development factors that l e d to success. Annex A 64 Notes (continued) Notes 1. http://www.worldbank.org/oedJsdstudy/ 2. Germany, Netherlands, Norway, and the U n i t e d Kingdom. 3. See Barney Glaser and Anselm Strauss’ social science classic, The Discovery of Grounded Theory, 1967. 4. Barney Glaser, The Basics of Grounded Theory Analysis, 1992. 65 Annex B Annex B. Definitions o f Social Development The Asian Development Bank (ADB) believes that social development “promotes people’s well-being alongside a dynamic process o f economic development. People must have equitable access to the economic and social benefits o f development. I t supports development interventions that build upon culture, social structures and institutions. Local initiatives and people’s needs are merged through a participatory approach. This helps improve the lives o f the poor and other vul- nerable sectors.” Source: ADB website: oesd.adb.org/socdprimer.asp. The Department for InternationalDevelopment (DFID) definition o f social development i s all encompassing: “Social development i s a growing discipline which aims to refine development work by making it more accountable, more equitable, and more appropriate. Throughout, the focus i s o n the needs and interests o f the poorest people. Social development brings together various strands that in recent years have contributed to development thinking and practice. These include: 0 an understanding o f h o w gender issues influence all aspects o f development practice and policy; 0 a realization o f the need to give poorer people a voice and encourage their participation; 0 an awareness o f the social and cultural factors affecting a l l efforts to improve people’s lives; and 0 an emphasis o n people’s rights. . ..Together with formal economics and formal political or institutional analysis, it covers every aspect o f human relations. I t i s central to every type o f development cooperation-as much with energy or transport as with health or education, and as much with macro-economic policy as with micro-enterprise. A social development approach aims to secure more equitable relations be- tween people-within and between households and communities. I t i s because the social devel- opment approach i s concerned with relationships that issues including social support networks, participation, social exclusion, gender and poverty can all be integrated within a single frame- work. In this case it can be seen that the specific outcomes expected define social development. DFID’s approach strives for increased proportions o f men and women w h o (i) are active partici- pants in social groups, communities, and structures o f governance that affect their lives; (ii) have reduced their vulnerability to natural and man-made shocks and disasters; and ( iii) have achieved sufficient livelihood security to be able to exercise choice in relation to their priorities and needs as they define them.” Source: DFID Social Development Factsheet. Can be viewed at: www.d$d.gov. uklindex. html The United Nations Development Program (UNDP) equates social development with “sustain- able human development” (1990). I t defines this as “enhancing human capabilities for enlarging human choices.” Source: Lincoln C. Chen and Meghnad Desai, “Paths to Social Development: Lessonsfrom Case Studies, ” Development with a H u m a n Face Experiences in Social Achievement and Economic Growth, Santosh Mehrotra and Richard Jolly, eds. (Oxford: Clarendon Press, 1997) 422. Participants o f the World Summit for Social Development, 1995, defined i t as: “The well-being o f individuals and the harmonious functioning o f societies in a context o f sustainable economic growth. Development o n a social level includes the provision o f a decent living standard for all, work and employment without discrimination or exploitation, gender equality, social cohesion, democracy and respect for human rights. The development o f the individual in a peaceful and Annex B 66 creative society and w o r l d also implies the sharing o f moral norms o f decency and responsibil- ity.” Source: www.geneva2000.0rg/index-en. html. Social development experts, Lincoln Chen and Megnand Desai believe that “Every level o f society i s engaged-the individual, the family, the community, and the nation state. Social devel- opment i s holistic, encompassing physical and psychosocial well-being, a healthy polity, and harmonious social relations-not simply the abundance o f material goods.” Source: Lincoln C. Chen and Meghnad Desai, “Paths to Social Development: Lessonsfrom Case Studies, ” Development with a Human Face Experiences in Social Achievement and Economic Growth, Santosh Mehrotra and Richard Jolly, eds. (Oxford: Clarendon Press, 1997) 422. Ignacio Garaycochea from International Cooperation for Development (ICD), Peru, believes social development i s “a process in which multi-dimensional activities, particular and general, take place in both quantitative and qualitative terms.” Source: Ignacio Garaycochea, “The methodology of social development, ” Evaluating Social Development Projects, Ed. David Marsden. (Oxford: Oxfam, 1990) 67. Seep 18for more information on the history o f the debate. David Marsden, a professor at the Center for Development Studies, University College, Swan- sea, Wales, states that social development “addresses problems o f access to resources, the provi- sion o f basic needs, the distribution o f those resources, the room to maneuver in straitened cir- cumstances, and the effectiveness o f the use o f those scarce resources. I t examines the different value premises o n which policy decisions are made, and the contexts in which they are elabo- rated. It takes as i t s starting point the willingness o f governments to intervene to direct develop- ment efforts, and to contribute resources to the satisfaction o f basic needs and the redistribution of assets on a more egalitarian basis. But it also recognizes the inability o f many governments to intervene effectively, and the rising importance o f what are termed non-governmental organiza- tions (NGOs) in the provision o f resources, to supplement those o f governments.” Source: Ignacio Garaycochea, “The meaning of social development evaluation: Thematic paper, ” Evaluating Social Develop- ment Projects, Marsden, David and Oakley, Peter, eds. (Oxford: Oxfam, 1990) 67. Gerard Rolland, o f the Centre Lebret, a research-action organization based in Paris, claims that social development “should enable each society to bring about the dynamics o f endogenous de- velopment and development from below.” Source: Gerard Rolland, “Social Development as a Vision of Societies, ” Development 2000, 43.2; 94. Paul Streeten, Professor Emeritus o f Economics at Boston University, describes it as a three- dimensional process: social service and social transfers; economic access and productive returns; and social integration. Neil Thin, Professor, University o f Edinburgh, School o f Political and Social Studies, defines the t e r m for the paper “Characteristics o f DFID-Funded Projects” as “any aspects o f Department for International Development’s (DFID’s) work which seek to improve social relations (structures, sys- tems, patterned relations among individuals and groups and through networks and organizations), andor social institutions (attitudes, values, informal networks, patterned behavior, identity)” H e notes difficulties in defining the term, stating that a “challenge for this review has been that o f iden- tifylng what i s understood as ‘social development’, a term which was barely used and not defined in the original project Concept Paper.” Source: Neil Thin, “Characteristics of DFID-Funded Projects. SD ” SCOPE Paper no. 12, DFID SD SCOPE Project. Bath: Centerfor Development Studies, University of Bath, 2000. 67 Annex C A n n e x C. W o r l d Bank Operational Policies Relevant to Social Development 1. OMS 2.20 Project Appraisal (1984) 2. OP 2.30 Development Cooperation and ConJict (2001) 3. OD 4.15 Poverty Reduction (199 1) 4. OD 4.20 Indigenous Peoples (199 1) 5. OD 8.60 Adjustment Lending Policy (1992) 6 . OP 8.50 Emergency Recovery Assistance (1995) 7. OP 4.0 1 Environmental Assessment (1999) 8 . OP 4.20 The Gender Dimension ofDevelopment (1999) 9. OPN 11.03 Cultural Property (1999) 10. OP 4.02 Environmental Action Plans (2000) 11. OP 4.07 Water Resources Management (2000) 12. GP 14.70 Involving NGOs in Bank-Supported Activities (2000) 13. OP/BP 4.12 Involuntary Resettlement (200 1) 14. OP 4.36 Forestry (2002) Annex D 68 Annex D. Key Characteristics o f Highly Satisfactory Projects 1. O f those projects that overlapped at least four themes, ten were rated highly satisfactory in outcome. The study team examined these ten projects (a desk review) to determine what success factors they might have in common.’ Project Performance Assessment Reports (PPARs), if avail- able, and Implementation Completion Reports (ICRs) for the projects were examined for factors and findings pertinent to social development success. 2. Within the referenced documents, success was attributed to several common project charac- teristics. These characteristics were noted and then separated into two groups: social develop- ment-related factors (such as “stakeholders, especially beneficiaries, participated in project de- sign”) and general factors o f success (such as “performed consistent monitoring and took correc- tive action when necessary”). As anticipated, these mirror findings o f the meta-evaluation quite closely. The l i s t o f general factors o f success i s incomplete; it has been edited down for brevity and to omit those that were not at a l l related to the topic under study (Box F1 l i s t s the social de- velopment-related characteristics o f success, along with the number o f projects in the set o f ten that noted that characteristic as important). 3. What emerges from this exercise i s that these highly successful projects agreed o n several things. First, that success stems from quality participation of a l l stakeholders. Quality participa- tion in the context o f these highly satisfactory projects means that the process i s inclusive, cultur- ally appropriate, and sustained. In the Peru Rural Roads and Rehabilitation Project, community leaders, the heads of NGOs, and community members were all involved in project process. Strong publicity efforts were carried out to reach community members and encourage their active participation, especially that o f women and others who might hesitate to express themselves. Par- ticipation o f various social groups helped improve structures o f accountability, encourage greater dialogue, and contribute to good governance at the local level, as in the India Maharashtra Earth- quake Recovery Project. The I C R for that project points out that women’s organizations are able to play an important and highly influential role in projects that involve intensive social interac- tion, and should be inducted into formal administrative structures for developmental programs to aid communication and foster participation. T o reduce the risk that they might try to undermine the project, the Kyrgyz Republic Health Project brought in important stakeholders with the abil- ity to cause potential harm, such as health facility managers, physicians, and other health person- nel, early o n in the process. 4. Another important factor was giving the participants the responsibility for structuring their involvement in the project, thus increasing the likelihood o f success and sustainability. The M o - rocco Emergency Drought Project ICR notes that the sustainability o f locally operated, commu- nity-based systems i s greatly enhanced i f communities are involved from the beginning in plan- ning their water supply system according to their needs and their willingness to pay. Likewise, the success o f the Peru Rural Roads and Rehabilitation Project was partially attributed to the fact that participants were in charge o f structuring their involvement in the project. 5. Especially important is that stakeholders-benejkiaries in particular-participate in project design (nine out o f the ten projects noted this) while their local needs and traditions were also in- corporated into the design. For example, in the Morocco Emergency Drought Project, the sustain- ability o f locally operated, community based water systems i s greatly enhanced if communities are involved in planning from the beginning, helping to design the project according to their needs and their willingness to pay.* The Peru Project documents note that project design was in tune with the 69 Annex D local ethos. I t grew from what already existed (traditions, practices, and groups), which was a factor that enhanced the likelihood o f community support and project sustainability. In the Bosnia- Herzegovina Local Initiatives Project, demand-driven and well-tailored technical assistance largely contributed to highly satisfactory performance o f partner organizations, thus contributing to the success o f the project. In addition, the Second Indonesia Village Infrastructure Project ICR notes that task managers should not simply look up “best practice,” but analyze local needs and find local solutions with the local people. On a similar note, an evaluation o f the Indonesia Village Infi-astruc- ture Project found that working directly w i t h village people rather than through local government produced sound, sustainable results and i s an approach worthy o f replication. Box F1: Characteristics of the 10 Highly Satisfactory Project Desk Study Cases S D Characteristicsfor Success General Characteristicsfor Success b Stakeholders, especially beneficiaries, b Planned for risks. participated in project design to help clearly b Maintained staff continuity. identify desired outcomes. (9110) b Created local consulting capacity. .Had a project team with relevant country b Enabled self-selection o f beneficiaries. (political and social), sector, and project experience. (8) b Carried out the project in conjunction with another donor agency. bProvided integrated attention t o social development themes. ( 7 ) b Created broad support and ownership. bMade participation inclusive and brought in b Avoided unnecessary complexity. participants early on. (6) b Performed consistent monitoring and took b Analyzed socially relevant aspects o f the corrective action when necessary. project. 1. Risks 2. Community participation 3. bProject preparation used lessons-past projects, NGOs. (6) country dialogue, the p i l o t project, Bank-wide b Viewed participation as a continued process. experience, and direct observations. (5) b A Pilot Project provides an ongoing example t o b Built institutional capacity at a l l levels, use for continued testing and honing o f project improved institutional design or procedures. ( 5 ) design and t o guide the planning and execution o f the project. b Involved beneficiaries in project implementation. (4) bA well-structured, decentralized management system, with clear responsibilities, roles, and b Secured community contribution to the project procedures. and i t s future maintenance. (4) b Frequent field visits. bBuilt social capital: the “success breeds success” approach. (4) b Maintained a database o f project information. bEmployed consciousness-raising activities as b Developed technical and operational guidelines appropriate. (4) and manuals for implementation o f project activities. .Had a livelihood focus. (3) ,Develops at a reasonable speed (consistent with .Drew o n community traditions. (2) the absorptive capacity o f the borrower). .Built o n local NGO capacity. (2) b Institutional development at all levels. bMade sure participation was culturally b The extensive use o f small local private firms. appropriate. (2) b On-the-job training opportunities. .Demonstrated the broad effects o f the project. (2) b Labor-based construction methods. 6. Project team composition was critical to the success o f the projects, in eight o f the ten cases. In- depth knowledge o f the political and social context and continuity o f the team members were cru- cial. In the India Maharashtra Earthquake Recovery Project, the Bank fielded teams with diverse Annex D 70 expertise for project supervision, including a sociologist, community participation experts, commu- nity housing experts, financial analysts, seismic engineers, an earthquake reconstruction specialist, a disaster management specialist, a small-scale industry specialist, a c i v i l engineer, a procurement specialist, and a disbursement specialist. In addition to team composition, team continuity was es- pecially important to success, according to project reports. So was time o n the ground. In the Chile Secondary Education Project, the continuity in Bank task management ensured consistency and a high degree o f f o l l o w up o n the recommendations o f supervision missions. It also helped to con- tinue building the already good working relationship with the borrower. 7. In project design, seven out o f these ten successful projects attributed their success toprovid- ing integrated attention to social development themes. The fact that the ten projects had four so- cial development components each and were a l l rated highly satisfactory i s in itself suggestive. Integrated approaches, providing they can avoid over-complexity, lead to better implementation and more socially sustainable benefits. In the case o f the (highly satisfactory) Second Indonesia Village Infrastructure project, the components were few and simple, but each focused o n achiev- ing the objectives o f the project. Comprehensive rehabilitation, including social and economic rehabilitation plans, as w e l l as long-term disaster management and planning were critical t o the success o f the India Maharashtra Earthquake Recovery Project. The China Iodine Deficiency Disorder Project’s success hinged o n delivering a social benefit through a technologically simple and very cost-effective intervention at the local level. The K y r g y z Republic Health Project files noted straightforwardly, “Simplistic solutions that only addressed one element o f the health sec- tor were n o t possible, and w o u l d not have produced any results o n the ground.” 8. M o r e than h a l f o f the projects partially attributed their success to the analysis o f socially rele- vant aspects ofprojects. Analysis considered and planned for risks in the long and short terms, mapped out community participation, and established a role for NGOs, for example. A major con- tribution o f the Kazakhstan Technical Assistance Project was t o carry out the technical studies re- quired for preparing subsequent policy-based operations. In the India Maharashtra Earthquake Re- covery Project, the Project Management Unit (PMU) produced extensive analysis o n the project, covering, among other things, community participation, NGOs, the media, and a beneficiary sur- vey. The Morocco Emergency Drought Project employed an expert panel to review project docu- ments and meet regularly with the task team to discuss potential risks and other sectoral issues, based on experiences elsewhere in the Bank. Likewise, the Chile Secondary Education Project drew i t s technical assistance from a p o o l o f worldwide technical experts provided by the Bank. 9. Granting that the study’s desk review o f highly satisfactory projects was merely exploratory, one conclusion that i s nevertheless nearly inescapable i s that reproducing many o f these success factors in n e w projects will require sustained social development support. Notes 1. The ten (highly satisfactoryhighly likely sustainability that address at least four social develop- ment themes) projects given intensive desk review were: Peru Rural Roads and Rehabilitation Project (Loan 3962-PE); Morocco Emergency Drought Project (Loan 3935-0, 3935-1 MOR); Bosnia- Herzegovina Local Initiatives (NOO2-0 BOS); India Maharashtra Earthquake Recovery (Credit 2594- IN); Kazakhstan Technical Assistance (Loan-36420); Indonesia Village Infrastructure Project (Loan 3888-IND); Indonesia Second Village Infrastructure Project (Loan 4100-IND); China Iodine Defi- More > 71 Annex D Notes (continued) ciency Disorder (Project 39140; 3914A; 27560); Kyrgyz Republic Health Project (IDA-28600); Chile Secondary Education (Loan-38830; Loan-38836) 2. The Rural Water Study found that a particularly effective way to include beneficiaries in project design and implementation i s to establish a role for project champions within project-sponsored ac- tivities. This can include employment by the project when possible. Annex E 72 Annex E. Selected Comments o f W o r l d Bank Staff on Social Development Issues These comments are gathered from the three surveys-Country Director, Task Manager, and SD Staf-and arranged by subject. Social development specialists should stop policing and join the tasWcountry team. (Task manager) “Increase ownership o f social assessment and internalize related processes to project implementation, supported by effective capacity development in the client countries. Re- duce zealous policing in the context o f safeguards, based o n templates and focus on the realities o f client countries.” (Task manager) “Finding the balance between project quality enhancement and policing Task Team Leaders; Social Development as a field needs to be more clearly defined.” (Task manager) “Currently their role i s policing, which does not provide much help.” (Task manager) “More focus on learning and cross-regional exchanges; less o n policing.” (Task manager) “It should be seen as a core part o f the Bank’s dialogue and work at the country level; and should not be considered an “extra” burden by both the client and the Bank teams. The reason it has become an extra burden i s lack o f teaming with the task teams, and a perception that S D has more o f an external and “negative policing function” rather than “positive value added h n c t i o n ” as part o f the team.” (Country director) “SD staff sometimes act l i k e police constables rather than broadly trained de- velopment professionals.. .” (Country director) “Shift skill mix from enforcement o f safeguards to sound social development impact.” (Country director) “Get rid o f the enforcement and clearance process entirely and focus instead o n development impact. Give professional advice o n policy compliance rather than policing.” (SD specialist) “They see u s [SD specialists] as policemen and nay-sayers instead o f people who bring solutions and help them get things done.” SD specialists should join the team and focus on problem solving rather than criticizing (Task manager) “Become more team-oriented instead o f an external critic. Become aware o f the project as a whole rather than treating social issues in a compartmentalized manner. Interact with project TT and make an effort to understand issues in other areas o f specialization.” (Task manager) “Join the staff working in operations (this applies t o social scientists working in operational units) ... It will be for the better for a l l concerned... We find that you are interested in a career as “critics” and don’t t r u s t you. You think we’ll do anything to avoid complications on social issues. We’re a l l working for development and should b e cooperating.. .” 73 Annex E (Task manager) “. . . social development scientist[s] should be working much more as part o f the project teams and have to learn to work better with people with hard skills (engineers and finan- cial analysts) .” (Task manager) “In case this means h o w can S D issues be more streamlined into project: The key approach i s t o have SD professionals acting as team members, accountable to the TTL for getting these things right, rather than acting as gatekeeper for the TTL, and the TTL having to trans- m o g i f y into an expert to meet WB requirements. T h i s may have improved since the last time I l e d a preparation team.” (Task manager) “Social development staff should be proactive and be a part o f the task team.” (Task manager) “ need such specialists’ help that can work in a multidisciplinary team, can start I without an agenda but can develop it and a program, together with the clients, and then w o r k so that the client implements the program by doing and learning (meaning, it i s not going t o b e per- fect and there are mistakes). I t needs a budget, too, and the client has to pay even if o n l y a l i t t l e . N o grants.” (Task manager) “Avoid working in isolation. Interact with TT to get pointers and to develop relevant approaches. NB: Developing a strategy paper and circulating it for comments i s NOT adequate interaction!)” (Task manager) “They should be an integral part o f country teams and should participate in de- velopment o f CASs and PRSCs, as well as be part o f the quality enhancement team for projects.” (Country director) “[Have a] sense of ownership for development outcomes; more training; more ‘being there’.” (Country director) “Less focus o n ‘mandating’ policies, more o n helping task teams to improve projects.” (Country director) “Good o n relations and process, less so o n analysis and dialogue. Sometimes focus too much o n problems with insufficient proactive problem-solving.’’ (SD specialist) “ Iw o r k in one o f the regions. There was a mutual learning curve. In the f i r s t op- erations I was involved in, I was marginalized and treated as a nuisance by the Task manager - I was there to do what I was told, and particularly to ensure that the project did not get into trouble with the safeguards compliance police. Gradually TMs began to see that a social scientist could improve project quality, and I was able to get gender, participation, poverty focus integrated into projects. There have been successes and failures, but I can l o o k back o n a number o f projects that have been quite significantly changed as a result o f SDV inputs, and at least one where the TM has said that without the social scientist, there would have been n o project at all. (become so fix- ated on resisting safeguards and the argument that a l l social requirements are onerous that many opportunities for value added and sustainable interventions are lost.)” (SD specialist) “[SD specialists] should be made accountable to the field units and operational units. They need to be more field oriented in their approach and focus their efforts at evolving practical solutions to day to day problems than just produce manuals and guidelines based o n a few examples! ! !” Annex E 74 SD input should come early and continue throughout the lifecycle o f a project (Task manager) “Involvement in earliest phase o f project, so that project design and project area can be easily adjusted, i.e. make social development staff less o f a ‘watchdog’ and more responsi- ble for decisions they make.” (Task manager) “Upstream participation o f social development specialists will ensure minimiz- ing costs later on.” (Task manager) “If carried out upfront during initial project preparation, the information could help influence quality o f design.” (Task manager) “Too much emphasis o n making the perfect plan, rather than o n establishing a baseline and adapting implementation as circumstances change.” (SD specialist) “We need to move more toward upstream analytical work at the country level, and w e need to pay more attention to policy dialog.” Focus m o r e o n development effectiveness, less o n f o r m (Task manager) “Focus o n impact o f projects, rather than on form.” (Task manager) “Too much advertising o n what can be done, just do it.” (Country director) “In order to mainstream social issues there has to be a shift to demonstrating development effectiveness o f S D issues.” (Country director) “ Ifeel strongly that our social safeguard policies are one dimensional- applied without intelligent discretion and without sufficiently broad perspective a l l over the globe. Policies designed to ensure inclusion o f S. American indigenous groups are applied by looking for tribal groups to protect non-existent tribes in XXRegion. M i l l i o n dollar studies are repeated to determine that a protected area project o f a few m i l l i o n dollars in a remote area does not affect a group o f 8000 people many miles away, despite field staffs repeated assurances. Re- peater protects that have been financing very benign interventions for decades are held up for several years to wait for the completion o f under supervised social assessments o f entire state.” (Country director) “We delude ourselves that our endless action plans can have any significant effect o n the societies where we work. M u c h o f the SDV work i s a response to special interests (NGOs, etc) in the north, not a response to needs o f our client countries.” SDV should fund what they recommend (Task manager) “To begin with, if the Bank thinks that we need to do it, a special budget should be given to Social Development experts and they should be responsible o f the stud- ies/assessments needed in the context o f project preparation and/or supervision. Task managers should not be responsible for this task!” 75 Annex E (Country director) “Oblige the safeguards people to go deal with the clients, instead o f issuing directives from the comfort and safety o f their Washington offices. Subject their work to some sort o f cost-benefit analysis.” (Country director) “The most important challenge i s to ensure that the S D staff are fully inte- grated into the process and understand the costs and benefits o f their proposals. I t works least well when the N W and the staff see themselves as advocates or prophets in the wilderness (al- though that m a y not always be their fault!)” (SD specialist) “The social departments should manage the social staff budgets, instead o f the relevant project task managers. L e t me ask a question, saying that who are willing to buy criti- cisms or problems. We can discuss the social development strategies, but do not forget to con- sider the ways o f h o w to reach the targets.” Tailor w o r k to the specific context (Task manager) “[SD input] improves project design and likelihood o f success by making pro- jects contextually relevant instead o f blueprint.” (Task manager) “Social development issues should be dealt with in a project specific manner-a project will not be able to solve a l l social development issues o f a country-the social input should help to strengthen the objective o f a project and develop mechanisms to best reach the target community-it should not a i m at having all social issues mentioned and included in each and every project.” W o r k at the country level. (Country director) “Moving from a project by project basis t o general agreement o n appropriate country policies might reduce cost and help identifying home grown solutions.” (Country director) “Greater reliance on borrower procedures and regulations when they are broadly consistent with our policy principles and objectives.” (Country director) “Disconnect o f Government and Bank policies and an unclear understanding and commitment by clients to social assessment processes and application o f safeguard policies.” (Country director) “There are high transaction costs associated with safeguard compliance. One way to reduce them would be through systematic and thorough Country-level Social Analysis.” (SD specialist) “Mainstream SD staff as country social scientists just l i k e the country economist.” (SD specialist) “To engage effectively in programmatic lending, social scientists need to have equal status to country economists. Their role at the policy level needs to be institutionally rec- ognized and effectively pursued with appropriate financial resources across the Bank.” (SD specialist) “While the participatory process i s w e l l understood to b e a needed one, i t needs to s t i l l be institutionalized in the country, and there has been generally little attention paid to the social development issues which underlie the poverty as currently experienced in that country nor Annex E 76 have the strategies to alleviate poverty been analyzed from a social development point o f view o f r i s k s and stakeholders buy in.” SD staff need experience on the ground. (Task manager) “Staff in S D V need to get into the trenches and work with staff at the field level.” (Country director) “Social development staff in the Center should more frequently do country w o r k and be closer t o where the social problems are, and to understand the real needs and the difficulties o f the client.” (Country director) “A closer examination o f staffing levels need to be undertaken both in the field as well as H Q s to ensure that proper m i x i s available to satisfactorily support social devel- opment interventions both at policy and project levels. M u c h more ‘hands on’ field work and ex- posure to ground realities needed. S D staff need to ‘own’ the Country Assistance Strategy o f countries and search for solutions rather than search for problems. This applies to S D staff at re- gional and resident missions.” (Country director) “This area needs to be decentralized as far as possible so that the country con- text i s fully understood.” (SD specialist) “The most important things I need for my work are extensive local knowledge in countries that Iwork. I t i s not a case o f adding additional skills, but continuously learning about new developments and experiences so that I can include them if appropriate in my work.” (SD specialist) “ Igot here because o f my skills and not because Ineed more and more training. S D staff i s always being trained and not given any management responsibilities. N o more train- ing, get us more managerial and higher profile tasks.” SD needs more contact, recognition, mainstreaming. (Task manager) “Frankly, Ido not really know h o w they are organized now, but think their im- pact could be increased.” (Task manager) “Don’t k n o w much about the center’s w o r k as based in field office where there i s hardly any interaction with the work o f the center.” (Task manager) “Not even aware that S D V exists!” (Task manager) “They should be more visible. Sitting in a country office Ido not k n o w any SD persons other than the one in our field office.” (Country director) “[I’m] not very much aware o f contributions o f the center-too many policy documents?” (SD specialist) “. . .We have in our region not one single social scientist employed by the Bank in a country office. Three social scientists sitting in Washington and visiting the region from time to 77 Annex E time, plus a handful o f consultants, can’t help shape the needed transformation. We have neither the detailed in country knowledge, nor the immediacy, nor the legitimacy.” (SD specialist) “[I do not feel that my w o r k has had a significant impact o n the Bank operations with which I have been involved] because the Country Team has not taken social development seriously enough. Unfortunately, the needed assistance from Washington has not been forthcom- ing. I t has tended to reinforce the peripheralization o f social development by not providing the needed support to staff in the field, opting instead to insist that field staff b e proactive. Our work needs to be streamlined and given the same importance that others take for granted.” (SD specialist) “Unfortunately because there i s n o formal requirement for social assessment or social analysis, many TMs really do not take it seriously at all.” Connect w i t h other networks, the regions, and operations. (Task manager) “Devolve more staff to operations, require frequent rotation, focus o n principles rather than grand strategies, best practices, guidelines and similar templates.. . Social issues must b e dealt with in their specific context; hence, avoid overgeneralizing in approaches to b e used.” (Task manager) “The center units should support the operational units-experts should be avail- able for a project with a budget if needed, otherwise it i s hard to bring in this expertise under the tight project preparation and supervision budgets. T h e center needs to advertise more what they offer to operations and what they can help to make a project more successful. This does not mean to create a new website, but more practical options that come with a budget.” (Task manager) “Have the SD staff get their heads out o f the clouds and back down to earth.” (Country director) “More staff, less Washington.” (SD specialist) “[The least u s e f u l aspect o f S D V is] mind numbing bureaucracy and discus- sionshnitiatives that go nowhere.” (Country director) “Resident mission social development staff would need t o be much closer linked to networks. Especially local staff are not properly integrated in network.” (Country director) “Bank networks need to collaborate AND networks need to b e formed with other partners and universities. For Bank to w o r k o n social development policies alone has little impact.” (SD specialist) “More than merging activities, there i s a need for a better system o f communica- tion between regions and anchor and o f working together.” (SD specialist) “The S D V anchor should b e more in sync with the work o f the regions.” (SD specialist) “SDV should be more close to the regions in order to k n o w and understand the conditions o f work and the possibilities in each one o f them.” Annex E 78 (SD specialist) “Too removed from staff in operations. Need greater engagement with operational staff both o n personal and professional levels. Should provide one-stop service for information (best-practice, sample TOR, case studies, application o f OPs, etc.) relevant to operations.” (SD specialist) “Programmatic lending requires a whole different set o f s k i l l s both in the bank and in the country, and we are just nibbling at the edges. I think institutes o f applied socio- economic research, sufficiently independent o f government and sufficiently grounded to be l i s - tened to, are needed if not at country level then as hubs-probably 3 (Iran i s big enough to sup- port one o f i t s own, but one francophone and one for the arabo-anglophone w o u l d be needed in the long-term.” Move SD specialists to the sectors. (Task manager) “Social development specialists skills should be imbedded in the sectoral de- partment. A generalist social development expert often does not understand sector specific or country specific issues and h o w to approach issues pragmatically.” (Task manager) “They should be part o f Sector Units and work with their colleagues as part o f interdisciplinary teams.” (Task manager) “They should be spread across all sector units, not concentrated in a SD units, the anchor would bring them together professionally, not operations.” (SD specialist) “An idea to make S D more effective-the S D sector should be dissolved into other sectors. SD staff w o u l d become members o f sectors units as infrastructure, rural, etc. This way S D issues become sector issues and S D problems become sector problems. S D work would then be more focused and S D staff would be specialists in fields as transport or energy. This would also give the opportunity to S D staff to pursue both technical and task manager roles. In effect, the Bank would increase i t s development impact by many times.” (SD specialist) “ Ithink i t i s ill-conceived to have social development staff isolated even in fami- lies. I t makes the most sense to m e for the various technical specialists-rural, environmental, social-to be grouped according to sectors/problem areas. We w o r k o n questions and problems in the Bank. Rarely are these main questions social ones exclusively. Rather the main questions might have to do with environment/poverty linkages, or natural resource management and so forth. Rarely are projects focused exclusively o n social questions or problems. Hence, social sci- ence expertise i s more o f a means to an end than an end in itself. Hence, in my view it i s very important that the Bank social scientists work in interdisciplinary teams, contributing their exper- tise to larger goals. For example, I primarily work in environment and rural o n rural and envi- ronmental questions. Hence, I think i t does not make sense to group people by a technical disci- pline in the Bank.” SD i s critical to sustainable development (Task manager) “It makes a key contribution to sustainability o f operations, and to having pro- jects with positive impact. Without attention to this, projects can become lots o f money spent with l i t t l e impact.” 79 Annex E (Task manager) “It i s the very basis o f development. If people are not educated, healthy and have access to jobs, services and have political voice, there will be n o sustained economic development.” (Task manager) “. . . If we did not pay attention to social issues projects would be worse, or worse, they could not be implemented.. .” (Task manager) “The social theme should be accompanying the hard core infrastructure invest- ments needed and develop a mutual understandable agenda o f the constraints and opportunities o f implementing a project-be it road, water supply, or other large scale investment.This might not be relevant to your survey but when large scale construction projects are implemented, there are bound to be social issues which need to be taken into account.” Attention to SDhafeguards reduces poverty. (Country director) “The application o f social safeguards increases the developmental impact o f sector reform projects. By taking into account the needs o f vulnerable groups, and protecting those categories that are affected by the implementation o f specific activities financed under a project, the Bank contributes to improve their living conditions and access to economic opportu- nities, thus reducing the country’s poverty levels.” (Country director) “Usually [benefits accrue to projects from the application o f the social safe- guards], but there i s a tendency to think that most bank investments should have a social compo- nent, even if that i s not the major objective.” (SD specialist) “It i s difficult to talk about specific impact. However, the inclusion o f social de- velopment as part o f the Bank agenda, which has been dominated by macro-economics and struc- tural adjustments, has certainly increased the focus o n poverty reduction.” Good social development takes time (Task manager) “The problems come in t e r m s o f timing-both for project preparation and project implementation. The Bank procedures do not seem to recognize that social development takes time, to implement and to get measurable results.” (Task manager) “Recognize that social development takes time and change the timing o f project preparation and implementation and time needed to get measurable results.” (Task manager) “Be more flexible and take into account the extra time needed to obtain social impacts out o f our projects.” (SD specialist) “Systematizing the Bank’s w o r k o n resettlement: W e learned from the Upper Krishna Project, which involved resettlement o f 40,000 people, that resettlement implementation does not work in sync with the construction o f physical structures. Relocating populations i s m u c h easier than rebuilding communities. One o f the operational implications has been a recog- nition that supervision o f these impacts can extend beyond a project’s closing date. This i s n o w provided for in the new OP o n resettlement. . . .” Annex E 80 Disseminate knowledge, lessons. (Country director) “At institutional level more effort needs t o be undertaken t o disseminate knowledge o n ‘Best Practices’ and effective case studies.” 81 Annex F Annex F. Survey Instruments STUDY SOCIAL DEVELOPMENT OED SOCIAL DEVELOPMENT STAFF SURVEY 1. H o w long have you been working as a member o f the Social Development Family? years 2. What unique contributions has social development, as a theme o f development, made to the w o r k o f the Bank? 3. What unique contributions has social development, as a theme o f development, made to the work o f poverty reduction? 4. Do you feel that your work has had a significant impact o n the Bank operations with which you have been involved? Yes 0No 5. W o u l d you b e most effective in furthering the SD agenda as a (check one): 0 Task Manager 0 Technical Specialist 6. What are you currently? 0 Task Manager 0 Technical Specialist 7. If you could be trained in one additional social development skill, what would i t be? 8. Are social development themes more likely to receive appropriate levels o f attention n o w than in the past? Yes 0 No 9. W h i c h phase/s o f the project cycle needs more attention from SD specialists than it i s currently receiving? 0 Project identification 0 Design and appraisal 0 Supervision 0 Monitoring and evaluation Completion reporting 10. I s there one particular topic related to social development that should receive more attention from SD specialists than it i s currently receiving? Yes 0 No Annex F 82 11. Has the work o f SD staff to forward the safeguard agenda reduced the willingness o f Bank colleagues in Operations to pursue other social development goals? 0 Yes 0 No 12. Has your w o r k changed (or does it need to change) as a result o f the PRSP process? 0 Yes 0 No 13. Has your w o r k changed (or does it need to change) as a result o f the Bank engaging in more programmatic lending? 0 Yes 0 No 14. Should staff working o n related social development topics (currently handled by various Departments) be merged into one unit? 0 Yes 0No 15. If, in the previous question, you supported a merging o f SD topic areas, should this new arrangement 0 B e part o f SDV 0 Be a new stand-alone unit Everything i s fine as is. (answered “No” o n question 14.) I 0 Other 16. What i s the most helpful aspect o f the social development work o f the center (SDV)? What aspect i s least useful (SDV)? 17. Please give any opinions you might have o n h o w the role o f the central social development department (SDV) should be changed. 18. What should the role o f the regional social development units be? 19. W o u l d increased decentralization o f social development staff (placing them a l l around the Bank fully integrated into operational units) be an improvement? 0 Yes 0 No 20. W o u l d relocating some HQsocial development staff to country or regional offices impact development outcomes? 0 Yes 0 No 83 Annex F 2 1. Which topids related to social development, if any, should receive more attention from Task Managers than i s the case in current practice? If there i s anything else that you would l i k e to say about the effectiveness o f Bank support for social development, please comment here. OED SOCIAL DEVELOPMENT STUDY OPERATIONALSTAFF SURVEY 1. Does the nature of your w o r k routinely require attention to social development themes? 0 Yes No 2. In about h o w many projects have colleagues specialized in social development collaborated with you? If this answer i s greater than 0, did you usually find them knowledgeable in the areas you required their assistance? 0 Yes 0No 3. D o you face any obstacles in incorporating social development inputs and concepts in your work? 0 Yes 0 No 4. A r e there social development-related skills that are under-represented in your Region? 0 Yes 0 No 5. Are there social development-related skills that are over-represented in your Region? 0 Yes 0 No 6. In your opinion, does attention to social development issues improve the effectiveness o f Bank operations? 0 Yes 0 No 7. Are there ways to apply due diligence o n social development with transaction costs that are more appropriate and without compromising project quality? 0 Yes 0 No Please elaborate: 8. Does social development project support need to change as a result o f the PRSP process? Annex F 84 Yes 0 No 9. H o w should social development w o r k be applied within the context o f Bank support for programmatic lending? 10. Should staff working o n related social development topics (currently handled by various Departments) be merged into one unit? 0 Yes 0 No 11. If, in the previous question, you supported a merging o f SD topic areas, should this new arrangement 0 Be part o f SDV 0 B e a new stand-alone unit 0 Everything i s fine as is. (I answered “No” o n question lo). 0 Other 12. What i s the most helpful aspect o f the ongoing work o f the center (SDV)? What aspect i s most in need o f improvement? What activity should be dropped, if any? 13. Please give any opinions you might have on h o w the role o f S D V and the regional social development units should be changed. Changes in the role o f SDV: Changes in the role o f regional social development units: 14. W o u l d increased decentralization o f social development staff (placing them all around the Bank in operational units) be an improvement? 0 Yes 0 No 15. W o u l d relocating some HQsocial development staff to country or regional offices impact development outcomes 0 positively 0 negatively 0 not significantly 16. Please give your recommendations for increasing the impact o f the work done by social development staff o n poverty reduction. 17. Please give your recommendations for increasing the impact o f the work done by social development staff o n issues related to vulnerable groups. 18. Ifthere i s anything else that you would like to say about the effectiveness o f Bank support for social development, please comment here. 85 Annex F STUDY COUNTRY OED SOCIAL DEVELOPMENT UNIT SURVEY To be answered by the Country Director or designated Country Team member. 1. What are the three most important social issues in your country? 1. 2. 3. 2. Do you get the professional assistance that you need from Bank social development staff to deal with these three issues? 0 Yes 0 No If anything needs to be done about this situation, please specify: 3. D o you get the support you need from Bank staff to l a y out participatory processes? 0 Yes 0 No 4. Are the governments you work with interested in confronting social problems (in Bank- financed projects) 0 more intensively 0 less intensively than i s normal under current Bank practice? 5. Do you perceive that significant benefits accrue to projects from the application o f the social safeguards? 0 Yes 0 No Please elaborate: 6. Are Bank and Borrower transaction costs associated with safeguard policies appropriate? 0 Yes 0 No Please give any ideas you might have about h o w to change the process: 7. What procedures, if any, associated with social development impede the progress o f your w o r k unnecessarily? How? 8. Do any recent experiences in your country suggest innovative ways in which social development themes should be addressed? 9. When you have asked social development staff for assistance, in general have you found their performance in the field o f their specialization to be adequate? Annex F 86 Yes 0 No 10. D o y o u find that social development staff have the necessary experience t o interact with k e y borrower institutions? 0 Yes 0 No 11. Please give your recommendations for increasing the utility o f the w o r k o f social development staff in the Center: in the Regions: in the Resident Mission: 12. D o y o u feel that Task Managers are adequately trained t o recognize those social development issues that arise during the project cycle that require specialized intervention? Yes No 13. In your opinion does attention t o social development issues improve the development outcome o f projects? 0 Yes 0 No 14. A r e there areas where increasing the investment in social development support w o u l d yield major benefits? 0 Yes 0 No Please specify: 15. D o y o u find the network system useful with respect t o social development? 0 Yes No 16. Should staff working o n related social development topics (currently handled by various Departments) be merged into one unit? 0 Yes 0 No 17. If, in the previous question, y o u supported a merging o f S D topic areas, should this n e w arrangement 0 B e part o f S D V B e a n e w stand-alone unit 0 Everything i s fine as is. (Ianswered “No” o n question 16.) 0 Other 87 Annex F 18. W o u l d relocating some HQsocial development staff t o country or regional offices impact development outcomes 0 positively negatively 0 n o t significantly 19. If there i s anything else that you w o u l d l i k e t o say about the effectiveness o f B a n k support for social development, please comment here. 20. The above answers reflect the experience in w h i c h country or countries? Annex G 88 Annex G. Management Response INTRODUCTION 1. Management welcomes the review by the Operations Evaluation Department (OED), with i t s overall assessment that social development has a significant impact o n the development outcomes o f Bank operations. By combining the lessons from past evaluations with additional data generated f r o m surveys and new reviews, the study provides timely evidence o f the value o f social development, while raising issues and challenges that has informed the preparation o f the draft paper, “Social Development in W o r l d Bank Operations: Results and Way Forward.” This paper lays out the Bank’s approach to the thematic area o f social development, the strategic pri- orities going forward and discusses h o w these priorities will be achieved.’ Management concurs with the findings o f the study, while recognizing the limitations o f the desk review, and notes that the Social Development Group2 has already initiated many o f the actions that follow from the recommendations. W h i l e we agree with O E D on most o f i t s findings, we would l i k e to highlight in this Management Response recent and ongoing actions related to social development that OED could not take into account, given the timing and scope o f i t s study. OED FINDINGS 2. The study consists o f a meta-evaluation o f recent and ongoing OED evaluations o f specific social development theme^,^ supplemented by analysis o f data from a portfolio review; literature review; individual interviews and surveys o f country directors, task managers, and social develop- ment staff; and a review o f Country Assistance Strategies (CASs). Social development themes, es- pecially participation, vulnerability/exclusion, gender, and NGOs/civil society, are highly relevant to the lending priorities that the Bank and borrowers agree on. B o t h country directors and task managers find social development issues highly relevant. According to the O E D review, 96 percent of country directors said that, in their experience, attention to social development improves the de- velopment outcome o f Bank operations. Also, 93 percent o f task managers said that the nature o f their work routinely required attention to social development themes (Para. 6.3). However, they would l i k e to see more systematic flow o f relevant social information to country and task teams. OED finds that among social development themes, greater attention i s needed o n institutional re- form (transparency/accountability and decentralization), anticorruption, and public services for the poor. I t also finds that until recently, social analysis has focused o n the project level, but priorities emerging from Poverty Reduction Strategy Paper (PRSP) processes indicate that there i s a role for country-level analyses, with additional focus o n sociopolitical realities. 1. The World Bank Board’s Committee on Development Effectivenessreviewed an earlier draft o f that paper together with the OED report. T h e revised draft will be used as a basis for consultations with borrowers, partners and other stakeholders during the first half o f 2004. 2. T h e Social Development Group consists of social development specialists grouped in six regional social development teams, an anchor department (Social Development Department in the Environmentally and Socially Sustainable Develop ment Vice Presidency) and staff in other Bank units. 3. The evaluations included in the meta-evaluationwere Gender (1995, 1997, 2001), Post-Conflict Reconstruction(1998), NGOs (1 999), Participation, (2000), Rural Water (2000,2002), Resettlement (2001), Cultural Heritage (200 l ) , Community- Driven Development in the Sahel(2002), and Forestry (1 99 1,2000). T h e team also drew on the 2002 Annual Review o f Development Effectiveness (ARDE) and the Social Investment Funds review (200 1). 89 Annex G 3. Conclusions. O E D concludes that the Bank has succeeded in drawing borrowers’ attention to a wide array o f social development themes. Since 1972,26 percent o f all rated projects have ad- dressed at least one social development theme (1060 out o f 4123). The range o f themes has ex- panded from the early focus on social safeguards4to broader social development issues, including participation, conflict, gender, community-driven development (CDD) and social capital. Over a 30-year period, projects that address social development issues have consistently been rated 3 to 4 percentage points better than projects that do not address these themes. For example, across the en- tire Bank portfolio, 72 percent o f those that addressed at least one social development theme were rated satisfactory, compared with 68 percent o f all rated projects. Likewise, compared to all rated projects, projects with social development themes were rated by OED to be three percentage points more likely to be sustainable (53 vs. 50 percent) and to do better on institutional development im- pact as defined by OED. Further, the review provides striking evidence that integrating multiple social development themes leads to even better outcomes, sustainability, and institutional develop- ment impact. For example, among projects that include two social development themes, 8 1 percent were rated satisfactory by OED, with three themes, 84 percent, and with four themes, 90 percent satisfactory. The clear implication i s that taking on more social development approaches improves project effectiveness. Despite this success, intense attention to social development was exceptional within the portfolio o f rated projects: roughly 1 percent or 61 out o f 4123 projects going back to 1972 addressed four or more social development themes. T h e review reports that many aspects o f project-level social development work are highly complementary and can be more systematically addressed by Bank operations in an integrated, coordinated fashion. The social impact o f develop- ment interventions at the community level can be substantially increased by helping borrowers de- velop a more inclusive, cohesive and accountable institutional framework at the country level. The review also finds that the Bank’s human and financial resources for social development are not adequate for the number and scope o f the mandates received. 4. Recommendations. The review recommends the following: (a) the Bank (through the Social Development Sector Board) should identify and promote integration among social themes whose combinations improve outcomes; (b) the Bank’s human capital and borrowers’ existing institutional capacity need to be deployed to provide task and country teams with relevant social development expertise throughout the project cycle; (c) country teams should have the capacity to identify critically needed social knowledge and to facilitate i t s flow; and (d) sector staff need to ensure that stated Bank or policy priorities receive adequate treatment across Regions and countries, and Bank strategic planning needs to address gaps in current skills and in monitoring and evaluation. The attached Management Action Record provides Management’s responses t o these recommendations, noting that many are followed up in more depth in the context o f the dis- cussion draft o f the Social Development strategic priorities paper (SPP), “Social Development in W o r l d Bank Operations: Results and Way Forward.” 4. T h e social safeguards arose out o f concem for involuntary resettlement, indigenous peoples and cultural property; they are govemed by the procedures laid out in OPL3P4.12, Involuntary Resettlement, OD4.20, Indigenous Peoples Policy, and OPNl 1.03, Management o f Cultural Property in Bank-Financed Projects. Annex G 90 MANAGEMENT COMMENTS 5. Management welcomes the report as an important backdrop for the draft SPP5 and ongo- ing w o r k o n planning and implementation o f Social Development activities in the Bank. This pa- per draws o n the OED review to propose strategic actions to integrate social development activi- ties more fully within the Bank’s development assistance work. Specifically, three o f the four strategic priorities proposed by the draft SPP are: improve the effectiveness of Bank-supported investment projects, with a more systematic approach to social development and stronger fol- low-through; strengthen the social development thematic portfolio, enhancing projects and stud- ies that directly address social development principles: and improve the Bank’s capacity build- ing, advocacy and research on social development to build a stronger implementation platform. 6. Added Dimension in the D r a j l SPP. However, the OED study acknowledges that it did not cover all aspects o f social development in Bank assistance; in particular, it did not examine the significant progress achieved since June 2000 in adapting social development to the macro or country level in the context o f Bank support to countries’ PRSP processes. The SDG’s work in assisting borrowing countries with participation in their PRSPs, their poverty and social impact analysis (PSIA) o f policy reforms, and their analysis o f social accountability to improve govern- ance-work that addresses some o f OED’s recommendations-is thus outside the purview o f the O E D review. The draft SPP, therefore, also draws o n this more recent experience to propose stra- tegic actions to enhance the Bank’s development assistance. Specifically, the draft paper’s f i r s t strategic priority i s to strengthen the Bank’s policy dialogue andpolicy-based lending, by in- creasing attention to the social dimensions o f development. 7. Conceptualizing Social Development. The study finds that, although the Bank does not yet have explicit social development objectives and a strategy to achieve them, there i s remark- able consistency across a wide range o f Bank documents. The study uses an innovative approach, creating an objectives matrix to compare the perspectives o n social development in different Bank policy and strategy documents, and it notes that “there i s a remarkable interconnectedness between the various documents o n nearly every subject” (Para. 1.IS). Overall, the picture that emerges i s largely consistent with the Copenhagen Declaration issued at the 1995 W o r l d Summit for Social Development, which put people at the center o f development and pledged to “make the conquest o f poverty, the goal o f full employment, and the fostering o f stable, safe and just socie- ties their overriding objectives.” That statement o f objectives i s s t i l l relevant today, particularly for the achievement o f the Millennium Development Goals. The draft SPP lays out the Bank’s strategic choices and business plan to help borrowers achieve these objectives. The O E D review observes that other organizations consistently consider gender as a key part o f social develop- ment. Recognizing the crosscutting nature o f the Bank’s w o r k in these areas, Management will seek to strengthen operational collaboration between social development and gender. 8. The Social Development Portfolio. Social development i s not a sector with a large stand- alone portfolio. Indeed, OED finds explicit attention to social development issues to be relatively routine across most sectors and Regions. It cites data showing that since 1990, when attention to social development dramatically increased, some $15.8 b i l l i o n in lending include attention to so- cial development (including gender) activities. Other sector units manage most o f these projects, 5. The discussion draft i s available at the following website: http://lnweb18.worldbank.orglESSD/sdvext.ns~62ByDocName/SocialDevelopmentS~ategy 91 Annex G with varying degrees o f inputs from social development specialists.6 Management appreciates OED’s effort t o identify and assess the social development component o f the Bank’s portfolio. SDG’s portfolio monitoring system tracks projects with primary or direct funding within six thematic classifications, under the general heading o f social development, gender, and inclusion (SDGI)7from the Bankwide database.’ Under this classification system, a total o f 922 projects with a commitment value o f $54.2 b i l l i o n and SD allocations o f $14.3 b i l l i o n have been identi- fied f r o m FY90-FY03. All C D D projects involve participation, but, as OED notes, i t was unable to include this overlap in i t s analysis because i t s data for these two themes come from t w o differ- ent time periods. Management notes that social development themes are found less frequently in the M i d d l e East and N o r t h Africa Region (MNA) and in the finance, private sector development, and economic policy sectors. The MNA social development strategy, currently being formulated, i s expected to draw greater attention to social development themes in the Region. M o r e system- atic attention to PSIA since 200 1 i s helping to demonstrate the relevance o f social development issues to private sector development and economic policy reforms. 9. CustomizingSocial Analysis. OED’s meta-evaluation highlights the significance o f knowledge o f the country context and local realities. Analysis o f sociopolitical realities and local institutions provides a more concrete basis for social development and economic development as- sistance. Management agrees with the study’s conclusions that sound country-level (or subnational and sectorwide) social analysis i s important to underpinproject-level analytical work. Therefore, the draft SPP emphasizes as part o f i t s first strategic priority that upstream analytical work i s more useful for policy dialogue and can reduce project preparation costs. Further, it proposes actions to (1) improve the process o f preparing and implementingpolicy dialogue and lending, and (2) im- prove the poverty and social impact analysis o f reforms. Management agrees that it i s important to build o n the positive experience o f decentralization o f Bank staff to countries such as India, Indo- nesia, and the Philippines. Further, it highlights the importance o f using local expertise to under- stand and support local institutions in both upstream work and throughout the project cycle. 10. Policy Guidance. The portfolio review indicates h o w the Bank’s work o n social devel- opment has evolved from the focus o n social safeguards in the late 1980s to a broader menu o f social development themes, including attention to conflict and gender.gManagement agrees with OED’s observation that lack o f clear guidance” has hampered consistency in application o f so- cial analytical tools to Bank projects. As a major step to rectify this problem, the Social Devel- opment Sector Board has issued a Social Analysis Sourcebook” clarifying the difference between the Bank’s role in social analysis and the borrower’s role in social assessment. The Sector Board has also produced a User’s Guide for Poverty and Social Impact AnalysisI2 laying out the Bank’s 6 . In FY03, there are a total o f 20 active stand alone social development operations with a combined original commitment value o f S 1.345 billion. 7. T h e themes are civic engagement, participation, CDD, conflict prevention and post-conflict reconstruction, gender, in- digenous peoples, social analysis and monitoring, and other social development. 8. Portfolio information under this classification scheme i s available only from FY90 to FY03. An OPCS team i s currently reclassifying operations dated before FY90 using current thematic and sector definitions. 9. Operational policies have been issued recently for conflict (OPBP2.30, Development Cooperation and Conflict) and gen- der (OP4.20, Gender and Development). 10. Current guidance dates back to 1984 when OMS2.20, Project Appraisal included a section on Sociological Appraisal. The OMS is outdated. T h e Bank’s portfolio has evolved and so has the range o f social analysis tools and instruments. 11. See http://www.worldbank.org/socialanalysissourcebook. 12. T h e User’s Guide, produced jointly with the PREM Network, i s available at http://www.worldbank.org/psia. Annex G 92 advice to countries o n tools and techniques for their analysis o f poverty and social impacts o f policy reforms. Management i s working to reorganize i t s lending polices under a few key themes, limited to core principles, with prescriptive advice moved to guidelines. Revisions to OMS 2.20 o n project appraisal, which covers appraisal o f social opportunities, constraints and r i s k s o f pro- jects, are part o f that work.I3 11. Due Diligence and Safeguards. W h i l e the OED review points out that projects with safe- guard policy issues (indigenous peoples, physical cultural heritage, and involuntary resettlement) have higher sustainability ratings than comparable cohorts o f Bank projects, it also suggests that the special emphasis o n existing safeguard policies may be crowding out other social development is- sues rather than making them the entry point for good social analysis. Management believes that the issue can be addressed by making better use o f the available room for exercising judgment in the application o f safeguard policies, making firther progress o n the work program to improve the country focus o f safeguards, and promoting an integrated treatment o f social development issues in country programs and projects. Management believes that social due diligence should be focused o n results rather than on policing. Management notes that, apart from the safeguard policies that are designed to mitigate the adverse impacts o f economic development, all the remaining social devel- opment themes are aimed at improving development outcomes. Social development specialists help borrowers and task teams identify which social development themes from the menu are most rele- vant and help to ensure that project benefits reduce poverty. By helping to design more responsive projects and more inclusive and accountable institutions, social development assistance helps bor- rowers broaden the beneficiary base and build support for reforms. The draft SPP builds o n an im- portant element o f the Bank’s pilot program to improve the effectiveness o f safeguards in calling for the use o f country or sectoral social analysis to scale up safeguard development impact, as well as to capture opportunities for social development, upstream o f project work. Further, it proposes having a single focal point for social development knowledge in a country (or a group o f smaller countries) to ensure that country teams can exploit synergies across different social dimensions. 12. Demand for Social Development. OED i s concerned about the mismatch between the apparent demand for social development and the resources available for those inputs. The study finds that 96 percent o f country directors and 83 percent o f task managers rated the importance o f social development quite high and said attention to social development improves the develop- ment outcome o f Bank operations. At the same time, OED notes that country directors control budgets and can affect the allocations for social development work. Management makes note o f OED’s observation that meeting the apparent demand for social development services w o u l d have significant resource allocation implications, and the implementation and business plan ( s t i l l under preparation) will examine the constraints, gaps, and competing priorities, through exten- sive internal consultations and six detailed, costed regional implementation and business plans. 13. Demonstrating Relevance and Monitoring Outcomes. The OED review describes the evolution o f the Bank’s social development work from an initial concern with social safeguards toward concern for transparency, accountability, decentralization, anticorruption, and public ser- vices for the poor. The Bank i s already responding to these demands through recent work o n con- flict, PSIA, and social accountability. The Social Development Sector Board has also initiated work o n social development indicators and will place much greater emphasis o n monitoring and evaluation o f social development outcomes in Bank operations. These efforts will be strength- ened in the next few years as priority activities identified in the draft SSP are implemented 13. See World Bank Lending Modemization and Simplification: An Update (CODE 2003-0086), December 24,2003. 93 Annex G 14. Social Development Staff within the Bank. The review attempts to estimate the number o f social development professionals who work for the Bank. OED reports that in January 2002, the Bank employed 175 social development professionals, both staff and short-term consultants (Box 2.1). This i s consistent with Human Resources records that currently l i s t 141 staff with a primary mapping in Social Deve10pment.I~ Over 40 percent o f these staff are not social scientists, indicating a diversity in the skills m i x . Other Networks employ a few staff who, because they w o r k o n social development, are included in SDG’s roster. An even more important finding i s that 249 staff w h o have professional degrees in non-economic social sciences are not working o n social development, and could be tapped for social development work. The draft SD SPP dis- cusses actions to ensure that responsibility for S D i s not concentrated within a small number o f specialists, but i s the province o f Bank staff across a l l sectors. The internal implementation and business plan will address the issue more concretely. IV. CONCLUSIONS 15. The Bank i s firmly committed to strengthening i t s focus o n social development. Man- agement welcomes this review as a contribution toward this end. Effective development depends o n borrowers’ ability to provide effective services and development opportunities to all in a transparent and accountable manner, and o n the ability o f all citizens to build o n their assets and exercise voice for a better future. As the O E D review indicates, the Bank-through i t s analytic and development services, and in close coordination with its partners-has made significant gains over the past decade and i s deeply engaged in this rapidly evolving agenda. Management will use the OED review in i t s implementation o f strategic priorities in social development w o r k in support o f borrowing countries. ACTION RECORD MANAGEMENT Recommendation Response Since the study shows the Management welcomes OED’s documentation o f the strong posi- strong positive tive interaction o f social themes. The OED report supports and sub- interaction of some social stantiates evidence o f the findings emerging from the review o f themes, the Bank poverty and social aspects in Q A G reviews. Management agrees (through the Social with the need to maximize the use o f integrated approaches to so- Development Sector cial development issues, and i s taking up this issue in the draft SPP. Board) should identify, In addition, the S D Sector Board already has undertaken work, un- and promote integration der Management’s direction, to develop policy and guidance to task within, the thematic teams o n an integrated approach t o social development. combinations that improve outcomes. 1. T h e Social Analysis Sourcebook, released in August 2002 aRer Bankwide review, provides explicit guidance o n an integrated approach. 2. The work o f the S D Sector Board o n social analysis issues i s an important input into the proposed new effort to clarify and simplify the investment lending process. 14. SDG’s staffing data, from the Human Resources Strategic Staffing Unit database, include IBRD staff at levels GE and above. Annex G 94 1 Recommendation Response 3. The ongoing update o f the Bank’s operational policy o n adjustment lending (OP/BP8.60) proposes an integrated approach to the treatment o f social and poverty impact at the country and opera- tional levels, reflecting SD Sector Board advice. Agreed Actions 1. Management plans to cover social analysis in i t s simplified policy framework for investment lending slated for issuance by the end o f FY05. 2. Management has recently revamped the processes for sys- tematic sectoral input, including from social development staff, into CASs and operational products. The draft S D SPP discusses tools and practical steps for improvements in the provision o f social devel- opment inputs. (See draft SPP, Chapter 1 1 1, Strategic Priority 1.) 3. The S D Sector Board will continue i t s strategic collabora- tion with QAG to monitor operational quality and document good practices o n the poverty and social dimensions o f Bank-supported operations. This will be covered in the internal business and imple- mentation plan under development. 4. Management will support a strategic research agenda on the linkages between social development and poverty, as outlined in the draft SPP. T h e SD Sector Board i s extracting further lessons o n inte- grated S D approaches, summarizing the results from Implementation Completion Reports and OED’s Evaluation Summaries for compila- tion in FY05. (See draft SPP, Chapter 1 11, Strategic Priority 4.) The Bank’s human Management agrees that effective social development outcomes capital and the depend o n attention to social development issues throughout the borrowers’ existing project cycle. To this end, the draft SPP explores the use o f S D ap- institutional capacity proaches in upstream E S W to inform the C A S and country pro- need to be employed to grams, and downstream in implementation and M&E. The internal provide task and country business and implementation plan will explore further the implica- teams with the relevant tions for the operational skills o f SD staff and the incentives for social development staff working o n SD issues to work across the structural boundaries expertise throughout the between PREM, HD and SD. I t also addresses the use o f local ex- project cycle. pertise, the demand for investment in client capacity building for social development, and WBI’s role in this regard. (See draft SPP, Chapter 1 11, Strategic Priority 2.) Agreed Actions Decentralization o f S D experts has yielded benefits in terms o f more in-depthand relevant knowledge o f countries’ social and institutional context, and has also lowered supervision costs. Man- agement will continue t o promote decentralization o f S D expertise to country offices (or subregions) with significant S D issues where there i s a strong business case to do so. In countries with a large portfolio, this m a y entail placement o f international expertise. In 95 Annex G Response others, it may entail deployment o f national staff with the s k i l l s to provide integrated S D inputs to operations. The internal business plan will address these staff deployment issues and will discuss the use o f local expertise to provide SD support to countries, for both upstream work and during the project cycle. (See draft SPP, Chap- ter 111, Strategic Priority 4.) Management will help to strengthen country capacity by supporting local networks o f social scientists (as in E C A and MNA). (See draft SPP, Chapter 111, Strategic Priority 4.) The draft SPP explores how best to promote projects and project com- ponents to develop the ability o f local government and nongovernmen- tal institutions to address SD issues and enhance project sustainability. (See draft SPP, Chapter 1 11, Strategic Priority 4.) Management agrees with this recommendation. The draft SPP sug- have the capacity to gests a major strategic shift from individual projects to the country as identify critically needed the most important unit o f analysis and action. This shift i s fblly con- social knowledge and to sistent with the CDF/PRSP framework, and entails a programmatic, facilitate its flow. long-term approach to country-owned social development at the na- tional (or in large countries, at subnational) and sectoral levels. W e have made significant initial strides at the macroeconomic level by helping countries facilitate participation in PRSP processes and con- duct their PSIA analysis, social accountability analysis, and conflict analysis. Assistance related to the first three areas i s provided in close collaboration with PREM, reflecting increased efforts to work across sectoral boundaries as well as the benefits o f a country-driven ap- proach. (See draft SPP, Chapter 1 11, Strategic Priority 1.) SDG i s developing social development indicators that are being tested and refined through cross-country and in-country research. The indicators will be integrated into the W o r l d Development Indi- cators (WDI), provided as background for Country Policy and Insti- tutional Assessment (CPIA) work, and incorporated into the Coun- try-at-a-Glance data. (See draft SPP, Chapter 1 11, Strategic Priority 4.) Finally, several regions are experimenting with assigning re- sponsibility to staff for integrating social development information and making it available to country teams in a timely manner - simi- lar to the role that country economists play on the economic side. The draft S D SPP takes stock o f these experiments; in the next stage, i t will examine a generic approach. Agreed Actions Management will strive to integrate social analysis and gender analysis with poverty assessments, and will take stock o f progress annually as part o f portfolio monitoring. (See draft SPP, Chapter Annex G 96 Recommendation Response 111, Strategic Priority 2.) The draft SPP explores h o w best to: (1) ensure that relevant social development information i s made available to country teams in a timely manner; (2) strengthen social analysis at the country level; and (3) strengthen, over time, borrowers’ capacity to undertake so- cial assessments as routine inputs into project preparation, while retaining Bank responsibility for due diligence on social develop- ment issues in Bank-supported operations. (See draft SPP, Chapter 111, Strategic Priority 2.) Sector staff need to The formulation o f Regional strategies and the draft SPP i s helping ensure that stated Bank to clarify and build consensus around strategic priorities. Manage- or policy priorities ment has increased i t s attention to high-priority issues for borrower receive adequate countries through SDG’s recent work o n conflict, community- treatment across Regions driven development, and the development o f PSIA and social ac- and countries, and Bank countability tools for use by borrowers. W o r k in each o f these areas strategic planning needs has involved creating a new knowledge base, broadening the to address current s k i l l s Bank’s o w n skills mix, and developing partnerships with a new se- and monitoring and ries o f institutions for effective implementation. evaluation gaps. Management agrees with the value o f increasing the skills o f all Bank operational staff to take account o f S D issues. This important topic i s being addressed in the internal business and implementation plan under preparation. Agreed Actions 1. The draft SPP addresses the shift in strategic priorities to the country level in support o f country-owned poverty reduction strategies, and the implications o f that shift for social development work in the Bank. (See draft SPP, Chapter I1and Chapter 1 11, Stra- tegic Priority 1.) 2. The S D s k i l l s mix i s being h r t h e r diversified by recruiting political scientists, policy analysts, institutional specialists and economists within the S D group. Given the institutional and re- source constraints within the S D group, other sectors will be en- couraged to recruit S D specialists to provide more specialized knowledge and S D inputs relevant to each o f those sectors. The skills mix strategy will be an important element o f the internal business and implementation plan under preparation. 3. The internal business and implementation plan under preparation will cover training issues, including the integration o f S D in core learning activities for Bank operational staff. 97 Annex H Annex H. Chairman's Summary: Committee on Development Effectiveness (CODE) 1. On December 17,2003, the Committee on Development Effectiveness (CODE) met to consider the findings from the report OED Review o f Social Development in Bank Activities and to provide feedback to Management o n the strategic priorities it proposed for Social Development. 2. Conclusions and next steps. The Committee commended O E D for the comprehensive- ness o f i t s report o n what was a complex area and appreciated the operational focus o f the report. Members were supportive o f the importance o f social development in Bank operations and em- phasized that the subject as a strategy was more in the nature o f a thematic approach. Members f e l t that i t would be more helpful if management were to focus o n thematic aspects o f Social De- velopment and on improving the quality o f Social Development practices and to mainstream more effectively SD themes into the Banks work. Accordingly, the Committee suggested that management prepare a paper more along the lines o f an implementation plan for mainstreaming o f social development in Bank operations. As next steps, the O E D Report will be disclosed along with a revised management response attached as an annex. Management also indicated that it would simultaneously disclose a strategic priority paper that took account o f the comments and suggestions received at the C O D E meeting and the recommendations in the OED review. Management will then prepare a business action plan highlighting i t s approach to social de- velopment for mainstreaming, detailing organizational and resource implications and submit i t to CODE for review. This paper, while not being a sector strategy per se, will present the Bank's thematic approach to social development and provide useful guidance to staff and would fill the gap noted by OED - the lack o f a single document that prescribed the Bank's priorities and goals in this area. OED Review o f Social Development in Bank Activities. The m a i n purpose o f the OED Review was to assess a range o f social development activities that the Bank has undertaken, based o n past evaluations, with a view to informing the design and review o f a Social Develop- ment (SD) Thematic Strategy Paper (TSP) and future Bank engagement in SD. The evaluation finds that there i s a positive association between greater attention to social development themes and successful project outcomes. The review, however, also finds: a lack o f broadly accepted definitions for the t e r m social development; highly uneven treatment o f the various social themes during implementation; slow integration o f social development practice and scarce quality assur- ance mechanisms; and that inadequate resources devoted to social development often hampers project implementation. Safeguards also present a number o f challenges. 4. The Review makes four m a i n recommendations to the Bank: It should (a) actively iden- tify and promote consideration o f social development themes and thematic combinations that im- prove outcomes; (b) utilize better the existing institutional capacity o f the Bank and borrowers t o provide timely and relevant expertise throughout the project cycle; (c) equip country teams with the capacity to identify critical social development issues; and (d) provide adequate attention in the institution's strategic planning process to such policy priorities across regions and countries as w e l l as to skills and monitoring and evaluation gaps. There i s broad agreement between M a n - agement and O E D o n the findings and the recommendations o f the review. Annex H 98 5. Strategic priorities for social development. In 2002, the CODE Subcommittee en- dorsed the development o f a thematic strategy paper (TSP) for social development (SD) in re- sponse to the growing scale and importance o f social development work within Bank activities. Management proposed a set o f operational strategic priorities for better meeting the existing pov- erty reduction "vision" o f the Bank. Based on the Bank's history in working o n social develop- ment, the approach i s guided by three operational principles: inclusion, cohesion, and account- abiZity. These principles, which articulate the Bank's existing broad framework for achieving sus- tainable poverty reduction, underpin four strategic priorities: 0 increasing the attention to the social dimensions of development to strengthen the Bank's pol- icy dialogue and policy-based lending; 0 improving the effectiveness of Bank-supported investment projects by mainstreaming of so- cial development activities; 0 strengthening the social development thematic portfolio by enhancing projects and studies that directly address social development impacts: and 0 strengthening implementation by improving the Bank's capacity building, advocacy, and re- search on social development. 6. These strategic priorities are intended to mainstream and scale up the Bank's efforts to address the social dimensions o f development in i t s work. They w o u l d be accompanied by an implementation plan and a monitoring and evaluation framework to be completed by the regions. Main points o f discussion. 7. Aim and focus o f the strategy. W h i l e members broadly agreed with the overall focus, strategic priorities, and the principles (inclusion, cohesion and accountability) that underpin the strategic priorities, they were unsure about what the aim o f a thematic strategy paper would be. First, members raised questions around the definition o f social development and the scope and fo- cus o f the strategy. They noted that "social development" was not a sector per se and that the strate- gic priorities did not constitute a sector strategy as was typically understood in the Bank. W h i l e some members believed there was a need for a more solid theoretical anchor to ground the strategy, some supported an operational focus and believed that it should go even fiu-ther to focus the Bank's efforts in areas where the poor could benefit most in concrete terms, such as health, education, housing, income and employment, gender and core labor standards. Others cautioned that the Bank steer clear o f areas that were in the purview o f other agencies. Some members felt it was important for the Bank to have a thematic strategy in this area and suggested that it should become part o f the regular scrutinizing exercise in the framework o f the annual sector strategy implementation update. 8. The committee noted that current focus o f the proposed strategy o n h o w the Bank could improve i t s work in the area o f social development -- ensuring consistent qualitykreatment o f SD topics across regions, improving operational instruments, deployng resources more effectively -- were primarily issues o f internal operational management. Accordingly, i t was suggested that the aim be consolidating best practice within the Bank and thus disseminating the Bank's approach to social development. 99 Annex H 9. The committee asked Management to return with a paper, including a business "action plan" that clearly spells out h o w the Bank will implement a more consistent, systematic approach t o social development, including the associated organizational and resources implications. Other suggestions made by different committee members included that: (a) the plan include monitor- able indicators building o n those discussed; (b) management focus on i t s five business lines and make clear the concrete ways the kinds o f social change the Bank can best assist clients in under- taking; and (c) management simply expand the management response to the OED report instead o f writing a social development strategy. 10. Management emphasized the need for a Board-reviewed strategy to guide i t s social de- velopment activities so the right message i s sent about the relative importance o f social develop- ment as compared to other thematic and sectoral areas. Similarly, management added that that it w o u l d be essential for it to consult publicly, especially with clients, o n the Bank's approach to social development issues for it to meet the committee request to return to CODE with a realistic implementation and business strategy. Management will report o n progress in implementing i t s approach to social development through the sector strategy implementation update reports. 11. What i s the business case for proposing a social development strategy?Second, some members raised a question about the additional value o f a new strategy paper noting that it would not break new conceptual ground nor propose any new mandates, requirements or policies. They felt that much o f what was proposed was already on-going and appeared to be mainstreamed. They sought clarification o n what was new and would be done differently. One speaker noted that treatment o f safeguards s t i l l posed challenges for staff and for clients and in his view, i t was unrealistic to think that these issues could simply be resolved by attempting to address safeguards more upstream. 12. Management said that there were s t i l l substantial gaps that remained to be filled in order to mainstream social development. Management noted that the OED only found a small propor- tion o f the Bank's total portfolio that addressed several social development issues ( o f about 4000 projects reviewed, only about 100 projects had addressed four or more social themes). OED also found that the language o f social development found in the country assistance strategies (CASs) had yet to translate into actual Bank operations. Furthermore other assessments have shown that there was a need to adopt a more wholesale approach to addressing social issues at the macro- level and more upstream as opposed to o n a project-by-project basis. Lastly, management noted that there had been gaps in the Bank's success in distilling i t s knowledge o n social development and in documenting best practice consistently. The intent o f the proposed strategy was to fill these gaps and therefore, in this regard, not to propose any new policies, but rather, respond to the OED findings and recommendations for the need to identify and make improvements, build o n best practices, be more systematic and scale up o n implementation, given the demonstrated high rates o f return to project outcomes o f addressing social issues. Management added that "do- ing things better, more systematically, and expanding best practice to a l l aspects o f social devel- opment" was, in i t s view, as strategic and as big a challenge as striking out into new areas. 13. Third, members were concerned about the disconnect between what the Board was given t o understand in other contexts, that social development has been internalized within the Bank, or what development partners were hearing in messages f r o m the President in terms o f claims o f substantial progress made by the Bank in this area since the Copenhagen Summit or W o r l d Summit o n Social Development (WSSD) and the Beijing Conference o n Women, and the mes- sages from the O E D findings and the strategy paper showing that much remained to be done. Annex H 100 Some noted the importance o f the strategy as a signal that the Bank was following up o n these major conferences and i t s commitment to implement the MillenniumDevelopment Goals. Man- agement noted that significant changes had occurred in the Bank's thinking o n social develop- ment since the Summits on Social Development in 1995 and 2000, including the establishment o f a Social Development Department and the current focus o n the principles o f inclusion, cohesion, and accountability. The OED report noted that while there i s clarity in the Bank o n some policy issues, such as safeguards, there i s less clarity o n other aspects o f social development. I t said that the review findings showed uneven treatment o f social development issues across countries, indi- cating that staff could use more guidance and that there was room to improve what the Bank i s currently doing with respect to social development. 14. Fourth, members also asked h o w the strategy would differ in i t s approach from the com- prehensive development framework (CDF) and the PRSP approaches. O E D clarified that the CDF approach focused o n "how to do things" using ownership and partnership modalities and noted that some social development instruments informed the C D F approaches while social de- velopment was an instrument and approach to achieve broader development objectives. 15. Resources, staffing, and strengthening institutional capacity. One member noted that what was proposed appeared in some respects to go in the direction opposite that implied by OED with regard to resource needs and staffing. The OED review recommends increased use o f local institutional capacity yet the proposal includes external recruitment and appears to omit the use o f local institutions, including c i v i l society. Management noted that they did include the use o f local expertise and underlined that there i s total agreement between O E D and Management. Some members voiced the view that n o significant additional budgetary resources are necessary to implement the proposed actions, since many were ongoing. Management noted that the re- source implications would be addressed in the regional strategies. One member ventured that perhaps instead o f more social development specialists, there may be a need to strengthen the social development skills o f more professional staff (for example economists, engineers). Others noted that additional and diverse s k i l l s in social development were needed. Others asked for a detailed costing o f the strategy when the business plan was submitted as well as the anticipated operational costs to clients. Management confirmed that it would return to C O D E with a paper including a business plan for mainstreaming. 16. Implementation in a matrix organization. Several members drew attention to the seri- ous organizational and managerial issues identified in the O E D report and cited the need to break "the silo mentality" and address the fragmentation across units while dealing with social devel- opment. Management noted that they had consulted widely with regions and sector boards and there had been success in working across various arms o f the Bank, for example between P R E M and Social Development o n developing and implementing support for Poverty and Social Impact Analysis by borrowing countries. Others noted that beyond deeper collaboration with D E C and PREM, management considers improving the alignment o f social development activities, includ- ing making fundamental changes to the matrix i f necessary. Management said it would continue to seek ways to operate more efficiently within a matrix system. Members supported Manage- ment's proposal that the business strategy be bottom-up, building o n the regional business plan, but cautioned that the approach remain flexible t o allow for regional, sub-regional, and country differences. As regards local institutions, Management said that it did intend to use local institu- tions to ensure that the Bank's initiatives were responsive to country and local conditions. 101 Annex H 17. OED Findings. Members were very appreciative o f OED's paper, which they f e l t was very u s e h l in clarifying the concepts in a complex area and believed that the findings and rec- ommendations laid out a good framework for a strategy. They commended the valuable and prac- tical lessons elicited in OED's analysis, which had a direct bearing for the Bank going forward. C O D E members especially welcomed the OED findings that social development was important for project success, and that "there was a strong positive association between including social development themes and project success". Some speakers, however, believed that more empirical evidence was required beyond the findings from the OED desk review to demonstrate the impact o f social development themes. They also noted the need to improve the l o w ratings for sustain- ability and institutional development for projects generally, including those with social develop- ment themes, noting that ratings o f less that 50 percent o f projects having substantial institutional impact was unacceptable. OED clarified that the analyses carried out by the study were statisti- cally robust. Management agreed with the need for more solid research o n the linkages between social and economic development o n the macro-level as w e l l as more data at the project level and added that consultation o n these issues would contribute towards country ownership. 18. A holistic approach. Members reiterated the need to take a holistic approach that inte- grated economic development and social development, which after all had a symbiotic relation- ship. Some cautioned that pursuit o f ambitious social development goals should not be "at the expense o f economic growth'' particularly since sustainable achievements as regards the former were predicated o n sustained economic progress. Others noted that social development was im- portant in and o f itself and believed that areas such as gender and social protection should feature more prominently in the strategy's goals. Management agreed and noted that the OED findings showed that attention to social development improved economic and development outcomes sig- nificantly and that there was n o trade-off between a focus o n good social development and a fo- cus o n economic development. 19. IFC and private sector development. Some members noted the importance o f private sector development and investment climate for good social development outcomes and suggested management explore possible linkages with IFC programs. Management informed the committee that I F C was in the process o f revisiting i t s o w n approach to social development and that mutual learning between the two agencies was on-going. Chander Mohan Vasudev, Chairman 103 Works Cited Works Cited Note: A full l i s t of all references can be found in the background paper, Social Development: A Review of the Literature. Cernea, Michael, and Ayse Kudat. 1997. “Social Assessments for Better Development, Case Studies in Russia and Central Asia.” Environmentally Sustainable Development Studies and Monographs Series N o . 16. Washington, D.C.: The World Bank. f Grounded Theory. Aldine de Glaser, Barney, and Anselm Straus. 1967. The Discovery o Gruyter. Glaser, Barney. 1992. The Basics of Grounded Theory Analysis. Mill Valley, California: Sociol- ogy Press. Lytle, Paula Franklin. “Preliminary Findings o n FYO1 Social Assessments.” PowerPoint presen- tation. World Bank Social Development Department website. Social Development Department. 2002. “Issues Paper for a World Bank Social Development Strategy.” Washington, D.C.: The World Bank. Social Development Department. 2002. “Uncovering the Operational Contribution o f Social Analysis: Lessons from Task Teams.” Washington, D.C.: The World Bank. Social Development Team ECSSD. 200 1. “Social Development in Europe and Central Asia Re- gion: Issues and Directions.” Washington, D.C.: The World Bank. Thin, Neil, Tony Good, and Rebecca Hodgson. 1997. “Social Development Policies, Results, and Learning: A Multi-Agency Review.” London: DFID SD-SCOPE Social Development Systems for Coordinated Poverty Eradication. Thin, Neil. 2000. “Characteristics o f DFID-FundedProjects.” SD SCOPE Paper no. 12, DFID SD SCOPE Project. Bath: Center for Development Studies, University o f Bath. Thin, Neil, Clare Lockhart, and Gil Yaron. 200 1. “Conceptualizing Socially Sustainable Devel- opment: A Policy Review.” Seminar o n Institutions for Sustainability. Washington, D.C.: The World Bank, December 5,200 1. The World Bank. 1992. Advancing Social Development, A World Bank Contribution to the Social Summit. Washington, D.C.: The World Bank. The World Bank. 2001. Draft OP 10.05, June 21,2001. Washington, D.C.: The W o r l d Bank. The World Bank. 2001. PRSP Sourcebook. Washington, D.C.: The W o r l d Bank. World Summit for Social Development. 1995. The Copenhagen Declaration and Program o f Ac- tion. Document o f the World Summit for Social Development, March 1995. Copenhagen: The United Nations. Bibliography 104 Bibliography This i s the complete bibliography o f OED publications dealing with social development themes com- piled for this report. The major OED evaluations used in the meta-evaluation, and background papers are indicated in boldface type. Campbell, J. Gabriel. 2000. Financing the Global Benejts of Forests: The Bank’s GEF Portfolio and the 1991 Forest Strategy. Washington, D.C.: The World Bank, Operations Evalua- tion Department. Candler, Wilfred, and Nalini Kumar. 1998. The Dairy Revolution. Washington, D.C.: The World Bank, Operations Evaluation Department. Gibbs, Christopher, Claudia Fumo, and Thomas Kuby. 1999. NGOs in World Bank- Supported Projects. Washington, D.C.: The W o r l d Bank, Operations Evaluation De- partment. Gopal, Gita. 1997. Integrating Gender in World Bank Assistance. Washington, D.C.: The W o r l d Bank, Operations Evaluation Department. Gopal, Gita. 2002. The Gender Dimension o f Bank Assistance, An Evaluation of Results. Washington, D.C.: The W o r l d Bank, Operations Evaluation Department. Gopal, Gita. 2003. Implementation of Operational Directive 4.20 on Indigenous Peoples: An Evaluation o f Results. Washington, D.C.: The W o r l d Bank, Operations Evaluation Department. Hanna, Nagy. 2000. “Analytical and Advisory Services for Comprehensive and Participatory Development.” O E D Working Paper N o 12. Washington, D.C.: The World Bank, Opera- tions Evaluation Department. Jayarajah, Carl, W i l l i a m Branson, and Binayak Sen. 1996. The Social Dimensions of Adjustment: World Bank Experience, I980-93. Washington, D.C.: The World Bank, Operations Evaluation Department. Jerve, Alf Morten, and Gunn H e l e n Softing. 2002. “The Organization o f the Social Devel- opment Dimension in M a j o r Aid Agencies: Lessons o f Relevance f o r the W o r l d Bank.” Bergen, Norway: The Christian Michelsen Institute. Johnston, Timothy, and Susan Stout. 1999. Development Effectiveness in Health, Nutrition, and Population: Lessons from World Bank Experience. Washington, D.C.: The World Bank, Operations Evaluation Department. Kreimer, Alcira, John Eriksson, Robert Muscat, Margaret Arnold, and Colin Scott. 1998. The World Bank’s Experience with Post-Conflict Reconstruction. Washington, D.C.: The W o r l d Bank, Operations Evaluation Department. Lele, Uma. 2000. The World Bank Forest Strategy: Striking the Right Balance. Washington, D.C.: The W o r l d Bank, Operations Evaluation Department. Plus special case stud- 105 Bibliography i e s published for 6 countries: Brazil, Cameroon, China, Costa Rica, India, and In- donesia. Little, Kristin S. Social Development: A Review of the Literature. 2002. Washington, D.C.: The World Bank, Operations Evaluation Department. Mackay, Keith, and Sulley Gariba. 2000. “The Role o f C i v i l Society in Assessing Public Sector Performance in Ghana.” O E D Workshop Proceedings. October 2000. Washington, D.C.: The World Bank, Operations Evaluation Department. Murphy, Josette. 1995. Gender Issues in WB Lending. Washington, D.C.: The World Bank, Operations Evaluation Department. Murphy, Josette. 1997. Mainstreaming Gender in WB Lending: An Update. Washington, D.C.: The World Bank, Operations Evaluation Department. Operations Evaluation Department. 199 1. “Forestry Development: A Review o f W o r l d Bank Ex- perience.” Precis N o 9. Washington, D.C.: The World Bank. Operations Evaluation Department. 199 1. Forestry Development: The World Bank Experience. Washington, D.C.: The World Bank. Operations Evaluation Department. 1992. “Natural Resource Management in Nepal.” Precis no 24. Washington, D.C.: The World Bank. Operations Evaluation Department. 1993 .“Early Experience with Involuntary Resettlement: Overview.” Washington, D.C.: The World Bank. Operations Evaluation Department. 1993. “Involuntary Resettlement.” Precis N o 52. Washing- ton, D.C.: The World Bank. Operations Evaluation Department. 1993. “Natural Resource Management in Bolivia.” Precis n o 44. Washington, D.C.: The World Bank. Operations Evaluation Department. 1994 “Community Development in Colombia.” Precis N o . 77. Washington, D.C.: The World Bank. Operations Evaluation Department. 1994. Conditional Lending Experience in World Bank- Financed Forestry Projects. Washington, D.C.: The World Bank. Operations Evaluation Department. 1994. “Gender Issues in WB Lending.” Precis N o 79. Wash- ington, D.C.: The World Bank. Operations Evaluation Department. 1994. “Revitalizing Tree Crops: Rubber in Thailand.” Precis n o 66. Washington, D.C.: The W o r l d Bank. Operations Evaluation Department. 1994. “Two Decades o f Lending for Urban Development.” Precis no 81. Washington, D.C.: The World Bank. Bibliography 106 Operations Evaluation Department. 1996. “Paddy Irrigation and Water Management in SE Asia.” Precis n o 126. Washington, D.C.: The World Bank. Operations Evaluation Department. 1996. “World Bank Lending for Large Dams: A Preliminary Review o f Impacts.” Precis N o 125. Washington, D.C.: The World Bank. Operations Evaluation Department. 1996. The World Bank’s Experience with Large Dams: A Preliminary Review of Impacts. Washington, D.C.: The World Bank. Operations Evaluation Department. 1997. “Grassroots Organizations for Resource Management in the Philippines.” Precis no 153. Washington, D.C.: The World Bank. Operations Evaluation Department. 1997 “Participatory Evaluation for Rural Development in S r i Lanka.” Precis N o 138. Washington, D.C.: The World Bank. Operations Evaluation Department. 1998. “Post-Conflict Reconstruction: Bosnia & Herzegovina Case Study Summary.” Precis N o 170. Washington, D.C.: The World Bank. Operations Evaluation Department. 1998. “Post-Conflict Reconstruction Overview.” Precis N o 169. Washington, D.C.: The World Bank. Operations Evaluation Department. 1998. “Post-Conflict Reconstruction: Uganda Case Sum- mary.” Precis N o 171. Washington, D.C.: The World Bank. Operations Evaluation Department. 1999. “Grassroots Pastoral Organizations in Mauritania.” Precis no 181. Washington, D.C.: The World Bank. Operations Evaluation Department. 1999. “Aid Coordination and Post-Conflict Reconstruction: The West Bank and Gaza Experience.” Precis N o 185. Washington, D.C.: The World Bank. Operations Evaluation Department. 1999. “NGOs in World Bank-Supported Projects: A Re- view.” Precis N o 177. Washington, D.C.: The World Bank.. Operations Evaluation Department. 1999. “Bangladesh: Progress Through Partnership.” Precis N o 191. Washington, D.C.: The World Bank. Operations Evaluation Department. 2000. “Evaluating Gender and Development at the World Bank.” Precis no 200. Washington, D.C.: The World Bank. Operations Evaluation Department. 2000. “Mainstreaming Participation in Development.” OED Working Paper N o 10. Washington, D.C.: The World Bank. Operations Evaluation Department . 200 1. Recent Experience w i t h Involuntary Resettlement: Brazil-Itaparica. Washington, D.C.: The World Bank. Operations Evaluation Department. 200 1. Recent Experience with Involuntary Resettlement: China-Shuikou and Yantan. Washington, D.C.: The World Bank. 107 Bibliography Operations Evaluation Department. 200 1. Recent Experience with Involuntary Resettlement: I n - dia-Upper Krishna. Washington, D.C.: The World Bank. Operations Evaluation Department. 200 1. Recent Experience with Involuntary Resettlement: Thailand-Pak Mun. Washington, D.C.: The World Bank. Operations Evaluation Department. 200 1. Recent Experience with Involuntary Resettlement: Togo-Nangbeto. Washington, D.C.: The World Bank. Operations Evaluation Department. 2002. The Meta-Evaluation: An Analysis o f OED Study Findings on Social Development. Washington, D.C.: The W o r l d Bank. Operations Evaluation Department. 2003. “OED Surveys o f Social Development Staff, Task Managers, and Country Directors: Summary o f Results.” Washington, D.C.: The W o r l d Bank, Operations Evaluation Department. Parker, Ronald, and Tauno Skytta. 2000. Rural Water Projects: Lessons from OED Evalua- tions. OED Working Paper Series no 3. Washington, D.C.: The W o r l d Bank, Opera- tions Evaluation Department. Parker, Ronald. 2001. Protecting and Enhancing Cultural Properties: A Review of World Bank Experience. Washington, D.C.: The W o r l d Bank, Operations Evaluation De- partment. Picciotto, Robert, W a r r e n Van Wicklin, and E d w a r d Rice. 2001. Involuntary Resettlement, Comparative Perspectives. Washington, D.C.: The W o r l d Bank, Operations Evalua- tion Department. Ridker, Ronald. 1994. The World Bank’s Role in Human Resource Development in Sub-Saharan Africa: Education, Training, and Technical Assistance. Washington, D.C. The World Bank, Operations Evaluation Department. Skytta, Tauno, Alcira Kreimer, and Roy Gilbert. 1994. Twenty Years of Lending for Urban De- velopment 1972-92. Washington, D.C. : The World Bank, Operations Evzluation De- partment. V a n Wicklin, Warren. 2000. OED Participation Process Review. Washington, D.C.: The W o r l d Bank, Operations Evaluation Department.