51214 The Challenge of Governance in Fragile and Conflict-Affected Countries: Supporting Leadership Development A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF Fragile and Conflict-Affected Countries Group Operations Policy and Country Services (OPCS) March 2009 ABBREVIATIONS AND ACRONYMS PCF Post Conflict Fund LICUS Low Income Countries Under Stress LICUS TF Low Income Countries Under Stress Implementation Trust Fund RRA Rapid Results Approach RRI Rapid Results Initiative BLTP Burundi Leadership Training Program CAD Capacity-Building Assistance and Development LCCNR Leadership and Communication Capacity for National Renewal LED Building Leadership Capacity for Economic Development GLCDP Governance and Leadership Capacity Development Program WBI World Bank Institute CAR Central African Republic PRSP Poverty Reduction Strategy Paper TTL Task Team Leader Table of contents I. EXECUTIVE SUMMARY ............................................................................................ 1 OVERVIEW OF LEADERSHIP DEVELOPMENT INTERVENTIONS................................................................................ 2 SUMMARY OF FINDINGS ..................................................................................................................................... 3 SUMMARY OF RECOMMENDATIONS ..................................................................................................................... 5 II. BACKGROUND .......................................................................................................... 6 A. REVIEW PURPOSE ............................................................................................................................... 6 B. LEADERSHIP DEVELOPMENT IN FRAGILE AND CONFLICT-AFFECTED COUNTRIES .................................... 6 C. RESEARCH DESIGN AND METHODOLOGY.............................................................................................. 8 III. FINDINGS ................................................................................................................. 11 A. CONTEXT AND INTERVENTION GOALS ................................................................................................. 11 B. INTERVENTION STRATEGY AND DESIGN .............................................................................................. 15 C. IMPLEMENTATION ARRANGEMENTS..................................................................................................... 19 D. MONITORING AND EVALUATION .......................................................................................................... 22 E. INTERVENTION IMPACT ...................................................................................................................... 23 IV. CONCLUSION, SUMMARY OF FINDINGS, AND SUMMARY OF RECOMMENDATIONS ............................................................................................. 25 FINDINGS ........................................................................................................................................................ 25 RECOMMENDATIONS ........................................................................................................................................ 27 This review was prepared by Susanna P. Campbell and Kelly M. Hannum as consultants to the World Bank March 2009 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 I. Executive Summary Qualities of effective leadership outlined in the literature can be broadly organized to include the ability to set direction, create alignment, and maintain commitment. These three leadership tasks roughly correspond with three dimensions articulated by an informal World Bank working group; Vision, Competence, and Integrity. Leadership development always takes place within a broader context; the specific challenges faced by leaders in fragile and conflict-affected countries include: vague and shifting leadership expectations, multiple and significant priorities to be addressed, lack of shared perspectives, weak state-society relations, and a culture of mistrust and myopia. In this review we examined the eight leadership development interventions funded by the PCF and LICUS-TF from 2000-2008. The sources of information included all available documentation about the interventions, over 100 interviews conducted with individuals representing a variety of stakeholder groups, and field visits to Burundi and Timor-Leste. The effectiveness of leadership development is defined by and dependent on different perspectives regarding the intentions and expectations of interventions. These perspectives, and the efforts guided by them, may or may not be articulated or be in agreement with each other. Such is the case with the interventions under review here. The organization of the evidence gathered about the eight interventions is guided by a framework of leadership development including the following five phases: analyzing context & identifying desired goals, clarifying intervention design to achieve desired goals, detailed design & intervention development, intervention implementation, and monitoring & evaluation. The eight leadership development interventions have been, for the most part, very well received by participants and observers. At a minimum, they have increased participants' awareness of skills, behaviors, and attitudes that could help them to strengthen the institutions of state and society and prevent the emergence of conflict. At a maximum, they supported the application of these new skills, behaviors, and attitudes in situations, groups, and organizations playing a critical role in moving the war-to-peace transition forward. At the individual level, the impact of the interventions was greatest when participants had the motivation, knowledge, opportunity, and authority to apply what they learned. At the group and organizational levels, the impact of an intervention seemed to be greatest when it trained a critical mass of people who were in constant communication with one another and could reinforce the new skills, attitudes, and behaviors. Table 1 on the following page provides summary information about the interventions. A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Table 1. Overview of Leadership Development Interventions Central African Central African Timor-Leste ­ Capacity- Timor-Leste ­ Building Timor-Leste ­ Leadership Burundi Leadership Burundi Governance Tajikistan ­ Republic - Republic ­ 2005 Building Assistance and Leadership Capacity for and Communication Training Program (BLTP) and Leadership Leadership Seminar 2004 Leadership Leadership Development (CAD) Economic Development Capacity for National Capacity on Strategic Seminar Workshop (LED) Renewal (LCCNR) Development Directions for Program Development Project # LICUS 15 LICUS 15-b PCF 171 PCF 258 LICUS 51 PCF 208 LICUS 52 PCF 350 Dates 08/04 ­ 02/05 09/05 ­ 06/06 08/01 ­ 06/02 06/04 ­ 01/07 11/06 ­ 06/08 12/02 ­ 07/04 01/07 ­ 01/09 03/07 ­ 04/07 Funding $129,380 $123,000 $249,993 $250,000 $1,078,100 $993,279 $423,800 Leadership Build capacity of senior Strengthen capacity Enhance participants' skills Build trust among urban youth Build leadership capacities of Strengthen the ability of the Improve the Enhance the capacity of Objective(s) leadership to manage of leadership to and techniques as effective groups and youth-at-risk and state and non-state actors Burundian Government to confidence, the Government of more effectively and articulate priorities national leaders and to engage them in policy with a view to strengthening develop, through a accountability, and Tajikistan to make carry out the short-term and deliver on short- managers; unite a broad dialogues with government trust and thus an ability to participatory approach ultimately the informed decisions on reforms necessary to term goals in the range of Timorese leadership and community leaders; and work more cooperatively, involving civil society, a vision effectiveness of the strategic directions ensure stability and context of the PRSP and society in building an anti- support young people through more collaboratively and for economic recovery as well national leaders and for development to continued engagement framework. corruption strategy; and help peer-to-peer non-formal therefore more effectively; and as a strategic and socio- through them, the achieve sustainable with the international participants acquire the tools training sessions to strengthen enhance the communication economic recovery plan for accountability of growth. community and lay the to understand and resolve self-confidence and a sense of skills and information-sharing the transitional period that thematic and groundwork for broader conflict in a non-political identity, and to value positive abilities not only of individuals would establish the conditions geographic teams. leadership-building context so that they may play expression of cooperation and but of state institutions with for sustainable economic activities following the an important role as mediators teamwork. civil society and the public development. transition. and conflict-managers. more broadly. Description of A two-day seminar A two-day seminar Two five-day leadership Multiple interventions Three three-day leadership Three six-day workshops in High-level leadership Two-and-a ­half-day training retreats plus including a 3 day youth retreats; six two-day follow-up Ngozi; retreat.; seminar mentoring/coaching retreat, two week leadership workshops; three three-day Seven follow-up workshops Training of Trainers; training; multi-skill training networking/training session; and distribution of small mentoring/coaching; and six Coaching for 60 Rapid grants. small sub-grants. Results Initiatives. Instructional Peer-to-Peer Peer-to-Peer; Transformational leadership; Strategy/Plan Development; Transformational leadership; Transformational leadership; Peer-to-Peer; Peer-to-Peer; Method(s) Rapid Results Anti-corruption Exposure; Advocacy; Conflict resolution exposure; Conflict resolution exposure; Rapid Results Method Case studies; Method Exposure; Conflict Resolution Exposure; Multi-skill exposure SIM SOC; SIM SOC; Exposure; Anti-corruption Use of Rapid Strategy/Plan Development Network Follow-up workshops; Follow-up workshops; Use of Rapid Results Results Method in Development projects; Network; Exposure Communication/PR efforts Method in projects projects; Communication/PR efforts Mentoring/coaching; Anti-corruption Development projects Exposure Number of 37 60 15 137 109 95 150 20 Participants* Participant Government and Civil Government and Government and Civil Society Youth & Govt. Leadership Government and Civil Society Government and Civil Society Government and Civil Government Group(s) Society Leadership Civil Society Leadership (Current) (Current and Emergent) Leadership (Current and Leadership (Current and Society Leadership (Current) Leadership Emergent) Emergent) Representatives (Current) Documented Tool exposure; Tool exposure; Enabled discussion among Development of Youth Policy; Strengthened capacity and Tool exposure; Tool exposure; Reform discussion prior Outcome(s) Commitment to common Completion of 2 key leaders during transitions. Strengthening Youth Councils confidence of individuals with Tool use; Supported significant to donor roundtable document (Electoral projects; the ability and opportunity to Catalyzed breakthrough in decrease in the time it Participants reported very peace process; took to pay Burundian Code of Conduct) Commitment to favorable impression, but apply conflict resolution and light planning skills to their life Network teachers; common document there was no monitoring or (Guiding Principles evaluation of this. and work. Implementation of 60 for Achieving Rapid Results Results) Initiatives. *The final number of participants was difficult to locate in many cases. The numbers provided here are estimates based on documentation. 2 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Summary of Findings A. Context and Intervention Goals Finding A.1: Each of the intervention goals were relatively vague, leaving room for different interpretations based on different perspectives. While the intention of the interventions was clear, the link between goals, context, intervention activities and outcomes was not clear. Only two project teams, the BLTP and LCCNR, revisited goals to investigate whether or not they were being met, or whether the goal should be revised to reflect new learning by the team or changes in the context. In the case of Tajikistan, the intervention lasted only a few days, leaving little time or need for revisiting the goal. Finding A.2: Creating and maintaining alignment between the intervention goals, design, and the context is particularly important for leadership interventions in conflict-affected states. None of the eight leadership development interventions established and maintained alignment between their goal, design, and context throughout the intervention. When the alignment was lost, the effectiveness and influence of the intervention was often reduced. Finding A.3: Leadership development in volatile and conflict-affected contexts is a politically-sensitive activity. This sensitivity should be addressed throughout the intervention. Finding A.4: The leadership interventions reviewed here lacked articulated links between their «big picture» strategic goal and their short-term implementation goals. The failure to regularly examine whether and how day-to-day activities contribute to the interventions larger strategic goal, reduces the likelihood that the intervention will achieve strategic goals. B. Intervention Strategy and Design Finding B.1: It is essential to have a shared understanding among the design, implementation, and management team of the leadership challenge to be addressed, the type of change desired, the method to be employed to enable that change, the process by which legitimacy and buy-in will be maintained with all critical stakeholders, and how all of this will be monitored and evaluated.Having a balanced combination of internal and external perspectives can create value by blending deep contextual knowledge with knowledge of generalities. Finding B.2: The pre-implementation data gathering and buy-in process is critical to an effective leadership development strategy and the legitimacy of the approach to participants and other key stakeholders. Finding B.3: Intervention staff need legitimacy with the participating leaders and other key stakeholders, to create buy-in for the project, serve as the project's interloctor, and to deliver the content of the intervention. Leaders do not want to be trained by someone who they feel is beneath them. The individual qualities or characteristics perceived as legitimate are specific to the culture, contexts, and the personalities involved. Finding B.4: A single identity or mixed identity approach can both work well as long as the intent is clear among stakeholders and the approach is aligned with the overall goal for the intervention and is culturally appropriate. Finding B.5: Issues of learning transfer are best addressed in the design phase in order to incorporate components that will enable and encourage transfer. Factors that determine whether participants transfer what they learn include: motivation to do so; opportunity to do so; support for doing so; and knowledge or skills that can be transferred (i.e., knowing how to translate or adapt what happened in the intervention to their context). 3 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Finding B.6: There is no clear evidence suggesting which dispersion approach (i.e., hierarichal v. horizontal/distributed) is more effective. The strategy used to implement the approach appears to be more important than the approach itself. Dispersion can happen formally by establishing the expectation and training for participants to share what they have learned with others (e.g. putting them in coaching roles, training of trainer programs, etc.) or informally (e.g., hoping participants will model what they have learned and share it with others). Finding B.7: It is important to have connections to and feedback from a representative group of individuals and groups, so support and credibility are based on multiple sources. The tradeoff is that broad based support is more complex and thus requires more time to develop, and in some cases may be exceedingly difficult to obtain. For example, working among groups who do not agree and have very different perspectives can raise suspicion. On the other hand working exclusively with one group also has risks, namely when they lose power and influence, your efforts will be quickly replaced. C. Implementation Arrangements Finding C.1: When desigining leadership development projects, the Bank should be mindful of how its administrative procedures could inhibit the effective implementation of these types of interventions. Additionally, if the Bank wants to continue to implement leadership development projects, then it needs to ensure that its administrative procedures are adapted to the specific needs of these intervention designs. Finding C.2: When designing a leadership development intervention, the Bank needs to carefully consider the costs and benefits of the different implementation arrangements. The choice should inevitably be determined by both supply- and demand-driven considerations in the context. Finding C.3: Regardless of who (or what group) manages or implements the interventions, it is important that these individuals (separately or collectively) have knowledge of the following: context (culture, history, politics, etc. of the region), methodology (subject matter expertise), and process (how to get things done in country and via Bank systems). Staff must also be clear about roles, responsibilities, accountability, and decision-making authority. Making the goals, desired changes and process explicit is particularly important when different individuals and groups are responsible for different aspects of the intervention, and all the more so when an intervention is implemented in a dynamic context. Regular discussion and information- sharing among all relevant staff is paramount. Finding C.4: There are two aspects of leadership development interventions that make the lack of high-level oversight worrying: 1) leadership development interventions are potentially seen as more political than other Bank projects, and thus require oversight to ensure that they do not have an adverse effect on the country context or the reputation of the Bank; 2) leadership development interventions may be the only contact that many individuals (or groups) have with the Bank and thus can significantly influence the Bank's reputation with a large group of people who may not have direct experience with the Bank otherwise. D. Monitoring and Evaluation Finding D.1: If the purpose of the PCF, LICUS-TF, and the new Statebuilding and Peacebuilding Fund is to support new and innovative types of interventions, and replicate the ones that work, monitoring and evaluation is essential so information is documented and learning is created and shared. M&E is particularly true for leadership development interventions because of the complex nature of the work. Without adequate M&E, the value of this type of funding is greatly reduced and the risk of unintended consequences of these types of projects is heightened. Finding D.2: Intervention monitoring and evaluation were not given high priority by staff and M&E skills and expertise were lacking. 4 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Finding D.3: The absence of systematic and informed oversight, support, and accountability of PCF and LICUS-TF grants during the project implementation phase significantly reduced the knowledge available to staff implementing these leadership development interventions, as well as the requirements for project monitoring and reporting that might have increased the effectiveness of several interventions. E. Intervention Impact Finding E.1: At the individual level, we found that the impact of the interventions was greatest when participants had the motivation, knowledge, opportunity, and authority to apply what they learned. Finding E.2: At the group and organizational levels, the impact of an intervention seemed to be greatest when it trained a critical mass of people who worked together or were in frequent contact and could reinforce the new skills, attitudes, and behaviors. Summary of Recommendations While in most cases leadership development interventions cannot address all of the leadership challenges in fragile and conflict-affected states, it is essential that these and other challenges are examined openly and honestly, both in the initial design process and throughout its implementation. It is important that the Bank continue to work towards a shared understanding of leadership development that takes into account the challenges facing leadership in fragile and conflict-affected states, acknowledges the potential value of all of the methods and approaches used in the eight interventions, and continues to refine criteria and processes for the selection and evlauation of leadership development interventions. To ensure that interventions add the greatest value, leadership development goals and designs should be based on both supply- and demand-driven considerations. To maintain the alignment between the strategic goals, the intervention design, the intervention implementation, and the context, the intervention should ensure buy-in of all critical stakeholder groups (i.e., Government, Bank staff, potential participants, interested international staff, interested community and non-governmental leaders, etc...) and gather feedback and provide input to these groups as the project advances. This feedback and buy-in process helps to maintain the legitimacy and credibility of the intervention with multiple changing contexts and stakeholders. A feedback and buy-in process also helps provide the intervention team with essential feedback. Because the sensitivity of leadership development interventions requires a high level of monitoring of the political context, the staff and managers of these projects should be actively engaged in oversight of the projects. If the TTL is located outside the country, we recommend a member of the Country Office staff be linked with TTL and the contractor or consultants working on the project, and that they have a plan for ensuring responsibility and oversight. We recommend clear guidance be provided to Country Managers/Directors and TTLs outlining the particular type of oversight that should be given to leadership development interventions. We recommend that if the Bank wants to continue investing in leadership development, it extend its grant period to four or five years, or allow for subsequent grants to the same project. We recommend the Bank develop criteria for selecting and funding leadership development interventions and allocate funds to document and review intervention processes and outcomes in order to learn from interventions. We recommend the Bank staff charged with the oversight, design, and implementation of leadership development activities are provided with information about relevant monitoring and evaluation methods. 5 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 II. Background A. Review Purpose The World Bank's Post Conflict Fund (PCF) and the Low Income Countries Under Stress Implementation Trust Fund (LICUS TF) have funded eight leadership development interventions in fragile and conflict- affected countries.1 In 2007, the PCF-LICUS-TF Committee recommended a review of these eight interventions to synthesize information and extract lessons learned from them. The purpose of this review is to understand the investment made in and the intended results of eight leadership development interventions funded by the PCF and LICUS-TF from 2000 to 2007; to identify common processes, challenges, and success factors in order to inform future Bank engagement in leadership capacity development in fragile and conflict-affected countries; and to provide information- and evidence-based suggestions to inform the Bank's future strategic and operational engagement in leadership capacity development in fragile and conflict-affected countries. B. Leadership Development in Fragile and Conflict-Affected Countries Plans for breaking the cycle of state fragility and violence have limited value without adequate leadership. While the core mission of the Bank is to provide financial and technical assistance often through interest loans, interest-free credits, and grants, the success of this assistance depends, in part, on effective leadership. This is Box 1: Senior Management Views "Development is a particularly true in fragile and conflict-affected situations tough business in the best of circumstances. In fragile states it is not only that much harder, but presents a where the challenges facing good governance and economic different type of challenge for which our traditional tool-kit development are greatest, and those charged with is ill equipped. Unfortunately, our primary institutional surmounting them often lack the necessary experience and tools ­ both national and multilateral ­ talk about innovation while stifling it with bureaucratic governance. incentives. Risk is only accepted if ventures succeed... We shouldn't retreat from risk, but mitigate and manage risks to the full extent possible in the clear knowledge that some projects According to Robert Zeollick, "Too often, the development will fail, but that without innovative and flexible community has treated states affected by fragility and conflict approaches none will succeed." simply as harder cases of development" (see Box 1 for more - Robert Zoellick, President, The World Bank Group - information).2 Breaking the cycle of fragility and violence, will January 8, 2009 require a different approach.3 While international actors have the potential to catalyze and support efforts to break this cycle, the sustainable formation and operation of effective institutions of state and society can only be achieved by its citizens. Leadership development interventions can play an important role in helping international and national actors to develop a shared understanding of what needs to be done to secure development4, building the relationships and trust necessary for effective governance, and strengthening the capacity of national leaders to demonstrate their leadership by delivering on their promises. Our review of data from four countries, including over 100 interviews with stakeholders, suggests that there is a strong desire and need for well-designed and implemented leadership development interventions in fragile and conflict-affected countries. Nonetheless, there was little agreement on how to create effective leadership in fragile and conflict-affected states, or a common concept of leadership development. While it is beyond the scope of this review to provide a comprehensive overview of the theory and practice of 1 The Post Conflict Fund (PCF) was created to enhance the World Bank's ability to support countries in transition from conflict to sustainable peace and economic growth. The Low Income Countries Under Stress Implementation Trust Fund (LICUS TF) finances integrated programs outlined in a country re-engagement note rather than discrete activities. The LICUS TF uses the same operational procedures as the PCF and is governed by an expanded PCF Committee and PCF and LICUS-TF are designed to complement each other. 2 Robert B. Zoellick, "Securing Development," United States Institute of Peace Passing the Baton Conference, January 8, 2009, p. 4. Accessed January 15, 2009 at http://www.effectivestates.org/Papers/zoellick.pdf. 3 Zoellick 2009. 4 Zoellick 2009. 6 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 leadership and leadership development, we briefly introduce some of the core issues and recommend additional reading in Annex L. Leadership development is a growing subset of capacity development. Fifty years ago, there were very few publications focused on leadership development; now there are thousands. Leadership cuts across concepts of authority, influence, integrity, capability, power, and wisdom, among others. As an academic topic, leadership is based on and studied from the perspective of many disciplines including psychology, sociology, management, education, and political science. There are trait theories, behavior theories, contingency theories, power-influence theories, transformational theories, and collaborative theories of leadership.5 The rapid expansion and interdisciplinary nature of the field has resulted in a wide, and often confusing, array of leadership theories and leadership development approaches. Accoring to the leadership development literature, effective leadership includes the ability to: set direction ­ including defining and vetting a vision among relevant individuals and groups; create alignment ­ including coordinating the knowledge and work of a collective in service of this broader direction/vision; and maintain commitment to this broader direction/vision ­ including through the expansion of the group's efforts to achieve collective goals, not just individual goals.6 These three qualities of effective leadership can be fulfilled, in both formal and informal ways, by individual leaders, collective leadership, or through a combination of both. They also correspond roughly to the three dimensions of effective leadership articulated by an informal World Bank working group: Vision, Competence, and Integrity (See Annex H for a complete description). The eight interventions reviewed here also align to some degree with these various dimensions of effective leadership (see Table 1 and the case studies in Annexes C-F for further information about each intervention). Cooperative leadership approaches - the CAD, LED, LCCNR, and BLTP were interventions lasting more than a year that focused on conflict resolution and communications content, aiming to improve individual skill sets and increase trusting behaviors between former adversaries and potential collaborators. By increasing their communication capacity, basic problem solving skills, and/or relationships among themselves, this type of intervention could help leaders to set direction, create alignment, and maintain their commitments across divergent stakeholder groups. Results-based and management approaches - the 2005 Leadership Workshop in the Central African Republic and the GLCDP in Burundi aimed to strengthen management and leadership skills, through exposure to the Rapid Results Approach (RRA). The 2005 Leadership Workshop in the Central African Republic lasted less than a year while GLCDP was funded for two years. 7 By increasing the capacity of leaders to manage their teams and deliver results, these leadership development interventions could help leaders to set direction, create alignment and maintain their commitments with regard to a specific result. Knowledge-exposure and strategy development workshops - the 2004 Leadership Seminar in the Central African Republic and the Tajikistan Leadership Seminar, which were funded for only a few months, used the peer-to-peer and other approached to support the development of new strategies and reforms. The 2004 Leadership Seminar in the Central African Republic was in preparation for the election while the Tajikistan Leadership Seminar was in preparation for a donor conference. By providing leaders with ideas about how to address challenges and a forum for developing a vision, these interventions could help leaders to individually and collectively set their direction and vision. In spite of the similarities between the concepts of effective leadership put forward in the literature, by the informal Bank working group, and in the eight cases under review, the Bank has not agreed on an overall 5 For more information about leadership theories see Hughes, Ginnett, & Curphy (2005) and Northouse (2008) 6 McCauley, C.D. & Van Velsor, E. (2004). Handbook of Leadership Development. 2nd Edition. Jossey-Bass. San Francisco. 7 The Rapid Results Approach is a set of management tools, processes, and skills that help leaders in organizations use a series of short-term projects to translate long-term goals into concrete actions, results, and impact. Small teams work on Rapid Results Initiatives, which are projects lasting 100 days. 7 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 definition of leadership or approach to leadership development. Not unlike the leadership development field itself, the Bank's understanding of leadership development is still emerging. As a result, the three qualities of effective leadership were not used by the World Bank to select or evaluate any of the eight interventions under review. Therefore, we did not examine interventions against them, or against another shared concept of effective leadership or leadership development (see Annex H for more information). Instead, we asked our interviewees in Timor-Leste and Burundi for their definition of effective leadership. Their response seems to align with the general understandings of effective leadership articulated in the literature and by the Bank's informal working group. They defined effective leadership as: listening to people that one is leading, Box 2: Leadership Challenges in Fragile and Conflict-Affected States and Situations Vague and shifting leadership expectations. Fragile and conflict-affected countries experience a great deal of cultural, political, and structural change. Frequently, there are a variety of perspectives about what constitutes "good leadership". These vague and shifting expectations increase the uncertainty that leaders have of their own position, and make it difficult to identify the appropriate population for leadership development. Multiple and significant priorities to be addressed. Fragile and conflict-affected countries face a multitude of critical priorities with a limited capacity to respond. The overwhelming number of competing choices can stagnate leadership processes. Lack of shared perspectives. Particularly in conflict-affected situations, there are highly different perspectives on what is needed and who is prepared to and can be trusted to lead. Weak state-society relations. The infrastructure of state and society through which leadership is exercised is often severely compromised in fragile and conflict-affected countries, governed by patrimonial and military culture rather than the liberal democratic culture that the international community and some leaders aim to create. Culture of mistrust and myopia. Particularly in conflict-affected situations, years of war often create a culture of mistrust and secrecy that inhibits leaders from developing and building a common approach to the country's problems. Decades of poverty create a culture of insufficiency, where individuals and groups are often intensely focused on their own interests and needs, not on the collective needs of society, inhibiting both leaders and society from finding and implementing solutions to collective problems. understanding and taking other perspectives into account, making difficult decisions, and delivering on promises. When we asked whether the interventions under review contributed to the achievement of this definition, they answered that the interventions may have contributed to this type of leadership, but that there were too many other leadership challenges in fragile and conflict-affected countries that prevented leaders from really embodying this definition of effective leadership. Box 2 contains a description of leadership challenges in fragile and conflict-affected states. Recommendations While in most cases leadership development interventions cannot address all of the leadership challenges in fragile and conflict-affected states, it is essential that these and other challenges are examined openly and honestly, both in the initial design process and throughout its implementation. In addition, it is important that the Bank continue to work towards a shared understanding of leadership development that takes into account these challenges, acknowledges the potential value of all of the methods and approaches used in the eight interventions, and continues to refine criteria and processes for the selection and evlauation of leadership development interventions. C. Research Design and Methodology This review examines the eight interventions individually and in comparison with each other. Our approach was not intended to confirm a theoretical model, but rather to identify common processes, challenges, and success factors of leadership development interventions. Our approach was not based on causal attribution, but rather on logical contribution. For reasons described briefly below and in more detail elsewhere8 we gathered primarily qualitative data. The research approached used is based, in part, on Grounded Theory.9 8 Craig and Hannum (2006). Experimental and Quasi-Experiential Evaluations. In Hannum, Martineau, & Reinelt Eds. The Handbook of Leadership Development Evaluation. Jossey-Bass Publications. 9 See Strauss and Corbin (2007). Basics of Qualitative Research: Techniques and Procedures for Grounded Theory. Sage Publications. 8 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 The challenges frequently faced by those evaluating leadership development interventions were important considerations in our design process. We share a list of these challenges as well as brief statement of the impact of that challenge on our review in the box below. Our review of the PCF and LICUS-TF investment in leadership development was designed with the challenges facing leadership evaluation and other constraints in mind, which are summarized in Box 3. The value of this review is primarily the gathering and integration of information across a wide variety of contexts and sources into a coherent framework in order to articulate success factors and identify patterns of supports and barriers. This team of researchers was created because of the experience and expertise that each member brought to the project. Ms. Campbell has studied the capacity of the international intervention to prevent violent conflict and build peace for the past 12 years. Dr. Hannum has over 15 years experience in leadership development across a variety of sectors, with a focus on the evaluation of leadership development efforts. A short biography of each researcher is available as Annex M. Both researchers were asked to allocate 60 days to complete this review. Box 3: Leadership Development Evaluation Challenges 1) Leadership and leadership development are abstract concepts and are not often well-articulated or understood at an operational level. In terms of this review, specific changes in leadership and progress towards a greater goal as a result, were often not articulated in intervention documentation. For that and other reasons a direct, quantitative comparison of interventions was neither possible nor desirable. 2) In fragile and conflict-affected situations there can be frequent and dramatic individual and contextual changes. Individuals are literally on the move both in terms of their positions as well as their physical location. These changes complicate the tracking of participants and other stakeholders in addition to complicating the chain of evidence linking intervention outcomes to impact and progress on goals. In terms of the impact on this review, context was not always explicitly addressed in the design and implementation phases and thus difficult to assess retrospectively and impossible to do adequately within the parameters of this review. 3) The benefits of leadership development are not likely to be evident immediately which requires a longitudinal approach of gathering and linking data across time to create a chain of impact. It is difficult to attribute measured changes to a single intervention, often because significant time typically must pass before leadership outcomes are measurable and multiple interventions and situational factors influencing leadership or leadership outcomes have occurred in the interim. In terms of the impact on this review, the logical sequencing of activities and accomplishments aligned with a goal was often not available which made measuring the chain of impact difficult in retrospect. For that reason and due to resource limitations, this review provides our analysis based on available data at this point in time. 4) The final challenge is that staff may not have the necessary skills, time, or motivation to effectively evaluate interventions creating an obvious challenge for a review of interventions. In terms of the impact on this review, there appeared to be no common concept of or guidance about evaluation and evaluations of interventions were often not present or were planned and implemented retrospectively making a standard review across interventions challenging. Our primary data sources for the review were existing documentation (See Annex G for more information) and through semi-structured interviews with participants, World Bank staff, observers, and other donors. We gathered data directly from stakeholders in order to obtain a variety of perspectives about the process and outcomes and to allow for flexibility in data collection; a copy of the interview protocol is available Annex A. Table 2 provides an overview of the how many individuals from the various stakeholder groups were interviewed as well as an indication of the countries in which the interventions examined took place. Table 2. Distribution of Interviewees by Stakeholder Group and Country* World Bank Staff Program Staff Participants Other Stakeholders Total Burundi 8 12 27 4 53 Timor-Leste 10 11 18 9 50 CAR 4 5 5 Tajikistan 2 2 Total 24 28 45 13 110 *In some cases the cell amount do not equal "total" values because interviewees may have represented multiple stakeholder groups or the best way to group the interviewee was unclear based on the interview. We conducted interviews during our field visits to Burundi and Timor-Leste, during our week-long visit to the Bank's HQ's office, and over the phone. The majority of interviews were conducted on-site in Burundi and Timor-Leste, which were selected for field visits because they represent the location of five of the eight 9 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 interventions. They also have the longest history of PCF- and LICUS-TF-funded leadership development interventions, and the greatest diversity of leadership development approaches. Our field visits were of approximately 10 days in each country. For each intervention, we gathered information about the context, the degree to which each of the eight interventions achieved its objectives, and investigated factors perceived as supporting or detracting from intervention effectiveness. To the extent possible, we compared the interventions to each other along several dimensions (e.g., objectives, context, instructional method, design, implementation, and outcomes), investigated key themes, highlighted common approaches, identified lessons learned, and suggested strategic and operational implications for future Bank engagement in leadership capacity development in fragile and conflict-affected countries. It was neither possible to gather full data about nor to control for the wide variety of factors that influenced each intervention. Leadership development interventions often involve personal and cultural change, and thus are complex and sensitive. This is especially true in fragile and conflict-affected countries where trust is typically low and risks are more pronounced. As a result, gathering accurate data from various stakeholders requires a level of trust and understanding that is difficult for researchers from outside that context to achieve during a short time-period. The data on which we base our findings is the best possible information available to us, but is insufficient for an in-depth evaluation of each intervention. Consequently, our analysis focuses on the core lessons learned across all eight interventions in terms of impact; alignment with the context; design, monitoring, and evaluation; and implementation arrangements. It is critical to note that the development of more rigorous monitoring and evaluation functions (both within and across interventions) would aide future efforts to examine the process and outcomes of leadership development in fragile and conflict-affected situations. Gathering detailed information about interventions occurring, in some cases, years ago produced substantial challenges; primarily in locating and accessing stakeholders and gathering accurate data about events in the distant past (about which memories and documentation may not have been complete or accurate). In order to guide our process, we outlined the core elements of a framework for leadership development interventions. The framework is based on the ADDIE instructional development model, Bloom's Taxonomy, the Organizational Elements Model, the CIPP model, and the D6 model10. The framework, which is more fully described in Annex K, provides an overview of the phases related to leadership development interventions and is informed by decades of research from the fields of organizational development, instructional design, educational psychology, and program evaluation. Elements of a Leadership Development Framework Analyzing Context and Identifying Desired Goals Clarifying Intervention Strategy To Achieve Desired Goals Detailed Design & Intervention Development Intervention Implementation Monitoring and Evaluation While the phases are organized in a linear fashion, progress through the phases is iterative and recursive. For example, as new information becomes available it is necessary to revisit phases to maintain integrity and alignment. Within each phase there are a wide variety of options regarding how information is gathered and documented, and who is involved in the process, all of which are guided by assumptions about the process. For example, the framework may be applied with the assumption that the process will be a "learning by doing" or rapid prototyping approach in which information is gathered and used in quick succession. Alternatively, the framework can be applied with an assumption of a defined causal, or 10 Resources describing each of these inputs are included in the "Further Reading" annex. 10 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 predictive, approach in which expectations are clearly articulated at the onset of the work and information is gathered to determine the legitimacy of assumptions and document impact. No matter the approach taken and the assumptions underlying that approach, the process of doing work that could be categorized into the five phases, though perhaps by other names and in different forms, is critical. III. Findings This section contains the findings and recommendations from our review of the eight leadership development interventions supported by the PCF and the LICUS-TF. It is organized according to the elements of leadership development framework described above. The goals of each intervention are summarized in Box 4. A. Context and intervention goals The eight leadership development interventions under review occurred in four fragile countries, three of which (Burundi, Central African Republic, and Timor-Leste) were conflict-affected at the time of the interventions. For the World Bank, states are fragile when their Country Policy and Institutional Assessment Box 4: Summary of Intervention Goals Central African Republic 2004 Leadership Seminar - Build capacity of senior leadership to manage more effectively and carry out the short-term reforms necessary to ensure stability and continued engagement with the international community and lay the groundwork for broader leadership- building activities following the transition. Central African Republic 2005 Leadership Workshop - Strengthen capacity of leadership to articulate priorities and deliver on short-term goals in the context of the PRSP framework. Timor-Leste ­ Capacity-Building Assistance and Development (CAD) - Enhance participants' skills and techniques as effective national leaders and managers; unite a broad range of Timorese leadership and society in building an anti-corruption strategy; and help participants acquire the tools to understand and resolve conflict in a non-political context so that they may play an important role as mediators and conflict-managers. Timor-Leste ­ Building Leadership Capacity for Economic Development (LED) - Build trust among urban youth groups and youth-at-risk and to engage them in policy dialogues with government and community leaders; and support young people through peer-to-peer non-formal training sessions to strengthen self-confidence and a sense of identity, and to value positive expression of cooperation and teamwork. Timor-Leste ­ Leadership and Communication Capacity for National Renewal (LCCNR) - Build leadership capacities of state and non-state actors with a view to strengthening trust and thus an ability to work more cooperatively, more collaboratively and therefore more effectively; and enhance the communication skills and information-sharing abilities not only of individuals but of state institutions with civil society and the public more broadly. Burundi Leadership Training Program (BLTP) - Strengthen the ability of the Burundian Government to develop, through a participatory approach involving civil society, a vision for economic recovery as well as a strategic and socio-economic recovery plan for the transitional period that would establish the conditions for sustainable economic development. Burundi Governance and Leadership Capacity Development Program (GLCDP) - Improve the confidence, accountability, and ultimately the effectiveness of national leaders and through them, the accountability of thematic and geographic teams. Tajikistan ­ Leadership Seminar on Strategic Directions for Development - Enhance the capacity of the Government of Tajikistan to make informed decisions on the strategic directions for development to achieve sustainable growth. (CPIA) is 3.2 or lower. Fragile states are recognized by having weak state policies and institutions, "making them vulnerable in their capacity to deliver services to their citizens, to control corruption, or to provide for sufficient voice and accountability."11 Partly because of their fragility, these states "face risks of conflict and political instability" and are also low-income countries.12 The goal of each of the eight leadership development interventions reviewed was to strengthen the country's leaders' capacity to break the destructive cycle of weak statehood, poverty, and instability.13 11 Fragile States: Good Practice in Country Assistance Strategies, Operations Policy and Country Services, December 19, 2005, p. 5. Accessed on January 21, 2008 at http://web.worldbank.org/WBSITE/EXTERNAL/PROJECTS/STRATEGIES/EXTLICUS/0,,contentMDK:21392462~pagePK:64171531~piPK:6417150 7~theSitePK:511778,00.html. 12 Ibid. 13 Robert B. Zoellick, "Securing Development," United States Institute of Peace Passing the Baton Conference, January 8, 2009. http://www.effectivestates.org/Papers/zoellick.pdf. 11 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Intervention goals create a distal focal point by which to guide an intervention and maintain clarity among stakeholders in the midst of contextual changes. The intervention goals of these eight leadership development interventions are summarized in Box 4 (See the table in the Executive Summary for full overview of the projects and case studies in Annex for detailed description). These interventions were all intended to develop leadership in a fragile and conflict-affected country, however they vared in the type and degree of change in leadership that they sought (see further discussion of intervention design below and in the case studies in the Annex). The interventions of shortest duration were those in CAR and Tajikistan, which aimed to expose leaders to different leadership and thematic approaches through presentations by peer leaders from other countries and contexts. The three interventions in Timor-Leste (CAD, LED, and LCCNR) and the first Burundi intervention (BLTP) aimed to transform the way current and emergent leaders related to their former enemies and new collaborators by helping them to empathize with and communicate more constructively with one another. The second Burundi intervention (GLCDP) aimed to build the capacity of leaders to deliver results, and demonstrate that their leadership could provide real change. Finding A.1 Each of the intervention goals were relatively vague, leaving room for different interpretations based on different perspectives. While the intention of the interventions was clear, the link between goals, context, intervention activities and outcomes was not clear. Only two project teams, the BLTP and LCCNR, revisited goals to investigate whether or not they were being met, or whether the goal should be revised to reflect new learning by the team or changes in the context. In the case of Tajikistan, the intervention lasted only a few days, leaving little time or need for revisiting the goal. The literature on leadership development interventions and the literature on intervention in conflict-affected states strongly emphasize the importance of alignment between the intervention goal and the context. For leadership development interventions, this alignment is particularly important because leadership in and of itself is not typically the desired end point of a leadership development intervention. The effectiveness of leadership development interventions is defined by and contingent on context. Leadership development interventions implicitly or explicitly intend to build leadership for a purpose. The purpose, and the kind of leadership developed, should be determined by the context as well as a vision of the change stakeholders want to create in the context. Similarly, almost all evaluations and studies of peacebuilding and conflict prevention underscore the importance of fit between the context and the intervention. To impact such a complex environment, and adapt to its changing dynamics, detailed knowledge of the causes of the conflict, its manifestations, and of the relevant players is essential.14 Our review of the eight leadership training interventions supported by the PCF and LICUS-TF indicate that alignment between the intervention goal and the context is likely to be particularly important for the success of leadership development interventions in conflict-affected countries for several reasons, indicated below. Because these interventions work with influential and political individuals the Bank must pay careful attention to who they are working with and ensure that there is no perceived bias in the participant selection that could undermine the legitimacy of the effort, Bank staff, or the Bank. Because interventions aim to alter behavior, interventions must develop a goal, method, and approach that stakeholders find relevant, appropriate and desirable. 14 Susanna P. Campbell, "When Process Matters: The Potential Implications of Organizational Learning for Peacebuilding Success," Journal of Peacebuilding & Development forthcoming, (2008); Mary B. Anderson and Lara Olson, Confronting War: Critical Lessons for Peace Practitioners, (Cambridge: The Collaborative for Development Action, 2003); Saferworld International Alert, and FEWER, Conflict Sensitive Approaches to Development, Humanitarian Assistance and Peacebuilding: A Resource Pack, (London: 2004); Peter Uvin, The Influence of Aid in Situations of Violent Conflict, Informal Task Force on Conflict, Peace and Development Co-Operation, vol. (Paris: Development Assistance Committee, OECD, September 1999). 12 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Unlike infrastructure projects, leadership development uses individual-level change as the entry point for the achievement of its goal. Changes in individual behavior are often subtler and require an understanding of the culture in order to understand how to change behaviors, to detect changes, and to know whether or not, and how, the intervention needs to adjust its approach. Because the leadership context is, by definition, unstable in fragile and conflict-affected states, leadership development interventions must be prepared to adjust to frequent and sometimes dramatic changes during the design and implementation of the intervention. An accurate "read" of context and stakeholders at one point in time may not remain so for long. Leadership development interventions working at an individual-level are particularly at risk because of the high turnover in leadership and administrative positions. An understanding of context provides information about the value of and desire for leadership development and clues about what approaches are likely to be appropriate, relevant, and practical. In addition, behavior changes of influential individuals in complex and dynamic contexts entail risk that can only be accurately assessed by those with deep contextual knowledge. An understanding of context can also help identify and therefore reduce the potential risks of negative consequences. The goals of all five interventions in Burundi and Timor-Leste were initially aligned with the context, but this changed over time. The BLTP purposefully shifted away from the initial goal of supporting economic development to focus on increasing the trust and relationships between leaders who were important to a successful transition, and then leveraging bilateral funding to provide support to the integration of the National Defense Forces as that need became more critical. However, when the political leadership in Burundi changed in 2005, the BLTP staff lost their connections and influence with the highest levels of leadership, and has been unable to meet its goal of training the highest-level leadership in Burundi; they no longer have the level of access required. In Timor-Leste, the initial LCCNR goal was envisioned by one group and the design and implementation were carried out by another group. While the goal itself was not necessarily ill defined, the alignment of the goal with the strategy and design of the intervention was unclear, to us and to several participants. Partly as a result, the LCCNR seemed to fall short of its goal of group-, institutional-, and societal-level change. Participants reported to us that they used the skills and approaches that they acquired in the LCCNR in their own personal and work lives, but did not report higher-level changes. For the LED, the initial goal was revised in light of the unrest of 2006. The evidence remains unclear, but there is speculation that the intervention was used to increase the influence of a particular political party, raising questions about the reputational risks associated with leadership development and underscoring the importance of involving and/or consulting multiple stakeholder groups under politically savvy (but not politically aligned) oversight. Finding A.2: Creating and maintaining alignment between the intervention goals, design, and the context is particularly important for leadership interventions in conflict-affected states. None of the eight leadership development interventions established and maintained alignment between their goal, design, and context throughout the intervention. When the alignment was lost, the effectiveness and influence of the intervention was often reduced. Finding A.3: Leadership development in volatile and conflict-affected contexts is a politically-sensitive activity. This sensitivity should be addressed throughout the intervention. All of the interventions had ambitious strategic goals, but generally lacked an articulated link between the overall strategic goal and the short-term implementation goals. Practical, short-term goals are often limited in scope, but are tangible and measurable. Big picture goals require more time to achieve and are difficult 13 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 to measure, but are often of higher value than short-term goals. Ideally, an intervention has both and there is a strategic and explicit link between "big picture" goals and short-term goals and that link is reexamined in order to maintain congruence. Linking short-term goals with big picture goals requires strategic thinking and alignment between decision-making criteria, funding practices, management practices, and evaluation approaches. Finding A.4: The leadership interventions reviewed here lacked articulated links between their "big picture" strategic goal and their short-term implementation goals. The failure to regularly examine whether and how day-to-day activities contribute to the interventions larger strategic goal, reduces the likelihood that the intervention will achieve strategic goals. These findings point toward several important recommendations for improving future leadership development interventions. Interviews conducted in Timor-Leste and Burundi revealed a high demand for leadership development interventions in a breadth of sectors and levels (e.g., high-level government leadership and young community leaders), which creates both an opportunity and a challenge for the development of leadership development goals and designs. To ensure that interventions add the greatest value, leadership development goals and designs should be based on both supply- and demand-driven considerations. o Demand-driven considerations include (among others): a critical area of articulated need for leadership development; an understanding of how leadership development will address a larger need (i.e., a peaceful transition, economic development, etc.) in the context; perceptions of effective leadership in the country; openness of potential participants to leadership development and readiness for leadership development; levels of trust among potential stakeholder groups; level of leadership continuity versus turnover; and possible negative consequences of the intervention. o Supply-driven considerations include (among others): level of local credibility and legitimacy of the team delivering the leadership development intervention; willingness of the Bank Country Director or Manager to support and invest time in the intervention; opportunities for leadership development to increase the effectiveness of other Bank activities; availability of resources to monitor the progress of the leadership intervention and benefit from the relationships and exposure to different groups that it provides; whether the appropriately qualified personnel are available to design and implement the project; and whether sufficient funding is available and sustainable for long enough to achieve the desired outcomes. The assumption of a single or a stable context is a tenuous one. To maintain the alignment between the strategic goals, the intervention design, the intervention implementation, and the context, the intervention should ensure buy-in of all critical stakeholder groups (e.g., Government, Bank staff, potential participants, interested international staff, interested community and non- governmental leaders) and gather feedback and provide input to these groups as the project advances. This feedback and buy-in process helps to maintain the legitimacy and credibility of the intervention with multiple changing contexts and stakeholders. A feedback and buy-in process also helps provide the intervention team with essential feedback. When the BLTP's goals, intervention design, and the context were aligned, it engaged in a similar feedback process with critical stakeholder groups to beneficial ends (see BLTP case study in Annex C for details). 14 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 B. Intervention Strategy and Design Leadership development strategy and design link the intervention to the stated goal and frequently address the following questions: What are the leadership knowledge, skills, or attitudes that need to be developed in order to reach the goal? What stakeholders groups need to be included to reach the goal? What level(s) of those stakeholders groups would benefit from the intervention? Can those groups be effectively identified and engaged in a leadership development effort? What instructional content and process is most important to employ in order to reach the goal? What level of change is needed in order to have the desired impact? Is there a reasonably adequate time and other resources to reach that level of change? Is there access to the right resources to design and facilitate that type of change? What information will be important to gather and share in order to inform and document effectiveness? (For more information see Annex K: Leadership Development Framework) Table 3: Intervention Designs Central African Republic Timor-Leste Burundi Tajikistan Leadership Focus of Change Seminar 2004 Workshop 2005 CAD LED LCCNR BLTP GLCDP Seminar Individuals Groups Organizations Society Leadership Target Group of Change Seminar 2004 Workshop 2005 CAD LED LCCNR BLTP GLCDP Seminar Political Leadership Government Administration Civil Society Leadership Domain and Type of Leadership Seminar 2004 Workshop 2005 CAD LED LCCNR BLTP GLCDP Change Seminar Communication Skills Conflict Resolution Skills Problem Solving Skills Strategy Development and Implementation Skills Management Skills Trusting Behaviors New Attitude of Self- confidence and Responsibility The core elements of the intervention designs of the leadership interventions reviewed here are shown in Table 3 (Further data on intervention design is contained in the case studies in the Annexes C-F). For all eight interventions, the focus of change and the entry point for change was at the individual level, with the LED, the BLTP, and the GLCDP extending this to specific groups and organizations. In addition, all eight interventions sought to impact political leadership, with LED, LCCNR, and the BLTP also including civil society leadership. In spite of general similarities in the leadership challenges, target group and domain and type of change desired between the eight leadership development interventions under review, each intervention required a unique strategy and design. Comparison between two approaches that used the same methods and a similar training team ­ the BLTP in Burundi and the LCCNR in Timor-Leste ­ demonstrates the importance, and difficulty, of developing a leadership strategy and design that aligns with the multiple supply- and demand-driven contexts that influence the success of leadership development interventions. These interventions are briefly compared in Box 5. 15 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Box 5: Comparison of the BLTP and LCCNR In spite of the fact that the BLTP and LCCNR faced similar leaderhsip challenges, and had similar intervention goals, a similar training team, and used the same training methods, the alignment between their intervention strategy, design, the country context, the critical stakeholders, and the World Bank were fundamentally different. People who we interviewed in Timor-Leste and Burundi described a similar set of leadership challenges. Both countries were emerging from a prolonged period of violent conflict, where patrimonial and military leadership styles combined to create a tendency toward top-down hierarchical leadership, disempowering almost everyone but the highest leaders. In addition, the leadership of each country was charged with a job that they had never had before ­ governing a state. As a result, most interviewees argued, there was a real need for leadership and management training at all levels, but particularly at the top. The need for improved leadership and management in each country was pronounced and almost unanimously accepted. The BLTP was widely perceived as contributing to Burundi's successful post-conflict transition, particularly through its support for security sector reform (which was catalyzed but not funded by the PCF). Nonetheless, many of the factors that contributed to the success of the BLTP were not present in the LCCNR team or in Timor-Leste. The BLTP targeted a key need in Burundi by focusing on improving communication and relationships between former enemies during Burundi's five-year transition following the signing of the Arusha Peace Agreement. The LCCNR did not insert itself into a similar transitional process, possibly because there was not a comparable one in Timor-Leste at the time. The BLTP team remained the same from the design through the implementation and into follow-up activities. The LCCNR team was much more fragmented. The people who designed it were not the same people who implemented it. There was turnover in the implementation staff. The BLTP trainers were an integral part of the team, and the main trainer was based in the country. The LCCNR trainers, who were also the BLTP trainers, were asked to deliver a technical service by the World Bank, and did not engage as full partners in the design or implementation of the intervention, nor were they based in the country. The BLTP team maintained a general shared vision, or at least a shared understanding, of what they were trying to accomplish and how they would accomplish it. The LCCNR team had internal communication and leadership problems, had different understandings of the intervention strategy and design as well as a lack of role clarity (i.e. who was responsible and accountable for what). The BLTP was recipient executed, while the LCCNR was Bank executed. One consequence of the LCCNR's lack of alignment between the country context, goal, strategy, design, and management was a seemingly poor understanding among the participants of the logic behind the participant selection or the purpose of the intervention. The mixed group of participants (i.e, young and old, civil society and government, diverse political parties) selected for the LCCNR challenged Timor-Leste's traditional social hierarchy. This reduced the receptivity that many participants had to the training. In some cases, interviewees almost seemed insulted by the mixed-level approach. One interviewee commented that you would not ask a graduate student and a preschooler to go to the same course, so why was someone of his level asked to participant alongside someone at a much lower level? On the other hand, some interviewees (usually those at lower levels) indicated how helpful it was to bring together a diverse group of individuals in order to hear different perspectives and learn from each other. Comments such as these indicate that it was unclear how well the participants understood the purpose or benefits of the selection process used. The LCCNR participants we interviewed did not seem to have a clear understanding of the purpose of the intervention beyond their exposure to the material. While many of the participants we interviewed felt privileged to be involved, and greatly appreciated the professionalism of the trainers, it did not appear that the training resulted in outcomes beyond exposure to concepts and the application of some of the materials by individuals who had the capacity, authority, and motivation to do so. Leadership development is a highly sensitive and complex15 endeavor that requires context-specific knowledge and a team that is skilled in designing and implementing this type of intervention. Based on the lessons learned from individual cases and comparison between them, this review found several factors that are likely to be critical in determining the alignment of the leadership intervention strategy and design with the multiple supply- and demand-driven contexts. Finding B.1: It is essential to have a shared understanding among the design, implementation, and management team of the leadership challenge to be addressed, the type of change desired, the method to be employed to enable that change, the process by which legitimacy and buy-in will be maintained with all critical stakeholders, and how all of this will be monitored and evaluated.Having a balanced combination of internal and external perspectives can create value by blending deep contextual knowledge with knowledge of generalities. Finding B.2: The pre-implementation data gathering and buy-in process is criticical to an effective leadership development strategy and the legitimacy of the approach to participants and other key stakeholders. 15 See Westley, F., Zimmerman, B. & Patton, M.Q. (2006). Getting to Maybe: How the World has Changed. Random House for a description of the difference between complex and complicated. 16 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Finding B.3: Intervention staff need legitimacy with the participating leaders and other key stakeholders, to create buy-in for the project, serve as the project's interloctor, and to deliver the content of the intervention. Leaders do not want to be trained by someone who they feel is beneath them. The individual qualities or characteristics perceived as legitimate are specific to the culture, contexts, and the personalities involved. Finding B.4: A single identity or mixed identity approach can both work well as long as the intent is clear among stakeholders and the approach is aligned with the overall goal for the intervention and is culturally appropriate. Our examination of intervention designs and strategies also indicated that there were limited measures of effectiveness integrated into intervention designs and strategies. One reason for this is Box 6: Elements of successful leadership development in fragile and conflict-affected countries that while these types of measures are well- known in the literature on leadership and Transfer of learning refers to the ability of the participant to apply what s/he has peacebuilding/conflict resolution monitoring learned within his/her context (e.g. within their home, organization, and/or community). Ideally issues of learning transfer are explicitly addressed in the and evaluation, they are not widely understood design phase in order to incorporate components that will enable and or used within the World Bank. If measures of encourage transfer. Factors that determine whether participants transfer what they learn include: motivation to do so; opportunity to do so; support for doing effectiveness are not addressed at the design so; and knowledge or skills that could be transferred (i.e., knowing how to stage, then it is difficult for the project team to translate or adapt what happened in the intervention to their context). In the measure or understand the impact that it is interviewees we conducted each of these factors emerged as relevant. making, or correct for misalignment during the Dispersion of effect refers to the spreading of knowledge, skills and/or attitudes from participants in an intervention to those not directly participating in the implementation phase, when it is still possible intervention. The goals for leadership development interventions, such as for the correction to make a difference in the these, stretch beyond the group of participants directly served. In many cases, outcome of the interveniton. Box 6 describes starting with change at the individual level is seen as a way to achieve overarching goals in organizations, institutions, or the society at large. There three elements of successful leadership are often explicit or implicit expectations that participants will create larger development in fragile and conflict-affected change by becoming a role model of a "new" way to enact leadership, a source of information for others, or by making systemic or structural changes that countries. These elements are not unique to enable or support others to lead differently. Participants are seen as multipliers; leadership development in fragile and conflict- the conduit by which other individuals and systems are changed in order to affected countries but may be more important achieve greater reach and impact. because resources are often severely limited Sustainability refers to the continuity of impact, rather than the sustainability (or ongoing implementation) of the intervention. In short, have the changes and outcomes among broad groups of people catalyzed by the intervention been enduring? Sustainability is typically are often desired quickly. dependent on transfer and dispersion of impact. Transfer of learning refers to the ability of the participant to apply what s/he has learned within his/her context (e.g. within their home, organization, and/or community). In the case of BLTP, the high contextual knowledge of the staff (see BLTP case in Annex C for elaboration) may have subtly enhanced learning transfer because materials and dialogue may have included nuances (such as familiar examples and terms) that underscored the importance of and the way to apply the learning. In LCCNR, participants we interviewed who were in conflict situations as part of their work could more readily share stories of how they were able to transfer what they learned to their job and have a positive impact. To increase the potential learning transfer of an intervention, lessons such as these should be considered in the design phase. Finding B.5: Issues of learning transfer are best addressed in the design phase in order to incorporate components that will enable and encourage transfer. Factors that determine whether participants transfer what they learn include: motivation to do so; opportunity to do so; support for doing so; and knowledge or skills that can be transferred (i.e., knowing how to translate or adapt what happened in the intervention to their context). 17 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 The goals of the eight interventions reviewed suggest a desire to achieve outcomes beyond just the individuals directly engaged, or to disperse their effect. Dispersion of effect refers to the spreading of knowledge, skills and or attitudes from participants in an intervention to those not directly participating in the intervention. Several of the leadership development interventions that we reviewed, including CAD, LCCNR, and BLTP, trained high-level leaders in order to prompt a dispersion effect called cascading. They assumed that change at the high levels would lead to, or enable, changes at lower levels. This is based on a hierarchical view of leaderhsip. Interviewees in Burundi and Timor-Leste suggested that the highest political leaders should be the focus of leadership efforts because they were the source of both the problems and the solutions. Interviewees, especially in Timor-Leste, articulated a perception and provided some evidence that until those in power change there is little the "common person" can do. Other interviewees suggested another perspective on dispersion; that those in power were stuck in their ways and had little motivation or ability to change things. These leaders were victims of circumstance and the entire system needed to be addressed in order to effect any "real" change. This perspective on dispersion is one informed by a more horizontal, or distributed or collective, view of leadership, in which leadership is seen as a shared task among many. The assumption is that the system needs to be changed and that changing individuals embedded in a leadership system can change the system. In this case the leader would disperse his or her learning to other leaders (those with whom the tasks of leadership are shared) in an almost viral fashion. In this approach one may not select the highest ranking person, but rather the individual with the most connections and the greatest ability to spread what he or she learned as a result of leadership development. The two workshops in the Central African Republic and the workshop in Tajikistan did not directly support dispersion, while the BLTP, LCCNR, and LED built mechanisms for dispersion, all of which used a largely horizontal or distributed approach. The LED dispersion attempts seemed to have been successful, but the perceived monopolization by one political party is likely an unintended negative consequence of the intervention16. The BLTP supported dispersion through the training of trainers. In order for there to be a multiplier effect, as with other intervention expectations, selecting the "right" people (skills, reputation, opportunity) and providing the "right" training (relevant, actionable) was essential. In other cases, such as with the LCCNR, participants shared what they learned informally with colleagues. In one case, an interviewee kept the hand-out from the intervention visible so that, when asked, she created an opportunity to share what she learned with others. Her approach also reflects her savvy in terms of context. She did not think her colleagues would attend a training event (which was interpreted as something only lower level people did in some circles), but she knew that they would be curious. Finding B.6: There is no clear evidence suggesting which dispersion approach (i.e., hierarichal v. horizontal/distributed) is more effective. The strategy used to implement the approach appears to be more important than the approach itself. Dispersion can happen formally by establishing the expectation and training for participants to share what they have learned with others (e.g. putting them in coaching roles, training of trainer programs, etc.) or informally (e.g., hoping participants will model what they have learned and share it with others). Sustainability refers to the continuity of impact, rather than the sustainability (or ongoing implementation) of the intervention. The BLTP achieved the greatest degree of sustainability partly because it found additional sources of funding after the PCF grant ended. With these funds, the BLTP opened a local office; developed 16 We were unable to gather strong evidence about the extent to which the intervention was politicized. However, it did not seem that people thought it was intention of the Bank, rather that the Bank was unaware of the situation because politics are not always openly discussed or revealed. 18 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 a conflict resolution and communication module that was integrated into the training curriculum for the new National Defense Forces and the police; developed a community-based training program targeting areas that would experience high degrees of refugee and IDP return; and helped the UN to develop and implement a dialogue series with political parties. It was able to sustain its impact by increasing the number of people who could transfer and reinforce these skills. Nonetheless, the BLTP's approach to sustainability had a downside. The BLTP staff who had influence and access to the Burundian leadership during the political transition no longer had the same access to the newly elected leaders and were eschewed because of their association with the previous leadership. This situation is not uncommon and underscores the importance of having a connections to and feedback from a representative group of individuals and groups, so support and credibility are based on multiple sources. The tradeoff is that broad based support is more complex and thus requires more time to develop, and in some cases may be exceedingly difficult to obtain. For example, working among groups who do not agree and have very different perspectives can raise suspicion. On the other hand working exclusively with one group also has risks, namely when they lose power and influence, your efforts will quickly be replaced. Finding B.7: It is important to have connections to and feedback from a representative group of individuals and groups, so support and credibility are based on multiple sources. The tradeoff is that broad based support is more complex and thus requires more time to develop, and in some cases may be exceedingly difficult to obtain. For example, working among groups who do not agree and have very different perspectives can raise suspicion. On the other hand working exclusively with one group also has risks, namely when they lose power and influence, your efforts will be quickly replaced. C. Implementation arrangements As illustrated in Table 4, seven of the leadership development projects were implemented by World Bank staff. Only the BLTP was implemented solely by the recipient organization, which was a non-governmental organization (Woodrow Wilson International Center for Scholars ­ WWICS). All but one intervention, LED, employed external facilitators to design and implement at least part of the intervention, rather than relying on in-house capacity. Table 4: Implementation Arrangements Central African Republic Timor-Leste Burundi Tajikistan Seminar Workshop CAD LED LCCNR BLTP GLCDP Leadership 2004 2005 Seminar World Bank Executed Recipient Executed External Facilitators Navigating the complex bureaucracy of the Bank was particularly difficult in the case of the small grants work in Timor-Leste because headquarters' systems are not designed to manage or support this type of grant. The LCCNR also reported serious bureaucratic challenges with hiring consultants through the World Bank system. Both of these factors caused significant delays in project implementation. The time staff spent tracking down information and creating "work around" solutions would have been better spent elsewhere. Finding C.1: 19 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 When desigining leadership development projects, the Bank should be mindful of how its administrative procedures could inhibit the effective implementation of these types of interventions. Additionally, if the Bank wants to continue to implement leadership development projects, then it needs to ensure that its administrative procedures are adapted to the specific needs of these intervention designs. In spite of the administrative challenges experienced by the the LCCNR, and the other Bank-executed leadership interventions, there is no clear evidence that one implementation arrangement is better than another (i.e., Recipient-Executed v. Bank-Executed). The decision is more a matter of assessing suitability and balancing risk. When projects are Bank-Executed, the Bank may feel that it can have more control over the project and ensure that it is effective, and thus face less risk. But, the Bank does not necessarily have the in-house knowledge or time necessary for the implementation of effective leadership development interventions. As stated above, in seven of the interventions under review, the Bank had to bring in consultants to fill important capacity-gaps in its own staffing. It is therefore reliant on external capacity, whether it executes the project or not. The more signficant distinction, it seems, is between Government-Execution and Bank-Execution of leadership development interventions. But, because none of the eight interventions under review were government executed, it is impossible for us to draw any lessons learned from that arrangement. One could come up with significant arguments as to why an intervention should not be located within the government (e.g., it needs to have the appearance of some political neutrality if it is to engage leaders across the political spectrum), and why it should be located within the government (e.g., improved government leadership capacity in the long-term requires imbedding the skills and approaches within the government, so that they can be mainstreamed within the system). Finding C.2: When designing a leadership development intervention, the Bank needs to carefully consider the costs and benefits of the different implementation arrangements. The choice should inevitably be determined by both supply- and demand-driven considerations in the context (see earlier discussion). The most significant debates that we heard about Bank-Execution v. Recipient-Execution were in relation to the LCCNR. At least partly because of the political history of Timor-Leste, a few individuals felt the World Bank, and other international groups, did not have a positive reputation. This led several people involved in the intervention to argue that the project should be placed outside of the Bank. On the other hand, participants told us that one reason they attended the LCCNR was because of the prestige of the Bank. Based on our interviews, we feel that the positives and negatives of Bank execution in terms of the Bank's reputation balance out. The biggest problem with the implementation arrangements of the LCCNR that we observed were those related to internal issues of leadership, competition, information-sharing, and oversight, partly resulting from the hybrid implementation arrangement of the LCCNR (e.g., an sub- contracted trainer based outside of the country; consultants running the project; different people designing and implementing the project; located within the Bank offices and with a Bank TTL). These internal management and information-sharing issues truly seemed to threaten the credibility and effectiveness of the project. 20 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 Finding C.3: Regardless of who (or what group) manages or implements the interventions, it is important that these individuals (separately or collectively) have knowledge of the following: context (culture, history, politics, etc. of the region), methodology (subject matter expertise), and process (how to get things done in country and via Bank systems). Staff must also be clear about roles, responsibilities, accountability, and decision-making authority. Making the goals, desired changes and process explicit is particularly important when different individuals and groups are responsible for different aspects of the intervention, and all the more so when an intervention is implemented in a dynamic context. Regular discussion and information-sharing among all relevant staff is paramount.. The oversight of the eight leadership development interventions was unclear. For example, neither the Country Manager in Timor-Leste nor the new TTL seemed to have significant knowledge of the focus or outcomes of the current leadership development project, the LCCNR. While the Country Manager in Burundi was very interested in their current leadership development work, the GLCDP, she did not have a great deal of knowledge about its focus or outcomes. The TTL was engaged in the project, but was outside of the country and the level of supervision that she provided was unclear. Given the number of projects within the purview of a Country Manager/Director, it is not surprising that he or she would not be entirely familiar with a relatively small intervention. Nonetheless, particularly for Bank-Executed projects, the Bank needs to ensure that high-level oversight exists. Because, TTLs frequently transition it is important that as new TTLs are introduced they are informed about and become directly involved in these interventions, especially because of the political sensitivity of leadership development. Finding C.4: There are two aspects of leadership development interventions that make the lack of high-level oversight worrying: 1) leadership development interventions are potentially seen as more political than other Bank projects, and thus require oversight to ensure that they do not have an adverse effect on the country context or the reputation of the Bank; 2) leadership development interventions may be the only contact that many individuals (or groups) have with the Bank and thus can significantly influence the Bank's reputation with a large group of people who may not have direct experience with the Bank otherwise. Because the sensitivity of leadership development interventions requires a high level of monitoring of the political context, the fact many of the interventions' Task Team Leaders (TTLs) were located at headquarters or were not directly engaged in the project on a daily basis was of concern. The staff and managers of these projects should be actively engaged in oversight of the projects. If the TTL is located outside the country, we recommend a member of the Country Office staff should be linked with TTL and the contractor or consultants working on the project, and they have a plan for ensuring responsibility and oversight. In addition, we recommend clear guidance be provided to Country Managers/Directors and TTLs outlining the particular type of oversight that should be given to leadership development interventions. The one- to two-year timeframe of all eight leadership development grants is short in relation to the context and the nature of the work. The logical consequence of this approach is staff can focus on a small subset of leadership development (either in terms of content, participant groups, or type of intervention), implement a novel idea (just to try something out), or seek additional funding. If only a portion of leadership development is addressed it is likely the outcomes will also be reduced. In some cases, participants we interviewed wondered why there had not been more follow-up. They felt a worthwhile process began, but then fell flat, in some cases they seemed to feel shortchanged. The BLTP addressed this problem by raising additional funding, and received its most significant impacts under funding given by other donors, not by the World Bank. If subsequent funding is secured for an intervention, the Bank name often remains attached to it and 21 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 the Bank's reputation may be influenced, for good or for bad, by the outcomes of an intervention funded by another donor. As a result, we recommend if the Bank wants to continue investing in leadership development, it extend its grant period to four or five years, or allow for subsequent grants to the same project. D. Monitoring and evaluation Because of the complex relationships between the elements of the intervention and between those elements and the context, the alignment between the analysis of the context and need, the understanding of different stakeholder perspectives, the design of the intervention, the selection of the participants, the selection of the trainers, and the monitoring and evaluation of the intervention is more subject to error and change than in standard Bank programs.. As a result, monitoring and evaluation of leadership development projects is particularly important. Monitoring and evaluation planning and processes can help clarify the intentions among stakeholder groups and provide a mechanism for keeping an intervention on track and understanding the impact of an intervention. Finding D.1 If the purpose of the PCF, LICUS-TF, and the new Statebuilding and Peacebuilding Fund is to support new and innovative types of interventions, and replicate the ones that work, monitoring and evaluation is essential so information is documented and learning is created and shared. Without this, the value of this type of funding is greatly reduced and the risk of unintended consequences of these types of projects is heightened. Only three leadership development interventions under review were required to be externally evaluated: the BLTP external evaluation took place in 2004, the LCCNR evaluation took place in August 2008, and the GLCDP evaluation is intended for 2009. Most interventions used surveys to gather participants' reactions to the workshop, and, in the case of the BLTP and LCCNR, asked for feedback during the follow-up workshops. Nonetheless, they did not establish a method to monitor or evaluate the degree to which they achieved the desired change in attitude, behavior, or skills, or made progress towards overarching goals. In other words, all eight interventions lacked a system to evaluate or understand whether they achieved their desired results and impact, reducing the likelihood that these results and impact would be achieved. Evaluative processes that provide ongoing information about quality and efficiency about the intervention (formative) as well as processes that document the impact and value of the intervention (summative) are needed. Formative evaluation processes can enhance the positive impact of interventions and reduce negative consequences of them by providing a means of systematically gathering, interpreting, and communicating information in order to maintain implementation quality and fidelity (activities contribute towards the overall intent of the intervention) as well as stakeholder alignment (those involved in or benefiting from the intervention are informed and included in the process). Summative evaluation processes provide information about the impact of an intervention; in short, what was achieved as a result of the intervention and was it worth the effort as well as what was learned and how can it be applied? Our review found that intervention design, monitoring, and evaluation were not strategically linked with one another. The reasons for this included: a lack of M&E expertise on the team, lack of incentives to and resources for M&E (reducing the priority of and opportunity to conduct M&E), and lack of a sense of value of and purposes for M&E. A number of the staff we interviewed mentioned that M&E seemed nice to do, but did not see any real value to it; it seemed to be primarily a hindrance. Monitoring and evaluation systems have a reputation for being bureaucratic processes that do not enhance the impact of or learning from interventions; the purpose and value of M&E is not evident to all. Some of the participants we interviewed who provided suggestions as part of a post-intervention evaluation conducted expressed dissapointment the leadership development team had not responded to their feedback. The information gathered as part of M&E efforts should enhance decision-making either at the intervention or the system level. The above 22 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 statement illustrates that gathering information that is not used wastes resources and potentially damages relationships. Gathering data for which there is no plan is akin to asking someone his or her opinion and then ignoring it. Finding D.2: Intervention monitoring and evaluation were not given high priority by staff and M&E skills and expertise were lacking. The review process for LICUS-TF and PCF grants was eligibility-based rather than criteria-based, creating the risk that the Bank would support many different types of interventions but never build knowledge or expertise from funded interventions. This possibility is exacerbated by insufficient guidance and support (as well as leverage to require effective monitoring and reporting) for the design, monitoring, and evaluation of leadership development interventions within the HQ teams responsible for the oversight of these types of funds. Without this global review and the subsequent workshop, the lessons learned from the millions of dollars spent by the PCF and LICUS-TF on leadership development would potentially have been lost. Finding D.3: The absence of systematic and informed oversight, support, and accountability of PCF and LICUS-TF grants during the project implementation phase significantly reduced the knowledge available to staff implementing these leadership development interventions, as well as the requirements for project monitoring and reporting that might have increased the effectiveness of several interventions. Based on the above findings, we recommend the Bank develop criteria for selecting and funding leadership development interventions and allocate funds to document and review intervention processes and outcomes in order to learn from novel approaches. We also recommend the Bank staff charged with the oversight, design, and implementation of leadership development activities are provided with information about relevant monitoring and evaluation methods. E. Intervention Impact While not the sole purpose of monitoring and evaluation, those systems should eventually provide evidence of impact. Intervention impact can be conceived of in a wide variety of ways. For example, have there been positive and measurable changes in individual behavior, skills or attitudes? Have there been positive and measurable changes in relationships among groups? Have there been positive and measurable systemic changes? Is there measurable impact on societal indicators? In what way do these changes logically and/or empirically relate the intervention? In most of the leadership development interventions that we reviewed, data that would provide answers to these questions was unavailable. Planning, implementing and interpreting an evaluative process that would lead to this information is time consuming, can be expensive and it is not always worth the expense. The need for and value of information about intervention impact is something that can only be decided by the Bank. Below, we summarize our main findings about the impact of the leadership development interventions under review. At the individual level, we found the impact of the interventions was greatest when participants had the motivation, knowledge, opportunity, and authority to apply what they learned. For example in Timor-Leste, the LCCNR participants who held jobs or positions requiring them to negotiate regularly with other people found the skills, attitudes, and behaviors contained in the training to be very useful. They reported a sense of empowerment because they were more successful at understanding and resolving disputes after the workshops. On the other hand, participants who were not in positions of power or responsibility requiring them to negotiate regularly were not as able to apply what they learned. The interventions also had a positive impact on the participants who were subsequently trained as trainers or became staff members. These individuals were easily able to transfer their learning to other individuals 23 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 and groups, or had the opportunity to apply what they learned within their workplace. In addition, having these individuals as resources supported continued use of intervention lessons and created a mechanism by which knowledge could be dispersed to more individuals. Nonetheless, there were a number of individuals who showed little interest in the training content (in a few cases interviewees exhibited little interest in learning anything new) and, particularly in Timor-Leste, were unreceptive to being in a group with other participants whom they viewed as being of a lower social status. These participants had little motivation to apply the new skills, attitudes, and behaviors in their home or work environment. Finding E.1: At the individual level, we found that the impact of the interventions was greatest when participants had the motivation, knowledge, opportunity, and authority to apply what they learned. At the group and organizational levels, the impact of these interventions seemed to be greatest when they trained a critical mass of people who worked together, or were in frequent contact, and could reinforce the new skills, attitudes, and behaviors. The GLCDP and the BLTP trainings targeting specific security-sector institutions (under non-PCF funding) most directly capitalized on this idea. However, even with these groups, it was important that the participants had the motivation, opportunity, authority, and support to apply what they learned. For example, evidence from the GLCDP in Burundi demonstrates the importance of a champion who believed in the RRA method and had the power to apply it. Another important success factor was the articulation of projects that could be implemented within the scope of available resources (i.e. without additional resources). A result continually cited as a success ­ the reduction of time to pay teachers' salaries from one year to three months ­ possessed both of these traits (see case studies in Annex for more details). Two of the interventions, the BLTP and LCCNR, established a network for the participants to help them continue to strengthen their relationships with one another and the skills, attitudes, and behaviors presented in the training. These networks, however, never delivered the expected results and participation slowly dissipated. Neither the participants nor the staff prioritized the network because its added value and purpose were unclear. In Burundi, communication between the network members also relied largely on electronic media, to which people are not accustomed. Finding E.2: At the group and organizational levels, the impact of an intervention seemed to be greatest when it trained a critical mass of people who worked together or were in frequent contact and could reinforce the new skills, attitudes, and behaviors. Participants suggested two ways that the impact of the interventions might have been increased. First, the applicability of the leadership development approaches to other Bank work was obvious to many Bank staff, who saw it as addressing several management, communication, and problem solving challenges they encountered. Leadership development efforts may have more effect if built into Bank work, while simultaneously improving the effectiveness of other Bank work. Leadership development efforts would then have a clear focus and improve the relationships and effectiveness of people that would be working together on a project. Nonetheless, only one of the interventions (the Burundi Governance and Leadership Capacity Development Program) is beginning to link leadership development efforts with core Bank programs. Second, several interviewees, particularly in Burundi, suggested that the only sustainable solution was to establish a leadership and management training institute. This institute would respond to the enormous need for these types of skills, attitudes, and behaviors in fragile and conflict-affected countries. It would also 24 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 address the challenge posed by the high leadership turnover in these contexts. Nonetheless, no one seemed to know where the resources for this type of venture could come from or how it would be staffed. In addition, while the integration of leadership development capacities into core Bank work and the development of a leadership institute would respond to the broad and pronounced need for leadership skills in fragile and conflict-affected countries, they represent different goals from those sought by the eight interventions under review. Each of the interventions under review aimed to address a specific problem with a specific group of people. Mainstreaming these approaches into core Bank work and/or through a sustainable service-provider in the country may increase the broader impact of the intervention, at the same time as it may reduce the potential for timely, targeted interventions toward key populations and issues in the fragile and conflict-affected country. Focusing on the creation of a management institute may be the best way to increase capacity in the long term, but would require a much greater investment of resources. One approach is not better than the other, it is just important to know and articulate the difference and evaluate it against supply- and demand-driven considerations. IV. Conclusion, Summary of Findings, and Summary of Recommendations Through the PCF and LICUS-TF support of eight leadership development interventions in fragile and conflict-affected countries, the World Bank has built important knowledge and expertise in this area, but that knowledge is not integrated or broadly shared. In addition, based on our interviews with other international donors the Bank has developed a reputation among those who work on fragile and conflict-affected countries as an organization making important investments in leadership development interventions. In our view, the Bank's investment in leadership development seems to be at a crossroads. Either it will continue to support leadership development on an ad hoc basis, slightly expanding the approaches it has used in the past, or it will decide leadership development work (e.g., visioning, conflict management, program management, strategy development) is an area in which the Bank wants to seriously invest, in which case it needs to expand its knowledge and capacity beyond what currently exists. This does not mean the Bank needs to expand the in-house capacity to implement leadership development interventions, but it does mean that the Bank needs to understand what effective interventions require and establish the necessary systems and criteria to enable effectiveness. Findings Below is a summary of findings; a complete list is available in the Executive Summary. A. Context and Intervention Goals Each of the intervention goals were relatively vague, leaving room for different interpretations based on different perspectives. While the intention of the interventions was clear, the link between goals, context, intervention activities and outcomes was not clear. Creating and maintaining alignment between the intervention goals, design, and the context is particularly important for leadership interventions in conflict- affected states. None of the eight leadership development interventions established and maintained alignment between their goal, design, and context throughout the intervention. When the alignment was lost, the effectiveness and influence of the intervention was often reduced. Leadership development in volatile and conflict-affected contexts is a politically-sensitive activity. This sensitivity was often not addressed throughout the intervention. 25 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 B. Intervention Strategy and Design It is essential to have a shared understanding among the design, implementation, and management team of the leadership challenge to be addressed, the type of change desired, the method to be employed to enable that change, the process by which legitimacy and buy-in will be maintained with all critical stakeholders, and how all of this will be monitored and evaluated. The pre-implementation data gathering and buy-in process is critical to an effective leadership development strategy and the legitimacy of the approach to participants and other key stakeholders. Intervention staff need legitimacy with the participating leaders and other key stakeholders, to create buy-in for the project, serve as the project's interloctor, and to deliver the content of the intervention. The tradeoff is that broad based support is more complex and thus requires more time to develop, and in some cases may be exceedingly difficult to obtain. On the other hand working exclusively with one group also has risks, namely when they lose power and influence, your efforts will be quickly replaced. It is important to have connections to and feedback from a representative group of individuals and groups, so support and credibility are based on multiple sources. Having a balanced combination of internal and external perspectives can create value by blending deep contextual knowledge with knowledge of generalities. The individual qualities or characteristics perceived as legitimate are specific to the culture, contexts, and the personalities involved. A single identity or mixed identity approach can both work well as long as the intent is clear among stakeholders and the approach is aligned with the overall goal for the intervention and is culturally appropriate. Issues of learning transfer are best addressed in the design phase in order to incorporate components that will enable and encourage transfer. Factors that determine whether participants transfer what they learn include: motivation to do so; opportunity to do so; support for doing so; and knowledge or skills that can be transferred (i.e., knowing how to translate or adapt what happened in the intervention to their context). Dispersion can happen formally by establishing the expectation and training for participants to share what they have learned with others (e.g. putting them in coaching roles, training of trainer programs, etc.) or informally (e.g., hoping participants will model what they have learned and share it with others). There is no clear evidence suggesting which dispersion approach (i.e., hierarichal v. horizontal/distributed) is more effective. The strategy used to implement the approach appears to be more important than the approach itself. C. Implementation Arrangements When designing a leadership development intervention, the Bank needs to carefully consider the costs and benefits of the different implementation arrangements and be mindful of how administrative procedures could inhibit the effective implementation of these types of interventions. The choice should inevitably be determined by both supply- and demand-driven considerations in the context. Regardless of who (or what group) manages or implements the interventions, it is important that these individuals (separately or collectively) have knowledge of the following: context (culture, history, politics, etc. of the region), methodology (subject matter expertise), and process (how to get things done in country and via Bank systems). Staff must also be clear about roles, responsibilities, accountability, and decision-making authority. Making the goals, desired changes and process explicit is particularly important when different individuals and groups are responsible for different aspects of the intervention, and all the more so when an intervention is implemented in a dynamic context. Regular discussion and information-sharing among all relevant staff is paramount. There are two aspects of leadership development interventions that make high- level oversight important: 1) leadership development interventions are potentially seen as more political than other Bank projects, and thus require oversight to ensure that they do not have an adverse effect on the country context or the reputation of the Bank; and 2) leadership development interventions may be the only contact that many individuals (or groups) have with the Bank and thus can significantly influence the Bank's reputation with a large group of people who may not have direct experience with the Bank otherwise. D. Monitoring and Evaluation If the purpose of the PCF, LICUS-TF, and the new Statebuilding and Peacebuilding Fund is to support new and innovative types of interventions, and replicate the ones that work, monitoring and evaluation is essential so information is documented and learning is created and shared. Without adequate and 26 A Global Review of Eight Leadership Development Interventions funded by the LICUS TF and PCF March 2009 appropriate M&E the value of funding is reduced and the risk of unintended consequences of interventions is heightened. However, monitoring and evaluation were not given high priority by staff. The absence of systematic and informed oversight, support, and accountability of PCF and LICUS-TF grants during the project implementation phase significantly reduced the knowledge available to staff implementing these leadership development interventions, as well as the requirements for project monitoring and reporting that might have increased the effectiveness of several interventions. E. Intervention Impact At the individual level, we found that the impact of the interventions was greatest when participants had the motivation, knowledge, opportunity, and authority to apply what they learned. At the group and organizational levels, the impact of an intervention seemed to be greatest when it trained a critical mass of people who worked together or were in frequent contact and could reinforce the new skills, attitudes, and behaviors. Recommendations While in most cases leadership development interventions cannot address all of the leadership challenges in fragile and conflict-affected states, it is essential that these and other challenges are examined openly and honestly, both in the initial design process and throughout its implementation. It is important that the Bank continue to work towards a shared understanding of leadership development that takes into account the challenges facing leadership in fragile and conflict-affected states, acknowledges the potential value of all of the methods and approaches used in the eight interventions, and continues to refine criteria and processes for the selection and evlauation of leadership development interventions. To ensure that interventions add the greatest value, leadership development goals and designs should be based on both supply- and demand-driven considerations. To maintain the alignment between the strategic goals, the intervention design, the intervention implementation, and the context, the intervention should ensure buy-in of all critical stakeholder groups (e.g.., Government, Bank staff, potential participants, interested international staff, interested community and non-governmental leaders, etc.) and gather feedback and provide input to these groups as the project advances. This feedback and buy-in process helps to maintain the legitimacy and credibility of the intervention with multiple changing contexts and stakeholders. A feedback and buy-in process also helps provide the intervention team with essential feedback. Because the sensitivity of leadership development interventions requires a high level of monitoring of the political context, the staff and managers of these projects should be actively engaged in oversight of the projects. If the TTL is located outside the country, we recommend a member of the Country Office staff be linked with TTL and the contractor or consultants working on the project, and that they have a plan for ensuring responsibility and oversight. We recommend clear guidance be provided to Country Managers/Directors and TTLs outlining the particular type of oversight that should be given to leadership development interventions. We recommend that if the Bank wants to continue investing in leadership development, it extend its grant period to four or five years, or allow for subsequent grants to the same project. We recommend the Bank develop criteria for selecting and funding leadership development interventions and allocate funds to document and review intervention processes and outcomes in order to learn from interventions. We recommend the Bank staff charged with the oversight, design, and implementation of leadership development activities are provided with information about relevant monitoring and evaluation methods. 27