31292 Impact Evaluation of Staff Learning in East Asia & Pacific and South Asia Regions Phase I Report Chaoying Liu Michael Hanson WBI Evaluation Studies EG04-80 The World Bank Institute The World Bank Washington, D.C. December 2003 ACKNOWLEDGEMENTS This evaluation report was prepared for the Learning Board under the overall guidance of Marlaine Lockheed, Manager, Evaluation Group. This report was prepared by Chaoying Liu (WBIEG) and Michael Hanson (The Geldfond group). A number of people contributed to this evaluation study and the report, including: Cristina Ling Chard, Elina Manjieva (WBIEG), Narendra Sharma, Jennifer Ross from the Asia Learning Group (ALG) and Jennifer Rankin (the Geldfond group). Assistance to the production of graphics and documents was provided by Elina Manjieva and Humberto Diaz (WBIEG). This report also benefited greatly from expert review comments provided by Jaime Quizon (WBIEG) and Kin Bing Wu (SAR). The report was discussed at a meeting with the ALG SLC chaired by Phyllis Pomerantz on November 24, 2003. WBI Evaluation Studies are produced by the Institute Evaluation Group (IEG) to report evaluation results for staff, client, and joint learning events. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors and do not necessarily represent the view of the World Bank Group. WBI Evaluation Studies are available on line at: http://info.worldbank.org/etools/WBIEG/publications/index.cfm?pg=getPubs&category= Publications&Intro=yes&instructions=no&showDetails=no&ID= Vice President, World Bank Institute Ms. Frannie Léautier Chief Learning Officer Ms. Phyllis Pomerantz Manager, Institute Evaluation Group Ms. Marlaine Lockheed Task Team Leader Ms. Chaoying Liu ii ACRONYMS ACS Administrative and Client Support AEA American Evaluation Association APA American Psychological Association CAS Country Assistance Strategy CDF Comprehensive Development Framework DAP Development Action Plan EAP East Asia and Pacific FY Fiscal Year GA-GD Staff in grades A through D GE Staff in grade E GF Staff in grade F GG Staff in grade G GH Staff in grade H HDN Human Development Network HQ Headquarters HR Human Resources IEG Institute Evaluation Group IT Information Technology OED Operations Evaluation Department OPCS Operations Policy and Country Services PMU Portfolio Management Unit RCT Random Controlled Trial SAR South Asia Region SARS Severe Acute Respiratory Syndrome SD Standard Deviation STC Short-Term Consultant STT Short-Term Temporary VPU Vice-Presidencial Unit WB World Bank WBI World Bank Institute WBIEG World Bank Institute Evaluation Group iii iv TABLE OF CONTENTS ACKNOWLEDGEMENTS ............................................................................................ii ACRONYMS.................................................................................................................iii TABLE OF CONTENTS................................................................................................v EXECUTIVE SUMMARY ..........................................................................................vii 1. INTRODUCTION...................................................................................................1 2. THE EVALUATION CONTEXT AND METHODOLOGY.................................3 The Evaluation Context and Objectives..................................................................3 The Evaluation Methodology..................................................................................6 3. REGIONAL STAFF LEARNING: A DESCRIPTIVE REVIEW .........................9 Data Issues ...............................................................................................................9 Learning Participation...........................................................................................10 4. REGIONAL STAFF LEARNING: ACCORDING TO STAFF AND MANAGERS .........................................................................................................23 Methodology..........................................................................................................23 Main Findings from Staff Survey..........................................................................24 Main Findings from Manager Interviews ..............................................................34 Summary .............................................................................................................36 5. CONCLUSIONS AND RECOMMENDATIONS................................................39 REFERENCES .............................................................................................................41 ANNEXES ....................................................................................................................43 Annex A: Staff Participating with Extreme High Number of Learning Days Annually (Extreme Outliners) ....................................................45 Annex B: Learning Events Participated by the EAP and SAR Regional Staff in FY 02 .....................................................................................46 Annex C: Summary of Average Days of Dtaff Learning in Each Course Field .....................................................................................................52 Annex D: Staff Survey Questionnaire ...................................................................53 Annex E: Manager Interview/Survey Questionnaire............................................61 Annex F: Demographic Characteristics of Survey Respondents....................64 v vi EXECUTIVE SUMMARY This evaluation study examines the impact of staff learning programs in the East Asia & Pacific (EAP) and South Asia (SAR) regions and reports the results of three separate evaluation activities undertaken during FY03: (a) A descriptive review of staff learning programs in EAP and SAR regions for the period of FY99-02; (b) A survey of over 400 regional staff and 19 managers on their views of staff learning and perceived impact; and (c) An analysis of impact of learning on staff and organizational performance. The main objective of this evaluation is to estimate the extent to which participation in learning has affected regional staff performance and regional operations. Rather than focusing on impact of specific staff learning programs, the evaluation was designed to address the following "generic impact" questions: · To what extent did the regional staff participate in learning? · To what extent did participants use the knowledge and skills acquired in the learning programs? · Did learning help participants and team/unit perform better? · What factors influenced regional staff learning and knowledge utilization? Several conclusions pertaining to the evaluation questions can be drawn from the evaluation analysis. Staff learning participation in the EAP and SAR regions was at a low level during FY99-02. The data suggest that learning participation of EAP and SAR regional staff during the study period (FY99-02) was at a low level ­ far below the Bank's standard1 (10 days per year). Despite the fact that a majority of regional staff participated in some learning, less than a third of regional staff was actively involved in regular learning events. 1This 10 day standard was used by the Bank prior to FY03, Since then this notional benchmark has not been used. vii Staff members who had a learning plan were more likely to participate in learning. GE+ staff and staff members stationed in the Headquarters were more likely to be frequent and regular learners, as were, to a lesser extent, staff with a postgraduate degree. Work commitment plus travel time/cost were the major barriers to the learning participation. The overall quality of learning events offered by Regions and by the Bank was high. Of more than 300 types of learning events, Bank operations, professional and technical, and IT related training were the top three learning events engaged by the regional staff. According to the staff survey, both Bank- and Region-sponsored learning events were well reviewed, but Region-sponsored events received more favorable ratings. The most effective training modes claimed by the regional staff were workshop/clinic, 5-day course and seminar. Participants utilized their knowledge and skills acquired from the learning events. The vast majority of learners utilized their knowledge and skills acquired from the learning events. Respondents indicated that lack of time and conflict with existing practices were the top barriers to knowledge utilization. Learning participation had impact on staff perceptions of their performance. Over half of learning participants agreed that learning contributed to the improvement in their performance and their service to clients. The specific benefits yielded from learning included: better understanding in an area, gaining of new knowledge, and improvement of existing skills. In view of the conclusions, it is recommended: · to supply managers with feedback from the survey that highlights participants' assessment of the impact of learning event attendance so that managers might be able to structure the workflow in a way that allows greater participation-- especially among those who have not participated; and · to develop effective strategies encouraging and directing new staff to participate in learning. Building upon the phase I results, more robust analysis should be undertaken in the next phase to investigate the relationship between learning and staff performance, and to estimate the extent to which staff learning has had an impact on regional operations. viii 1. INTRODUCTION 1.1 The impact evaluation of staff learning programs in EAP and SAR regions was conducted by the Institute Evaluation Group (IEG), following a request from the Bank's Learning Board. The evaluation is being undertaken in two phases. This phase I evaluation was conducted in FY03 and the second phase will be completed in FY04. This phase I report summarizes three separate activities undertaken during FY03 and is structured in the following four sections. 1.2 Chapter 2 begins with an overview of the World Bank staff learning programs, the context for the current evaluation and the study's scope and objectives. It then describes the evaluation methodology, including evaluation questions, approach and implementation. Terms and concepts used in this evaluation are also reviewed in this chapter. 1.3 Drawing on HR training records, Chapter 3 presents a descriptive analysis of EAP and SAR regional staff learning participation over the past four years (FY99-02). 1.4 Chapter 4 presents the result of a web-based staff survey focusing on individual learning experiences, motivations and perceptions, including perceived benefits of participating in the learning. The reasons for staff non-participation in learning and factors that may affect staff utilization of their knowledge and skills gained from learning are also examined in this chapter. Additionally, this chapter presents a summary of telephone interviews with regional managers, focusing on their perception of learning and the impact of learning on team performance and productivity. 1.5 The final chapter offers conclusions and recommendations that are drawn from the study findings. 1 2 2. THE EVALUATION CONTEXT AND METHODOLOGY THE EVALUATION CONTEXT AND OBJECTIVES The World Bank Staff Learning Programs 2.1 The benefits of education and training for individuals and organizations are widely acknowledged (Moy & McDonald, 2000). The World Bank (WB), like many organizations, has invested heavily in developing the knowledge and skills of its people ­ the most critical resource. Between FY98 and FY00, the Bank invested approximately $75-80 million per year in staff learning, equivalent to 5.8-6.4 percent of the net administrative budget. This is well above the five percent benchmark that was endorsed at the start of the Strategic Compact, which is based on the investments of comparable "knowledge" firms. Even under budget pressures between FY01 and FY02, the Bank's total investment in staff learning was still more than $50 million, equivalent to 4.2 percent of the budget (The Learning Board, WB, 2001). 2.2 The Bank's learning agenda has been managed and implemented by several groups since FY02: · Learning Board has provided governance, including the overall strategy for knowledge and learning, and has allocated resources for the design and delivery of learning, as well as the monitoring of performance and quality. · Networks are largely responsible for operations as well as professional and technical learning, including the design and delivery of a variety of learning events including sector weeks, clinics, seminars, and study tours. Networks also promote knowledge sharing through thematic groups, in conjunction with Operations Policy and Country Services (OPCS), Operations Evaluation Department (OED), and other units. · Human Resources (HR) is responsible for behavioral learning and leadership development, including new manager orientation, coaching skills, teamwork, communication skills, and action learning. · World Bank Institute (WBI) is responsible for client learning and for the evaluation of both staff and client learning. · Regions and other Vice-Presidency Units (VPU) fund staff time and travel costs for staff to attend learning activities and for monitoring those funds. While no specific policies were in place in the past, both individuals and units were allowed to engage in either internal or outside learning events that met 3 their learning needs. On the supply side, regions design and deliver their own learning events related to specific VPU priorities. The number of regionally designed courses has increased rapidly since FY02. 2.3 Like other forms of investment undertaken by an organization, training represents a significant investment of the Bank in the people, which is expected to bring returns to the Bank. Over the years, IEG has utilized Kirkpatrick Level 1 and Level 2 evaluations to assess the short-term returns of staff and client learning. Since FY02, efforts have been made to evaluate the long-term impacts of training on knowledge/skills utilization, and on changes in performance and productivity (Level 3 and Level 4 evaluations). However, to date evaluations of staff learning have largely concentrated on formal courses offered by Networks and HR, studied at Levels 1 and 2, as well as at an individual level of behavioral change. No evaluation studies were conducted looking at the ultimate impact of the learning investment on the organization. There has been especially a lack of information about how learning programs have affected regional staff, and their learning experience, as well as impacted regional operations. This situation is the impetus for the current evaluation study. The Scope and Objectives of the Evaluation 2.4 A number of factors were considered in determining the scope of this impact evaluation. First, because the Learning Board required that the evaluation be completed by FY04, we chose evaluation objectives and procedures based on their ability to deliver useful information in a short period of time. A retrospective design, assessing the impact of past activities in the selected regions would permit the evaluation to achieve the delivering timeframe. Second, because most regions have just started building up their capacity to systematically design, deliver and monitor staff learning activities, the evaluation had to work with limited information sources and sometimes poor quality data. Third, time and resource constraints also limited this evaluation to two regions. 2.5 Ultimately, the primary objective of this evaluation was determined to be flexible and realistic -- to estimate the extent to which participating in learning has affected staff performance and regional operations in two regions for the period of FY99-03. Evaluation Regions 2.6 IEG contacted six regions for the purpose of identifying one or two regions for this evaluation. The selection criteria to become an evaluation site include requirements that the region: · has shown an interest in participating in this evaluation on a voluntary basis; · has a manageable staff size; · was relatively active in training over the past few years; · has established training management and governing mechanisms; and 4 · was willing and able to provide available training data. 2.7 Based on the available information and after consulting with all regions, the East Asia & Pacific (EAP) and South Asia regions (SAR) were selected for this evaluation. Regional Context: EAP and SAR Staff Learning Programs 2.8 The EAP region formed a Learning Committee in September 2000 to develop a learning strategy and business plan for EAP staff. In order to provide a clear, comprehensive assessment of the learning environment and the learning needs in the region, the region conducted a staff survey, various focus groups and interviews in FY01. The survey showed that: · almost half of the regional staff were new to the Bank (within their first three years); · a larger proportion (65 percent) of the field staff were new to the Bank; · training in the Bank was very supply-driven; · only a third of staff had learning and development plans (DAPs); and · less than 50 percent of staff believed that training was an institutional priority. 2.9 The survey also revealed the three top barriers to learning: · very little commitment or support from management; · limited guidance; and · being field-based. 2.10 To address the above issues, EAP's Learning Committee adopted the mission: "to create a culture and environment that supports continuous learning," and through this mission, the following learning agenda was formed for EAP in FY02: · learning was driven by the region's priorities and business imperatives, while taking into account the development needs of individual staff; · maintaining a balance in providing training to both Headquarters (HQ) and field staff, reducing any dichotomy in terms of equity, coverage, and opportunities; · more dynamic, team-based, on-the-job and just-in-time learning; and · eliminating the barriers to staff learning. 5 2.11 Under the direction of this learning agenda, the region's priority targets in FY02 were: building operational skills, training new staff members and providing more learning opportunities to field staff. The EAP Learning Group also began assisting the regional units in developing Unit Learning Plans that reflected their unit's business priorities and staff's learning and development plans. 2.12 Due to the similarity in demographics, the high rate of decentralization of staff, and the success of EAP's FY02 regional learning program, the Bank's management appointed its first Regional Learning Advisor, Mr. Narendra Sharma, to oversee the learning programs in both EAP and SAR regions in November 2002. A central "hub" for the delivery of training courses was established in Bangkok, Thailand, which has easy access for staff from both regions. In FY03, the new Asia Learning Group delivered over 15 courses and held over 10 operational video-clinics tailored specifically to the two Asia regions. In addition to courses, Asia Learning Group launched a series of Task Team Leader interviews focusing on good practices and lessons learned from actual Bank projects, and made this available to staff via the intranet. THE EVALUATION METHODOLOGY Some Concepts/Terms 2.13 Before turning to a detailed discussion of the evaluation methodology, there is a need to clarify some concepts and terms used in this report. Learning 2.14 Throughout this report, the terms "learning" or "learning event" are used to represent all learning related activities attended by regional staff. "Learning" in this study can refer to the act, process, or experience of gaining knowledge or skill, the knowledge or skill gained, or resulting behavior changes.2 Likewise, a learning program or offering is not typically a single event but a description of a learning activity or set of learning activities.3 A learning offering produces participant training days and may be delivered face-to-face, via Distance Learning or Electronic Learning, or a combination of the three modalities. Learning offerings are delivered in one of the following formats: Workshop/Clinic, Conference, Course, Global Dialogue, Seminar or Study Tour.4 Impact 2.15 A typical impact evaluation is concerned with the long-term effect of an intervention program such as a learning offering. A rigorous impact evaluation requires the program to have a relatively high evaluability such as when data are collected through Random Controlled Trials (RCTs). Such controlled evaluations tend to be focused on 2Source: the American Heritage Dictionary of the English Language, 4th edition 3Source: The World Bank Learning Board, Learning Catalogue, June 2003 4Source: World Bank Institute Policy & Procedures, Last Updated August, 2002 6 assessing the "net" impact of a narrow set of outcomes. This type of controlled design was not possible for the current evaluation, given available resources and information. Instead, the evaluation looks at the learning experience of all regional staff, without a comparison or control group of non-learners, but with the ability to cover issues wider than the net "technical impact" of learning programs. It attempts to examine the "generic impact" of regional learning programs in terms of: · individual utilization of knowledge and skills learned; · enhancement staff performance; and · enhancement of team performance. Evaluation Questions 2.16 As just discussed, the current evaluation is interested in the broader impact of learning, and addresses the following "generic impact" questions: · To what extent did the regional staff participate in learning? · To what extent did participants use the knowledge and skills taught in the learning programs? · Did learning help participants and team/unit perform better? · What factors influence regional staff learning and knowledge utilization? Evaluation Approach 2.17 The evaluation took a three-step approach to answering the evaluation questions. The approach and steps are illustrated in Figure 1below: · Description: to review learning records of EAP and SAR regional staff over the past four years (FY99-FY02). The purpose of the review was to describe the nature of regional staff learning experience. · Staff and manager survey: to obtain information from both staff and managers about their views of staff learning in general, their utilization of knowledge and skills learned, and perceived benefits of staff learning, as well as the factors that influence staff learning participation and knowledge utilization. · Analysis: to analyze the possible impact of learning on staff and organizational unit) performance. 2.18 Detailed discussion on each approach and the methods employed is presented later in each corresponding chapter. 7 Figure 1: Evaluation approach and steps D escription S urvey Learners' profiles Learning experiences Learning records Motivation Learning programs Perceived impact Learning patterns Feedback Performance outcome A nalysis Relations & condition Individual level Learning and outcome Regional level Outcome and condition Conclusions Recommendations Evaluation Phase and Implementation 2.19 The impact evaluation was designed to be implemented in two phases. Phase I was undertaken from November 2002 through June 2003. IEG worked with an independent consulting firm that was responsible for conducting the staff survey and manager interviews. While the data collection remained independent, IEG worked consultatively with study regions and formed a strong partnership to ensure a successful evaluation implementation. Consultations were held with the Asia Learning Group at varying stages of the evaluation. Phase II will be carried out in FY04, focusing on assessing the impact of learning programs on individuals and on regional operations. 8 3. REGIONAL STAFF LEARNING: A DESCRIPTIVE REVIEW 3.1 This analysis looks at the regional staff learning experience according to the level of their learning participation, and types of learning events that regional staff were involved in over the past four years. It also explores the relationship between staff learning and performance outcomes. In doing so, the analysis draws on a previously unavailable source of staff learning data provided by HR. DATA ISSUES The Data Set 3.2 The HR data set contains unit records for all Bank staff members (with open and term appointment) listed on November 30, 2002. For the purpose of this study, only learning related information for over 1200 EAP and SAR staff members was extracted to create a data set for analysis. Table 1 summarizes the major data domains and items relating to learning that are included in the analysis data set. All information in the HR data set is de-identified, i.e., it does not contain names of staff. However, every record contains a unique personal identification key (CODE_ID) that is assigned by HR. The CODE_ID-to-person correspondence is maintained by HR and has not been made available to the evaluation team. This strategy ensured the confidentiality of individual- level information. In addition, the evaluation team subscribes to the professional standards of the American Evaluation Association (AEA) and the American Psychological Association (APA) regarding confidentiality of data. Table 1: Summary of data items in the HR provided data file Data domains and timeframe Key data items Training/learning information Name (coded) (FY99 through November 2002) Type of course/event attended (internal vs external) Duration for individual courses/events Demographic information Age (Current) Gender Years working in the Bank Education Grade level and job title Duty country station and duty unit Performance ratings from the OPE Items from the result assessment section (FY99-FY03) Items from the behavioral assessment section (client orientation, drive for results, teamwork, and learning and knowledge sharing) Items from the manager behavioral assessment section 9 Data Quality and Limitations 3.3 Data quality is a key issue for any data source, and is a particular concern for a data source not originally designed for evaluation purposes. As the documentation of past individual learning largely relied on staff self-report, there is no guarantee that all historical learning events were accurately recorded. Unfortunately, such a limitation cannot be easily addressed by simple statistical strategies. In addition, there may be other data quality issues, such as missing data either at the case or item level. To understand the basic quality of the data, simple consistency checks were conducted, including: · Comparisons with regionally supplied data with respect to some key demographic variables (total number, gender and grade level distribution). The results showed that the discrepancy was minor (less than 1 percent). · Logical consistency of all activity dates documented (date of birth, appointment date, course staring and ending date). Of 1200 staff members, nine staff members showed appointment dates later than their first training course date, thus leading to an error in the calculation of the length of their service with the Bank. For these records, a correction was made using their first training date as a proxy for their appointment date. LEARNING PARTICIPATION 3.4 This section presents the results of our analysis of regional staff learning participation over the past four years. The key indicators of learning participation used are: · Regularity of learning participation, as measured by number of years participating; · Intensity of learning participation, as measured by number of events or days; and · Field and type of learning. Analysis 3.5 In many of the results presented in this section, whenever possible four types of analyses are made independently: · Description of the learning participation indicator. · Comparisons between the EAP and SAR regions. Although staff learning in EAP and SAR regions has been consolidated recently under one management umbrella, historically staff learning activities were planned and conducted separately in the two regions. While the evaluation treated these two study regions as one evaluation site, most analyses were done using each individual 10 region as the unit of analysis. Comparisons between the regions were helpful in explaining how staff learning was affected by each region's geographical location, organizational culture and policies. For example, the staff in one region had a lower participation rate than staff in the other region. If this difference occurred under the same levels of financial support, then it is likely to be due to the organizational culture, policy or demand factors. · Comparisons between different demographic groups. Such comparisons helped understand the participant-level determinants of staff learning, as well as the level of their influence. · Trends over time for regional staff learning participation. This analysis visualizes whether staff learning participation generally increased or declined and if it was worse or better in some years and some areas rather than others. For instance, the level of learning participation in both regions declined over the past four years. While this trend may be partly explained by changes in the Bank's learning policy, it may be also be related to the reform of the regional learning management mechanism that occurred during the period under consideration. Regularity of Learning Participation 3.6 Regularity of learning participation is concerned with how many of the past four years (FY99 through FY02) a staff member participated in a learning/training activity. It is not concerned with the intensity/quantity of the learning experience5. As can be seen from Figures 2a and 2b, as of November 2002, of the total 1,278 EAP and SAR staff members: · The lowest proportions (27 percent in each Region) participated in learning either none or one year. · The highest proportion participated three or four years. (This is despite the fact that nearly 30 percent of the participants have been with the Bank less than four years ­ see Table 2, hereafter.) · The pattern and level of participation is very similar for both regions, generally increasing from none to four years, with between a quarter and a third of staff participating all four years. 5The regularity category is structured according to the fact that if a staff member participated in learning during the period of FY99 and 02 or not. If a staff reported to participate in learning in FY99 and FY02, his/her learning regularity is coded as "two years", regardless how many events and how many days he/she attended. 11 Figure 2a: Regularity of staff Figure 2b: Regularity of staff learning in FY99-02 learning in FY99-02 (EAP=690) (SAR=588) None None 4 yrs 12% 4 yrs 16% 29% 28% 1 yr 1 yr 15% 11% 2 yrs 3 yrs 2 yrs 3 yrs 21% 27% 18% 23% 3.7 When broken down by grade level, as shown in Figure 3 (a, b, c and d): · ACS staff learning participation was lower, compared to GE+ staff. · From 60 to 70 percent of GE+ staff participated in learning regularly (three of four years or annually). By contrast, only 34 to 41 percent of ACS staff participated in learning at this level. · The lowest regularity was found in SAR, where 30 percent of ACS staff did not participate ­ five times greater than SAR GE+ staff with no participation. Figure 3a: Regularity of staff Figure 3b: Regularity of staff learning in FY99-02 learning in FY99-02 (EAP GE+=395) (EAP ACS=262) none none 7% I yr 4 yrs 21% 4 yrs 25% 12% 32% I yr 2 yrs 3 yrs 18% 21% 16% 3 yrs 2 yrs 28% 20% Figure 3a: Regularity of staff 12 Figure 3c: Regularity of staff Figure 3d: Regularity of SAR learning in FY99-02 staff learning in FY99-02 (SAR GE+=327) (SAR ACS=234) none 6% I yr 4 yrs none 7% 4 yrs 16% 30% 37% 2 yrs 17% 3 yrs 18% 2 yrs I yr 3 yrs 19% 17% 33% 3.8 Table 2 presents a number of participant demographic characteristics broken down within the five levels of participation (regularity). The key findings include: · Large majorities of non-participants were either stationed outside the USA (84 percent), male (77 percent), or ACS (70 percent). · By contrast, large majorities of those participating all of the four years were characterized as either Masters level (75 percent), stationed in the USA (72 percent) or GE+ (68 percent). · Both non-participants and higher regulars (3 or 4 years) were highly likely to be either over age 40 (over 71 percent each) or with the Bank at least four years (69, 75 and 80 percent). Table 2: Learning participation by demographic groups FY 99-02 None N=176 1 year 2 years 3 years 4 years N=169 N=248 N=324 N=361 Total N=1278 Variables N (%) N (%) N (%) N (%) N (%) N (%) EAP 84 (47.7) 102 (60.4) 143 (57.7) 162 (50.0) 199 (55.1) 690 (54.0) PMU SAR 92 (52.3) 67 (39.6) 105 (42.3) 162 (50.0) 162 (44.9) 588 (46.0) Female 41 (23.3) 88 (52.1) 131 (52.8) 148 (45.7) 197 (54.6) 605 (47.3) Gender Male 135 (76.7) 81 (47.9) 117 (47.2) 176 (54.3) 164 (45.4) 673 (52.7) <30 13 (07.4) 19 (11.2) 11 (04.4) 7 (02.2) 7 (01.9) 57 (04.5) 30-39+ 37 (21.0) 56 (33.1) 89 (35.9) 88 (27.2) 92 (25.5) 362 (28.3) Age group 40-49+ 65 (36.9) 59 (34.9) 82 (33.1) 111 (34.3) 139 (38.5) 456 (35.7) 50+ 61 (34.6) 35 (20.7) 66 (26.6) 118 (36.4) 123 (34.1) 403 (31.5) <2 21 (11.9) 26 (15.4) 10 (04.0) 0 (00.0) 0 (00.0) 57 (04.5) 2-3+ 33 (18.8) 53 (31.4) 85 (34.3) 80 (24.6) 71 (19.7) 322 (25.2) Year with the Bank 4-9+ 26 (14.8) 35 (20.7) 59 (23.8) 98 (30.2) 113 (31.3) 331 (25.9) 10-14+ 30 (17.0) 26 (15.4) 53 (21.4) 67 (20.7) 86 (23.8) 262 (20.5) 15+ 66 (37.5) 29 (17.2) 41 (16.5) 79 (24.4) 91 (25.2) 306 (23.9) (Table 2 continues on next page.) 13 (Table 2 continued.) None 1 year 2 years 3 years 4 years N=176 N=169 N=248 N=324 N=361 Total N=1278 Variables N (%) N (%) N (%) N (%) N (%) N (%) GA-D 123 (70.3) 87 (51.5) 96 (38.7) 85 (26.2) 105 (29.1) 496 (38.8) GE-F 15 (08.6) 35 (20.7) 56 (22.6) 69 (21.3) 64 (17.7) 239 (18.7) Grade level GG-J 30 (17.1) 36 (21.3) 84 (33.9) 150 (46.3) 183 (50.7) 483 (37.8) Other 7 (04.0) 11 (06.5) 12 (04.8) 20 (06.2) 9 (2.5) 59 (04.6) Bachelor/- 17 (40.5) 23 (34.8) 51 (33.1) 52 (22.9) 75 (25.3) 218 (27.7) Education Master/+ 25 (59.5) 43 (65.2) 103 (66.9) 175 (77.1) 222 (74.7) 568 (72.3) Outside USA 147 (83.5) 137 (81.1) 185 (74.6) 175 (54.0) 102 (28.3) 746 (58.4) Station USA 29 (16.5) 32 (18.9) 63 (25.4) 149 (46.0) 259 (71.7) 532 (41.6) Intensity/Quantity of Learning Participation 3.9 Intensity of learning participation of the regional staff is analyzed at two levels: · The number of learning events; and · The number of learning participation days. Learning Events 3.10 Tables 3a and b present the mean and standard deviation of learning events that the regional staff attended. · On average, the regional staff in each region participated in roughly two learning events per year during the study period. · The number of events engaged by the regional staff declined over the first three years and perhaps stabilized during year four. · There was a great variation in the level of participation among individual staff in each of the two regions. Table 3a: Learning events attended by Table 3b: Learning events attended by SAR staff (N=588) EAP staff (N=690) FY Mean SD Min Max FY Mean SD Min Max 99 2.5 3.2 0.0 20.0 99 2.2 2.8 0.0 22.0 00 2.4 3.1 0.0 23.0 00 2.2 2.6 0.0 17.0 01 1.9 3.3 0.0 25.0 01 1.7 2.7 0.0 18.0 02 1.8 2.4 0.0 21.0 02 1.8 2.2 0.0 15.0 3.11 Moreover, looking at the level of participation by number of events per year (as shown in Figure 4a and 4b) reveals that: 14 · There was a high proportion of the regional staff (nearly 40 percent) inactive in learning each year, and the size of this non-participating group increased considerably in FY01 for both regions, remaining high for SAR in FY02. · The proportion of staff participating in learning events one to three times per year (low level) increased over time. · The proportion of staff at the highest level of participation (four times plus per year) declined significantly in FY01 and remained relatively low in FY02. · The concurrent decrease in "high learners" and increase in "low learners" illustrates a kind of "substitution effect" of decreasing intensity over time. · Additional analyses reveal that a very small proportion (one to four percent) of the regional staff participated in more than 10 learning events (extremely high intensity) each year. Figure 4a: Learning events attended Figure 4b: Learning events attended EAP SAR 50 50 40 40 30 30 20 20 10 10 0 0 99 00 01 02 99 00 01 02 % Fiscal year % Fiscal year 0 1-3 4+ 0 1-3 4+ Learning Days 3.12 Detailed statistics about learning days are summarized in Tables 4a and b. It can be seen from these tables that each region presents a decrease in learning days attended over the four years. 3.13 However, the distribution of the participation levels by individual participant is severely skewed toward the high end, with one to three percent of staff participating in learning at an extremely high level each year. As this group has strongly inflated and distorts the average and standard deviation, it was removed from the analysis done for Figure 5. Figure 5 shows the average number of the learning days participated in by the EAP and SAR staff during each of the past four years, with outliers removed6. Findings from Figure 5 include: 6The method of dealing with the outliers and analysis of this group is presented in Annex A. 15 · After removing the outliers from the analysis, the highest average learning days attended by regional staff during the study period was around a week (5 days). · The average number of the learning days declined over the years: the most dramatic drop is seen between FY00 and FY01 -- 52 percent for EAP and 27 percent for SAR. · Average learning days for both EAP and SAR appears to stabilize in FY02. Table 4a: Learning days attended by EAP staff Table 4b: Learning days attended by SAR staff (N=690) (N=588) FY Mean SD Min Max FY Mean SD Min Max 99 6.1 14.1 0.0 197.0 99 7.0 10.6 0.0 118.0 00 4.7 8.2 0.0 95.5 00 7.2 18.3 0.0 360.0 01 3.8 8.6 0.0 107.0 01 3.3 6.9 0.0 85.0 02 3.9 10.7 0.0 223.0 02 3.5 6.7 0.0 104.4 Figure 5: Learning days* 7 6 5.9 5.2 5 4.8 4 4.1 3.0 3 2.9 2 2.5 1 SAR EAP * Extreme outliers removed 3.14 The intensity of learning is broken down into different demographic groups, as summarized in Table 5: · Consistent with the previous analysis (regularity), GE+ staff and staff stationed within HQ/USA attended more than 20 days learning during the study period. · There is little difference at the highest level of participation by education, gender or age. · The least participation by length of service with the Bank appears to occur for participants who have been at the Bank less than two years (and who of course have fewer years to accumulate learning days) and, to some degree, for those who have reached 15 or more years of service. 16 · By region, participation by SAR was about 10 percent higher on the high- participation end (>20+ days); EAP has about 12 percent more participants in the lowest participation category (<=10 days). Table 5: Learning days by demographic groups FY 99-02 <=10 >10-20 >20+ Total N=289 N=364 N=449 N=1120 Variables N (%) N (%) N (%) N (%) PMU EAP 193 (31.8) 195 (32.2) 218 (36.0) 606 (100.0) SAR 96 (19.4) 169 (34.1) 231 (46.6) 496 (100.0) Gender Female 128 (22.7) 201 (35.6) 235 (41.7) 564 (100) Male 161 (29.9) 163 (30.3) 214 (39.8) 538 (100) Age group <30 11 (25.0) 18 (40.9) 15 (34.1) 44 (100) 30-39+ 82 (25.2) 111 (34.2) 132 (40.6) 325 (100) 40-49+ 104 (26.6) 128 (32.7) 159 (40.7) 391 (100) 50+ 92 (26.9) 107 (31.3) 143 (41.8) 342 (100) Year with the <2 18 (50.0) 14 (38.9) 4 (11.1) 36 (100) Bank 2-3+ 62 (21.5) 103 (35.6) 124 (42.9) 289 (100) 4-9+ 69 (22.6) 92 (30.2) 144 (47.2) 305 (100) 10-14+ 61 (26.3) 84 (36.2) 87 (37.5) 232 (100) 15+ 79 (32.9) 71 (29.6) 90 (37.5) 240 (100) Grade level GA-D 98 (26.3) 153 (41.0) 122 (32.7) 373 (100) GE-F 51 (22.8) 70 (31.3) 103 (45.9) 224 (100) GG-J 113 (24.9) 129 (28.5) 211 (46.6) 453 (100) Other 27 (51.9) 12 (23.0) 13 (25) 52 (100) Education Bachelor/- 44 (21.9) 67 (33.3) 90 (44.8) 201 (100) Master/+ 135 (24.9) 155 (28.5) 253 (46.6) 543 (100) Station Outside USA 169 (28.2) 218 (36.4) 212 (35.4) 599 (100) USA 120 (23.9) 146 (29.0) 237 (47.1) 503 (100) 3.15 Both regularity and intensity measures may indicate relatively low levels of participation. FY01 appears to be a particularly bad year: a considerable number of staff members was either inactive or had withdrawn from active learning. Field and Type of Learning 3.16 While no precise definition is available, the learning fields attended by the regional staff can be categorized into the following six groups7: Bank operations 3.17 The event focuses on helping participants to familiarize themselves with the Bank's operational framework, function, policies, and various instruments and to develop knowledge and skills in relation to Bank operations. 7When this coding exercise was conducted, the concept of SLC was not implemented in the Bank. The category applied here was based on the definition used by the study regions in describing their learning activities. A consultation of the course code list with the Learning Board members also took place as part of this coding exercise. 17 Professional and technical 3.18 The event provides staff with new or latest knowledge, information, development and skills relating to particular professional fields/topics such as finance, statistics, education, etc. This category includes office skills, where the event focuses on helping participants develop efficient and effective knowledge, and skills in relation to daily office duties and tasks such as meeting planning, travel basics and minute writing. Behavioral and social science (including management) 3.19 The event focuses on influencing staff social and behavioral skills such as communication, teamwork, leadership and management. Information and technology (IT) 3.20 IT training focuses on helping staff master or optimize the use of computer technology tools (software application, data bases or programming). Language (mainly refers to learning foreign languages) Other (refers to unclassified learning events) 3.21 Among these fields of learning events, Bank operations, Professional and technical, and Behavioral and social science related learning events could be functionally considered as "core courses." IT, Office skills, Language and other learning events could be described as "facilitating courses" as skills developed through these learning events play a role in facilitating and enhancing individual ability to utilize more effectively their operational and professional skills. Annex B provides a list of events attended by the EAP and SAR regional staff in FY02. A completed list of learning events participated by the EAP and SAR regional staff during FY99 and FY02 can be provided upon request. Learning Participation by Fields 3.22 Total number of learning events by each course field and its proportion to the total learning events are presented in Table 6 and Figures 6a and b. Figure 6a: EAP Staff Learning Field Figure 6b: SAR Staff Learnign Field Other 02 Other 02 Information Technology 01 Information Technology 01 Professional & Technical 00 00 99 Professional & Technical Behavioural & Social 99 Behavioural & Social Bank Operations Bank Operations (%) 0 10 20 30 40 50 (%) 0 10 20 30 40 50 18 The most notable features of the regional staff learning participation by field can be summarized as: · Regional staff participated in over 300 learning events across seven classified learning fields each year and there was a slow increase over time during the study period and for SAR through FY01. · Between 70 percent and 80 percent of learning events participated in by the regional staff each year were "core courses." · Over time, participation in Professional and technical field learning increased gradually (except for SAR in FY02), accompanied by increases, decreases and then rebounds in the Bank operations learning activities in both regions. The ratio of Professional and technical to Bank operation learning activities increased to nearly two-to-one in FY02, particularly for SAR. · IT related learning events constituted approximately 15 percent of total learning events attended, making this category by far the highest among the "facilitating courses" and third among all fields of regional staff learning. · Participation in Behavioral and social science related learning events increased nearly 30 percent from FY01 to FY02 in EAP. However, its proportion remained at approximately 10 percent of total learning events attended by the EAP and SAR regional staff. · For the most part, both regions presented comparable levels and trends. A summary of average days of the EAP and SAR staff learning on each course field is provided in Annex C. Table 6: Events participated in by regional staff during FY99-FY02, by Year and Field F i s c a l y e a r T o t a l P M U C o u r s e f i e l d 9 9 0 0 0 1 0 2 B a n k o p e r a t i o n s 9 6 1 0 2 6 7 8 9 3 5 4 B e h a v i o u r a l & s o c i a l 3 4 4 1 3 5 4 5 1 5 5 P r o f e s s i o n a l & t e c h n i c a l 9 6 1 3 6 1 5 8 1 6 5 5 5 5 I n f o r m a t i o n t e c h n o l o g y 5 0 3 8 7 1 4 7 2 0 6 E A P L a n g u r a g e 1 3 1 6 8 7 4 4 O f f i c e s k i l l s 1 2 1 9 1 5 1 0 5 6 O t h e r 8 7 2 3 2 0 T o t a l 3 0 9 3 5 9 3 5 6 3 6 6 1 3 9 0 B a n k o p e r a t i o n s 8 2 1 0 2 7 3 8 5 3 4 2 B e h a v i o u r a l & s o c i a l 3 0 4 1 3 6 3 6 1 4 3 P r o f e s s i o n a l & t e c h n i c a l 1 0 8 1 4 9 1 9 9 1 6 0 6 1 6 4 8 5 0 7 3 5 7 2 2 8 S A R I n f o r m a t i o n t e c h n o l o g y L a n g u a g e 1 4 1 2 8 7 4 1 O f f i c e s k i l l s 1 8 2 0 1 1 1 0 5 9 O t h e r 8 6 3 1 1 8 T o t a l 3 0 8 3 8 0 4 0 3 3 5 6 1 4 4 7 19 Type of Learning 3.23 In terms of the suppliers of learning activities, as illustrated in Figures 7a and b, the majority of specified learning events (70-80 percent) were offered internally. The ratio of internal to external courses relating to Bank operations changed very little over time, especially for EAP. Attendance at externally offered Behavioural related courses was as much as nearly 50 percent higher in FY01 than other years. The proportion of externally offered Professional and Technical related learning decreased by 50 percent, from 16 percent in FY99 to seven percent in FY02. Both regions presented a similar trend. Figure 7a: Type of learning Figure 7b:Type of learning (external vs internal), EAP (external vs internal), SAR 100 100 90 90 80 80 70 70 60 60 50 50 Percent 40 Percent 40 30 30 20 20 10 10 0 0 Ext Int Ext Int Ext Int Ext Int Ext Int Ext Int Ext Int Ext Int Operations Behavioural IT Professional Operations Behavioural IT Professional 99 00 01 02 99 00 01 02 Summary and Discussion 3.24 The analysis of regional staff learning participation from FY99 through FY02 reveals that learning participation of regional staff during the study period was at a low level ­ far below the Bank's 10 days standard (The Learning Board, WB, 2001). Despite the fact that the majority of regional staff participated in some learning, less than a third of regional staff was actively involved in regular learning events, and around 12 to 16 percent of staff did not participate in any learning over the four-year period. 3.25 The low level of learning was evidenced by the low level of intensity of participation. The average length of learning participation was lower and declined from five days in FY99 to 2-3 days in FY01 and FY02. (Even adjusted for outliers, participation during the study period is only at about five days.) 3.26 Bank operations and Professional and technical topics were the most popular learning fields for the regional staff. There was a notable shift from operations related learning to Professional and technical related learning in the most recent year. Among the "facilitating courses," IT training remained as the leading field. The choices made by the regional staff reflected the importance of these learning events in helping them to conduct the Bank's business. A majority of learning events that the regional staff 20 attended were provided internally and the proportion of internally delivered Professional and technical learning events had grown significantly in the last two years. 3.27 Of the several demographic variables analyzed, the influence of geographical location, grade level and, to a lesser extent, education degree on learning participation seemed substantial. Staff stationed in the USA and GE+ staff were more likely to be the regular learners. Additional analyses show that the difference between ACS and GE+ staff in the level of learning participation tended to get wider with the increasing length of service in the Bank. 3.28 The reasons for the low levels of learning participation of the regional staff, particularly by the ACS staff, can be very complex, relating to both supply (available learning events) and demand factors (time and finance necessary for participation in learning). The low participation by regional staff stationed outside the USA may imply a lack of local learning opportunities (supply) and barriers or a lack of capacity to access centralized/outside learning opportunities (demand -- time and finance). Moreover, over time, the shift from the Bank operations-related learning to Professional and technical related learning may suggest that supply and demand of regional staff learning programs was unbalanced: i.e., more variety of Professional and technical related learning available for the professional staff, compared to other types of learning events. Regional staff, particularly ACS staff, may attend the Bank operations related learning events intensively at the beginning of their assignment with the Bank, but such intensity may then be hard to maintain unless Bank operations courses with new content are made available to them on a regular basis and locally. Finally, international and local political environments could also possibly influence regional staff learning participation. This may be a particular issue for some regional country offices where civil war or other international events frequently occurred during the study period. 21 22 4. REGIONAL STAFF LEARNING: ACCORDING TO STAFF AND MANAGERS METHODOLOGY General Context and Approach 4.1 The survey population comprised all staff from EAP and SAR regions. There was a total of 1,611 staff members invited to participate. The survey was conducted online via the Web. Staff received an emailed invitation to participate which included a link to the vendor's survey website. Subsequently, non-respondents were sent follow-up reminders, also via email. Up to three reminders were sent by the surveyor. 4.2 Managers were surveyed as well to determine what benefits they saw in having their staff members attend learning events. This survey was administered primarily by telephone interviews conducted by the external consulting firm. Participants were notified by email of the proposed interview. For those whose time zone or schedule would not allow a personal interview, the survey was sent to them as an email attachment requesting it be returned by fax. Non-respondents, including those for whom a voicemail message was left, received up to two additional phone attempts to complete the survey. 4.3 It should be noted that both surveys were administered during one of the busiest times of the year, late-May through late-June. Additionally, there were unforeseen circumstances such as evacuation of the China office and travel restrictions that resulted from the Severe Acute Respiratory Syndrome (SARS) epidemic. These issues are important considerations as to the response rates on both surveys (see below). Evaluation Instruments 4.4 The Regional staff learning questionnaire contained 35 quantitative questions, four qualitative questions, and eight demographic questions. The web-based survey employed skip logic, where necessary, to take the respondent to the next appropriate question. The survey covered participation in learning, quality of the learning events, utilization of the knowledge and skills learned, and self-perceived benefits and impact. 4.5 The manager instrument had 25 quantitative items and two open-ended questions. It covered the following topics: effectiveness of learning events, impact on unit performance, benefits of the investment in learning, obstacles to staff participation, and extent to which learning events meet the needs of staff. Both staff and manager survey/interview questionnaires are included in Annexes D and E. 23 Response Rates 4.6 The overall response rate for the learning events participants was 27 percent. In total, there were 432 respondents. Among EPA staff, there were 262 respondents for a response rate of 28 percent. For SAR staff there were 170 respondents for a response rate of 25 percent. 4.7 Of the 432 respondents, a majority (68 percent) was located in the field as opposed to Headquarters (31 percent). Gender had a fairly equal representation between Female (43 percent) and Male (34 percent).8 The full demographic breakout of respondents is shown in Annex F. 4.8 The level of participation among managers of learning event attendees was much lower than that for the attendees. Despite repeated telephone calls and the fax-back alternative, only 19 managers took the time to respond to the survey. The results from these calls are included but should not be considered statistically representative of the manager population. MAINFINDINGS ­ STAFF SURVEY Participation in Learning Learning Plans and Budget/Funding 4.9 Respondents were asked if they had a Learning Plan endorsed by their manager in each of the last three fiscal years. The responses for the 432 survey participants: FY03: 53 percent had a Learning Plan endorsed by their manager 39 percent received sufficient budget/funding to implement their Learning Plan FY02: 42 percent had a Learning Plan endorsed by their manager 31 percent received sufficient budget/funding to implement their Learning Plan FY01: 32 percent had a Learning Plan endorsed by their manager 20 percent received sufficient budget/funding to implement their Learning Plan 4.10 The increasing percentages for past fiscal years largely represents decreases in the "Don't Know" response. This relative lack of certainty in earlier fiscal years supports the survey strategy of focusing on respondents' experience with learning events in the last 12 months. 4.11 Across all three fiscal years, there were a larger percentage of people in HQ who do not have a manager-endorsed learning plan in comparison to those in the Field. The 823 percent missing value for this item. 24 percentage gap between the two locations increased from FY01 through FY03. Another point to note is that those in the youngest (<30) and the oldest (60+) demographic groups were least likely to have learning plans; this was particularly evident in FY03. Over half the combined STC/STT/Other grade level indicated they did not have learning plans for any of the fiscal years. As with the age comparisons, the highest percentage was in FY03 where 70 percent did not have a learning plan. Other demographic comparisons seemed to be in line with the findings of all staff surveyed. 4.12 As with all respondents, there was a high percentage among demographic subgroups of those who simply "don't know" when it comes to whether or not funding was made available (Q.2). Learning participation 4.13 Among survey participants, almost three-quarters (73 percent) said they participated in learning events over the last 12 months. When comparing grade levels, GA-GD and GE+ had similar degrees of participation (76 percent and 81 percent respectively). Those who were STC/STT/Other had only 58 percent participating. Within the Regions, only half (8 out of 16) who indicated they belonged to South Asia's Central Units participated in learning; other Units had participation rates similar to total staff. When comparing those in the field to those in HQ, there was a slightly higher percentage of participation within HQ (80 percent) compared with those located in the field (70 percent). (t=2.28, p<.05). Other demographic breaks such as job/position, gender, age, education level, and tenure showed no significant difference from that of total respondents. 4.14 The average amount of time spent attending learning events in the last 12 months per participant was 11.1 days overall. For EAP the average was 10.7 days; for SAR, 11.6 days. For HQ, it was 9.8 days; for field, 11.7 days. In general, HQ staff attended slightly more learning events while field staff attended events of slightly longer duration. 4.15 For the most part, those who attended learning events had a learning plan (62 percent); those who did not attend were less likely to have a learning plan (70 percent). Table 8 provides a summary of staff learning plan and learning participation. 25 Table 8: Learning plan and learning participations EAP FY02 SAR FY02 Learning Plan (%) Participation (%) Learning Plan Participation (Q. 1a) (Q. 3) (%)(Q. 1a) (%)(Q. 3) Station HQ 13 26 15 22 FLD 38 47 39 49 Country 15 16 12 17 Regional Units Central 8 10 5 5 Sectors 23 34 31 39 Gender Male 27 37 22 26 Female 16 22 25 32 GA ­ GD 16 20 21 25 Grade Level GE+ 24 32 25 32 STT/STC/Other 3 6 Insufficient 5 < 30 5 6 Insufficient 4 > 30 ­ 40 18 24 15 16 Age Group > 40 ­ 50 13 17 16 19 > 50 ­ 60 7 9 14 18 60+ Insufficient Insufficient Insufficient Insufficient Reasons for not participating 4.16 Those respondents who did not participate in a learning event in the last 12 months were asked to select from a list of possible reasons. Of the 117 respondents who were invited to provide one or more reasons, 76 responded as follows: Figure 8: Barriers to participation (n=76; more than one choice possible) Unable to get away from work schedule 42% Not enough budget allocated 29% Supervisor did not approve 14% Not interested in the events offered 12% Timeframe of event was too lengthy 4% 0% 10% 20% 30% 40% 50% 4.17 Of the remaining respondents, another 42 percent gave various other reasons for not attending. It should be noted that the sizes become fairly small when looking at demographic breakouts of these 76 individuals, but that there was a higher percentage who selected "other" reasons among those newest to the Bank (<1 year) as well as those who were the combined STC/STT/Other grade level (71 percent for each). Regional Units such as South Asia's Country Units and East Asia and Pacific's Sector Units specified "unable to get away from work schedule" as a significant reason (71 percent and 57 percent respectively). Other demographic subgroups appeared to have a similar distribution in their reasons for not participating in learning events. 26 4.18 Respondents who had not participated in the past 12 months were not asked any further questions about learning events but were skipped to the final section on demographics. Learning fields 4.19 Respondents who did participate in learning events in the last 12 months were asked to provide a breakdown of the fields of learning event they had attended. Out of the 315 staff eligible to respond, 279 respondents provided information. The results are as follows: Table 9: Participation by course type (n=279) Type of Learning % Participants Average number of events Average number of days Professional and technical 58% 2.2 8.3 Bank operations 53% 1.5 6.0 Information technology 32% 1.6 4.2 Behavioral and social 27% 1.3 3.2 Office skills 11% 1.5 3.5 Language 8% 1.0 14.2 Sources of learning 4.20 Respondents were asked how many of the learning events they participated in over the last 12 months were internal (organized by the Bank or their Region) and how many were external (organized by an outside party). Respondents classified 478 events as internal or external. The breakdown of events is as follows: 4.21 Staff in the field were somewhat more likely than HQ staff (38 percent vs. 28 percent) to participate in learning events offered by the Region, compared to those offered by the Bank. HQ staff were somewhat more likely than field staff (54 percent vs. 44 percent) to participate in learning events offered by the Bank than by the Regions. For both field and HQ staff, there were external sources for 19 percent of the learning events they attended. Figure 9: Proportions of internal versus external courses (n=478) Internal ­ Bank offered 47% Internal ­ Region offered 35% External 19% 0% 10% 20% 30% 40% 50% Motivation 4.22 Survey respondents who had participated in learning events were asked what, in general, were their main reasons or motivation to participate. They were provided with a 27 list to select from and invited to select up to three responses. A total of 287 respondents provided answers listed below. Figure 10: Main reasons for learning participation (n=287) For professional interest and growth 51% To learn new knowledge 49% To improve existing skills To prepare for greater responsibilities 49% To learn new skills 44% To improve understanding in the area 40% To network and share information 31% To obtain a pre-requisite for further Coursework 21% Other, miscellaneous reasons 2% 2% 0% 10% 20% 30% 40% 50% 60% 4.23 The total respondents' top-ranked reasons as to why they chose to participate in learning were the same throughout all demographic subgroups. Quality of the Learning Events Mode of delivery 4.24 Respondents were asked in what modes the learning events they had attended were offered. A total of 268 respondents provided information on the mode of delivery. There were no statistically significant differences by region or station. Table 10: Participation by delivery mode (n=268) % Total Number of Mode of Delivery Participating Events Attended Workshop/Clinic (3 days or less) 60% 349 Course (5 days or more) 53% 226 Conference 23% 90 Seminar (1 to 5 days) 22% 102 Online (self-paced CD-ROM/web) 13% 60 Global dialogue (video conference) 9% 51 Online (net meeting) 4% 15 Study Tour 3% 7 Most effective mode 4.25 Respondents were asked to identify the most effective mode of delivery for the learning events they participated in. They were invited to select up to three most effective modes. The top four modes are as follows: 28 Figure 11: Delivery modes rated by effectiveness (n=287) Workshop/Clinic 42% 5-day Course 40% Seminar 6% Study Tour 4% 0% 10% 20% 30% 40% 50% 4.26 The remaining modes were selected by 6 percent or fewer respondents. HQ staff were more likely to say Workshop/Clinic was most effective while field staff preferred a 5-day Course. Travel 4.27 Respondents were asked if they had to travel outside their duty station to participate in learning events over the past 12 months. Close to two-thirds of the respondents (65 percent) said they traveled outside their duty station. Slightly more than one-third (35 percent) did not. In some cases, the travel was extensive, with a number of individuals traveling to Washington, DC for learning events. 4.28 The necessity for travel varied slightly by region. In EAP, some 61 percent of participants traveled beyond their duty station for learning events. In SAR, 70 percent of participants traveled beyond their duty station. (The difference is not statistically significant at the .05 level.) Country Units in SAR did a comparable amount of travel (59 percent) to those in East Asia and Pacific (65 percent). However, the Sector Units did a greater amount of travel in SAR compared to EAP (75 percent vs. 56 percent ­ t=2.54, p<.02). Similarly, Central Units in SAR did a greater amount of travel compared to their counterparts in EAP (100 percent, N=8 vs. 68 percent, N=25). 4.29 Presumably, the difference in those at Headquarters that had to travel (31 percent) is much less than those in the field (80 percent). By job/position, managers did the highest amount of travel (85 percent). Among grade levels, those who were the combined STC/STT/Other did the least (35 percent). Staff that had been with the Bank <1 year or 15+ years did the least amount of traveling (46 percent and 51 percent respectively). The traveling percentages of other tenure breaks are in line with that of total staff. Other demographic breaks seem to be consistent with total respondents. 29 Quality of learning events offered by Region vs. Bank 4.30 As shown in Figure 12, the vast majority of respondents (83 percent) rated Region-sponsored learning events they attended as "good" or "very good". The same percentage of respondents (83 percent) rated Bank-sponsored learning events they attended as "good" or "very good". Somewhat fewer respondents considered the Bank's events "very good" compared to the region's events. The assessment did not vary greatly by the regional location or by any other demographic comparison. Figure 12: The overall quality of learning events by type of offering (Region n = 247, Bank n = 265) 100% 58% 42% 41% 25% 16% 15% 2% 0% 1% 1% 0% Very Good Average P o o r Very G o o d P o o r Region Bank Assessment of regional vs. Bank learning events 4.31 Most respondents believed that the events offered by both their regions and the Bank contributed to their learning across the six areas rated. Nearly one-quarter of the respondents (23 percent) said that the regional learning events impacted their overall performance "to a great extent". There are no statistically significant differences between the ratings of regional and Bank-sponsored events. 4.32 Gender, age, tenure, and job/position comparisons provide similar data to that of all respondents for both types of sponsored events. However, when looking at education, those who indicated they were associate level gave very favorable ratings to certain aspects of Regional events (i.e., addressing learning needs, increasing knowledge/skills, impacting overall performance) but rated Bank events on par with those of other educational backgrounds. Those who were the combined STC/STT/Other grade level gave slightly lower ratings to both Regional and Bank-sponsored events than those who were GA-GD or GE+. When looking at HQ versus field, those in the field gave more favorable ratings to both Regional and Bank-sponsored events. 4.33 Eight in ten respondents considered Regional and Bank learning events as relevant to their work. The small difference in ratings is not significant. 30 4.34 Approximately seven in ten respondents said that their learning needs were addressed to a "considerable" or "great" extent. The difference in favorable ratings for the Region and Bank was not statistically significant. 4.35 Seven in ten respondents favorably evaluated both the region and the Bank on providing learning events that increased their knowledge and skills. The difference in favorable ratings, with the region at 74 percent and the Bank at 65 percent is statistically significant (t=2.16, p< .05). 4.36 The majority of respondents favorably evaluated both the region and the Bank with regard to providing learning events that met their expectations. The difference in favorable ratings between the Region (68 percent) and the Bank (59 percent) is statistically significant (t=2.06, p< .05). 4.37 Seven in ten respondents favorably evaluated both the region and the Bank with regard to learning events providing useful skills required in your job. The difference in favorable ratings between the region (73 percent) and the Bank (65 percent) is not statistically significant. 4.38 The majority of respondents thought that both the region's and the Bank's learning events impacted their overall performance. (Nearly one-quarter of the respondents said that the region's learning events impacted their overall performance "a great deal.") The difference in favorable ratings of the region and the Bank s not statistically significant. Utilization of the Knowledge and Skills Learned Ability to utilize knowledge and skills 4.39 The vast majority of respondents (87 percent) said that they had been able to make use of the skills and knowledge from the learning events they participated in over the past 12 months. This assessment does not vary greatly by region or by any other demographic comparison. Barriers 4.40 Respondents were asked if any of the factors listed below were barriers to utilizing the knowledge and skills learned over the past 12 months. For the 233 respondents to this question, the majority said it was a "lack of time". 31 Figure 13: Barriers to knowledge utilization (n=233) Other various 14% factor Manager and 15% peer support Lack of budget 18% Existing practices 38% Lack of time 59% 0% 10% 20% 30% 40% 50% 60% 70% 4.41 The top total staff responses of "lack of time" and "existing practices" are repeated as the top responses among all of the demographic subgroups. Supervisor encouragement 4.42 Most respondents (74 percent) said that their supervisor encouraged them to seek out appropriate regional staff learning experiences. There is no statistically significant difference by Region on this item. Additionally, the demographic subgroups provided ratings that corresponded to total staff. Having adequate opportunities 4.43 The majority of respondents (58 percent) said they had adequate opportunities to learn new skills to do their job better; a much smaller percentage (14 percent) disagreed. As with encouragement (Q. 17a), there is no statistically significant difference by region or noteworthy differences in ratings by demographic subgroups for this item pertaining to adequate opportunities. Self-Perceived Benefits and Impact of Learning Improve service to clients 4.44 Respondents were asked the extent to which their participation in learning events over the past 12 months contributed to improving service to clients. The majority (57 percent) believed the events contributed to a "great" or "considerable" extent. There is no statistically significant difference by region on this item. 32 Figure 14: Contribution of learning participation on improvement in client services (n=274) 40% 36% 34% 35% 30% 25% 20% 20% 15% 8% 10% 5% 1% 0% To a great To a To some extent To a limited Not at all extent considerable extent extent 4.45 Ratings among tenure, gender and education levels seemed to be in line with total respondents. Field staff gave a higher rating (62 percent) to improved client service as opposed to HQ staff (44 percent) (t=2.78, p< .01). In addition, 30 percent of those whose grade level was the combined STC/STT/Other indicated that their client service improved "to a limited extent" or to "no extent" as a result of participation in learning events. Value to others of learning events 4.46 Almost all respondents (97 percent) believed that participation by others in the learning events they participated in during the last 12 months will be "very" or "somewhat" valuable. There is no statistically significant difference by region on this item. Other demographic subgroups are also in line with that of total staff participants. Specific benefits 4.47 Participation in learning events resulted in specific benefits to the participants. Respondents were invited to select as many of the benefits listed as applied. The following list of benefits is displayed in descending order of selection. The top benefits named by all staff are representative of those given by the demographic subgroups as well. Figure 15: Benefits of learning activities to participants (n=270) It improved my understanding in an area 70% It improved my existing skills 66% It taught me new knowledge 66% It contributed to enhancing my own productivity 51% It taught me new skills 49% It prepared me for greater responsibilities 48% It contributed to enhancing my team productivity 44% It served as a pre-requisite for future coursework 11% Other, miscellaneous reasons 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 33 Open-Ended Questions 4.48 Respondents were invited to submit answers to three open-ended questions. Many respondents provided various answers to these questions. The most common comments include: · In what areas do you feel you perform better now that you have participated in learning over the past 12 months? Areas that EAP respondents felt they performed better (Q. 21) included a wide variety of responses, with the most frequently named being Trust Fund Management and Bank operations. SAR also gave several different answers, with Bank operations being the most common. · What other areas of learning do you think would be valuable for you in the future? Other areas that both regions wrote they would like available were Communications skills, IT skills, and to a lesser extent, Procurement knowledge. · What other comments or suggestions do you have for staff learning? As far as any other comments pertaining to learning, the primary theme evident in responses from both regions related to availability. They felt that there should be more regional events, more budget, and a greater number of alternative methods of learning since going to HQ/USA is not always a viable choice. 4.49 A detailed list of responses to the above open-ended questions can be provided upon request. MAINFINDINGS ­ MANAGER INTERVIEWS 4.50 Over 60 managers from EAP and SAR regions were contacted for the interview/survey. Only 19 managers responded to the survey9. Unit Learning Plan 4.51 Of the 18 out of 19 who responded to the question about number of staff in their unit, slightly less than half (44 percent) said fewer than 30 while slightly more than half (56 percent) said 30 or more staff. Most of the 19 managers' units (68 percent) had a learning coordinator in FY03. This was a higher percentage by far than for FY02 (47 percent, n=17) and FY01 (38 percent, n=16). In addition, managers were asked if their units had a Learning Plan in each of the last three fiscal years. For FY03, the majority (68 percent, n=19) had a Learning Plan; for FY02, there were Learning Plans for half the 9The results for the manager survey may be summarized, but they should not be considered representative of the larger population since the response rate was so low for managers. 34 units (50 percent, n=16); and for FY01, there were Learning Plans for fewer than one- third (31 percent, n=16) of the units. 4.52 Fewer than half of the 19 managers' units (47 percent) had a monitoring system for unit learning activities. The vast majority (82 percent, n=17), however, were supported by the Asia Learning Group. In nearly half (47 percent, n=19) of the units, 30 or more staff participated in learning events in the last 12 months. Effectiveness of Learning Events 4.53 Managers assessed the effectiveness of the learning events attended by staff in various areas (see Figure 16). In general, the 18-19 respondents were not often likely to see the events as "very effective." The most common response was "somewhat effective." Impact on team productivity was the least favorably rated. Figure 16: Effectiveness of learning activities in team and personal development (n=18) Team productivity 67 33 Individual productivity 89 11 Individual professional development 79 21 Services provided to clients 74 26 10 20 30 40 50 60 70 80 90 100 Somewhat or Very Effective Not Very or Not all Effective Impact on Unit Performance 4.54 For the most part, managers' assessments of the impact of participation in learning events was neither positive nor negative. Most of the 19 respondents (63 percent) saw a neutral impact. Somewhat more saw a positive impact (21 percent) than saw a negative one (16 percent). 4.55 Nearly one-quarter of 17 managers (24 percent) said they "often" observed staff practicing the skills/knowledge they gained from learning events. Over half (59 percent), however, said they saw staff "sometimes" practicing the skills/knowledge. Benefits vs. Investment 4.56 Most of the 18 respondents (61 percent) gave a neutral response when asked about the benefits associated with sending staff to learning events. None of the managers felt that the investment outweighed the benefits. 35 Obstacles 4.57 Managers were asked whether various factors were obstacles in sending staff to learning events (Figure 16). The most frequent obstacles were the "demanding work program," and the related issue of "too much time away from job responsibilities." Figure 16: Barriers to training participation of staff as perceived by managers (n=18) Demanding work program 58% Too much time away from job 53% Cost/budget constraints 32% Having a single, centralized learning event location 22% Concerns about travel safety 5% Staff resistance to learning 5% 0% 10% 20% 30% 40% 50% 60% 70% 80% Effectiveness of Learning Events in Meeting the Needs of Staff 4.58 Managers stated that learning events offered by the region as well as the Bank had mostly met needs of their staff. Out of 19 respondents, only five percent said that the Bank learning activities had failed to meet the needs of their staff. The proportion of those dissatisfied with the regional offerings is slightly higher (16 percent). Open-Ended Questions 4.59 Managers were asked to answer two open-ended questions: · What learning event have you perceived to be the most beneficial to your unit? · What other topics or skills would you like to see covered in future learning events? 4.60 Responses to both items varied and, given the small number of participants, there does not appear to be any predominant response to either of the questions. A detailed list of responses can be provided upon request. SUMMARY 4.61 This chapter provides a summary of regional staff and manager surveys regarding the staff learning programs in EAP and SAR regions. These survey findings provide one snapshot of staff learning experience and their perceived impact of staff learning. 36 4.62 Slightly over one-half (53 percent) of the staff population had a learning plan in FY03. Additionally, 73 percent of the staff participated in learning within this year. Nearly two-thirds of those who participated had a learning plan for the year. Those who did afford themselves the opportunity to attend learning spent an average of 11 days in various learning events. 4.63 The field's learning participation (70 percent) was slightly lower than Headquarters (80 percent), which could be partially attributed to the large difference in traveling (Field travel = 80 percent; HQ travel = 31 percent) required of those who did participate. Field staff who did not participate answered "unable to get away from work schedule" (43 percent) and "other" (45 percent) as their reasons why. Staff with the least tenure (<1 year) was the highest percentage group (71 percent) selecting "Other" as the reason for not participating. This may imply that the group had limited knowledge of what learning events were available or was hesitant to inquire about learning given their new position. 4.64 Job commitments plus travel time/cost seem to be factors as to why field participation in learning may not have been as easily facilitated. 4.65 Of those who did not attend learning events, 70 percent did not have or know about a learning plan. As far as reasons for not participating, this group cited their inability to get away because of their work schedule (42 percent). To a lesser extent, budget allocations prevented attendance (29 percent). Neither lack of supervisor support nor lack of interest in the events (14 percent and 12 percent respectively) functioned as barriers to attending. 4.66 When comparing Bank with region-sponsored events, both were well reviewed, but region-sponsored events received more favorable ratings. It appears that region- sponsored events offered a more customized approach to learning. It would be a fair assumption that regional events also saved on time and costs associated with attending Bank-sponsored learning events. 4.67 A majority of the staff that participated in learning felt they were able to make use of the skills and knowledge acquired from attending learning events. A lack of time and existing practices were barriers to utilizing the knowledge and skills they learned. 4.68 Both regional staff and their managers acknowledged a value in staff participation in learning events, with staff more likely to have seen a positive impact on performance than managers (keeping in mind that the limited data from managers was less statistically reliable). Presumably managers were also assessing the balance between the productivity loss while staff were attending learning events with the higher level of performance that might have followed attendance. Over half of participants believed the events contributed to a "great" or "considerable" extent to improving service to clients. Additionally, almost all respondents felt that learning events they participated in would be "very" or "somewhat" valuable for others. Some of the specific benefits gained from learning include: better understanding in an area, gaining new knowledge, and improvement in existing skills. 37 4.69 Most managers interviewed/surveyed felt that individual productivity and development were improved by the effectiveness of learning, while team productivity and services to clients were slightly less improved. In terms of overall unit performance, most felt that learning had a neutral impact, and saw staff practicing skills/knowledge learned. 38 5. CONCLUSIONS AND RECOMMENDATIONS 5.1 The report presents the results of three separate evaluation activities conducted in FY03 regarding the staff learning programs in EAP and SAR regions. Several conclusions pertaining to the evaluation questions can be drawn from the analysis. Staff learning participation in EAP and SAR was at a low level during FY99-02. 5.2 The examination of learning participation indicators suggests that learning participation of regional staff during the study period (FY99-02) was at a low level ­ far below the Bank's standard (10 days per year). Despite the fact that a majority of regional staff participated in some learning, less than a third of regional staff was actively involved in regular learning events. 5.3 The level of regional staff learning participation seemed to be related to -- and may be influenced by -- staff grade level, education degree and geographic location. GE+ staff and staff members stationed in HQ were more likely to be the more frequent and regular learners, as were, to a lesser extent, staff with a postgraduate degree. Additionally, staff members who had a learning plan were more likely to participate in learning. Work commitment plus travel time/cost seem to be the major barriers to the learning participation. The overall quality of learning events offered by regions and by the Bank was high. 5.4 Of more than 300 various learning events engaged in by the regional staff each year, participation was highest for three learning fields, namely Bank operations, Professional and technical related events and IT. Both Bank and region-sponsored learning events were well received, but region-sponsored events elicit higher ratings. The most effective modes according to the regional staff are workshop/clinic, 5-day course, and seminar. Participants utilized their knowledge and skills acquired from the learning events. 5.5 The vast majority of learners utilized their knowledge and skills acquired from the learning events. The participants indicated that lack of time and conflict with existing practices were the top barriers to knowledge utilization. Learning participation had impacted staff performance 5.6 More than half of learning participants agreed that learning contributed to the improvement in their performance and their service to clients. The specific benefits from learning include: better understanding in an area, gaining of new knowledge, and improvement of existing skills. 39 5.7 As a general recommendation, it may be useful to supply managers with feedback from the survey, highlighting participants' assessment of the impact of learning event attendance. In this way, managers might be able to structure the workflow in a way that allows greater participation--especially among those who have not participated. There is a considerable gulf between the average participation of participants and the significant number of staff who have 0 days of participation in learning events in the past year. It would be particularly detrimental if the most productive staff are denied learning event opportunities because they are relied upon to deliver client services. 5.8 The group to target for increased participation would be new staff members (<1 year). Effective strategies should be developed to encourage and direct them to participate in learning. 5.9 The present phase of the evaluation primarily focuses on a descriptive analysis and gives one snapshot of regional staff learning experience during FY99-02. For a more complete picture, further analysis is needed. It is suggested that the next phase evaluation should focus on: · examining the relationship between learning and performance at individual levels; · assessing the learning impact at organization levels (unit and regions); and · assessing the quality of regional offered courses in FY03. 5.10 The second phase analysis should provide more robust evidence of the nature and benefits of staff learning. 40 REFERENCES Moy & McDonald, (2000) Analysing enterprise returns on training. National Centre for Vocational Education Research, Australia. Accessed at: http://www.ncver.edu.au/research/papers/index.htm, August 15, 2003. Washington, D.C The Learning Board, The World Bank (2001). Building Staff Capacity for Development: A New Learning Framework. 41 42 ANNEXES 43 44 ANNEX A: STAFF PARTICIPATING WITH EXTREME HIGH NUMBER OF LEARNING DAYS ANNUALLY (EXTREME OUTLINERS) Steps in Determining Outliers Calculate the "lower fence" as: Q1 ­3 Calculate the "upper fence" as: Q3 +3 Any data smaller than the lower fence is an outlier; and any data bigger than the upper fence is an outlier. Characteristics of the extreme outliers As presented in Table 7 there are approximate 100 staff members identified as "extreme outliers" according to their number of learning days participated. They were more likely to be female, GE+ staff and staff with a Master's degree. As the average learning days attended by them was 10 times of the rest of the study population, thus exceeding the "upper fence," the performance of this group should be analyzed separately (see Table 7). Table 7 Characteristics of extreme outliners Extreme Outliers (learning day) Total Variables 99 (N=21) 00 (N=25) 01 02 (N=24) (N=28) N=98 Min 37.3 33.5 21.0 20.1 Max 197.0 360.0 107.0 223.0 Learning day Mean 77.7 68.3 46.5 40.2 SD 47.1 65.3 24.2 41.9 Sum 1630.9 1707.4 1114.8 1126.1 EAP 9 8 12 16 45 PMU SAR 12 17 12 12 53 Outside USA 12 8 8 14 42 Station USA 9 17 16 14 56 >1-2 2 2 >2-3 1 1 2 Year >3-4 yrs 4 4 6 10 24 Bank >4-10 yrs 9 8 4 6 27 >10-15 yrs 4 6 4 4 18 15+ 4 7 9 5 25 ACS 6 7 7 9 29 Grade GE+ 15 17 17 19 68 <30 1 2 3 30-39 yrs 6 3 5 11 25 Age 40-49 yrs 10 10 9 5 34 50-59 yrs 5 12 9 10 36 F 10 15 18 16 59 Gender M 11 10 6 12 39 Bachelor/(-) 3 7 4 5 19 Degree Master 10 11 10 10 41 Doctor 4 5 4 8 21 45 ANNEX B: LEARNING EVENTS PARTICIPATED BY THE EAP AND SAR REGIONAL STAFF IN FY 02 # BANK OPERATIONS (119 ) REGION # BANK OPERATIONS (119 ) (cont.) REGION 1 A WORKSHOP ON OP 4.09 - THE SAFEGUARD POLICY ON PE EAP 53 MANAGING WORLD BANK RESOURCES - CONTROLS EAP 2 ACS - A DAY IN THE VIETNAM COUNTRY OFFICE EAP 54 MULTI-SECTORAL LENDING OPER. O SUPPORT SME SERVICE EAP 3 ACS CORE CURRICULUM FY02 FIELD EAP 55 NEW IBRD FINANCIAL AND HEDGING PRODUCTS EAP 4 ACS INTEGRATED OPERATIONS & TECHNOLOGY SKILLS TRNG EAP 56 NEW IBRD LOAN AND HEDGING PRODUCTS: GENERAL EAP SAR 5 ADM. SERV. REQ. & MISC. EXPENSE REIMB. COURSE EAP SAR 57 PFG RISK MITIGATION/WB GAURANTEES BBL EAP 6 ADVANCE CONTRACT ADMINISTRATION EAP 58 PIC HUB TRAINING EAP 7 ADVANCED PROCUREMENT EAP SAR 59 PILOT INTEGRATED OPERATIONS AND TECHNOLOGY SKILLS EAP 8 ANALYSIS & EVALUATION OF INVESTMENT OPERATIONS EAP SAR 60 POLICY BASED LENDING AND IMPLICATIONS FOR ESSD EAP SAR 9 APPROVING FOR MANAGERS EAP SAR 61 PORTFOLIO SAR 10 ASSET MANAGEMENT EAP 62 PORTFOLIO I (PDS) SAR EAP 11 BASIC PROCUREMENT EAP SAR 63 PORTFOLIO II (PSR, LEGAL AND DISBURSEMENT ESTIMATE SAR EAP 12 BUDGETING: OPERATING BUDGET THAT REALLY WORKS EAP 64 POVERTY ASPECTS OF THE PRIVATE SEC. AND MARKET DEV SAR 13 CLINICS-WB GUARANTEES/RISK MITIGATION SERIES EAP 65 PRINCIPLES OF PROJECT COSTING SAR EAP 14 CONTRACT IN SUCCESSFUL PROJECT MANAGEMENT EAP 66 PROC. TRNG. FOR DISB. ANALYSTS & SR. FMD STAFF SAR 15 CORE CURRICULUM - PILOT I EAP 67 PROCUREMENT FOR FM STAFF SAR 16 COST AND PRICE ANALYSIS SEMINAR EAP 68 PROCUREMENT CONFERENCE: GLOBAL PROCUREMENT SAR 17 DETERMINING WHETHER HNP PROJ. REACH THE POOR EAP 69 PROCUREMENT DATA MANAGEMENT SAR EAP 18 DEVELOPMENTAL ASSIGNMENT EAP 70 PROCUREMENT FILING SYSTEM SAR 19 DISBURSEMENT FOR FM STAFF EAP 71 PROCUREMENT HUB RETREAT SAR 20 DISBURSEMENT FOR OPERATIONAL STAFF GA-GD EAP SAR 72 PROCUREMENT IN CDD (FORUM TRAINING) SAR 21 EAP ACS HUB TRAINING EAP 73 PROCUREMENT OF GOODS SAR EAP 22 EAP ADVANCED OPERATIONAL SKILLS HUB TRAINING EAP 74 PROCUREMENT REFORM CAPACITY BUILDING (FORUM TRG.) SAR 23 ENV. & SOCIAL SAFEGUARD POLICIES FOR NDO STAFF EAP 75 PROJECT CYCLE MANAGEMENT W/ THE LOGICAL FRAMEWORK SAR EAP 24 ESSD - WEEK CORE SKILLS TRNG.- FIN. MNT. FOR TLS EAP 76 PROJECT FINANCIAL MANAGEMENT FOR TASK LEADERS SAR EAP 25 ESSD INPUTS & INFLUENCES ON PRSPS: TAKING STOCK EAP 77 PROJECT FINANCIAL MANAGEMENT MODULE 1 SAR EAP 26 ESSD INPUTS AND INFLUENCE IN PRSPS EAP 78 PROJECT FINANCIAL MANAGEMENT MODULE II SAR EAP 27 ESSD WEEK CORE SKILL TRAINING - PROJ. CYCLE MNGT. EAP 79 PROJECT PLANNING AND PREPARATION SAR EAP 28 ESSD WEEK CORE SKILLS TRG.-TF TRG. FOR PROJ.LEADER EAP 80 PROJECT SUPERVISION AND IMPLEMENTATION SAR EAP 29 ESSD WEEK RELATED TRAINING - PROJ.SUPERV. & MONIT. EAP 81 QUALITY ASSURANCE & ARRANGEMENTS SAR 30 ESSD WEEK TRAINING-DISCLOSURE OF SAFEG. POL. DOC. EAP 82 QUALITY AT ENTRY & QUALITY OF SUPRV. FOR AFRI. REG SAR 31 ESSD WEEK TRAINING-WORKSHOP ON SAFEG.POLICIES EAP 83 QUALITY AT ENTRY FOR THE HD SECTORS SAR 32 ESSD WEEK: INTEGRATING GENDER INTO OPERATIONS EAP 84 RATIONALE & PROCES. OF INTEGR. SAFEG. DATA SHEETS SAR 33 FISCAL POLICY EAP 85 RESOURCE MANAGEMENT FORUM SAR 34 FUNDAMENTALS OF DESIGN II EAP 86 SAFEGUARD POL. WKSHOP FOR ECA MGRS.,SECTOR LEADERS SAR 35 FY02 FIDUCIARY FORUM EAP 87 SAFEGUARD POLICIES IN ORIENTATION NEW LEGAL STAFF SAR 36 HD CROSS-SECTORAL: QUALITY OF SUPERVISION EAP 88 SAFEGUARD POLICY & ENVIRONMENT ASSESSMENT SAR 37 HD WEEK EAP SAR 89 SAFEGUARD TRAINING FOR BANK STAFF SAR 38 HD WEEK 2002 EAP 90 SAFEGUARD WORKSHOP FOR THE ECA MANAGEMENT TEAM SAR 39 HISTORY OF DESIGN EAP 91 SAR HUB TRAINING LEVEL 1 - ACS SAR (Annex B continues on next page.) 46 (Annex B continued) 40 HUB LEVEL 1 EAP 92 SAR HUB TRAINING LEVEL 1 - H/L SAR 41 HUB TRAINING ON INTEGRAGED IT & OPERATIONAL SKILLS EAP 93 SELECTION OF CONSULTANTS SAR EAP 42 IBRD LOAN AND HEDGING PRODUCTS (SPECIAL) EAP SAR 94 SEMINAR ON DISPUTE REVIEW BOARD SAR 43 INCORPRATING SAF. NETS IN PRSP: REFLEC. ON SRI LAN EAP 95 SHORT-TERM CONSULTANT/SHORT-TERM TEMP. APPOINTMENT SAR 44 INTEGRATING CARBON FINANCE IN BANK OPERATIONS EAP 96 SPECIAL ISSUES IN PROCUREMENT OF IT SAR 45 INTRO. TO CLIMATE CHANGE AND ITS RELEV. TO BANK OP EAP 97 STRENGTH. OPERATIONAL SKILLS IN COM.-DRIVEN DEV. SAR 46 INTRO. TO RISK & DECISION MAKING IN PROG. & PROJ. EAP 98 TECHNICAL ASSISTANCE PLANNING AND PREPARATION SAR 47 INTRODUCTION TO BANK OPERATIONS EAP SAR 99 TEXTBOOKS OPERATIONAL GUIDELINES AND PROCUREMENT SAR 48 INTRODUCTION TO BANK OPERATIONS FOR FM STAFF EAP 100 TRAINING PROGRAM FOR RESIDENT REPRESENTATIVES SAR EAP 49 INTRODUCTION TO COMMERCIAL CONTRACTING EAP 101 TRUST FUND SAR 50 LEGAL OPERATIONS FOR TASK TEAMS EAP SAR 102 TRUST FUND STATUS REPORT SYSTEM FOR IDF SAR 51 LENDING PROJECT CYCLE EAP SAR 103 TRUST FUND TRAINING FOR PROJECT TEAM LEADERS SAR EAP 52 LETTER OF CREDIT BASICS AND ITS LATEST DEVELOPMENT EAP 104 TRUST FUND WORKSHOP SAR # BANK OPERATIONS (119 ) (cont.) REGION # BEHAVIORAL AND SOCIAL (58) (cont.) REGION 105 TRUST FUNDS & TRS SAR 37 INTEGRITY AWARENESS SEMINAR EAP SAR 106 TRUST FUNDS LOR (LETTER OF REPRESENTATION) SAR 38 LEADING & MANAGING IN A TEAM-BASED ENVIRONMENT I SAR EAP 107 TRUST FUNDS MONITORING DEMOS SAR 39 MANAGERIAL EFFECTIVENESS AND PERSONAL GROWTH SAR 108 TRUST FUNDS TOOLKIT SAR 40 MANAGING YOUR TIME PRODUCTIVELY SAR EAP 109 USING KNOWLEDGE FOR DEV.: A COURSE FOR BANK STAFF SAR EAP 41 MASTERING THE MEDIA INTERVIEW EAP 110 WBS ENV. & SOCIAL SAFEGUARD POLICIES FOR NDO SAR 42 MCD 3- COMMUNICATION AS POWER EAP 111 WB'S ENVIRONMENT &SOCIAL SAFEGUARD POLICIES SAR 43 ORAL PROFICIENCY INTERVIEW SAR EAP 112 WORKSHOP FOR COUNTRY OFFICE HR STAFF SAR EAP 44 PRESENTING IDEAS EFFECTIVELY SAR EAP 113 WORKSHOP ON COORD. COUNTRY ENVIRONMENTAL ANALYSIS SAR 45 PURSUING A DEGREE SAR EAP 114 WORKSHOP ON OD 4.20: INDIGENOUS PEOPLES SAR 46 RESOURCES TO IMPROVE YOUR COMMUNICATION SKILLS SAR EAP 115 WORKSHOP ON OP 4.37: SFETY OF DAMS SAR 47 SASRD/ACS RETREAT/ENHANCING COMMUNICATION SKILLS SAR 116 WORKSHOP ON SAFEGUARD POLICIES FOR SAR STAFF SAR 48 SEXUAL HARASSMENT: ISSUES & CONCERNS SAR 117 WORKSHOP ON THE STATE-OF-THE-ART IN OP 4.12 SAR 49 SKILLS CRITICAL ADDR.& RESOLVE WORKPLACE DISPUTES SAR EAP 118 WORLD BANK/DTT INTERNATIONAL ACCOUNTING WORKSHOP SAR 50 STRESS MANAGEMENT: FUNDAMENTALS FOR EMPLOYEES EAP 119 WORKSHOP ON SAFEGUARD POL.: A GEN. OVERV. SAR 51 TASK PLANNING TOOL IN SAP SAR # BEHAVIORAL AND SOCIAL (58) REGION 52 TASK TEAM LEADER MODULE I EAP 1 ACHIEVING PEAK PERFORMANCE EAP 53 TEAM BUILDING SAR 2 ADMIN. SEMINAR SAR 54 TEAM WORK: GETTING PEOPLE TO WORK TOGETHER SAR 3 ANTI HARASSMENT ADVISORS TRAINING SEMINAR SAR 55 THE CHALLENGE OF LEADERSHIP- ETHICS SAR EAP 4 ANTI HARASSMENT AWARENESS SEMINAR EAP 56 THE ISG MASTER SERVICE LEVEL AGREEMENT SAR 5 ANTI-HARASSMENT BRIEFING EAP 57 TIME MANAGEMENT EAP SAR 6 ATTAINING BUSINESS GOALS THROUGH PEOPLE MANAGEMENT EAP SAR 58 TLLLAB - LEADING & MANAGING IN A MATRIX ENV. EAP 7 AUTHENTIC MANAGER EAP # INFORMATION TECHNOLOGY (80) REGION 8 BASIC MEDIATION SKILLS EAP SAR 1 3 C'S OF SUCCESSFUL IT PROJECTS EAP 9 BRINGING NEW MEMBERS INTO AN EXISTING TEAM EAP 2 ACCESS 2000 INTRODUCTION SAR EAP 10 BUILD. YR. SKILLS FOR WORKING IN A TEAM BAS ENV BLE EAP SAR 3 ADOBE PHOTOSHOP EAP 11 BUILD.YOUR SKILLS FOR WORKING -TEAM BASED ENV. COH EAP SAR 4 ADVANCED EXCEL SAR 12 BUILD.YOUR SKILLS-COHORTI EAP SAR 5 ADVANCED LEVEL COURSE ON EXCEL/POWERPOINT SAR 13 BUILDING YOUR SKILLS - COHORT II EAP SAR 6 ADVANCED MS WORD SAR 14 BUILDING YOUR SKILLS IN TEAM-BASED ENVIRONMENT II SAR EAP 7 BEGINNING FLASH 5 EAP (Annex B continues on next page.) 47 (Annex B continued) 15 COACHING SKILLS FOR LEADERS EAP SAR 8 BUSINESS RELEVANCE OF TELECOM. TECHNOLOGIES SAR 16 CONDUCTING MEETINGS: EFF. MEETING COMMUNICATION EAP 9 BUSINESS WAREHOUSE SAR EAP 17 CONFLICT ANALYSIS SAR EAP 10 BUSINESS WAREHOUSE (BW) PART 1 SAR 18 CREATING EFFECTIVE PRESENTATIONS SAR EAP 11 BUSINESS WAREHOUSE FOR RM STAFF EAP SAR 19 CREATIVITY & INNOVATION: THINKING CREATIVELY EAP SAR 12 BUSINESS WAREHOUSE TRAINING PART 1 EAP SAR 20 CRITICAL THINKING SAR EAP 13 DEMYSTIFYING SEARCH FOR INFORMATION SAR EAP 21 DEALING WITH CONFLICT EAP SAR 14 DIPLOMA IN MULTIMEDIA & WEB DESIGNING (SEMESTER-1) SAR 22 DEALING WITH CONFLICT FOR MANAGERS SAR EAP 15 E.DESKTOP 3.0, MS OFFICE 2000, LOTUS NOTES R5 SAR 23 EFFECTIVE WORKPLAN CONVERSATION SAR EAP 16 ED 3.0 AND LOTUS NOTES R5 SAR 24 ESSENTIALS OF DECISION MAKING SAR 17 E-HR: USING THE INTRANET AND YOURNET SAR EAP 25 ETHICS EAP 18 ELECTRONIC DESIGN II (QUARKXPRESS, PHOTOSHOP,ILLUS EAP 26 ETHICS AWARENESS TRAINING EAP 19 ENTERPRISE 3.0. MICROSOFT OFFICE AND LOTUS NOTESR5 SAR 27 ETHICS AWARENESS TRAINING, WORLD BANK OFFCE MANILA EAP 20 ENTERPRISE DESKTOP 3 EAP 28 ETHICS INTEGRITY AWARENESS TRAINING EAP 21 ENTERPRISE DESKTOP 3 (ED3) TRAINING EAP 29 EXEC DEV PRGM FOR TEAM LEADERS: MODULE I SAR EAP 22 EXCEL 2000 CHARTS AND DATABASES EAP SAR 30 EXEC DEV PRGM FOR TEAM LEADERS: MODULE II SAR EAP 23 EXCEL 2000 GOAL SEEKING, AUDITING & CONSOLIDATION SAR EAP 31 GIVING & RECEIVING FEEDBACK EAP 24 EXCEL 2000 INTERMEDIATE EAP 32 HARVARD EXECUTIVE DEVELOPMENT PROGRAM SAR EAP 25 EXCEL 2000 INTRODUCTION SAR EAP 33 HOW THE BANK IS HARDWIRED EAP 26 EXCEL 2000 MACROS EAP 34 IDENTIFYING STAFF TRAINING NEEDS SAR EAP 27 EXCEL 2000 PIVOT TABLES SAR EAP 35 IMPROVING COMM. BY UNDERSTANDING PERS. STYLE EAP 28 HDM-4 AND RED TRAINING EAP SAR 36 INITIATIVES IN DIVERSITY IN THE WORLD BANK SAR 29 HTML AND JAVASCRIPT COURSES EAP # INFORMATION TECHNOLOGY (80) (cont.) REGION # PROFESSIONAL AND TECHNICAL (224) REGION 30 HTML INTRODUCTION SAR EAP 1 2002 - THE YEAR OF THE MOUNTAIN EAP SAR 31 HTML LEVEL 1 SAR 2 7TH ANNUAL MICROFINANCE TRAINING PROGRAM EAP 32 HTML LEVEL 2 SAR 3 A COMM.-BASED INTERVENTION STUDY OF INDOOR AIR POL EAP SAR 33 IMPLEMENTING & ADMINISTERING MS-W2K DIRECTORY SERV EAP 4 A DRAMA IN AT THE MIDST OF SUCCESS: POLLISH EMPLOY SAR 34 IMPLEMENTING AND ADMINISTERING WIN 2000 AD EAP 5 A PRO OFFERS PROJ. DES. INSIGHTS: PRI FERNANDO EAP 35 INTRODUCTION - DREAMWEAVER 4 SAR EAP 6 ACCUM. OF EDUC. IN MODERN ECON. GROTH: A COMPARATI EAP 36 INTRODUCTION TO COSTAB SAR EAP 7 ADAPTING TO CHANGE: CORE COURSE ON POP.,REPR.HLTH SAR 37 INTRODUCTION TO LOTUS NOTES R5 EAP SAR 8 ALTERNATIVES FOR IMPLEM. HUMAN CAPITAL CONTRACTS EAP 38 INVENTORY MANAGEMENT BARCODE SCANNER SAR 9 AN INTEGRATED APPROACH TO THE PROC. OF HEALTH SEC. EAP SAR 39 IRIS EAP SAR 10 ART OF LIVING COURSE SAR 40 IRIS 3.0: INTEGRATION RECORDS & INFORMATION SYSTEM SAR 11 ASSESSING DAMAGES AND NEEDS SAR 41 IRIS3 SAR 12 ATTACKING POVERTY COURSE EAP SAR 42 IRIS3 INFORMATION ASSISTANT'S COURSE SAR 13 BANK LENDING INSTRUMENTS & FINANCIAL PRODUCTS SAR EAP 43 ISN IT LEARNING SERIES ON MICORSOFT POWERPT EAP 14 BASIC ECONOMICS IN A GLOBAL PERSPECTIVE SAR EAP 44 IT LEARNING SERIES FOR COUNTRY OFFICE STAFF EAP 15 BIOTECHNOLOGY, BIODIVERSITY, AND FOOS SECURITY SAR EAP 45 IT LEARNING SERIES FOR HEADQUARTERS STAFF EAP SAR 16 CDD PART I-SAFEGUARDS FOR CDD IN SOCIAL FUNDS SAR EAP 46 IT ORIENTATION FOR NEW STAFF SAR 17 CERTIFICATE EN DROITS DE L'HOMME (HUMAN RIGHTS) EAP 47 MAIL MERGE IN WORD SAR EAP 18 CHILD MALNUTRITION AND FOOD SUBSIDES IN ASIA SAR 48 MICROSOFT ACTIVE DIRECTORY EAP 19 CHILD. AND YOUTH IN DEV.: BACKGROUND, APPROACHES SAR 49 MICROSOFT EXCEL 2000 VISUAL BASIC FOR APPLICATION EAP 20 CHILDREN & YOUTH IN DEV.: ISSUES AND PERSPECTIVES SAR 50 MS ACCESS, MS POWERPOINT 2000 SAR 21 CHILDREN, YOUTH, THE UNGASS, & THE BANK TOWARDS HD SAR 51 MS EXCEL SAR 22 CLEAN AIR STRAT. WORKSHOP IN ASIA AND THE PACIFIC EAP (Annex B continues on next page.) 48 (Annex B continued) 52 MS EXCEL 2000 FUNDAMENTALS EAP 23 CLEAN FUELS/CLEAN AIR EAP SAR 53 MS POWER POINT SAR 24 CLIENT CAPACITY BUILDING IN FINANCIAL MANAGEMENT SAR EAP 54 MS PROJECT -BASIC & ADVANCED CONCEPTS SAR 25 CLINIC ON FIN. INCENTIVES FOR SMES IN RURAL ENERGY SAR 55 ON-LINE LEARNING TOOLS FOR SUCCESS SAR EAP 26 CLINIC SERIES ON OUTPUT-BASED AID - OBA BASIS SAR EAP 56 ON-LINE VIDEOCONFERENCE: WEB-CT EAP 27 COFFEE CRISIS: CONSEQUENCES & POSSIBLE ANSWERS EAP 57 ORACLE - DEVELOP PL/SQL PROGRAM UNITS EAP 28 COMMON ISSUES, SHARED SOLUTIONS (CISS) ON MID-TERM SAR EAP 58 ORACLE - INTRODUCTION TO SQL & PL/SQL FUNDAMENTALS EAP 29 COMMUNITY BASED PROCUREMENT AND SMALL CONTRACTS SAR EAP 59 PHOTOSHOP 6 INTRODUCTION SAR EAP 30 COMMUNITY-BASED APPROACHES TO HEALTH: DO THEY WORK SAR EAP 60 PHOTOSHOP BASICS & GRAPHIC DESIGN FOR THE WEB EAP 31 COMMUNITY-DRIVEN DEVELOPMENT & COMMUNICATIONS EAP 61 POWERPOINT 2000 CHARTS SAR EAP 32 CONDITIONAL CSH TRANSF.: DEMAND-SIDE INNOV. IN S.P EAP 62 POWERPOINT 2000 INTRODUCTION SAR EAP 33 CONFLICT & NATURAL RES. MGT.: PERSPECTIVES ON NRM SAR EAP 63 POWERPOINT 2000 SLIDESHOWS AND ANIMATION SAR EAP 34 CONTRACTUAL SAVINGS CONFERENCE SAR 64 PROCESSING OF INTEGRATED POL. DATA SHEETS (ISDS) SAR EAP 35 CORE CORSE: POPUL, REPR. HEALTH AND HEALTH REFORM SAR 65 SAP - INTRODUCTION ETC. SAR 36 COUNTRY FINANCIAL ACCOUNTABILITY ASSESSMENT SAR EAP 66 SAP ACTIVITY PLANNING/MONITORING SAR 37 COUNTRY TRADE UNION PERSPECT. ON THE PRSP PROCESS SAR EAP 67 SAP -TIME & LARS SAR 38 CULTURE & INSTITUTIONS-INCORP. CRITICAL ELEMENTS EAP 68 SAP: BUDGET PLANNING & MONITORING SAR 39 DATA MANAGEMENT: CASE STUDIES IN THE LDB SAR EAP 69 SOFTWARE PROJECT DEVELOPMENT SAR 40 DECENTRALIZATION & INTERGOVERNMENTAL FISCAL REFORM SAR EAP 70 SOFTWARE TRAINING: ED 3.0, NOTES R5, OFFICE 2000 SAR 41 DELIVERING COMMUNITY-BASED SOCIAL SERVICES EAP 71 THE INTERNET SERVICES PROGRAM SAR 42 DESIGN. & BLDG. PERF.-BASED MONIT. & EVAL. SYSTEMS SAR 72 TRAINING ON MS PROJECT SAR 43 DESIGN. EFFC. SOCIAL FUNDS FOR A DECENTR. CONTEXT SAR EAP 73 USE OF MS PROJECT 2000 FOR MANAGING PROJECTS SAR EAP 44 DESIGNING AN EFFECTIVE TRAINING CURRICULUM EAP 74 USING MS WORD TO CREATE AND ORGANIZE WEBSITES EAP 45 DESIGNING RURAL POVERTY REDUCTION STRATEGIES SAR EAP 75 WEB DESIGNING-HTML,JAVASCRIPT,DEARMWEVERALTRADEV SAR 46 DEV. AGRIBUSINESS AND AGRO-FOOD SYS.: IMPACT ON PO EAP 76 WORD 2000 COMPLEX DOCUMENTS SAR EAP 47 DEVELOPMENT DATA TOOLS SAR 77 WORD 2000 DESKTOP PUBLISHING SAR EAP 48 DOC CREDIT & ELECTRONIC UNIFORM CUSTOMS & PRACTICE SAR 78 WORD 2000 INTERMEDIATE EAP 49 DYNAMIC PANEL ECONOMETRICS I EAP 79 WORD 2000 MAIL MERGE SAR 50 ECONOMIC ANALYSIS IN THE EDUCATION SECTOR SAR EAP 80 WORD 2000 TABLES SAR EAP 51 ECONOMIC FUNDAMENTALS IN COMPREHENSIVE DEVELOPMENT SAR 52 ECONOMIC STATISTICS SAR EAP # PROFESSIONAL AND TECHNICAL (224) (cont.) REGION # PROFESSIONAL AND TECHNICAL (224) (cont.) REGION 53 ED-SIDA: A PLAN. TOOL FOR PROJ. THE IMPACT OF HIV/ SAR 105 INVESTMENT CLIMATE POLICY EAP 54 EDUCATION IN AFGHANISTAN SAR EAP 106 ISSUES IN PUB. SECTOR FINANCIAL MANAGEMENT SAR EAP 55 E-FINANCE IN EMERG. MARKETS: IS LEAPFROGGING POSS. EAP 107 KNOWLEDGE AND SKILLS FOR THE NEW ECONOMY: A CONVER SAR EAP 56 E-LEARNING: UNIV OF PHOENIX ON-LINE ASYNCH LEARN EAP 108 LABOR MARKETS-LIBERALIZATION (LABOR CODE)/PASIVE SAR 57 EMERGING FROM ETHNIC CONFLICT: CHAL. FOR SP DESIGN SAR 109 LIFELONG LEARNING AND KNOW. ECON. THE DANISH EXAMP SAR EAP 58 ENERGY BBLS AND CLINICS SAR EAP 110 MACROECONOMETRICS USING EVIEWS EAP SAR 59 ENVIRONMENTAL & SOCIAL ASSESSMENT REVIEW EAP 111 MACROECONOMIC MODELING FOR POVERTY REDUCTION SAR 60 EXECUTIVE GRADUATE PROGRAM IN FINANCE EAP 112 MADRASSAS (KORANIC SCHOOLS) IN MUSLIN SOCIETIES SAR EAP 61 EXPLAN. OF RECENT PH.D. STUDY WHICH ANAL. BALOCHIS SAR EAP 113 MAINSTREAMING THE ENVIRONMENT IN PRSPS EAP 62 FINANCE FOR GROWTH EAP 114 MASTER SERIES IV - STAKEHOLDER CONSULTATIONS WKSHP SAR EAP 63 FINANCIAL MARKETS & DEVELOPMENT CONFERENCE SAR EAP 115 MEASURING THE EFFECT. OF PUBL. HEALTH PROGRAMS SAR 64 FLAGSHIP CORE COURSE ON HEALTH SECTOR REF. & SUST. SAR 116 MEDIA EXPERTISE TRAINING SAR EAP 65 FOLLOW-UP TO THE WORLD COMMISSION ON DAMS REPORT SAR EAP 117 MEETING THE EDU. MDG: A NEW POLICY & FIN. FRAMWORK SAR EAP 66 FOREST LAW ENVORCEMENT AND GOVERNANCE EAP 118 MERIDIAN 1 OPTION 11C TECHNICIAN COURSE (PABX) EAP (Annex B continues on next page.) 49 (Annex B continued) 67 FROM WHOM SHOULD PUBLICLY FIN. MED. SERV. BE PURCH SAR 119 MICROF. IN THE TAJIKISTAN SOCIAL INVEST. FUND EAP 68 FSE/INT'L. SEM. ON LEGAL INSTIT.ASPECTS OF FIN.-SW EAP 120 MICROF. INSIT.SELF-SUFFICIENCY & COMM. TO THE POOR SAR EAP 69 FUNDAMENTALS OF FINANCE SAR EAP 121 MICRO-MACRO LINKAGES IN THE DESIGN OF GROWTH & POV SAR EAP 70 GET. THE BOOKS TO THE KIDS: NEW DIR. IN TEXTBOOKS EAP 122 MIGRATION FOR CHANGE & SUSTAINABLE DEVELOPMENT SAR 71 GETTING PILLS TO PATIENTS SAR 123 MONEY MATTERS: A FINANCIAL MARKET LITERACY COURSE EAP SAR 72 GLAGSHIP CORE COURSE ON HEALTH SEC.REF. & SUST.FIN SAR 124 MONIT. & EVAL. FOR HD SECTORS (HD WEEK CROSS-SECT. SAR EAP 73 GLOBAL ALLIANCES FOR RURAL DEVELOPMENT SAR 125 MULTI-TOPIC HOUSEHOLD SURVEYS SAR 74 GOODS REQUISITIONS & QUICK POS SAR 126 MULTI-TOPIC HOUSEHOLD SURVEYS-POV. & INEQUALITY AN SAR EAP 75 GROUNDWASTER MNGT. STUDY TOUR-NEW MEXICO, ARIZONA SAR EAP 127 NATION BLDG.: HYPE AND HOPE. THE CASE OF THE EAST SAR EAP 76 HEALTH OUTCOMES IN POOR COUNTRIES & POL. OPTIONS SAR 128 NEW CHALLENGES FOR TERTIARY EDUCATION: A DISCUSSION SAR EAP 77 HIGHER EDUC. AND THE STATE IN MONGOLIA: DILEMAS" EAP 129 NHA & PUBLIC EXPENDITURE SAR EAP 78 HIV/AIDS AND HNP LEARNING DAYS SAR EAP 130 OFFICE OF THE OMBUDSMAN EAP 79 HIV/AIDS PREVENTION AND WOMEN AND GIRLS EMPOW. PRO SAR 131 OPERATIONALIZING THE EMPOWERMENT AGENDA SAR EAP 80 HR FORUM SAR EAP 132 PART I: POV. & SOC. IMPACT ANAL.: FRAMEWORK & SOC. SAR EAP 81 HR TRAINING FOR HR COUNTRY OFFICE ADMINISTRATORS SAR EAP 133 PARTICIPATORY MONITORING AND EVALUATION SAR EAP 82 HUMAN RESOURCE FIELD ADMIN RETREAT, MANILA EAP 134 PARTICIPATORY RESEARCH ASSESSMENT EAP 83 HUMAN RIGHTS & SUSTAINABLE DEVEL. BKS. ROLE SAR 135 PILOT PROJECT FOR THE PREQUALIF. OF HIV/AIDS DRUGS EAP 84 ICT IN EDUC. APPR. TO BRIDGING THE DIGITAL DIVIDE SAR EAP 136 POLICY CHANGES FOR THE FINANCIAL SECTOR IN CONTEXT EAP 85 IMPACT EVALUATION TOOLS AND TECHNIQUES SAR EAP 137 POLIO ERADICATION SEMINAR SAR 86 IMPROV. BASIC HEALTH SERV.FOR THE POOR BY ENGAGING SAR 138 POVERTY AND ENVIRONMENT LINKS EAP 87 INDIA'S DISTRICT PRIM. EDUC. PROG: HOW SUCCESSFUL" SAR EAP 139 POVERTY MON. IN THE CONTEXT OF POV. RED. STRATEGY SAR 88 INNOVATIONS IN COMM. DEV.: HUMAN-SCALE DEV. A PRAC SAR 140 POVERTY MONITORING & EVAL. IN RURAL AREAS - II EAP 89 INNOVATIVE FINANCING FOR SUSTAINABLE DEVELOPMENT SAR EAP 141 POVERTY MONITORING AND EVAL. IN RURAL AREAS - I EAP 90 INSECT LIFE (GRADE: A) EAP 142 POVERTY REDUCTION STRATEGIES FORUM FOR BANK/FUND SAR EAP 91 INTEGRATED APPROCH TO LEARNING SAR EAP 143 POVERTY, AIDS & CHILDREN'S SCHOOLING: A TARGET DIL EAP 92 INTEGRATED OPERATIONAL & TECHNOLOGY SKILLS EAP 144 PRACTICAL MANAGEMENT SKILLS FOR THE EXEC.SECRETARY EAP 93 INTEGRATED OPERATIONAL & TECHNOLOGY SKILLS FOR ACS EAP 145 PRODUCTIVITY AND SECURITY: THE GENDER CONNECTION EAP 94 INTEGRATED OPERATIONAL AND TECHNOLOGY TRAINING, VN EAP 146 PROJECT ANNOUNCEMENT: REACHING THE POOR SAR 95 INTEGRATING THE ANALY. OF POV. AND SOCIAL CAPITAL SAR EAP 147 PROMOTING GROWTH AND DEV. IN LAGGING REGIONS SAR EAP 96 INTERNAL COMMUNICATIONS: WHAT'S IN IT FOR ME? EAP 148 PUBL. ATTIT. MATTER: A CONCEPTUAL FRAMEWK. FOR ACT SAR 97 INTERNATIONAL E-COMMERCE SAR EAP 149 PUBLIC EXP. ANAL. AND MNGT. CORE COURSE SAR EAP 98 INTERNATIONAL WATERS: CHALLENGES AND OPPORTUNITIES SAR 150 PUBLIC EXPENDITURE POLICY IN THE RURAL CONTEXT SAR EAP 99 INT'L TRADE SEMINAR - GATS SAR EAP 151 PUBLIC HEALTH FUNCTIONS & DISEASE SURVEILLANCE SAR EAP 100 INT'L TRADE SEMINAR -ANTI-DUMPING AND SUBSIDIES SAR EAP 152 PUBLIC PRIVATE PARTNERSHIPS IN EDUCATION EAP 101 INT'L WORKSHOP: GLOBAL DEVELOPMENTS IN ACCOUNTING EAP 153 PUTTING KNOWLEDGE TO USE IN OPERATIONAL WORK-AFRIC SAR 102 INT'L. MACROECON. TOOLS FOR FISCAL SUSTAIN. ANAL. SAR EAP 154 QUALITATIVE & MIXED METHODS SAR EAP 103 INTRODUCTION TO PROGRAM EVALUATION EAP SAR 155 QUALITATIVE & MIXED METHODS - MOD 2 SAR EAP 104 INVEST. EFFEC. IN MATERNAL HEALTH: MALAYSIA & SRI SAR EAP 156 QUALITY ASSURANCE IN THE GLOBAL EDUCATION MARKET EAP # PROFESSIONAL AND TECHNICAL (224) (cont.) REGION # PROFESSIONAL AND TECHNICAL (224) (cont.) REGION 157 RAPID LABOR REALLOC. WITH A STAGNANT UNEMP. POOL SAR 210 VOCATIONAL SKILLS DEV. AND UNSCHOOLED ADULTS EAP 158 REACHING THE RURAL POOR SAR EAP 211 VULNERABILITY: MEAS. & POLICY IMPLICATIONS, EXP. SAR EAP 159 READING: MATHEMATICS & SCIENCE ACHIEV. IN 32 DEV. SAR EAP 212 WATER & SANITATION BBLS SAR EAP 160 RES. FOR HNP STAFF WORKING ON PRSPS AND/OR ON EQUI SAR EAP 213 WATER FORUM EAP 161 RISK AND VULNERABILITY ASSESSMENTS EAP 214 WATER RIGHTS AND THEIR APPLICATION EAP 162 RISK MANAGEMENT WORKSHOP FOR REGULATORS SAR 215 WEATHER AND CLIMATE (GRADE: A) EAP (Annex B continues on next page.) 50 (Annex B continued) 164 ROUNDTABLE FOR W&S FOR PRIVATE PARTICIPATION SAR EAP 216 WHAT IS EMOTIONAL INTELLIGENCE? AND DO I HAVE IT? SAR EAP 165 ROUNDTABLE SERIES SAR EAP 217 WHAT'S NEW AT JOLIS SAR 166 RURAL POVERTY DIANOSTICS SAR EAP 218 WILL FERTILITY EVER DECLINE IN SUB-SAHARAN AFRICA? SAR 167 S.A. NATIONAL ASSESSMENT OF EDUC. ACHIEVEMENT SAR 219 WKSHOP ON THE STATE OF THE ART IN OP4.04: NATURAL SAR EAP 168 SAFETY NETS CORE COURSE SAR 220 WORKING ACROSS SECTORS IN RURAL SPACE SAR EAP 169 SAFETY NETS: TARGETING AND PROXY MEANS TESTING SAR 221 WORKSHOP ASSESSMENT INDUCED & CUMULATIVE EFFECTS EAP 170 SCH. RESOURCES, EDUC. INST. & STUDENT PERFORMANCE SAR 222 WORLD BANK & BROOKINGS INST. CONF.ON FIN. MARKETS SAR EAP 171 SCHOOLING PROFILES: NEW METHOLOGY TO ESTIMATE SCH. SAR 223 YOUTH AND DEVELOPMENT SAR EAP 172 SECURITY: VULNERABILITY & ADAPT. TO CLIMATE CHANGE EAP 224 ZAT/LAC, ISDS/RACMS & DISCLOSURE POLICY TRNG. SES EAP 173 SMALL AND MEDIUM ENTERPRISE DEVELOPMENT SAR EAP # LANGUAGE (9) REGION 174 SOC. & PUB. ACCOUNTABILITY: PARTICIP. APPROACHES SAR EAP 1 ENGLISH - ORAL SKILLS EAP SAR 175 SOC. PROTEC. IN ASIA & THE PACIFIC: THE ADB'S S.P. SAR EAP 2 ENGLISH SKILL BUILDING: GRAMMAR IN CONTEXT SAR EAP 176 SOCIAL ANALYSIS FOR INVESTMENT PROJECTS SAR EAP 3 FRENCH - GROUP CLASSES SAR EAP 177 SOCIAL PROTECTION EXPENDITURE, PERFOR. & FIN. REV. EAP 4 FRENCH - LANGUAGE TESTING SAR EAP 178 SOCIAL RISK MANAGEMENT AND VULNERABILITY SAR EAP 5 FRENCH - PRIVATE CLASSES SAR EAP 179 STAKEHOLDER CONSULTATION WORKSHOP SAR EAP 6 PORTUGUESE - PRIVATE CLASSES EAP 180 STAKEHOLDER CONSULTATIONS EAP 7 SPANISH - GROUP CLASSES SAR 181 STANDARDS & AGRIC. TRADE: THE DEV. DIMENSION SAR 8 SPANISH - LANGUAGE TESTING SAR EAP 182 STATEMENT OF EXPENDITURE (SOE) EAP 9 SPANISH - PRIVATE CLASSES SAR EAP 183 STRATEGIC CHOICES FOR EDUCATION REFORM EAP SAR # OFFICE SKILLS (13) REGION 184 STRATEGIC CHOICES FOR TERTIARY EDUCATION REFORM SAR 1 CHINA EAP REPORT WRITING HUB EAP 185 STRATEGIC ENVIRONMENTAL ASSESSMENT SAR EAP 2 EDITORIAL SKILLS I SAR EAP 186 SUCC. & FAILURES OF PENSION PLANS: FROM ARGENTINA EAP 3 EDITORIAL SKILLS II SAR EAP 187 SUSTAINABLE FOREST MNGT. GLOBAL POLICY EXCHANGE 1 SAR EAP 4 EVERYTHING YOU EVER WATED TOKNOW ABOUT WB PUB EAP 188 TAGETING AND SEFT-TARGETING OF SOCIAL ASSISTANCE SAR 5 MEMO WRITING WORKSHOP: PART I EAP SAR 189 TEXTBOOKS, CURRICULA AND SOCIAL COHESION SAR EAP 6 MINUTE WRITING EAP SAR 190 THE ART AND SCIENCE OF COMMUNICATIONS SAR EAP 7 REPORT WRITING (TUTORIALS) SAR EAP 191 THE ECONOMICS OF CIVIL WAR SAR EAP 8 REPORT WRITING: PART I SAR EAP 192 THE EMERGENCE OF HYBRID PEN. PLANS AND THEIR IMPL. SAR 9 SPEED READING & READING SKILLS PROGRAMS EAP 193 THE MILLEN.DEV. GOALS - MONIT. THE HEALTH, NUTRIT. SAR EAP 10 THE 10 SECRETS OF BUSINESS WRITING SAR 194 THE POLITICAL ECON. OF PENSION REFORM: THE OPINION SAR 11 TYPOGRAPHY II EAP 195 THE RELATION. BEWEEN FORMAL AND FAM. SUP. OF ELDER EAP 12 WRIT.YOUR FUTURE - HOW TO WRITE A TOP NOTCH APPL. EAP SAR 196 THE ROLE OF UNESCO INSTIT. FOR STATISTICS (UIS) SAR EAP 13 WRITING WORKSHOP EAP SAR 197 TOO MUCH AND TOO LITTLE: THE EXPL. IN NUTRITION SAR EAP 198 TOOLS FOR IMPLEMENTING THE GENDER STRATEGY SAR EAP 199 TOOLS FOR TRADE POLICY ANALYSIS: QUANTIFYING TRADE SAR EAP 200 TOWARD AS SOCIAL PROTECTION STRATEGY OF ZIMBABWE SAR 201 TRAININ GIN HOSPITAL REFORM AND AUTONOMY SAR 202 TRAINING SESS. ON SAFEGUARD POL. IN WATER RESOURCE SAR EAP 203 TRANSPORT & POVERTY TOOL KIT AND CLINICS SAR 204 TRANSPORT LUNCHTIME LECTURE SERIES EAP 205 UNDERSTANDING THE ENERGY CHALLENGES SAR 206 UPDATES ON INTERNATIONAL ACCOUNTING STANDARDS EAP 207 URBAN FORUM 2002 SAR EAP 208 URBAN THEMATIC GROUPS BROWN BAG LUNCH SERIES SAR EAP 209 VIDEOCONF. CLINIC: MICROF. & VILLAGE SELF-HELP SAR 51 ANNEX C: SUMMARY OF AVERAGE DAYS OF STAFF LEARNING IN EACH COURSE FIELD Fiscal year Learning Days 99 00 01 02 99-02 Operations 2.6 1.9 0.9 1.6 6.9 Behavioral 0.9 0.5 1.2 3.2 15.5 Professional 1.3 1.4 1.0 1.3 5.0 IT 0.4 0.3 0.6 0.2 1.6 Language 0.4 0.2 0.1 0.0 0.8 EAP Office Skills 0.2 0.2 0.1 0.1 0.6 Other 0.3 0.0 0.0 0.0 Internal 4.6 3.7 3.0 2.9 7.4 External 1.5 1.1 0.9 1.0 0.4 99-02 6.1 4.7 3.8 3.9 18.5 Operations 3.4 2.8 0.8 1.7 8.7 Behavioral 1.0 1.0 0.6 0.4 5.8 Professional 1.5 2.0 1.4 1.0 3.0 IT 0.4 1.0 0.3 0.3 1.9 Language 0.2 0.2 0.0 0.1 0.6 SAR Office Skills 0.2 0.1 0.1 0.0 0.5 Other 0.3 0.1 0.0 0.0 0.1 Internal 5.8 5.2 2.7 2.9 8.0 External 1.2 2.0 0.6 0.6 0.6 99-02 7.0 7.2 3.3 3.5 21.0 52 ANNEX D: STAFF SURVEY QUESTIONNAIRE I. About your participation in learning 1. Did you have a Learning Plan endorsed by your manager for: Yes No a. FY03 ? ? b. FY02 ? ? c. FY01 ? ? 2. Did you receive sufficient budget/funding to implement your Learning Plan in: Yes No Don't know a. FY03 ? ? ? b. FY02 ? ? ? c. FY01 ? ? ? 3. Did you participate in any learning events over the past 12 months? ? Yes (Survey will skip to Q. 5) ? No (Survey will skip to Q. 4 and then the demographics) 4. What were your reasons for not participating in a learning event? (Please, select as many as apply) q Not interested in the events offered q Unable to get away from work schedule q Timeframe of event was too lengthy q Not enough budget allocated q Supervisor did not approve q Other (specify) ______________________________________ 53 5. Please provide a breakdown of your participation in learning events over the past 12 months: Type of learning Number of Total Events Number of days Professional and technical ___________________ __________________ Bank operations ___________________ __________________ Information technology ___________________ __________________ Behavioral and social ___________________ __________________ Office skills ___________________ __________________ Language ___________________ __________________ 6. Of the learning events you have participated in over the past 12 months, how many of them are internal (organized by the Bank or your region) events and how many are external (organized by an outside party) events? Internal External Region Offered __________ Overall Total__________ Bank Offered __________ 7. What are your main reasons or motivation for participating in learning in general? (Please select up to three responses) q To learn new knowledge q To learn new skills q To improve your existing skills q To improve your understanding in the area q To prepare you for greater responsibilities q To obtain a pre-requisites for further coursework q To network and share information q For professional interest and growth q Other (specify) _______________________________________ 54 II. Your opinion on the quality of the learning events 8. By what modes were the learning events you participated in over the past 12 months presented? Number of events Workshop/Clinic (3 days or less) __________ Conference __________ Course (5 days or more) __________ Global dialogue (video conferencing sessions) __________ Seminar (1-5 days) __________ Study tour __________ Online (self-placed CD-ROM/web based) __________ Other (specify) __________ 9. Of the above delivery modes, which three did you find to be the most effective? Most effective __________ Second most effective __________ Third most effective __________ 10. Did you have to travel outside your duty station to participate in learning events over the past 12 months? ? Yes ? No 10a. If yes, what was the furthest you had to travel? ________________ 11. How would you rate the overall quality of learning events offered by your region in general? ? Very good ? Good ? Average ? Poor ? Very poor 55 12. When recalling the regionallearning events that you participated in over the past 12 months, how would you rate the degree to which the learning events: Select one button for each item To a To To To a Not at Great Considerable Some Limited All Extent Extent Extent Extent a. Were relevant to your work ? ? ? ? ? b. Addressed your learning needs ? ? ? ? ? c. Increased your knowledge and skills ? ? ? ? ? d. Met your expectations ? ? ? ? ? e. Were useful to the skills required ? ? ? ? ? for your job 13. How would you rate the overall quality of learning events offered by the Bank in general? ? Very good ? Good ? Average ? Poor ? Very poor 14. When recalling the Bank learning events that you participated in over the past 12 months, how would you rate the degree to which the learning events: Select one button for each item To a To To To a Not at Great Considerable Some Limited All Extent Extent Extent Extent a. Were relevant to your work ? ? ? ? ? b. Addressed your learning needs ? ? ? ? ? c. Increased your knowledge and skills ? ? ? ? ? d. Met your expectations ? ? ? ? ? e. Were useful to the skills required ? ? ? ? ? for your job 56 III. Utilization of the knowledge and skills learned 15. You were able to make use of the skills and knowledge from the learning events you participated over the past 12 months in your job: ? Strongly agree ? Agree ? Neither agree nor disagree ? Disagree ? Strongly disagree 16. Please indicate whether the following factors were barriers to your utilization of the knowledge and skills learned over the past 12 months? (Please select as many as apply) q Existing practices q Lack of budget q Lack of time q Manager and peersupport q Other (specify) ___________________________ 17. How strongly do you agree or disagree with each of the following comments? Select one button for each Neither Statement Strongly Agree nor Strongly Agree Agree Disagree Disagree Disagree a. Your supervisor encourages you to seek out appropriate regional staff learning experiences. ? ? ? ? ? b. You have adequate opportunities to learn new skills to do your job better ? ? ? ? ? IV. Self-perceived benefits and impact 18. To what extent your participation in learning over the past 12 months contribute to improving services to clients? ? To a great extent ? To a considerable extent ? To some extent ? To a limited extent 57 ? Not at all 19. How valuable do your think the learning events you participated in over the past 12 months would be for others? ? Very valuable ? Somewhat valuable ? Not very valuable ? Not valuable at all 20. What were the specific benefits to you of participating in learning events? (Please select as many as apply) q It taught me new knowledge q It taught me new skills q It improved my existingskills q It improved my understanding in an area q It prepared me for greater responsibilities q It served as a pre-requisite for further coursework q It contributed to enhancing my own productivity q It contributed to enhancing my team productivity q Other (specify)______________________________ 21. In what areas do you feel you perform better now that you have participated in learning over the past 12 months? 22. What other areas of learning do you think would be valuable for you in future? 23. What other comments or suggestions do you have for staff learning? 58 Demographics 24. Your job/position is: ? ACS, non-manager ? ACS, office management staff (Office Administrator, Office Manager, and Executive Assistant) ? Non-ACS, non-manager ? Manager, below VP level (level GE-GI) ? Manager, VP and above (level GJ and above) ? Other 25. Your grade level is: ? GA-GD ? GE ? GG ? STC ? STT ? GF ? GH or above ? Other____________ 26. Please indicate the month and year of when you began your service with the World Bank Group: Month (MM)_______________ Year (YY)_______________ 27. Your region is: ? East Asia and Pacific ? South Asia 28. Your Business Unit/Work Group is: ____________________________ 29. Your gender is: ? Female ? Male 30. Your age group is: ? <30 ? >30-40 ? >40-50 59 ? >50-60 ? 60+ 31. Your highest education level is: ? Associate level ? Bachelor ? Master/Certificate/Diploma ? Doctorate ? Other Thank you for your participation in this study. 60 ANNEX E: MANAGER INTERVIEW/SURVEY QUESTIONNAIRE STAFF LEARNING MANAGER INTERVIEW Hello __________________, my name is __________________ of the Gelfond Group, and I am contacting you on behalf of the World Bank Institute Evaluation Group (WBIEG). You should have already received a notification that I would be calling you for your help in evaluating staff learning programs. This survey is intended to gain feedback on learning events from your personal perspective and should not take longer than 10-15 minutes to complete. Is now a good time for you to participate? Yes à Proceedwithsurvey No à Askwhatdayandtimewouldbebesttocontactagain:_____________ 1. How many staff are in your unit? For the next several questions, please answer "Yes" or "No" 2. Did your unit have a learning coordinator in Yes No a. FY03? 1 2 b. FY02? 1 2 c. FY01? 1 2 3. Did your unit have a Learning Plan in... Yes No a. FY03? 1 2 b. FY02? 1 2 c. FY01? 1 2 4. Does your unit have a monitoring system for your unit learning activities? Yes No 5. Is your unit supported by the Regional Learning Services Group? Yes No 6. Approximately how many of your staff have you approved to participate in learning events during the past 12 months? ___________________________ 61 7. How effective do you feel learning events are in improving the following: (After each lettered item read ­ "Would you say it is 1 ­ Not At All Effective, 2 ­ Not Very Effective, 3 ­ Somewhat Effective or 4 ­ Very Effective?") Not At Not Very Somewhat Very All Effective Effective Effective Effective a. Team productivity 1 2 3 4 b. Individual productivity 1 2 3 4 c. Individual professional 1 2 3 4 development d. Services provided to 1 2 3 4 clients 8. On a scale of 1 to 5, with 1 being the most "Negative" and 5 being the most "Positive", please rate the impact that participation in learning events has had on your overall unit's performance. Negative Positive 1 2 3 4 5 9. How often do you observe trained staff practicing the skills/knowledge gained from learning events? a. Never Would you b. Rarely say it is... c. Sometimes d. Often e. Always 10. On a scale of 1 to 5, with 1 being "Not At All" and 5 being "To A Great Extent", please rate to what extent you believe the benefits associated with sending staff to learning events are worth the investment of their participating in the events. Not To A Great At All Extent 1 2 3 4 5 62 11. Again, on a scale of 1 to 5, with 1 being "Not At All" and 5 being "To A Great Extent", please rate to what extent the following factors represent obstacles in sending staff to learning events. Not To A At Great All Extent a. Staff resistance to learning 1 2 3 4 5 b. Concerns about travel safety 1 2 3 4 5 c. Cost/Budget constraints 1 2 3 4 5 d. Having a single, centralized 1 2 3 4 5 learning event location e. Too much time away from 1 2 3 4 5 job responsibilities f. Demanding work program 1 2 3 4 5 12. Has the range of learning events offered by the region met the needs of your staff? Would Yes you Somewhat say No 13. Has the range of learning events offered by the Bank met the needs of your staff? Would Yes you Somewhat said No 14. What learning event have you perceived to be the most beneficial to your unit? 15. What other topics or skills would you like to see covered in future learning events? 63 ANNEX F: DEMOGRAPHIC CHARACTERISTICS OF SURVEY RESPONDENTS Demographic Characteristic % of Respondents Female 43 Gender Male 34 Missing 23 <30 8 >30-40 26 >40-50 24 Age >50-60 16 60+ 3 Missing 22 Associate 4 Bachelor 24 Educaion Master/Certificate/Diploma 33 Doctorate 11 Other 5 Missing 22 ACS, no manager 23 ACS, Office management staff (Office Administrator, Office Manager and 6 Executive Assistant) Non ACS, non-manager 28 Position Manager, below VP level (levels GE-GI) 12 Manager, VP and above (levels GJ and above) 0 Other 10 Missing 21 GA-GD 28 GE-GF 22 Grade level GG-GJ 18 Other 9 Missing 23 <1 year 15 1 to <4 years 18 4 to <10 years 25 Year with the Bank 10 to <15 years 10 15 + 12 Missing 20 East Asia and Pacific (EAP) 61 EASES Sector unit 8 EASHD Sector unit 3 Business Unit/Work EASIN Sector unit 1 Group EASEG Sector unit 1 EASEGP Sector unit 0 EASTR Sector unit 1 EASUR Sector unit 3 EASPR Sector unit 5 EASFS Sector unit 1 EASPS Sector unit 1 EASRD Country unit 3 EACSM/EACSQ Country unit 2 64 EACNF/EACNQ/EACDF/EACGF Country unit 2 EACKM Country unit 0 EACCF/EACCQ Country unit 3 EACIF/EACIQ Country unit 2 EACVF/EACVQ Country unit 2 EACPF/EACPQ Country unit 2 Business Unit/Work EAPCO Central unit 6 Group EAPVP Central unit 0 EAPCA Central unit 1 EAPRF Country unit 0 South Asia Region (SAR) 39 SACAG/SACAA Country unit 2 SACBD/SACBA Country unit 1 SACIN/SACIA Country unit 2 SACNP/SACNA Country unit 1 SACPK/SACPA Country unit 2 SACSL/SACSA Country unit 2 SACBT Country unit 0 SACMV Country unit 0 Business Unit/Work Group SAREX Central unit 1 SARFM Central unit 1 SAROQ Central unit 0 SARPS Central unit 1 SARRM Central unit 0 SARVP Central unit 0 SASEI Central unit 6 SASES Sector unit 3 SASFP Sector unit 1 SASHD Sector unit 2 SASPR Sector unit 3 SASRD Sector unit 3 SARIM Central unit 0 Missing 20 65