98927 Ethiopia Women Agribusiness Leaders Network Impact Evaluation Baseline Survey Report1 January 2015 1This document was prepared for USAID by the World Bank’s Africa Region Gender Innovati on Lab. Team members who worked on this report include Tigist Ketema, Gautam Bastian, Ombeline Gras, Zewdu Abro, Katherine Manchester and Eliana Carranza. 1 THIS PAGE IS INTENTIONALLY BLANK. 2 Contents I. INTRODUCTION ............................................................................................................................................... 5 II. IMPACT EVALUATION DESIGN................................................................................................................... 6 Randomization design ............................................................................................................................................. 6 Source of data........................................................................................................................................................... 7 III. BASELINE SURVEY: METHODOLOGY .................................................................................................. 7 Identification of respondents .................................................................................................................................. 8 Preparing the survey team .................................................................................................................................... 10 Selection and agreement with a survey firm ........................................................................................................................... 10 Survey teams: composition ..................................................................................................................................................... 10 Training and piloting .............................................................................................................................................................. 11 Impact of preparatory steps on the timeline ........................................................................................................................... 11 Survey instruments .................................................................................................................................................................. 11 Developing the questionnaire based on the Program Impact Strategy ............................................................. 11 A more focused questionnaire for the male partners .............................................................................................................. 12 Programming an efficient device-based questionnaire ........................................................................................................... 13 In-field data collection ........................................................................................................................................... 14 Survey team organization ....................................................................................................................................................... 14 Survey monitoring................................................................................................................................................................... 15 IV. BASELINE SURVEY: STATISTICAL ANALYSIS ................................................................................. 15 Individual and household characteristics of respondents .................................................................................. 15 Age .......................................................................................................................................................................................... 16 Marital status and household headship .................................................................................................................................. 16 Household size ........................................................................................................................................................................ 17 Education................................................................................................................................................................................ 18 Business characteristics ......................................................................................................................................... 19 Previous occupation of respondents ....................................................................................................................................... 19 Experience in income-generation and business activities ....................................................................................................... 20 Reasons to start the WALN business ....................................................................................................................................... 21 Value chains ........................................................................................................................................................................... 21 Number of Workers ................................................................................................................................................................. 23 Competitors and customers .................................................................................................................................................... 24 Location of Business Operation .............................................................................................................................................. 25 Formality ................................................................................................................................................................................ 26 Business practices and financial management .................................................................................................... 26 Business practices................................................................................................................................................................... 26 Financial habits and behavior ................................................................................................................................................ 27 Leadership and confidence ................................................................................................................................... 28 Role in business ...................................................................................................................................................................... 28 Decision-making power: perceptions and checks with spouses’ perceptions ......................................................................... 29 V. BALANCE BETWEEN TREATMENT AND CONTROL GROUPS.......................................................... 32 Individual, household and Education variables (Sections A & B) ..................................................................... 32 Business variables (Section C) .............................................................................................................................. 32 Business practices and financial management variables (Section D) ................................................................ 33 Revenue, profit, asset and inventories (Section E) .............................................................................................. 33 Leadership and confidence variables (Section F) ................................................................................................ 33 Overall Joint Test of Significance (Section G) ..................................................................................................... 33 VI. LIST OF ATTACHMENTS ......................................................................................................................... 34 3 List of Figures Figure 1: Impact Evaluation Sample Selections and Randomization (As Planned) ....................................................... 6 Figure 2: Impact Evaluation Sample Selections & Randomization (Actuals) ............................................................... 9 Figure 3: Profile of survey teams: level of education and subjects of their degrees .................................................... 10 Figure 4: WALN Program Impact Strategy ................................................................................................................. 12 Figure 5: Screenshot of the WALN SurveyToGo Program ......................................................................................... 13 Figure 6: Geographical distribution of respondents and survey teams......................................................................... 14 Figure 7: Age by respondent's characteristics .............................................................................................................. 16 Figure 8: Respondents by marital status ...................................................................................................................... 17 Figure 9: Average number of household members by gender...................................................................................... 18 Figure 10: Average number of household members by age group ............................................................................... 18 Figure 11: Summary of respondents educational level ................................................................................................ 19 Figure 12: Previous occupation of mentors and mentees ............................................................................................. 20 Figure 13: Reasons why respondents have started the WALN business ...................................................................... 21 Figure 14: Summary of the value chains in which respondents are operating their business ...................................... 22 Figure 15: Number of workers ..................................................................................................................................... 23 Figure 16: Competitors and customers ......................................................................................................................... 24 Figure 17: Location of business operating area ............................................................................................................ 25 Figure 18: Reasons why the business is operated from home ...................................................................................... 25 Figure 19: Summary of respondents’ business practices ............................................................................................. 26 Figure 20: Separation of business and household money is an important financial management principle ................ 27 Figure 21: Top four places where respondents say they save their money .................................................................. 27 Figure 22: Amounts held on accounts with financial institutions ................................................................................ 28 Figure 23: Role of respondents in their business ......................................................................................................... 28 Figure 24: Respondents' decision-making power in their business .............................................................................. 29 Figure 25: Assessing respondent’s self-confidence ..................................................................................................... 30 Figure 26: Confidence in organizations and/or individuals.......................................................................................... 31 List of Tables Table 1: Summary of mentors ........................................................................................................................................ 8 Table 2: Summary of mentees by ................................................................................................................................... 8 Table 3: Summary of age and household headship of respondents .............................................................................. 15 Table 4: Headship versus marital status ....................................................................................................................... 17 Table 5: Summary of number of income generating activities .................................................................................... 20 Table 6: Total number of workers and their summary by gender ................................................................................ 23 Table 7: Registration, tender proposal and export license............................................................................................ 26 Table 8: Business decision making by the WALN member & spouse responses ........................................................ 29 4 I. Introduction The World Bank’s Africa Region Gender Innovation Lab (GIL) is conducting a randomized controlled trial (RCT) impact evaluation of the Women in Agribusiness Leadership Network (WALN), a transformational project implemented in Ethiopia by ACDI/VOCA, and supported by USAID2: WALN is a framework for delivering business training, mentoring and networking activities, targeted to high- potential women leaders in the Ethiopian agribusiness sector. The mentoring component of WALN–which relies on members of the network receiving specific training to become mentors–is particularly innovative. Indeed, the theoretical and some preliminary empirical literature suggests that one-to-one counseling and tailored guidance can improve work and business performance; in this regard, the mentoring component of WALN provides a relevant alternative to classic business training programs. WALN, as a project, aims to increase participants’ business skills and self-confidence, enabling them to be community leaders and change makers. WALN also seeks to improve agribusiness outcomes by addressing gender differences in productivity, profitability, participation and leadership in the sector. In general, the development hypothesis of the project is that if peer training and mentoring can expand women’s capacity to play leadership roles in sector organizations, manage businesses profitably and effectively, and network with their peers, then women’s participation and contribution in leadership and decision-making in the agriculture sector will increase, catalyzing economic impact. The ongoing impact evaluation is assessing the impact of participating in WALN activities on the overall performance of the selected high-potential women leaders in the agribusiness sector. Specifically, the study examines the impacts of mentoring3 on both mentors and mentees on four broad research outcomes: knowledge and skills; psychosocial outcomes (e.g. self-confidence, trust etc.); business outcomes (e.g. business performance, resource access, asset ownership etc.); and networking outcomes (e.g. participation in business and community networks). Previous studies suggest that the impact of business training and consulting is highly sensitive to geographical and business contexts. Thus, in order to assess and interpret the impact evaluation outcomes, it is important to understand the specific business environment, as well as the needs and constraints faced by local entrepreneurs. Special attention needs to be paid to the particular needs of women in such a context. This baseline survey of all participants in WALN should shed some light on these questions. 2WALN is embedded in the Agribusiness and Market Development (AMDe) component of the Government of Ethiopia’s USAID and World Bank-funded Agricultural Growth Program (AGP), which is itself part of USAID’s wider Feed the Future (FtF) programming. 3To be precise, this study will be able to evaluate the impact of being part of WALN plus being a mentor or being part of WALN plus being a mentee. It won’t strictly be able to isolate the impact of mentoring from the impact of being part of the overal l WALN program. 5 II. Impact Evaluation Design Randomization design The principle of a study based on a randomized controlled trial (RCT) is to compare, over time, the evolution of two groups, which can be considered as similar at the starting point. The “program” group is benefitting from an intervention that the impact study intends to evaluate; the “control” group provides information on how the program group would have evolved in the absence of the intervention in question: in other words, it provides a counterfactual for the group of participants in the program. Only eligible applicants were considered at the time of the random allocation into the “program” or “control” groups. The implementer used pre-established selection criteria and the responses to long-from questions to determine the eligibility of applicants and have the potential mentees they recommended. Figure 1: Impact Evaluation Sample Selections and Randomization (As Planned) 300 Treatment Mentees (3 per mentor) 600 Eligible Mentees (6 per mentor) 100 Treatment 300 Control Mentees Mentors (3 per mentor) 200 Ineligible Mentees (2 per mentor) 200 Eligible Applicant Mentors 600 Eligibles Mentees (6 per mentor) 100 Control Mentors 200 Ineligibles Mentees (2 per mentor) The randomization itself was performed in two waves. The applicant pool of would-be mentors has been considered in the first stage of the randomization, using a stratified and blocked design. This means that the random allocation of potential participants to “program” and “control” groups has been performed while ensuring that both groups would be balanced in terms of regions (“blocks”) and size of the business of the participants (measured as number of employees, divided into three “strata”). Isolating the effect of these two important factors improves the statistical power of the study, leading to more precise estimates of any intervention impact measured. The mentees nominated by these eligible mentors have then been considered in the second stage of the randomization, and also assigned to “program” and “control” groups. In terms of sample size, the impact evaluation (IE) design (see Figure 1) was based on the assumption that a pool of 200 eligible applicant mentors would be gathered, so that 100 could be (randomly) selected into the program, and 100 would form a control group, together with their mentees (“pure control group”). Among the mentees recommended by the mentors of the “treatment” (program) group, half of the eligible ones would also be assigned to the program (to receive the mentoring), while the other group would be an 6 “alternative control group”; the latter, who would be in the network of program participants but not in the program themselves, would allow for measuring program spillover effects. Source of data Outcome data for this impact evaluation is being collected through survey instruments, administrative data (from implementer’s M&E systems and government data sources), intensive qualitative interviews and implicit association tests (IATs). The main source of data for demographic, programmatic, psychometric, social and economic indicators is the survey data. Three rounds of survey data (baseline and two follow-up surveys) from different subsets of the sample will be collected throughout the study. For the baseline survey, the plan was to interview at least all the 200 participants in mentors treatment and control groups, as well as all the 1,600 eligible and ineligible mentees. By the time survey field work started, mid-April 2014, just at the closure of the application period and before the selection of eligible applications, the sample included 234 applicant mentors; adding recommended mentees, the total number of households to be surveyed had become close to 1,600 in total. The survey covered all these respondents, with the idea that we would be using our data to refine the program eligibility screening. Two follow-up surveys are planned (midline and endline surveys). The midline survey will measure the outcomes shortly after the completion of the mentoring program, and the endline will be run at the end of the program.4 Each round of surveys will comprise: (i) a main questionnaire that will be addressed to the agri-businesswomen concerned with the program, and; (ii) a second, shorter, questionnaire addressed to the husband or male counterpart of the woman (when applicable), focusing on intra-household allocation of resources and decision-making. Mentor and mentee participants are being administered similar surveys, while separate questionnaires will be developed for the supply-chain sub-sample, surveyed at midline and endline. III. Baseline survey: Methodology The baseline survey covered the entire sample of applicants to the program. That is, the mentors and the recommended mentees were all interviewed at the end of the application phase, but before the announcement of selection results. The objective of the WALN baseline survey was to build a comprehensive dataset, which would serve as a reference point for the entire sample, before treatment and control assignment and program implementation. To get a better understanding of the context of the survey, this section describes the survey preparation steps and methodology. 4 Assuming a balanced randomization, it is not necessary to collect baseline data for this group. 7 Identification of respondents The program implementer ACDI/VOCA launched a recruitment phase in December 2013. In major towns of the four regions targeted by the program–Amhara, Oromia, SNNPR and Tigray–women were invited to orientation sessions, during which they were introduced to the WALN program and handed out application forms. In addition to radio and newspaper advertisements, the staff in charge of the program also used their professional networks to reach out to influential Ethiopian agri-businesswomen who might be interested in participating. All women who had sent a proper application form to become a WALN mentor to ACDI/VOCA were registered in our database with all the information filled in the form. They were then given a unique identification code, to be kept for the duration of the IE. The form required all applicants to provide the names of up to eight female agribusiness entrepreneurs that they knew and whom they would be willing to mentor; all mentees identified through this procedure were also recorded and associated with a unique ID (see the application form in Attachment 3). This record has served as a roster of all the respondents surveyed at baseline. Tables 1 and 2 below summarize our baseline final sample size after removing ineligible respondents. The total of 231 mentors5 and 1,129 mentees6 are included in the baseline survey. Table 1: Summary of mentors Table 2: Summary of mentees by by their treatment status their treatment status Mentors Mentees treatment status Mentors treatment treatment Treatment Control Total status status Mentees Mentees Mentees Treatment 50.25% Treatment 50.08% 49.92% 100% Mentors [99] Mentors [295] [294] [589] Control 49.75% Control 0% 100% 100% Mentors [98] Mentors [0] [540] [540] Total 100% Total 26.13% 73.87% 100% Mentors [197] Mentors [295] [834] [1,129] Note: Row percentages are in parentheses. 5Baseline data were collected for a total of 231 Mentors including 197 Mentors included in the impact evaluation (99 Treatment & 98 Control), 5 Mentors excluded from experiment and 29 ineligible mentors. For mentors included in the impact evaluation subsequent rounds of midline and endline data will also be collected. Please see detailed notes in Figure 2 for more information. 6Similarly, Baseline data were collected for a total of 1364 Mentees including 589 Mentees of Treatment Mentors, 540 Mentees of Control Mentors, 33 excluded Mentees and 202 ineligible Mentees. Please see detailed notes in Figure 2 for more information. 8 Figure 2 (A) below illustrates the actual sample sizes that we were finally able to achieve for the impact evaluation, and Figure 2 (B) provides a breakdown of the observations which were not included in the impact evaluation sample but for whom we have either received application information or collected baseline data. Detailed notes are provided describe the observation categories in Figure 2 (B). Figure 2: Impact Evaluation Sample Selections & Randomization (Actuals) A. Impact Evaluation Study Sample 295 Treatment Mentees 99 Treatment 589 Eligible 197 Eligible Mentors Mentees 294 Control Applicant Mentors 98 Control 540 Eligibles Mentees Mentors Mentees B. Observations not included in Impact Evaluation Excluded from Experiment (a) 33 Mentees of 5 Mentors (5 Mentors & 33 excluded Mentors Mentees) 139 Mentees of 29 Mentors Not in Experiment (b) Ineligible Mentors Ineligible (37 Mentors & 336 (29 Mentors & 202 Ineligible on their Mentees ) Mentees ) own(c) (63 Mentees) Not Interviewed 3 Mentors (3 Mentors & 101 Mentees) 101 Mentees (d) Notes: (a) Baseline data were collected for this group. In each of the 5 regions, the mentor largest business size (measured as the # employees) was excluded from the impact evaluation; the 5 mentors still participate in the program, as well as 18 among their 33 eligible mentees. (b) Baseline data were collected for this group. To be eligible, mentors had to have at least 1 employee; mentors and mentees also needed to be working in the targeted value chains. (c) In the time between application and randomization, 15 mentee nominations were withdrawn by mentors; 2 mentors from the Treatment & Control sample had just recommended 1 mentee, making the randomization impossible for these 2 mentees, thus we have 63 = 46 wrong Value Chain + 15 cancelled + 2 withdrawn (note that, among those 63, 2 would have been excluded anyway, and 15 among the mentees of ineligible mentors). (d) For mentees, of 101, 94 were not interviewed and 7 were added to roster after the interview (1 of the non- interviewed cancelled by her mentor). These 101 non-interviewed can also be broken down by the status of their mentors: 55 mentees were nominated by Treated mentors, 34 by Control mentors, 5 by excluded mentors and 7 by non-eligible mentors. 9 Preparing the survey team Selection and agreement with a survey firm The IE team compared the qualifications of international and local survey firms in order to find the most suitable partner to train and lead survey teams into the field. One local firm, Waas International Consultants, stood out as the best in terms of quality/price, and came recommended by both internal colleagues and trusted, external partners. This firm was also chosen for its experience and willingness to work with devices to collect electronic data (instead of paper questionnaires), which improves data quality and shortens the process of data delivery by cutting out the data entry stage. Based on a detailed Terms of Reference (TOR) that included strict quality controls, it was agreed that the firm would be in charge of developing the electronic version of the questionnaire to be prepared by our team. The firm would also be responsible for the recruitment, training and supervision of enumerators, and for collecting the datasets after a survey was administered, sent electronically to their server and shared with the IE team after two days of preliminary cleaning. After the completion of fieldwork, including revisiting some of the respondents to check the quality of the surveys, back checks would also be performed by telephone. Clean datasets would be finally delivered to us, with the understanding that mentors’ data would be given priority. Survey teams: composition In total, from their pool of regular employees, the survey firm recruited 25 enumerators, seven supervisors and two telephone-based quality controllers (QC).7 Most of their survey staff were recent college graduates, and 40% of them were female. Figure 3 below shows the distribution of the survey team’s education, in terms of level and subject, across enumerators hired for the WALN baseline survey. Figure 3: Profile of survey teams: level of education and subjects of their degrees Diploma Bachelors Masters + 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Sociology & Economics & Business IT Others Psychology Statistics Management 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 7Tele-QCs are in charge of calling back certain respondents, either from a random selection of interviews (to compare the datasets with the answers provided on the phone) or to fill in missing information in the interview. 10 Training and piloting A detailed training on the content of each questionnaire is always key to the success of a survey. In the case of the WALN baseline survey, the entire team met daily for close to two weeks, which is more than twice the frequency that was planned, to make sure that all survey questions would be appropriately covered and that issues raised by the survey teams would be entirely addressed. The program for the training, as well as supporting material developed by the survey firm, can be found in Attachments 1 and 2. The most important session, in terms of time and energy, was a careful walkthrough of the paper questionnaire. The discussion was both on the content of each question and its translation from English to Amharic. Beyond a better understanding of the aims of the survey and of the rationale behind the questions asked, another benefit of in depth training of the survey team is to provide feedback on the questionnaire itself and improve it. The survey instrument thus evolved continuously during the training, enriched by daily discussions. After completing the training on the paper questionnaire, a full day of training was conducted on the device (tablet) that would be used in the field. Parallel to the electronic device training, the survey team checked the consistency, formatting and skip patterns of the questionnaire. Finally, the programmers incorporated the feedback from the training of the survey team into the electronic version of the questionnaire. Moreover, a half-day session on respondent’s approach was undertaken. This session focused on effective communication among the interviewers, supervisors, respondents and other stakeholders in the process. Impact of preparatory steps on the timeline A consequence of this very intense training period, and of the rich exchanges with the survey team, was a delay of two weeks in finalizing the survey instrument. Changes were made not only to the structure or content of the questions, but also to the number and type of sections that needed to be incorporated into a readable paper format. Each new change also triggered: (i) its translation in the 3 regional languages, and; (ii) amending the program for the electronic version of the survey. Survey instruments The baseline survey intended to provide a detailed picture of women applying to WALN, combining personal and business information, as well as quantitative and qualitative dimensions. Based on the research questions, the IE team identified key outcome variables and developed a comprehensive questionnaire to be administered during face-to-face interviews by enumerators. A male version was also prepared, and both were translated into local languages and converted into an electronic program, for a better interview experience and higher data quality. Developing the questionnaire based on the Program Impact Strategy The purpose of the randomized controlled trial is to provide researchers with an objective “control group”, whose outcomes will be compared to those of program participants. The baseline survey provides information on the starting level of those variables for the entire sample, both control and future program 11 participants, in order to set their respective “starting points”.8 According to the program implementer (ACDI/VOCA), the WALN Impact Strategy expects changes in a couple of key areas to be observed among participants of the program (see Figure 4). Figure 4: WALN Program Impact Strategy The questionnaire referred to psychological and sociological literature, as well as to similar studies conducted by experimental economists, to identify the most relevant and unbiased questions, include best practice instructions to the enumerators or respondents, and select the most accurate response options. The debriefing sessions with experienced surveyors during the survey team training, as well as the various pilot exercises, also greatly informed the design of the questionnaire, ensuring that it is tailored to the local context and to respondents’ level of understanding. The entire questionnaire was translated into the main local languages: Amharic, Afan-Oromo and Tigrigna. As explained above, the questionnaire went through a series of revisions before reaching its final format for fieldwork. That version includes seven main sections, flowing from the most objective (demographic information, detailed questions on business) to the most sensitive topics (self-confidence, household decision making). The entire questionnaire can be found in Attachments 4 and 5. On average, it took 2.5 hours to administer the main questionnaire to a respondent. To compensate for this time, it was decided that an equivalent of ETB50 in airtime would be credited to each respondent’s cellphone account. A more focused questionnaire for the male partners As an essential part of the information to be collected from WALN participants and the control group, the answers provided by their spouses, if any, deserved the design of a specific questionnaire. The main objective was to gather data on the same interpersonal/intra-household dimensions asked to the main respondent, but from the partner’s point of view. Therefore, beyond contact details and contextual information on a husband’s official positions and economic activities, sub-sections of the main 8The random assignment of participants to the “control” or the “treatment” groups (which occurred after the baseline survey to avoid any interaction with the “starting point”) should ensure that both groups are, on average, equal on all those dimensions. 12 questionnaire were adapted for male respondents to provide information on a wife’s business, in the household’s decision-making and information-sharing processes, and on his perceptions of intra- household violence in his own home and related norms. This amounted to 82 questions (15% of the main questionnaire), for an average interview duration of 30 minutes. To compensate for this shorter survey, the airtime credit offered to each respondent was set to ETB 25. Programming an efficient device-based questionnaire The survey firm was assigned to translate the paper questionnaire into a device-administered interview. The software used, SurveyToGo, is meant to be user-friendly (see Figure 5 below), with responses to be ticked or entered as on paper. Moreover, the software also allows for embedding all sorts of checks, such as consistency with previous answers (e.g. end date which cannot precede a start date); automatic skips of irrelevant questions (e.g. if the respondent is single, skip all questions involving a spouse); and restrictions of response options given previous answers (e.g. if a respondent lives in a certain region, options for cities are restricted to those in that region). The objective of transferring the questionnaire to an electronic device was not only to ensure a more pleasant experience for the respondent and the enumerator, but also to restrict errors as much as possible, thereby improving data quality and reducing data cleaning. Figure 5: Screenshot of the WALN SurveyToGo Program 13 All of the interviews took place with the help of a tablet,9 with a few exceptions: for less than 2% of the respondents, paper questionnaires were administered when the table failed to function properly (e.g. did not update to a correct version of the survey, or due to a power shortage). In-field data collection Survey teams spent a total of 70 days in the field administering interviews to WALN respondents and their spouses. Data collection was split in two waves, starting with seven weeks between mid-April and early June 2014, and completed with three additional weeks in August 2014. Survey team organization The survey team was split into seven different units of 3-4 enumerators, each of them assigned to a more experienced supervisor. The responsibility of the supervisor was not to conduct interviews but to organize the fieldwork and to check the quality of the collected data by re-visiting randomly selected respondents. Figure 6: Geographical distribution of respondents and survey teams As with the survey team, respondents were also divided into seven groups based on their geographical location (see Figure 6). Every unit was directed to follow a pre-defined itinerary, intended to cover their entire area of enumeration. It was necessary to set up a well-monitored schedule for interviews, as there was only a defined period of availability for each team in a given area before they moved to the next point on their itinerary. However, those routes were planned so that important places could be revisited on the way back to Addis Ababa, should there be any respondent left to interview. 9 Specifically, a 7" touch screen tablet, operated with Android 4.0 Ice Cream Sandwich. 14 Survey monitoring The survey firm assigned one contact person from their Addis Ababa office to closely follow each team’s activities, including the field team’s travel schedule and interview progress. A member of the research team also travelled along with the enumerators to most of the survey sites, in order to see each of the teams in action. This continuous monitoring allowed the survey team to quickly react to any issue detected on the ground, such as a duplication of respondents or changes in their residence. The enumerators’ thorough understanding of the sample structure allowed them to adjust to unforeseen logistical challenges. For example, when recommended mentees were not reachable and mentors had to switch to other mentees, this required some flexibility from survey teams in acquiring this new information and updating their respondent list accordingly, always in agreement both with the program implementation and the research team. IV. Baseline Survey: Statistical Analysis The main objective of the baseline survey is to establish a comprehensive dataset that can serve as a benchmark for the midline and endline impact analysis. Understanding respondents’ individual, household and business characteristics provides valuable insight when evaluating and seeking to understand any changes caused by the intervention. This section presents a profile of WALN participants at baseline. Individual and household characteristics of respondents As can be seen in Table 3 below, the average age gap between mentors and mentees is only two years. Mentors are slightly more likely to be married and to consider themselves to be the head of their household. Table 3: Summary of age and household headship of respondents Average age Married Head of household 37 years 57% 71% Mentors (9 years) [113] [140] 35 years 54% 63% Mentees (10 years) [605] [720] 36 years 54% 65% All (10 years) [718] [860] Note: For Average age, figures in parentheses are standard deviation. For other columns, figures in parentheses are the size of the relevant sample. 15 Age A further analysis of the age of respondents is shown in Figure 7. In addition to the total sample, the figure disaggregates the respondents by their headship, marital status (married respondents and divorced or separated respondents) and official position. The disaggregation, however, does not change the trend observed: mentors are still, on average, older than mentees in all categories. Figure 7: Age by respondent's characteristics Marital status and household headship In proportion, there are slightly more married women among mentors than among mentees (57% versus 54%), while there are more single women among mentees (16% versus 13% for the mentors). Mentees (17% and 16% respectively) are slightly more likely to report being divorced or separated from their husband than mentors (15% and 13% respectively). 16 Figure 8: Respondents by marital status Mentors 113 30 24 4 26 Mentees 605 192 132 17 183 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Married Divorced Widowed Seperated Single Overall, 71% of mentors and 63% of mentees report being the head of their household. It is important to disaggregate these figures into married and unmarried respondents because, traditionally, the husband is the head of the household in the case of a married couple, whereas a woman is the head of the household in the absence of a male spouse, that is, when she is single, divorced, widowed, or separated. . Table 4: Headship versus marital status Mentor Mentee Marital Status Yes(a) No(a) Total Yes(a) No(a) Total 62% 38% 100% 48% 52% 100% Married [70] [43] [113] [293] [312] [605] 93% 7% 100% 95% 5% 100% Divorced [28] [2] [30] [182] [10] [192] 100% 0% 100% 96% 4% 100% Widowed [24] [0] [24] [127] [5] [132] 100% 0% 100% 82% 18% 100% Not Together [4] [0] [4] [14] [3] [17] 54% 46% 100% 54% 46% 100% Single [14] [12] [26] [98] [85] [183] 71% 29% 100% 63% 37% 100% All [140] [57] [197] [714] [415] [1129] Notes: Row percentages in parentheses. (a) “Yes” and “No” refer to the response of the respondent to the question: “Are you the head of your household?” As can be seen in Figure 8 and Table 4 above, 71% of mentors and 63% of the mentees reported that they are the head of their household. This can be contrasted to 22% of the households reporting women as heads in a recent national survey representative of rural areas and small towns (DHS, 2014). Household size The average household size for mentors and mentees is six and five members, respectively. Disaggregating by gender indicates that, on average, mentors’ households have 3.5 females and 2.5 males 17 (with an average gender ratio of 1.8 women per man), while mentees’ households have 2.8 females and 2.1 males (average gender-ratio of 1.5 women per man) (see Figure 9). Figure 9: Average number of household members by gender Mentors 3.5 2.5 Mentees 2.8 2.1 0 1 2 3 4 5 6 7 Females Males As indicated in Figure 10 below, disaggregating the household members by age group indicates that, for both mentors and mentees, the majority of household members are adults above 25, followed by children under 15 and then adolescents (aged 16-24). Figure 10: Average number of household members by age group Mentors 1.8 1.7 2.0 0.5 Mentees 1.7 1.3 1.5 0.5 0 1 2 3 4 5 6 7 Children (under 15 yrs old) Young Adults (aged 16-24) Adults (aged 25-49) Senior Adults (aged >=50) Education Although a significant number of mentors and mentees have completed at least middle school – at 66% and 57% respectively -, there seems to be a clear difference in the educational background of the two groups (See Figure 11). For instance, 34% of the mentees did not complete primary school, when it is the case for only 24% of the mentors. On the other end of the educational scale, mentors are 23% to have a higher education or university diploma, when it is the case for only 13% of the mentees. 18 Figure 11: Summary of respondents educational level 13% 10% 15% Mentors 28% 10% 12% 12% 4% 9% 13% Mentees 31% 9% 18% 16% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No schooling Some schooling Primary completed Middle School completed High School completed Completed higher education (not university) Completed university education Business characteristics Previous occupation of respondents A large share of the respondents have previous business experience, prior to starting the business that was the basis of their application to WALN. As can be seen in Figure 12, this applies to 45% of mentors (35% in a different sector and 10% in the same sector), and 35% of mentees (28% in a different sector and 7% in the same sector). 19 Figure 12: Previous occupation of mentors and mentees Mentors 35% 10% 13% 1% 6% 12% 19% 3% Mentees 28% 7% 15% 2% 5% 19% 20% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Other business (Different sector) Other business (Same sector) Wage employment (Different sector) Wage employment (Same sector) Unpaid work (Family business/farm) Staying at home/Studying Unemployed Other Experience in income-generation and business activities On average, mentors have had experience in 1.6 income-generating activities, while for mentees it is 1.4 (see Table 5). Although this difference is not significant, the amount of income they generate shows a significant gap. On average, mentees make only about 4% of what mentors earn. This could be attributed to the relatively lower educational level of mentees, which might force them to engage in lower paying activities. Table 5: Summary of number of income generating activities Average number Average of income Total Total number of Average age generating earning in earning in own of WALN activities million (a) million (b) business business 1.6 5.35 6.77 1.4 5 Mentors (0.82) (71.40) (80.30) (0.87) (4.64) 1.4 0.23 0.26 1.2 4 Mentees (0.64) (6.01) (6.41) (0.70) (4.18) Note: Standard deviations are in parenthesis. (a) Including observations reporting zero earnings. (b) Excluding observations reporting zero earnings, in other words, it only takes into consideration respondents reporting strictly positive earnings. When we consider only own businesses instead of all income-generating activities (which includes business as well as wage employment), mentors run on average 1.4 businesses, versus 1.2 for mentees. When we further narrow our analysis to businesses registered for WALN (“the WALN business”), it appears that the gap between mentors and mentees, in terms of experience running this specific business, is very small: on average, mentors only have one more year of experience than mentees (the same holds if the question is expanded to running any business). 20 Reasons to start the WALN business Mentees and mentors had more or less similar motivations when they started their current business. As shown in Figure 13, the main reason for both groups is to generate additional money for the household. Figure 13: Reasons why respondents have started the WALN business Mentors 38% 15% 10% 8% 7% 6% 16% Mentees 38% 10% 15% 11% 6% 4% 17% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Household needed additional money Saw a market opportunity Wanted to make money Could not find a job Wanted to benefit from my hobby Wanted to be my own boss/have own business Other Value chains Figure 14 shows the value chains in which the respondents are operating. The first eight value chains most represented (maize to dairy) logically correspond to those elected for eligibility to the WALN (explicitly mentioned on the application form). Twenty-five percent of mentors work in the dairy value chain, followed by wheat (19%), whereas for mentees the most popular value chains are first wheat (17%) and then dairy (14%) Among the eight WALN value chains, the least represented for both mentors and mentees is sesame. This might be due to sesame production being concentrated in Tigray (60%) and Amhara (23%) regions. 21 Figure 14: Summary of the value chains in which respondents are operating their business 25% Dairy 15% 19% Wheat 17% 17% Livestock 11% 15% Coffee 12% 13% Maize 14% 12% Chickpea 9% 9% Honey 6% 1% Baltina (a) 10% 7% Sesame 4% 9% Others (b) 15% Mentors Mentees Note: (a) Baltina is not a WALN Value Chain, but is included as a separate category because there are a number of mentees who report participating in this activity; (b) Others include services, food processing, beverages, fruits, vegetables and general merchandize. However, many respondents, and particularly mentees, are involved in other value chains, including Baltina, other grains, and service industries such as hotels, restaurants, bakeries and other food processing activities. 22 Number of Workers Among mentors, the average number of workers at their business is more than three times that at businesses owned by mentees. However, the high standard deviation in both cases (see Table 7 below) indicates a high dispersion in the distribution of workers and the likelihood of extreme values distorting the average. Figure 15: Number of workers To avoid the impact of extreme values, we removed the top 5% of results, reducing the average number of workers to five and two, for mentors and mentees, respectively (See Table 6). Indeed, as can be seen in the fourth column of the Table 6, the average number of workers in the top 5% for both mentors and mentees is very high, and also has a high standard deviation, showing that there is high variation even within the top 5%. Table 6: Total number of workers and their summary by gender Average Average Total Average Total Average Average Total Workers without Workers only top Female Male Workers top 5% 5% workers (a) workers (a) 13 5 191 39 32 Mentors (51) (5) (187) (97) (51) 4 2 39 33 19 Mentees (15) (2) (57) (48) (44) Note: Standard Deviations are reported in parenthesis. (a) Data only for businesses with more than 10 workers. The table also summarizes the number of male and female workers for businesses with more than 10 workers. About 13% of the mentors have more than 10 workers, and they have on average 39 female workers and 32 male workers. However, only about 5% of mentees have more than 10 workers, and the average number of female and male workers for this group is 33 and 19, respectively. The distribution of the total number of workers is also summarized in Figure 15 above. 23 Competitors and customers Understanding businesses’ competitive landscape is another important factor when seeking to understand their margin of maneuverability. The survey collected information about the number of immediate competitors that respondents reported they have within their markets. Figure 16 (A) shows distribution of the number of competitors reported by respondents. The left hand panel shows all the data, while the right hand panel excludes outside values in the display to show the 5- point summary more clearly. Similarly Figure 16 (B) shows a gender break down of the number of customers the respondents reported they served in the week prior to the survey. Both groups seem to serve similar numbers of customers in both gender groups on average, however the Mentors with a larger number of customers seem to be skewing the distribution. Both groups however serve more female customers than male customers on average. Figure 16: Competitors and customers A. Competitors B. Customer Gender 24 Location of Business Operation The baseline data reveals that the majority of mentors (63%) and mentees (56%) operate their businesses outside of the house (see Figure 17), although a significant number also operate from home, in particular among mentees (40%, while for the mentors this is only 28%). Figure 17: Location of business operating area Mentors 28% 8% 63% Mentees 40% 4% 56% Inside the house Both Outside the house If respondents reported operating their business from home, they were asked why they did so, and whether their decision was a choice or a constraint. Figure 18 summarizes their answers. The first explanation given, by both mentors and mentees, is the financial constraint they face to have a new business premise; the second most important reason is that it is more convenient to operate their businesses from home. Childcare and household chores are also important reasons together they account for 23% of Mentors responses and 31% of Mentees responses. Figure 18: Reasons why the business is operated from home Mentors 40% 28% 13% 10% 10% Mentees 40% 20% 15% 16% 8% Can't afford it Business convinience Childcare Household chores Other 25 Formality For the success of a business, its legal status also matters, especially in terms of access to finance and avoiding harassments from government officials. Out of the 197 mentors, 161 of them have responded to this question, meaning a non-response rate of 18%. Among those who answered, 65% had their business registered. Regarding mentees, only 44% of the 85% who responded to the question had a license to conduct their business (see Table 7). Table 7: Registration, tender proposal and export license Business Registration Yes No Total 105 56 161 Mentors (65%) (35%) (100%) 423 537 960 Mentees (44%) (56%) (100%) Note: Row Percentages are in parentheses. Business practices and financial management Results presented in this section explore the skills and financial knowledge of respondents to ensure a sound financial management of their business. Business practices Figure 19 present the answers to simple checks about the business (“does this business have a business plan/an annual budget/maintain financial records”), demonstrating basic practices of its manager. As indicated in the graph, mentors perform better on these three indicators. Figure 19: Summary of respondents’ business practices 61% 36% 39% 35% 14% 12% Business plan Annual budget Financial records Mentors Mentees Separating business and household money was identified by over 60% of both Mentors and Mentees as one of the most important financial management principles; (Figure 20). 26 Figure 20: Separation of business and household money is an important financial management principle Mentors 22% 45% 9% 20% 4% Mentees 52% 12% 11% 21% 4% Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Financial habits and behavior It also seems crucial to understand respondents’ usage of formal financial services and more generally their saving habits. 89 and 73% of mentors and mentees, respectively, reported that they have an account in either a bank or any other formal financial institution (the question was not restricted to accounts opened in the name or for the business). On average, the average number of bank accounts, respondents, is 1.3 and 0.88 for mentors and mentees, respectively. Holding an account may not necessary mean saving (and conversely). Hence, respondents were asked about how they save the money earned from their business: more than half of the mentors and 35% of mentees save in a bank, while 37% of mentors and 29% of mentees save in an MFI (see Figure 21). These are the main two means of saving, while a non-negligible fraction of the sample (12% of the mentors and 22% of the mentees) declares not saving business earnings. Figure 21: Top four places where respondents say they save their money 53% 35% 37% 29% 22% 11% 12% 6% Bank MFI At home Don’t save Mentors Mentees Figure 22 summarizes the amounts held by respondents who do have an account with a financial institution. 27 Figure 22: Amounts held on accounts with financial institutions 38% 31% 25% 26% 10% 11% 10% 9% 8% 6% 5% 5% 3% 3% 3% 2% 4% 1% Over-drawn 0-1 1-5 5-10 10-20 20-50 50-100 100-300 300+ Mentors Mentees Note: Horizontal axis is in Thousands of Ethiopian Birr (ETB). Market Exchange Rate on 1st May 2015 was 19.5ETB to 1 USD (Source: xe.com). Taking into account the Purchasing Power Parity conversion factor (Source: World Bank, International Comparison Program database 2011) the effective exchange rate is estimated to be 7.8ETB to 1 USD. Also, because of its sensitivity, respondents usually don’t want to disclose precise information about the amounts they hold. Hence, to reduce the non-response rate, the questions are designed in a range format rather than asking the actual values. Leadership and confidence One of the main objectives of WALN is to improve beneficiaries’ confidence and leadership skills. Hence, these outcomes were part of the information collected as part of the baseline survey, allowing the measurement of any change due to the program later on. Role in business Figure 23 shows respondent’s role in their business. More than 90% of both mentors and mentees are at the same time the owner and manager of their business. This reflects the small scale of these businesses, where the owner is also the main person running the business. Figure 23: Role of respondents in their business 91% 90% 6% 1% 1% 2% 3% 7% Owner and manager Manager (not the owner) Employee Other Mentors Mentees 28 Decision-making power: perceptions and checks with spouses’ perceptions In order to understand how much actual decision-making power respondents have in their business, they were asked about the persons making major business decisions. Figure 24 confirms that a majority of both mentors and mentees make the decisions alone, though there are still a significant number of women who declare consulting their spouse and/or their business partners. Figure 24: Respondents' decision-making power in their business 64% 51% 21% 26% 19% 10% 1% 3% 4% 1% Exclusive Control Jointly with Spouse Jointly with Partner Jointly with No Control, others Management decide Mentees Mentors To better understand the role played by respondents’ spouses and check their perceptions, Table 8 crosschecks the response given by the respondents against the response provided by their spouse (only for married women). The frequency is shown in blue when their responses match, while it is in red when there is a clear discrepancy. Table 8: Business decision making by the WALN member & spouse responses A. Mentors Male Spouse's View WALN Member’s View WALN Consult Consult Mostly Member Total spouse partners spouse alone WALN Member alone - 100% - - 6 Consult spouse 8% 77% 8% 8% 13 Consult partners - 100% - - 1 Mostly spouse - - - - 0 Any combination - - - - 0 Total 1 17 1 1 20 B. Mentees Male Spouse's View WALN Member’s View WALN Consult Consult Mostly Member Total spouse partners spouse alone WALN Member alone 33% 64% - 3% 30 Consult spouse 4% 93% 2% 2% 56 Consult partners - 67% 33% - 3 Mostly spouse - - 50% 50% 2 Any combination - - 100% - 1 Total 12 74 3 3 92 29 All the spouses of married mentors who report that they alone make all business decisions believe that they are consulted by their wives in such decisions. Similarly, mentees who consider that they were making all decisions alone, only 33% of their spouses confirm that it is the case while 64% report being consulted. Regarding respondents who declare consulting their spouses, spouses’ answers seem to match: 77% and 93% of mentors’ and mentees’ spouses confirmed it. Using well-known survey techniques, a set of questions was asked in order to assess respondents’ self- confidence (see Figure 25), which seems broadly similar for mentors and mentees. Figure 25: Assessing respondent’s self-confidence A. My business success is because of [Allah/God] B. I never try anything that I am not sure of Mentees Mentees Mentors Mentors C. I am at least smart as most others my age D. I make good decisions about managing money Mentees Mentees Mentors Mentors E. When treated unfairly, I speak or take action against it F. I feel I don’t have the same opportunity as other people Mentees Mentees Mentors Mentors G. I don’t need more business training Mentees Mentors Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Don't know 30 One important observation is that both mentors and mentees declare wanting further training to manage their business, showing that they believe they do not have all the necessary skills and knowledge to run their business. This is what ACDI/VOCA’s intervention is rightly trying to tackle. The success of a business might be affected by the relation of its manager with different stockholders that are operating in their environment. The impact might be direct or indirect depending on both the type of the business and/or the organization/individual. As shown in Figure 26, both mentors and mentees seem to have a great deal of confidence in their spouse, family and religious authorities followed by government and employees. Figure 26: Confidence in organizations and/or individuals A. Mentors Spouse Family Male Female Businesswomen Businesses Government Police Courts Religious authority 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% B. Mentees Spouse Family Male Female Businesswomen Businesses Government Police Courts Religious authority 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% A great deal of confidence Quite a lot of confidence Not very much confidence None at all 31 V. Balance between treatment and control groups Attachment 1 contains the WALN Baseline Summary Statistics and Balance Tables. The table is organized into 6 sections and 10 columns. Each of the subsections contains a group of related variables. The purpose of examining the balance of baseline variables is to test whether the difference between the treatment and controls is statistically significant at baseline. In other words, if their differences are insignificant, it means that the randomization strategy has succeeded in delivering two comparable groups (at least on observable characteristics), making it possible to attribute any significantly different outcomes to the intervention. Column 0 is the variable name and brief description. Columns 1a-1c provide data regarding the first experiment comparting Treated Mentors and Control Mentors, Columns 2a-2c regarding the second experiment Treated Mentees vs. Control Mentees of Treated Mentors, and finally Columns 3a-3c regarding the third experiment, All Mentees of Treated Mentors vs. All mentees of Control Mentors. Columns 1a, 2a and 3a provide the pooled sample mean for the both Treatment and Control observations in each of the experiments. Columns 1b, 2b and 3b provide the difference between the means of the Treatment and Control observations in each of the experiment. Finally, Columns 1c, 2c and 3c report the number of observations which were used to calculate the values in other two sets of columns. The remainder of this section discusses the balance of each subsection of Attachment 1. Individual, household and Education variables (Sections A & B) For the Mentors experiment there are only two cases of significant differences in characteristics between treatment and control mentors: 1) The number of mentors who have completed at least high school is higher by close to 11% for treatment mentors than for control mentors; 2) Almost 13% more treatment mentors hold a leadership position; For mentees of treatment mentors, schooling is the only variable for which we can observe significant differences, as mentees in the treatment group are close to 9% more likely to have completed at least primary school and, as a complement, they are less likely, by close to 6%, to have no schooling at all. Finally in the comparison between all mentees from treated mentors with those from control mentors, there are some differences on schooling: mentees of treatment mentors are about 6% less likely to have any schooling; and 4% more likely to have completed high-school. Also, on average in households of mentees of control mentors seem to have be fewer men (in absolute terms) and be less likely to be married. However, the joint test of the significance of these differences – can we find that these variables, jointly explain a significant difference between the two groups that are compared? – confirms that each of those groups are on average balanced enough to be comparable. Business variables (Section C) In general, the sample seems well balanced between the treatment and the control group, as far as business-related variables are concerned. There are only a few variables for which there is a significant 32 difference between treatment and control mentors, or between treatment and control mentees of treatment mentors. However, differences in a larger set of characteristics is observed between mentees from treatment mentors and mentees from control mentors. Business practices and financial management variables (Section D) In addition to the summary graphs and tables presented above, it is also important to see if there is a significant difference among different treatment groups’ business and financial management skills at baseline. As indicated in Section D, there are about 15 and 13% difference between treated and control mentors in terms of having annual written budget and saving in a bank. This shows that mentors in the treatment group perform better in these two outcome variables. Apart from these two cases, there is no significant difference on the other outcome indicators in the table. Revenue, profit, asset and inventories (Section E) Survey variables provide detailed information about respondents’ earnings, as well as on the outcomes of the WALN business, in terms of revenue and profit. The balance between treatment and control groups along those variables is presented in Section E. It shows that there is no significant difference between treated and control mentors, except on inventories. Similarly, there is little statistical significant difference between treatment and control mentees from treated mentors. On the other hand, there are a couple of significant differences between mentees from treatment and control mentors: this will affect the statistical power to estimate the impact of the intervention on these variables. Leadership and confidence variables (Section F) As with the other outcome indicators, we also tried to see if the treatment and control groups are similar on leadership and decision making characteristics at baseline. As it is indicated in Section F, mentors from the treatment group are less likely to make business decisions alone while the opposite is true in terms of consulting their business partners. On the other hand, treatment and control mentees from treated mentors have on average similar characteristics in these outcome variables. In addition, a statistically significant difference has been observed between mentees from treated mentors and mentees from control mentees in terms of consulting business partner in decision making and delivering speech in front of more than 5 persons. However, as explained in previous sections, it should be noted that our randomization didn’t attempt to balance these two groups. Overall Joint Test of Significance (Section G) The joint test of significance including all the 86 variables in the Baseline Summary table finds no significant differences in any of the experimental samples. However, it should be noted that the number of observations utilized to generate these statistics is a small subset of the total observations available in each experimental sample. 33 VI. List of Attachments 1. WALN Baseline Summary Statistics and Balance Tables 2. Enumerators Training Program 3. Enumerators Field Manual 4. WALN Application Form 5. WALN Mentors & Mentees Questionnaire 6. WALN Male Spouses Questionnaire 34