35742 A F R I C A R E G I O N H U M A N D E V E L O P M E N T W O R K I N G P A P E R S S E R I E S N o . 5 2 THE WORLD BANK Improving Educational Quality through Interactive Radio Instruction A Toolkit for Policy Makers and Planners Titles in the Africa Region Human Development Working Paper Series: No. 24 Better Health Outcomes from Limited Resources: Focusing on Priority Services in Malawi (April 2002)* No. 2 Engaging with Adults: The Case for Increased Support to Adult Basic Education in Sub-Saharan Africa No. 25 Secondary Education in Africa: Strategies for Renewal (February 2001) (May 2002) No. 3 Enhancing Human Development in the HIPC/PRSP No. 26 Financing Education for All by 2015: Simulations for 33 Context: Progress in the Africa Region during 2000 African Countries (June 2002) (March 2001) No. 27 World Bank Support for Provision of Textbooks in Sub- No. 4 The Implications of Private Safety Nets for Public Policy: Saharan Africa (1985-2000) (June 2002) Case Studies of Mozambique and Zimbabwe (April 2001) No. 28 Human Development in Africa: An Action Plan (June 2002) No. 5 Early Childhood Development in Africa: Can We Do More for Less? A Look at the Impact and Implications of No. 29 Public Expenditures on Education in Nigeria: Issues, Preschools in Cape Verde and Guinea (July 2001) Estimates, and Some Implications (August 2002) No. 7 Systematic Shocks and Social Protection: Role and No. 30 Uganda Post-Primary Education Sector Report Effectiveness of Public Works Programs (August 2001) (September 2002) No. 8 AIDS, Poverty Reduction, and Debt Relief: A Toolkit for No. 37 Cost and Financing of Education Opportunities and Mainstreaming HIV/AIDS Programs into Development Obstacles for Expanding and Improving Education in Instruments (August 2001) Mozambique** (June 2003) No. 9 Social Protection of Africa's Orphans and Vulnerable No. 38 Health Care on the Frontline: Survey Evidence on Public Children: Issues and Good Practice Program Options and Private Providers in Uganda (July 2003) (August 2001) No. 39 The Mauritania Health System and Implementation of the No. 10 Community Support for Basic Education in Sub-Saharan Poverty Reduction Strategy: Medium-Term Expenditure Africa (September 2001) Framework (2002-04) for Improving the Efficiency and Equity of Public Health Expenditure (February 2004) No. 11 Can Africa Reach the International Targets for Human Development? An Assessment of Progress toward the No.41 Health and Poverty in the Gambia; A Background report Targets of the 1998 Second Tokyo International to the National Poverty Reduction Strategy Paper* Conference on African Development (TICAD II) (September 2001) No. 42 Guinea: A Country Status Report on Health and Poverty (Health, Nutrition, and Population inputs for the PRSP and No. 12 Education and Training in Madagascar: Toward a Policy HIPC process)* (forthcoming) Agenda for Economic Growth and Poverty Reduction: A Summary (September 2001)** No. 43 Early Childhood Development in Kenya No. 14 Issues in Child Labor in Africa (September 2001) No.44 Skills and Literacy Training for Better Livelihoods-- A Review of Approaches and Experiences (French No. 16 Rapid Guidelines for Integrating Health, Nutrition, and Translation) Population Issues into the Poverty Reduction Strategies of Low-Income Countries (December 2001) No.45 Education in Rwanda: Rebalancing resources to accelerate post-conflict development and poverty reduction No. 20 Improving Health for the Poor in Mozambique: The Fight (Executive Summary) (August 2003) Continues (February 2002)* No. 53 School Education in Nigeria: Preparing for Universal No. 21 Skills and Literacy Training for Better Livelihoods: A Basic Education ** (February 2004) Review of Approaches and Experiences (March 2002) No. 54 Analyse des Indicateurs de Santé et Population au Mali * No. 22 The Impact of Adult Mortality on Primary School (May 2004) Enrollment in Northwestern Tanzania (March 2002) No. 55 Behavior Change Communication for Better Health No. 23 Enhancing Learning Opportunities in Africa: Distance Outcome (forthcoming) Education and Information and Communication Technologies for Learning (March 2002) * Country Status Report (continued on next page) AFRICA REGION HUMAN DEVELOPMENT WORKING PAPER SERIES Improving Educational Quality through Interactive Radio Instruction A Toolkit for Policymakers and Planners Development Research Group The World Bank IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION March 2005 Human Development Sector Africa Region The World Bank This toolkit was developed for the World Bank by Stephen Anzalone and Andrea Bosch, Education Development Center (EDC), Washington, D.C., with editorial assistance from Douglas Lehman and Nancy Levine. The views expressed in this paper are those of the original authors and do not reflect the opinions of the World Bank or any of its affiliated organizations. Photos courtesy of EDC, Open Learning Systems Education Trust (OLSET), and First Voice International. The work was supervised by Shola Aboderin, Senior Education Specialist, Africa Region, World Bank. Comments and advice on draft versions were received from Alan Dock, Brigitte Duces, Paud Murphy, David Harding, Robert Prouty, Maman Sidikou, and Adrianna Jaramillo. The Bank thanks the Norwegian Trust Fund for Distance Education in Africa and the Irish Education Trust Fund for Africa for providing the financial resources to de- velop this toolkit. ii Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix 1 What is Interactive Radio Instruction? . . . . . . . . . . . . . . . . . . . . . . . . . 1 Key Characteristics of IRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 How does IRI differ from other learning systems? . . . . . . . . . . . . . . . . . 2 What does the word "interactive" actually mean for a radio broadcast? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Why is IRI considered a learning system? . . . . . . . . . . . . . . . . . . . . . . . . 6 What subjects and audiences have been taught? . . . . . . . . . . . . . . . . . . . 8 Can IRI programs be developed for use in multiple countries? . . . . . . . . 9 What about hard-to-reach and out-of-school populations? . . . . . . . . . . 10 How do teachers fit in? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Can IRI help address the challenge of multigrade classrooms? . . . . . . . . 13 Quality in Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 How does IRI affect achievement? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Can IRI reduce equity gaps? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 What is the effect of IRI over time? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Going to Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 How are costs distributed? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 What is known about cost-effectiveness? . . . . . . . . . . . . . . . . . . . . . . . . 24 iii IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Have cost-sharing strategies been tried? . . . . . . . . . . . . . . . . . . . . . . . . . 25 Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Where have IRI efforts been sustained? . . . . . . . . . . . . . . . . . . . . . . . . . . 27 What are the critical factors for success? . . . . . . . . . . . . . . . . . . . . . . . . . 28 2 Feasibility of IRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Educational Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Audiences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 What subjects and grades can be taught with IRI? . . . . . . . . . . . . . . . . . 40 How does the content of IRI programs relate to an existing curriculum? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Has a decision been made on the language of instruction to be used in the IRI program? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Can IRI content be adapted from another country? . . . . . . . . . . . . . . . . 45 Program Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Are the key inputs needed to develop an IRI initiative available? . . . . . . 47 What kind of institution should lead the development of an IRI program? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 What relationships are needed with other stakeholders? . . . . . . . . . . . . . 50 What level of technical assistance is required? . . . . . . . . . . . . . . . . . . . . 53 Should programs be broadcast, or delivered on cassettes or CDs? . . . . . 55 What kind of pilot project is required to test IRI? . . . . . . . . . . . . . . . . . 57 Cost and Finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 What does it cost to develop and deliver IRI? . . . . . . . . . . . . . . . . . . . . . 59 How do IRI costs compare with those of other technologies? . . . . . . . . 61 How can recurrent costs be contained? . . . . . . . . . . . . . . . . . . . . . . . . . . 61 How are costs distributed and covered? . . . . . . . . . . . . . . . . . . . . . . . . . 63 What sources may be used to finance an IRI program? . . . . . . . . . . . . . 66 Planning for Institutionalization and Sustainability . . . . . . . . . . . . . . . . . . . 67 What is the plan for institutionalization? . . . . . . . . . . . . . . . . . . . . . . . . 68 What is the plan for sustainability? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3 Designing IRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Project Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Why is it important to understand the development process? . . . . . . . . . 74 iv CONTENTS What does audience research include? . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 What are the educational objectives? . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 What should the design document contain? . . . . . . . . . . . . . . . . . . . . . . 78 What is involved in scriptwriting? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 What is formative evaluation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 What are the options for audio production? . . . . . . . . . . . . . . . . . . . . . . 83 What are the options for print production? . . . . . . . . . . . . . . . . . . . . . . 84 What are the delivery options? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Marketing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Summative Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Designing for Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 A Final Note . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Appendix A. Summary Table of IRI Programs . . . . . . . . . . . . . . . . . . . ` 91 Appendix B. IRI Vignettes from Sub-Saharan Africa . . . . . . . . . . . . . 97 Ethiopia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Lesotho . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Portuguese-Speaking African Countries (PALOP) . . . . . . . . . . . . . . . . . . . . . 102 South Africa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Zambia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Online Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Index Tables 1. Mathematics scores in IRI centers and public schools, Zambia, 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2. Status of 29 IRI programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3. Responsibility for IRI in the context of decentralization, by level of authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 v IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION 4. Comparative per-student costs of selected educational technologies, 1998 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5. Distribution of costs in an IRI project in Africa . . . . . . . . . . . . . . . . . . . 65 6. Sources of funding: Five IRI programs . . . . . . . . . . . . . . . . . . . . . . . . . . 67 A-1. Subjects covered by IRI programs in Latin America and the Caribbean, Asia, and Africa, 1974 to present . . . . . . . . . . . . . . . 91 Boxes 1. Key IRI terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. IRI script excerpt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. The pedagogy of Interactive Radio Instruction . . . . . . . . . . . . . . . . . . . . 7 4. A South African teacher on IRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 5. Shadow audiences in Guinea's IRI program . . . . . . . . . . . . . . . . . . . . . . 39 6. Local community involvement in IRI in Zambia . . . . . . . . . . . . . . . . . . . 51 7. Pros and cons of wind-up radios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 8. Understanding the costs of IRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Figures 1. Post-test data on achievement of students in selected IRI projects . . . . . 16 2. The effect of IRI programs on urban-rural equity gaps . . . . . . . . . . . . . . 18 3. Typical IRI learning system and interaction of functions . . . . . . . . . . . . 49 4. IRI per-student cost at various program scales and for one subject . . . . 60 5. Development of an IRI program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6. Formative evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Checklists Educational goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Program development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Cost and finance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Planning for institutionalization and sustainability . . . . . . . . . . . . . . . . . . . . 68 vi Foreword This toolkit, Improving Educational Quality through Interactive Radio Instruction, is toolkit, "Improving Educational All and Universal Primary Education. part of the ongoing effort by the World Bank'sStudies have documented its effectiveness Quality through Interactive Radio Instruc- Africa Region to deepen understanding of howisdistanceineducation of well as information communication technologies (ICTs) tion," one a series as initiatives in improving quality and retention rates can support by the WorldAfrica. The toolkit and reducing absenteeism at "Enhancing undertaken education in Bank's Africa builds on the strategy paper the primary Learning Opportunities in Africa" (Murphy etschool level. Region Human Development Department al. 2002), which reviews past experience with distanceof increasing understanding the aim education in Africa and reaffirms its strong potential, with judicious inte- gration of ICTs. education and informa- of how distance tionThe Dakar Framework for Action, adopted at[TALK ABOUT HOWForum in 2000, and communication technologies the World Education IT HELPS IN- anticipates the use ofeducation education and ICTs as part of the strategy for achieving (ICTs) can support distance in Africa. SERVICE TRAINING AND SUPPORT] Education for All (EFA) goals in Africa. Rapid advances in technology and the increas- That effort is part of the Bank's overall ing accessibility of the Internet are kindling increased interest in alternative and innovative education sector strategy for Africa, which XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX models of education. These include "e-learning" and more recently, "blended learning" emphasizes the achievement of universal [More to COME] which combineseducation anddistance learning with conventional delivery. Traditional primary school new modes of improved technologies also continue to evolve and offer new ways to support educational im- access and quality at all levels. provements.be aexample oftool in Africanis interactive radio instruction (IRI). IRI extends IRI can An valuable this progress the use of broadcast radio the targets set being a tool to increase access to becoming a countries' efforts to meet from primarily means of promoting educational quality. The authors note that since IRI's introduction for achieving the goals of Education for nearly 30 years ago, studies indicate its effectiveness in improving educational quality as well as retention rates at the primary school level. Studies also demonstrate IRI's value as an effective mechanism to introduce new pedagogy and curriculum. Cost-effec- tiveness is also offered as one of IRI's primary benefits. In the section on "Cost" in chapter 1 of the toolkit, the authors cite findings on the cost-effectiveness of IRI in improving the quality of primary school instruction, compared with that of alternative investments, such as providing additional textbooks or teacher training. These findings make IRI a potentially powerful tool for African governments as they seek to meet the challenge of EFA by improving educational quality and undertaking in- vii IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION service teacher upgrading. Evidence suggests that while access to primary education has increased in many African countries, following reforms launched in the drive toward EFA, improvements in quality have not matched the gains in access. The biennial re- ports for 2003 of the Association for the Development of Education in Africa (ADEA) warn that improvement of educational quality remains the most critical challenge for Africa's educational systems. The need to upgrade and train millions of teachers across the region is a significant part of that challenge. It is therefore encouraging to learn from the toolkit that IRI programs are now emerging as an effective means of support- ing teacher development, in addition to their original focus on the learner. The use of IRI as a teacher support tool may also help justify the add-on costs associated with IRI interventions. IRI is clearly an important option to consider in improving educational quality. This is particularly true given the high penetration of radio in many African countries. Still, it is important to note that in spite of its successes in Africa and internationally, IRI generally remains outside the mainstream relative to other distance education interven- tions. This may be due to the relatively high initial investment costs and the fact that IRI has been funded mainly by one donor, the U.S. Agency for International Development (USAID). However, the growth of IRI interventions in Africa, and the national scale of the IRI program in Guinea may spur further interest among other African nations and a larger number of funders. The introduction of innovative technologies and pedagogies must always be based on a critical assessment of experiences and options. This toolkit offers a timely compi- lation and analysis of recent experience of IRI in Africa and elsewhere. It explains the IRI concept. It guides the reader through the process of determining when IRI might be appropriate in a given context, and it describes the steps in setting up an IRI program. The authors emphasize the critical factors for success, including appropriate policy frame- works as well as planning for the necessary investment and sustainability. It is hoped that the toolkit will lead to greater appreciation of IRI's potential and that it will make a useful and enriching contribution to the work of policymakers and edu- cators in Africa. Yaw Ansu Sector Director, Human Development Africa Region The World Bank viii Preface Interactive radio instruction (IRI) has emerged as an important option for improving educational quality in primary school classrooms in developing countries around the world. Studies of the IRI experience in more than two dozen countries during the past 25 years have shown that the use of IRI has led to significant and consistent improve- ments in school achievement and has helped overcome equity gaps between urban and rural children and between boys and girls. Because IRI can be broadcast to large audi- ences across a country, the cost per student is often low in comparison with other options for improving the quality of instruction. In addition, IRI is a proven, effective strategy for reaching children and youths who do not have access to formal schooling, and a significant number of "secondary" audiences also benefit from the radio lessons. This toolkit is intended for African policymakers, education planners, and pedagogi- cal specialists who may be considering the feasibility of using IRI in their education systems. Although, as is documented in the appendixes, considerable use had been made of IRI in Africa, there has been relatively little application of it in francophone coun- tries. The recent successful implementation of IRI in Guinea could serve as a catalyst for countries with similarly structured education systems. The growing interest in IRI has contributed to increased regional cooperation and collaboration as countries share ex- periences and resources. Since IRI focuses on a relatively modest, low-cost use of technology, it can have broad coverage and impact. It is especially useful in resource-lean environments and in settings where access to fundamental quality education in isolated areas is a problem. A particular benefit of IRI is its ability to improve classroom practices by exposing teach- ers, many of them untrained or undertrained, to the principles of learner-centered, child-friendly education. IRI also helps build capacity within education communities to enable them to embrace new approaches, innovations, and pedagogical thinking. ix IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Chapter 1 of the toolkit presents a general description of the IRI methodology and discusses the effectiveness, scale, and sustainability of the IRI approach. Chapter 2 then outlines steps for assessing the potential and appropriateness of IRI for improv- ing educational quality in primary school classrooms in a given country, and chapter 3 leads the reader through a guided decisionmaking process for identifying the issues involved in applying the IRI methodology in particular country contexts. Two appen- dixes give summary data and illustrate, through brief vignettes, how IRI programs have worked in Sub-Saharan African countries. The Resources section contains fur- ther useful information, much of it available online. The CD accompanying this book allows readers to see and hear IRI in action and provides a database of resources. x Chapter 1 What Is Interactive Radio Instruction? Main Points s Interactive radio instruction (IRI) is a distance education system that com- bines radio broadcasts with active learning to improve educational quality and teaching practices. IRI has been in use for more than 25 years and has demonstrated that it can be effective on a large scale at low cost. s IRI programs require teachers and students to react verbally and physi- cally to questions and exercises posed by radio characters and to participate in group work, experiments, and other activities suggested by the radio program. IRI builds on local resources and knowledge. s IRI has been used to teach nearly all basic primary subjects and audiences of all ages, as well as hard-to-reach and out-of-school populations. s There is consistent and significant evidence that IRI can increase learning across subject matter, age, gender, and rural or urban location. Students show progressively greater learning with time. s In Guinea, South Africa, and some Latin American countries, IRI pro- grams have demonstrated that they can enlarge their audiences, sometimes reaching a million or more students. s There is evidence that the benefits of IRI can be sustained over the long term and can be adapted for other countries. s Incentives for investing in IRI include its cost-effectiveness compared with other technologies and the willingness of donors and lenders to assist with up-front development costs. 1 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION T his chapter introduces IRI methodology--what it is, how it evolved, and how it can contribute to educational quality. Over the past several decades, radio has been used extensively around the globe to facilitate formal and nonformal learning, in places as diverse as Australia, Canada, Ghana, India, and Zambia. The programs have been directed at schoolchildren, teach- ers, illiterate adults, farmers, health workers, and many other groups of learners. Radio has proved to be a medium that is pedagogically effective and cost-efficient. Interactive radio instruction combines broadcast radio or another audio medium with an emphasis on active learning to improve educational quality. IRI programs guide the teacher or facilitator and the students through activities, games, and exercises that teach specific subject matter and offer the teacher models of how to organize effective learning activities. Teachers and students participate throughout the programs, during pauses that are built into the design of the radio scripts. These principles of active learn- ing and the particular style of instructional design distinguish IRI from other distance learning techniques. The original IRI model was a mathematics series that was created in Nicaragua in the early 1970s by a team from Stanford University with the support of the U.S. Agency for International Development (USAID). The program, IRI Math, combined the low cost and broad reach of the radio medium with a clear understanding of how people learn. Since then, more than two dozen countries around the world have engaged in developing IRI programs for a variety of subjects, audiences, and learning environ- ments. Many of these programs have been in operation for a decade or more. These experiences have shown that IRI can be used effectively to enhance learning in a large audience and can help reduce equity gaps between girls and boys and between rural and urban populations by standardizing certain learning processes. Key Characteristics of IRI IRI has a number of significant characteristics that distinguish it from other distance learning approaches. How Does IRI Differ from Other Learning Systems? Many distance learning efforts are primarily intended to increase access to education. IRI programs have as their main goal the improvement of educational quality within 2 WHAT IS INTERACTIVE RADIO INSTRUCTION? existing structures. Programs are designed to engage a specific audience through the use of entertaining characters and activities and an active learning pedagogy (see box 1 for terminology). The use of radio broadcast increases the ability of IRI programs to reach a large audience of students without dramatically increasing the cost, and so IRI is able to improve educational quality on a large scale. What Does the Word "Interactive" Actually Mean for a Radio Broadcast? Because radio is a one-way broadcast medium, IRI programs are not truly interactive in the sense of two-way communication. Short pauses provided throughout the lessons allow teachers and learners to stop and react to questions and exercises through verbal and physical responses to radio characters, group work, experiments, and other physi- cal and intellectual activities while the program is on the air. For example, an IRI program might achieve interactivity by having a group of students manipulate locally gathered materials such as rocks and bottle caps; participate in games; listen to and respond to information, stories, and exercises that are guided by radio characters; and interact with other students in the class. In some cases, student workbooks, posters, and wall friezes are provided to support and enhance the learning. Box 1. Key IRI Terms Active learning pedagogy. A teaching approach in which students learn through "hands- on" activities. Active learning involves physical activity with mental engagement that keeps students productively interested and focused on learning. Students are encouraged to learn by doing, experimenting, touching and manipulating objects, moving around, making things, solving problems, and working together. Formative evaluation. An evaluation conducted during the development and implemen- tation of a project. Formative evaluation looks at how well a program is doing in operation; it shows what is working and what has to be changed to accomplish program goals. It also tests the chosen audience during the pilot and implementation phases with a view to improv- ing script production, technical quality, and delivery to ensure maximum learning. Formative evaluation contributes throughout the IRI development process and so helps shape the final product. (continued on next page) 3 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Box 1. Key IRI Terms (continued) Summative evaluation. An assessment carried out at the beginning and end of the IRI program to measure how much the audience has learned from the program and how it is using this knowledge. Learning system. A network of activities that supports learning. It can include a variety of media, as well as activities inside and outside the classroom. Distance education. A set of practices for planning and implementing educational activi- ties when a separation, caused by distance, time, or other barriers, exists between teaching and learning. Distance education offers a means of overcoming this separation, chiefly through the characteristics of its learning materials, the application of communication technologies to provide tutoring and link learners to the system and to each other, and the use of feedback and student support systems (Murphy et al. 2002). Learning environment. The place where learning occurs--for example, a school, a church or mosque, the home, or a community center. Holistic learning. An approach to learning that takes into account the complexity of the learner as a whole person. Instead of focusing on content in one subject area, a holistic program integrates content across disciplines. The holistic approach emphasizes interconnec- tions, the relationship between theory and practice, and the development of critical thinking. It encourages the learner to develop to his or her full potential. Multichannel learning. The use of several different channels of communication to facili- tate the learning process. Learning is understood not just as what is transmitted in a formal setting such as a classroom but also what is learned in the home, the church or mosque, and the overall community. It also encompasses learning that takes place as a result of access to various media, such as radio, newspapers, television, or the Internet. It builds on prior learn- ing and indigenous knowledge. Multigrade teaching. A situation in which a teacher simultaneously provides instruction to two or more grade levels in the same classroom (as distinguished from multiclassroom teach- ing, in which a teacher instructs different grades in their individual classrooms). Analog and digital recording and editing. Analog technology is the older of the two. A simple example of analog technology is the cassette recorder, which records sounds of differ- ing frequency and amplitude on magnetic tape. Editing then involves manipulating the tape. With digital technology, all sounds are recorded directly in a computer file, as binary code. Editing is much simpler, and there is less chance of corruption of the sound over time. Note: See also De Fossard et al. (1995). 4 WHAT IS INTERACTIVE RADIO INSTRUCTION? The short pauses provided throughout the lessons after questions and during exercises ensure that students have time to think and to respond adequately. The example in box 2 is a drill and practice leading into a team activity. For different types of exercises there are different lengths of pauses and different roles for teachers and students. The stories and educational situations are developed locally and are designed to fit into the IRI methodology. This approach results in educational activities that are engaging and in- teractive. Box 2. IRI Script Excerpt Learning Tree: Children, we will play the letter game. I will call out a word, and you will tell me what letter the word starts with if your teacher picks you. We will use letters from the alphabet before the letter P. Teacher, please remind the students how to play the letter game in mother tongue. Sound Effects: MUSIC BRIDGE INSTRUMENTAL, 10 SECS Learning Tree: Teacher, please choose a girl to answer a question. Pause for Teacher Response: 3 SECS. Learning Tree (exaggerating B in banana): Here we go . . . what letter does the word BANANA start with? Pause for Student Response: 3 SECS Learning Tree: B, very good. B makes the sound /b/. BANANA starts with the letter B. Teacher, pick a boy this time. Pause for Teacher Response: 3 SECS Learning Tree: What letter does the word ICE start with? Pause for Student Response: 3 SECS Learning Tree: I! Letter I starts the word ICE. Now let's all play. Please divide the students into four teams. When I say a word, please ask one child in each group to write down the letter the word begins with on their slate or a piece of paper and hold it over their head. If they get the letter right, their team gets a point. 5 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Why Is IRI Considered a Learning System? IRI programs are designed to be a part of a comprehensive, multichannel learning sys- tem that generally includes written materials, teacher orientation, and other instructional activities. In most cases, IRI does not alter existing education structures; rather, it acts as a tool for changing the quality of interaction and learning that takes place within them. It works to integrate the elements of teaching and learning by building on re- sources that are already available--broadcast capability; radios; teachers; local songs, games, and activities; other students; community members; and locally available mate- rials--in specific ways that have been associated with good teaching and learning practice. Increasingly, however, IRI is also being introduced as a means for creating a learning system where the education system has been either dysfunctional or nonexistent. As an add-on to improve quality or as a fundamental means of generating the basic interac- tions of teaching and learning, the IRI program is designed to bring together existing local structures into a meaningful system of interactive learning and, in doing so, to enhance or generate a higher quality of teaching and learning. The audio medium serves Students, a teacher, a radio, and supporting visuals make up the typical IRI classroom scene. 6 WHAT IS INTERACTIVE RADIO INSTRUCTION? Box 3. The Pedagogy of Interactive Radio Instruction A key factor in the success of IRI learning systems has been the active learning pedagogy that is used in the design of an IRI series and is incorporated into every script. This is what sets IRI apart from other distance education strategies that use radio broadcasts, audio CDs, or audio tapes. The Nicaragua IRI Math series created in the 1970s by Stanford University developed the original parameters for the pedagogical principles that became IRI. Over time, some of the strategies changed as innovative methods were tested and different subject matters and audiences were targeted. Yet a number of consistent pedagogical principles define IRI and are present in IRI series: s Active learning pedagogy. Students take an active role in the learning process by doing some- thing--answering, measuring, singing, working in groups, counting, practicing, and so on. s Structured learning. IRI programs structure the learning process so that teachers and students know what to do. This method serves to guide activities and to ensure that the overall curriculum is addressed. s Recognition of existing knowledge. The IRI series builds on the existing knowledge of local audiences. Where a great disparity in knowledge is found among potential listeners, the IRI series begins by laying down a foundation of knowledge that can be drawn on in future lessons. In many programs, teachers are instructed to use the extended pauses to make connections between local knowledge and what is being discussed in the lesson. s Distributed learning. In the context of IRI methodology, this term describes a curriculum design that spreads the acquisition of new knowledge across several lessons so that stu- dents can learn gradually and practice new skills. Students are not expected to master new skills on the first try. s Diversified educational activities. Throughout the programs, students are invited to par- ticipate in varied learning activities--individual work, group work, experiments, communicative approaches to languages, word problems in math, and so on. This re- duces reliance on rote learning methods and diversifies the learning experience. s Engaging characters, stories, and exercises. IRI uses local activities, stories, and games to capture students' interest and motivate them to participate. s Reinforcement. The lessons that are taught are reinforced through repetition and posi- tive feedback. s Defined role of the teacher. Although IRI series differ in the degree to which the program is a teacher-training tool, all IRI programs build in specific and active roles for the teacher or facilitator so that he or she takes the leadership role in the teaching and learning process. s Formative evaluation. Individual IRI programs and the entire series are evaluated on an ongoing basis throughout the life cycle of the program to determine whether students are actively participating and are mastering skills. If shortcomings are found, additional attention is given to the specific skill in subsequent lessons. 7 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION as the core component that integrates learning across subject areas and guides student involvement (see box 3). The linkages of IRI to existing or new learning channels can have positive effects. For example, a study in Honduras showed that the combination of IRI radio programs with other educational interventions may have synergistic impacts. When IRI programs are introduced along with new textbooks, for example, the effect on gains in learning is almost double that of only providing textbooks. Thus, if multiple learning approaches are deliberately aligned, the combination of traditional and new techniques may have the maximum impact on learning (Tilson et al. 1991; van Steenwyk et al. 2002). What Subjects and Audiences Have Been Taught? IRI has been used to teach almost all basic primary subjects to audiences of all ages. IRI programs have been created for mathematics, science, health, English, French, Spanish, Portuguese, reading, environmental education, early childhood development, and adult basic education. In most cases, the IRI series closely follows the national curriculum for a particular subject and is produced in coordination with the ministry of education. Where there is no national curriculum for a particular subject, such as environmental education, health education, or early childhood development, IRI curriculum develop- ment may become part of the curriculum development process. IRI has been targeted to diverse populations of learners in different learning environ- ments, including preschool children and their caregivers, primary school students, school-age stu- dents who lack access to schools, out-of-school youths, teachers, and popu- lations that are vulnerable because of HIV/AIDS, refu- gee status, or other hardships. It is being used in southern Sudan to build capacity in a postconflict education system. In each IRI is suitable for learning in case, the series has been de- different environments. signed by local specialists to be engaging and to meet 8 WHAT IS INTERACTIVE RADIO INSTRUCTION? learning objectives. Table A-1 in appendix A shows subjects covered by IRI programs in Latin America, Asia, and Africa during the past three decades. Can IRI Programs Be Developed for Use in Multiple Countries? Almost all of the IRI activities reviewed here have been applications in a single coun- try, although some programs have involved adaptations of lessons developed elsewhere. An exception is the multicountry effort sponsored by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) to develop mathematics and Portu- guese instruction to serve lusophone countries in Africa (Angola, Cape Verde, Guinea-Bissau, Mozambique, and São Tomé and Principe--commonly referred to as the PALOP countries). A final evaluation of the program was expected to take place in 2004. Although it would be unwise to generalize from one example, this experience highlights an important question for the future: are multicountry, regional efforts to use IRI feasible? The advantages of being able to spread the often high fixed costs of program devel- opment across several countries are compelling, particularly where the same ethnic or linguistic groups reside in neighboring countries, potentially enlarging the pool of learners using a single set of programs. IRI programs for Somali-speaking people in Ethiopia are being developed with this goal in mind. The programs are designed to be suitable for Somali-speaking children throughout the Horn of Africa, including the target group in Ethiopia, where the radio programs are currently being delivered. Satellite broad- casting makes this breadth of coverage possible. Discussions with Somali and Ethiopian government officials on the shared use of these programs have taken place, but no agreement has been reached as yet. Although the situation may change, and new IRI programs are still being developed, building collaboration across borders remains a challenge. Many circumstances affect the feasibility of multicountry curriculum use, however strong the economic advantages. One of the most difficult obstacles is the bilateral nature of donor- or lender-assisted funding (grants and loans). Even where donors and lenders see the advantages of multicountry efforts, organizational structures and proce- dures often make such initiatives cumbersome. Another problem is that even small differences between countries in the curriculum or language, or in such things as the names of characters and places, can take on considerable significance. Still, there are ways to overcome these obstacles, given sufficient will and the resources to do so. It would be necessary, for example, to replicate in each participating country certain con- ditions related to critical success factors such as the presence of local IRI champions, sturdy institutional arrangements, and the ability to cover local recurrent costs. 9 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION What about Hard-to-Reach and Out-of-School Populations? Although IRI has generally been employed to improve quality in conventional classrooms, it has often been used with hard-to- reach populations. Studies of IRI show that the level of achievement among ru- ral students served by the programs has been as high as or almost as high as for urban students. Because ru- ral students generally have IRI programs can be similarly fewer resources, less well engaging and effective for both trained teachers, and lower rural and urban students. achievement levels, gains in learning among this group represent more significant total increases in overall learning. IRI programs have increasingly been developed in such a way as to capitalize on their ability to serve hard-to-reach populations, provide education where it has been nonexistent or poor, and increase learning gains. In areas where no schools exist or where the schools that do exist have not been able to meet the needs of particular popu- lations, IRI is being introduced as a means of serving hard-to-reach populations. Remote areas that are benefiting from IRI include agricultural areas of the Dominican Republic; worksites of adults and youths in Honduras; communities hard hit by HIV/AIDS in Africa, where the number of trained teachers is being depleted; areas of Tanzania where the demand for child labor undermines school attendance; areas in Honduras and Nepal where early childhood development programs are nonexistent; and refugee and no- madic areas of Ethiopia where schooling is not available. IRI programs in these settings address educational and social issues that go beyond quality in the classroom, to the life skills and basic knowledge that these children will need if they are to use their educa- tion. In all cases, the programs function in collaboration with ministries of education, rather than in competition with them. Several of these IRI programs were initiated within the past five years and still re- ceive donor funds to institutionalize and sustain them. Others have long graduated and 10 WHAT IS INTERACTIVE RADIO INSTRUCTION? are being sustained locally. The RADECO program, for example, has been functioning in parts of the western Dominican Republic for almost 20 years without donor support. The Honduran adult basic education and secondary education programs are the result of agreements between the ministry of education and local business and community organizations. Five of the newest IRI projects in Africa emphasize hard-to-reach popu- lations--primary-age children in rural and urban Guinea; children without formal schools in Zambia (grades 1­5); Somali refugees in Ethiopia (basic literacy and numeracy); child laborers leaving their work and integrating into formal schools in Tanzania (grades 1­ 4); and students in Qur'anic schools in Nigeria (grades 3­6). An IRI project is under way in southern Sudan, where children and teachers have been without formal schools for over a decade. On balance, an IRI program is one of the few strategies for improving the quality of education that has a relatively greater impact on the least-well-served segments of the population. This equity-enhancing characteristic is critical as countries strive to attain Education for All by focusing on the needs of poorly served communities, particularly in rural areas and among disadvantaged groups. How Do Teachers Fit In? IRI programs differ in the specific ways that teachers or facilitators are involved in the instructional process during broadcast and the degree to which they use IRI as a teacher- training tool. No IRI series has been designed to replace the teacher. Instead, the design of IRI requires the involvement of a teacher or advanced mentor to guide interaction. Most often, the programs aim to build the teacher's skills and enable the teacher to play a more active role in a student-centered and interactive teaching and learning process. The teacher's role in IRI programs has evolved over the years. Most of the earlier IRI series were organized so that teachers would mainly follow along with the activities and provide follow-up according to the radio characters' instructions. The original IRI Math series in Nicaragua, for example, guided student instruction and then asked the teacher to provide individual support to students who needed it. Since then, most IRI programs have given teachers a more central role and have introduced games, exercises, and group work that are directed by the IRI characters and managed by the teachers. Teachers or mentors consistently guide questions and answers, organize educational activities, and serve as role models. Since the mid-1990s, IRI programs have emerged as specific teacher-training tools. For example, in 1997 the Nepali government, with support from the United Nations Children's Fund (UNICEF), developed the "dual audience" IRI approach to help teach- 11 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Teachers in IRI programs generally find it relatively easy to improve their methods by following the radio teachers' approach and by observing the students' responses to the lesson. ers learn active teaching methods for primary mathematics and English. Teachers were given explicit directions about why certain types of activities could improve learning and how they might adapt them when IRI was not on the air. Since 1998, Guinea's IRI programs for French and mathematics instruction have incorporated a reflective teacher- training series that helps teachers assimilate student-centered instructional practices into their own teaching styles. In the mid-1990s, the Open Learning Systems Education Trust (OLSET) English in Action program in South Africa helped establish a professional development and support system for teachers that became integral to IRI delivery, moni- toring, and evaluation (see "South Africa" in appendix B). It included teacher-training workshops, manuals, and regional coordinators whose role was to nurture, advise, and support the teacher. The approach is also being applied in Nigeria, where IRI broad- casts provide teachers with recurrent hands-on training that complements other in-service or pre-service teacher-training efforts. One of the great synergies of IRI is that it can be an effective vehicle for teacher training, even though students, teachers, and the public perceive it as being oriented toward students. 12 WHAT IS INTERACTIVE RADIO INSTRUCTION? Can IRI Help Address the Challenge of Multigrade Classrooms? Although most IRI programs are designed for a single grade level of a national curricu- lum, there has been some experience in addressing the special needs of multigrade classrooms. In Costa Rica, a short series of IRI programs for environmental education was organized with multigrade classes in mind. Teachers with students from first to sixth grade were taught how to adapt particular types of activities or discussions so that groups at different levels would have different tasks. More advanced students were called on to assist their less advanced classmates, helping teachers learn how to organize several streams of activities simultaneously. The series was abandoned because of a change in administration in the late 1990s, but the experiment led to the creation of a ministry office on multigrade teaching, and the method of using IRI for multigrade classrooms resulted in innovations in other learning systems. In an IRI series in Karnataka State, India, a USAID-supported effort is developing a multigrade mathematics and science series for fourth/fifth grade classrooms that fo- cuses on "hard spots" in the curriculum. This dual audience series is used as a teacher-training tool and as direct instruction for students in the two grades. The pro- gram evaluations will address both learning processes and outcomes. The results should be available in late 2005. IRI can help the teacher organize different streams of classroom activities simultaneously. 13 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Quality in Education All parents want the very best education for their children that can be managed, but socioeconomic circumstances, geographic disadvantages, poorly trained and untrained teachers, and lack of access to educational resources often hamper this aspiration. Most governments are striving to meet Education for All (EFA) and Universal Primary Educa- tion (UPE) targets in the face of considerable difficulties--not the least of which is the impact of HIV/AIDS on the education sector. All these constraints compromise quality education in the classroom. Educational reform and improvement are high on the agenda of many ministries of education in Sub-Saharan Africa. The United Nations Convention on the Rights of the Child clearly states that chil- dren have a right to a quality education. Among the elements in the definition of quality education developed by UNICEF are: s Content that is reflected in relevant curricula and materials for the acquisi- tion of basic skills, especially in the areas of literacy, numeracy, and life skills, and knowledge in such areas as gender, health, nutrition, HIV/AIDS preven- tion, and peace s Processes through which trained teachers use child-centered teaching ap- proaches in well-managed classrooms and schools and employ skillful assessment to facilitate learning and reduce disparities s Outcomes that encompass knowledge, skills, and attitudes and are linked to national goals for education and positive participation in society (UNICEF 2000). The adoption of an IRI model should work toward meeting these goals for education and should, where possible, take a holistic and integrated approach that addresses the needs of the whole child, inside and outside the classroom (see box 4). All aspects of the learning system need to be taken into account: the learners, the learning environment, the curriculum, learning outcomes, teachers and their own development, technologies, and resources. 14 WHAT IS INTERACTIVE RADIO INSTRUCTION? Effectiveness Because of its innovative nature and the enthusiasm it can generate among students and teachers, IRI is often quite popular and develops a large and loyal following. Delivery of the program, however, is only an intermediate result along the critical path to improved learning outcomes and enhanced educational quality. What needs to be asked is whether, when compared with other learning systems, IRI produces the desired results consis- tently, and whether it helps reduce unevenness in the quality of teaching across education systems. How Does IRI Affect Achievement? After almost three decades of experience, the ability of IRI to improve the quality of education is established and well documented. Evaluations have yielded consistent and significant evidence that IRI can increase learning (Murphy et al. 2002). Changes in educational quality have been measured through controlled studies of learning gains, conducted by external evaluators, and through assessments of classroom activity in projects around the world. These studies have found increases in achievement across subject matter, student ages and genders, and rural or urban location of the project. Students in IRI classes, on average, outperform students in control groups with an effect size of 0.5 standard deviations (Tilson et al. 1991; Leigh and Cash 1999). This is a consistent and notable gain in learning. Figure 1 shows a cross-national sample of studies conducted in the early and mid- 1990s that looked at programs for various subjects and audiences in six countries. Although the figure does not include pre-test data, it does show the consistency of increased achieve- ment of students using IRI, compared with control students. Since the mid-1990s, numerous IRI initiatives have been launched in Africa and are now reporting results. In Guinea, where IRI is part of a larger USAID-funded education program that concentrates on reducing equity gaps, enrollment figures and external evaluation results indicate that improvements in educational quality are coupled with increased enrollment, attendance, and achievement for both boys and girls in urban and rural settings (Creative Associates International 2002). Enrollment has also increased considerably among the school-age population, particularly for girls. As of 2003, ap- proximately 65 percent of Guinea's school-age children begin first grade and approximately 44 percent of school-age girls enroll, whereas in 1995 only about 30 percent of all school-age children started first grade and a mere 22 percent of all school-age girls ever 15 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION enrolled (International Edu- Figure 1 cation Systems, ies.edc.org/ Post-Test Data on Achievement projects/ guineaFQEL.htm, of Students in Selected IRI Projects December 2003). To be sure, IRI is one part of a 90 much bigger set of educa- 80 tional interventions, and 70 these results cannot be solely 60 attributed to the IRI inter- vention, but according to score 50 estT Ministry of Education 40 evaluators, the introduction 30 of IRI has led to a funda- 20 mental transformation of 10 the culture of education that 0 Bolivia Nicaragua Thailand-- Papua New South Africa Honduras has been instrumental in (math) (math) northeast Guinea (English) (adult education, (math) (science) Spanish; females) these educational changes. size 0.94 0.58 0.24 0.36 0.72 0.54 To assess Nigeria's grade Effect Control IRI 3­5 literacy and mathemat- ics IRI program, evaluators Sources: Tilson et al. (1991); Corrales (1995); Leigh (1995). compared student perfor- mance in control and IRI classrooms. For the first year Box 4. A South African (school year 2002­03), a significant Teacher on IRI positive influence of IRI on student achievement was found in the third grade but not in the fourth. Evalu- "My learners are able to do projects work- ators cautioned that since the IRI ing as teams. The program encourages them to enquire about their environment, and as effects are cumulative, conclusions a result, grade 3 learners conduct their own regarding the effectiveness of the research on various topics--for example: ca- program beyond the first year of ex- reers, languages spoken within their posure should await the results of communities, sharing information both in- side and outside the classroom, bringing into another year of implementation the classroom interesting books, materials, (Royer 2003). and articles. The lessons are well integrated, In a Zambian IRI program de- addressing language across the curriculum." signed for out-of-school children of --Veliswa Noarana, grade 3 teacher, primary school age, 28.8 percent of Letsatsing Primary School whom are orphans, effectiveness has 16 WHAT IS INTERACTIVE RADIO INSTRUCTION? been studied in terms of access, learning gains, and equity gaps. Two evaluations (midpilot and summative) of the pilot sample were conducted in 2000, and a summative evalua- tion was again conducted in 2001. In 2002, the Examination Council of Zambia (ECZ) evaluated the program, and an external evaluator from the University of Zambia con- ducted an appraisal of the IRI system (Zambia 2004). The ECZ evaluation compared the out-of-school IRI learners with a corresponding sample of learners in government primary schools. The grade 4 evaluation was particu- larly important because it became part of the formal countrywide competency testing at the end of the foundation phase of primary schooling. In both grades 2 and 4, IRI learners were found to be performing as well as students in formal schools, if not better. For instance, the results of the grade 4 evaluation revealed significant differences in mean scores in mathematics between IRI learners and those in formal government schools. This finding is particularly significant because, unlike the students in the regular schools, the IRI students represented an otherwise out-of-school, vulnerable population, with- out trained teachers, textbooks, or classrooms, who received only one to two hours of IRI per day. Table 1 presents the grade 4 mathematics results as reported by the ECZ for IRI and regular government schools. Although these findings should not be taken as suggesting that the Ministry of Edu- cation should not open new formal schools, they do provide important data on the Table 1 Mathematics Scores in IRI Centers and Public Schools, Zambia, 2002 Subject matter IRI Public schools Notation 83.0 73.0 Numeracy 50.0 43.0 Addition and subtraction 67.4 67.2 Multiplication 52.0 44.4 Division 57.0 46.0 Number patterns 55.0 46.0 Decimal measures 37.0 34.0 Shapes and measurements 39.0 18.0 Fractions 59.0 38.0 Total mean score 57.0 46.6 17 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION effectiveness of IRI in producing learning gains even in the most difficult situations, where there are no formal schools or trained teachers, and in doing so efficiently. Can IRI Reduce Equity Gaps? A variety of studies have been conducted on IRI programs around the world to deter- mine their impact on reducing equity gaps. Figure 2 shows studies of learning gains conducted in Bolivia, South Africa, and Thailand that compare levels of achievement for rural and urban students. The gap between rural and urban student achievement decreases with exposure to IRI. These results, which follow a pattern that has been observed in many other countries, indicate that IRI programs are not only increasing quality, as reflected in gains in achievement, but are also having an effect on urban-rural equity gaps. The Guinea Fundamental Quality and Equity Levels (FQEL) educational program found similar results (see "Guinea," appendix B). Despite their remote and less advantaged situation, students in rural Guinean classrooms that had access to the FQEL program's IRI and teacher-training materials scored as high as or almost as high as their urban counterparts on the French language and math achievement tests (Creative Associ- ates International 2002). Figure 2 A retrospective analysis of several mid- The Effect of IRI Programs on 1990s IRI programs showed that gender Urban-Rural Equity Gaps gaps were reduced. In fact, since the girls' starting place, or baseline achievement, 80 was much lower than the boys' and their 70 post-test achievement was approximately the same as for the boys, the overall gain 60 for girls was significantly greater 50 (Hartenberger and Bosch 1996). This find- score estT 40 ing was demonstrated for upper primary 30 school science in Papua New Guinea, 20 lower primary school English in South Africa, and adult basic education in Hon- 10 duras, indicating that neither the age of 0 Bolivia-- Bolivia-- Thailand-- Thailand-- South South the learner nor the subject taught was an urban rural central northeast Africa--urban Africa--rural obstacle to girls' achievement. Similar find- Control IRI ings were reported in later studies in Source: Bosch (1997). Guinea, suggesting that the earlier cases 18 WHAT IS INTERACTIVE RADIO INSTRUCTION? were not isolated or ran- dom (Creative Associates International 2002). Other combined tests of achievement and equity, such as those on the Zam- bia IRI program that serves out-of-school school-age children, indicate that IRI has been effective in improving quality and re- ducing equity gaps in particularly difficult envi- IRI programs have enabled gains ronments. Evaluations of in achievement and reduced the Dominican Republic's urban-rural equity gaps. RADECO project found that achievement among out-of-school children receiving, per week, just 5 hours of integrated IRI and 30 min- utes of follow-up activities was comparable to that of students who were in regular formal schools for more than twice the time. First-graders using the RADECO pro- grams gave 51 percent correct responses on post-tests, compared with only 24 percent for the control group in regular schools. In spite of the obstacles faced by these out-of- school children, their learning was comparable to the control group's in reading, writing, and language, and they performed significantly better in math (Goldstein and de De Jesús 1995). The success of the RADECO IRI programs in the Dominican Republic and the Taonga Market IRI programs in Zambia in delivering education comparable to that of formal schools has reduced equity gaps in places without schools in both Latin America and Africa. What Is the Effect of IRI over Time? In a few studies that have analyzed the effectiveness of IRI as measured by learning gains over a series of lessons or grade levels, researchers have found that student achieve- ment has increased progressively over time. In South Africa, for example, students who received fewer than 33 English in Action lessons improved by 6.7 percent; those who received between 34 and 66 lessons improved by 13 percent; and those who received more than 66 programs improved by 24 percent (Leigh 1995). Similar results were found in Bolivia; evaluators found that between 1988 and 1991 the average score of 19 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION A well-developed IRI program holds a child's attention and elicits numerous student responses in a 30-minute lesson, making IRI far more learning-intensive than the standard classroom lesson. second-graders using IRI Math jumped from 47.0 to 66.23 percent. (The mean score of the control group was 35 percent.) Those experimental students who had already com- pleted one year of the radio lessons did much better than the control group (51.9 percent correct), and those who had completed two years scored 61.6 percent (Tilson et al. 1991). The cumulative effects of IRI exposure on learning gains and equity gaps have not been studied across many projects at this point, however, and more studies of efforts in Africa are needed. Going to Scale Due to the structure of IRI development costs, IRI's comparative advantage in relation to other educational interventions is greatest when the number of beneficiaries is larg- est. While the number of students may differ according to the particular educational 20 WHAT IS INTERACTIVE RADIO INSTRUCTION? needs or activities, the ability to scale up is critical. Because of the broadcast capacity of IRI programs, learning systems can reach a large population if administrative systems and other essential elements are in place. In several countries, including Bolivia, Guinea, Papua New Guinea, South Africa, and Venezuela, IRI programs have expanded to a million or more students over time. Many other programs have reached hundreds of thousands of students. Although IRI is still in pilot stages in many of its applications in Africa, at least three countries--Guinea, Lesotho, and South Africa--have taken an IRI project to scale. A 1999 World Bank­USAID study highlighted several issues related to the use of IRI in projects that had gone to scale and had encountered some of the associated challenges (Dock 1999): s Any large-scale program with multiple interacting components (in the case of IRI, lesson and script production, broadcast transmission, radio purchase and maintenance, teacher training, and system management) requires confi- dent, knowledgeable leadership and management. Even where these are present, they may be transitory. s Large-scale IRI programs demand wide political support because they are competing with other uses of public funds. Such support takes time to de- velop and may erode when there is a change of government. s It is important that appropriate policy frameworks be established to facili- tate the integration of IRI into the government's education plan and to ensure effective cooperation between broadcasting authorities, ministries, and other agencies. s Teachers' interest and momentum must be maintained. Unless programs are revised from time to time and teachers' enthusiasm is reinvigorated, the teachers can lose interest and may not be diligent about tuning in to broadcasts. New teachers must be trained and supported in the use of radio lessons. Expecta- tions that programs will never be revised or that familiarizing teachers with the methodology is a one-time event are unrealistic and can reduce the chances of long-term sustainability. s The diverse components of radio education must be well integrated into the administrative, budgetary, and professional structures of educational and broadcasting institutions. Changes in these structures (for example, as a re- sult of decentralization of education systems) may jeopardize the continuity of the program. s Successful national-level IRI programs can generate relatively large audiences of enthusiastic young listeners. Although this will likely be seen as a credit to 21 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION IRI captures its audience through the interest it generates rather than through obligation. the education system, there is a risk of uneven radio reception and coverage or of interruptions in broadcasts. Where not all listeners have effective ac- cess to good-quality broadcasts on a regular basis, the use of audiocassettes or CDs should be considered. Cost The cost structure of an IRI program differs significantly from that of other learning systems and initiatives for improving educational quality. It is important to understand the relationship between the costs of initial program development investments and re- current costs. Furthermore, the costs of an IRI program have to be looked at in light of 22 WHAT IS INTERACTIVE RADIO INSTRUCTION? how IRI is being used. In most applications, an IRI program is developed as an improve- ment of the regular school curriculum, and IRI costs are to be viewed as add-ons to the regular costs of instruction at a particular level. The use of IRI does not produce a savings in the education budget, but the additional cost is justified by the benefits IRI produces or because IRI is cheaper than other ways of improving quality. In many coun- tries, the costs of using IRI are roughly equivalent to the cost of a textbook. (Actual IRI costs and how they are distributed are discussed in chapter 2.) In other cases, IRI is developed to provide a core of instruction in what may be termed an alternative delivery system for education, where providing conventional instruction (usually through schools) is not practicable for certain areas or populations. Examples of such IRI programs include RADECO in the Dominican Republic, the EDUCATODOS basic education program and PREPI preschool program in Honduras, and the IRI learn- ing centers in Zambia and Tanzania. In applications such as these, it is necessary to include the costs of the other elements of the system, not just the radio programs. It is common for alternative delivery programs to rely on community volunteers as instruc- tors and to operate in facilities made available by the community, resulting in lower costs than would be incurred by expansion of conventional education. In Honduras, the use of alternative delivery systems increases enrollment at about half the usual cost. IRI alterna- tive programs do not usually cover the entire school curriculum or a full school day, and the issue of their equivalence to conventional school programs is something that must be assessed. How Are Costs Distributed? Two major incentives for investment in IRI in many countries are the cost-effectiveness of using radio and the willingness of donors and multilateral development institutions to absorb up-front development costs. The following summarizes key cost issues relat- ing to implementation of IRI: s Distribution of costs. IRI projects are front-loaded; that is, the initial fixed costs for creating management and training systems, producing audio pro- grams and associated print materials, and providing technical assistance are relatively high. The recurrent costs associated with salaries, dissemination, training, and maintenance are far lower. In some countries, adaptation of programs developed elsewhere can significantly reduce the investment costs. Pilot projects and startup phases of IRI programs have been financed by external funding agencies, primarily USAID and, more recently, the Norwe- gian Agency for Development Cooperation (NORAD), UNICEF, the 23 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Inter-American Development Bank, the World Bank, the Asian Development Bank, the U.S. Department of Labor, and the U.K. Department for Interna- tional Development (DfID). Even if external investment covers initial costs, it is important that countries determine whether they can meet the recurrent costs of program delivery and maintenance. s Cost per student. IRI is designed primarily to improve the quality of conven- tional education programs. Efforts to improve quality usually increase the per-student costs of providing education, even if, as discussed in chapter 2, the additional costs are relatively low. s Scale. Regardless of the initial investment, the per-student cost of IRI goes down as more students are reached. Most other interventions require pro- portionately higher numbers of new school facilities, textbooks, or teachers as learners are added. In an IRI project, these extra factors do not dramati- cally influence the cost of the program, and because the primary products--radio programs--are broadcast, the cost per learner decreases pro- portionally as the number of users increases. The secondary beneficiaries who participate in the program also have to be factored in. s Recurrent costs. Teacher orientation and other recurrent costs of IRI projects stay relatively constant over time after the development stage. The amounts might depend on how much training is integrated into the program, the sub- ject being taught, and the special circumstances of the country. Other recurrent costs include airtime, simple supplementary print materials such as one-page worksheets inserted into local newspapers or distributed at the beginning of the year, batteries and radios, and the maintenance of a management system or unit focused on IRI. What Is Known about Cost-Effectiveness? Few comparisons of the effects of various educational interventions on quality have been undertaken, but those that have been carried out provide some evidence of the cost-effectiveness of IRI compared with alternative investments in improving the qual- ity of primary school instruction, such as providing additional textbooks or teacher training. Lockheed and Hanushek (1988) showed that IRI was the most efficient invest- ment of those evaluated. In cost analyses conducted in South Africa, the evidence suggests that IRI is proving cheaper and more effective than alternative English-as-a-second- language programs. Cobbe (1995) found that the cost of South Africa's English in Action program ranged from one-third to one-half that of other tools. Adkins (1999: 43) com- pared the cost-effectiveness of IRI programs in math and language with investments in 24 WHAT IS INTERACTIVE RADIO INSTRUCTION? The use of wind-up radios reduces the recurrent cost burden, especially where there is no electricity grid. textbooks and teacher training and determined that IRI was the most cost-effective. This is not to suggest that IRI should be used instead of other inputs but, rather, that the costs of educational tools and teacher training should be weighed carefully and that the addition of, for example, textbooks cannot be assumed to be always the best approach for improving the quality of teaching and learning. For teacher-training programs, IRI may be a good substitute for some cost-heavy in-service training activities, particularly for reaching teachers in the most remote schools. Alternatives to IRI now include high-technology tools such as computers in schools. A recent World Bank analysis (see chapter 2) shows that computers are an expensive choice for primary schools and that radio and IRI are much more affordable for poorer countries (Bakia 2000). Have Cost-Sharing Strategies Been Tried? Governments have experimented with various cost-sharing and income generation schemes to pay for the recurrent costs of IRI projects. These strategies have been 25 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION specific to country circumstances. Three interesting examples are those of Lesotho, where a tax covered a portion of the costs; Honduras, which is trying out cost-sharing strategies involving the private and public sectors and nongovernmental organiza- tions (NGOs) on the municipal level; and Bolivia, where the early childhood development series is experimenting with decentralized methods of sustaining IRI programs through local municipalities. Most IRI projects are at least partially supported through part- nerships between ministries of education and ministries of communication and broadcasting. Sustainability Experience over 27 years of experimentation with different designs and applications of IRI suggest that in many cases IRI programs have shown the ability to endure long after pilot projects and the initial development stages have been completed. In Africa, Asia, and Latin America, of 35 significant IRI programs that have been designed over the past three decades, 25, or roughly two-thirds, are currently used as part of a for- mal or nonformal learning system, with tens of thousands of students, or more, listening each year. In 6 cases of the original 35, where the projects were smaller or their devel- opment took an unusual course, few details are known about how the projects have evolved. Of the remaining 29 projects, 9 (in 7 countries) were sustained in their origi- nal form for at least a decade, which minimizes the per-student investment (see table 2). In 9 programs, over 500,000 students have benefited, and in 6, more than a million students have participated. Nine of the 35 IRI programs are still in the development or pilot stage. While these figures suggest that sustainability, measured in reach and in longevity, is possible, not all countries have had equal success. Some pilot projects have encountered insurmountable obstacles due to recurrent costs, poor institutionalization, insufficient local ownership, or changes in policies or political leadership. Even with repeated dem- onstrations of learning gains and narrowed equity gaps, and with significant investments during development stages, some projects have not lasted for more than a few years after external funds were phased out. This inconsistency presents a conundrum to re- searchers and planners hoping to use IRI to improve educational conditions in developing countries: why do some efforts last while others do not? 26 WHAT IS INTERACTIVE RADIO INSTRUCTION? Table 2 Status of 29 IRI Programs Pilot phase Adapted or Programs delivered (delivered in delivered in in original form original form) new form Abandoned 1. South Africa 1. Haiti math/ 1. Honduras mental 1. Nicaragua math English literacy math (revised 2. Kenya English 2. Honduras adult 2. Zambia English/ and rebroadcast 3. Costa Rica envi- education math after 10-year hia- ronmental 3. Dominican Rep. 3. Ethiopia English tus) education integrated 4. Nigeria literacy/ 2. Thai math (cas- 4. Guatemala math 4. Lesotho English math settes used in 5. El Salvador math 5. Nepal ECD 5. Somali refugees hill-tribe area) 6. Venezuela math 6. Bolivia ECD in Ethiopia math/ 3. Bolivia math 7. Guinea French/ literacy (broadcast locally numeracy 6. Tanzania on smaller scale) 8. Nepal dual integrated 4. Bolivia health audience 7. Honduras (broadcast lo- (teachers and preschool cally) students) 8. India science/ 5. Papua New 9. Honduras jr. math multigrade Guinea science secondary 9. India English (broadcast lo- cally) Note: The 29 projects listed are those for which information about the current use of the IRI programs was available at the time of writing. ECD, early childhood development. Where Have IRI Efforts Been Sustained? Over the past 27 years, IRI projects have endured in different ways and for different lengths of time. In some cases, IRI programs are still broadcast or used on cassette in much the same way as originally intended, while the number of beneficiaries continues to increase. The series and approaches may have been institutionalized on a national level or integrated into a training and implementation system that has not greatly changed and that can sustain the original delivery system. In these instances, the structures and benefits related to the original IRI program still exist. In other cases, series that were originally to be provided nationally may now be delivered at a regional, municipal, district, or school level due to decentralization, privatization of communication sys- tems, or changes in educational trends that forced changes in the delivery system or in the scheme for covering recurrent costs. In a few cases in Bolivia, private or religious 27 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION organizations picked up the responsibility for delivering programs when government decentralization and reforms put an end to centralized broadcasts. Still other programs were abandoned prematurely due to poor planning or unforeseen circumstances. In these cases, core elements of the IRI program remain, but parts of the original structures have changed. Some of the early IRI efforts were not sustained at a substantial level for more than a few years. These projects concentrated much of their time on research to improve the quality of education through active learning and then encountered obstacles--in Nica- ragua, revolution, and, in Kenya, the government's unpreparedness to cover recurrent costs. Issues related to sustainability were either out of the project's control or were not well understood at the time. The IRI program in the Dominican Republic's Barahona region has lasted two de- cades, and, after 23 years, community demand remains strong (Miranda 2002; Helwig, personal interview 2004). Four international IRI projects--Papua New Guinea science, Honduras mental math, Bolivia math, and Lesotho English--broke new ground by in- troducing inquiry in learning. They were sustained for long periods of time, and they reached from 500,000 to a million students. As these IRI projects began to encounter difficulties related to sustainability, the 1987 Honduras mental math project experi- mented with a public-private collaboration scheme to distribute costs. This effort was probably ahead of its time in Honduras and did not succeed, but it was revitalized in 1999, and the program is now broadcast across the country through a network of com- munity radio stations. The 1987 IRI English project in Lesotho introduced a tax to pay for recurrent costs associated with the programs, and this system continues to sustain the project today. These projects yield valuable lessons about planning for sustainability. Based on early successes, nine pilot projects using IRI were initiated during 1990­ 92. Five were in Latin America, and seven of the nine were funded by USAID. Three were relatively small efforts that received very little financial support. Six were used effectively for a while but were not sustained. Two--Honduras adult basic education and South Africa English in Action--became large, sustainable projects; the others re- main dormant. This period may be have been one in which enthusiasm overshadowed the investment of time, expertise, and funds needed for the development and careful planning of an IRI initiative. It also came just before a period when donors were de- creasing funds for certain areas and governments were tightening their belts. It is not known exactly why so many of the projects that were developed at this time were not sustained; this was an unusual period in IRI's short history. Over the next 10 years, 17 IRI projects were developed; 13 of these are currently being used in their original form, and 3 have been abandoned. The projects differ in size, audience, subject matter, and region, and there is no particular pattern to their 28 WHAT IS INTERACTIVE RADIO INSTRUCTION? development or their demise. Over this period, IRI expanded geographically. During the first two decades, most IRI efforts were concentrated in Latin America, where there was a long history of receptivity to radio education. More recently, increasing numbers of projects have been implemented in settings in Africa where the overall economic situa- tion has not improved over the past few decades and per-student education budgets and teacher-training resources are limited. In these circumstances, IRI offers a viable means of increasing educational quality. Six of the 10 African IRI initiatives have been launched within the past seven years. Six have been introduced in environments characterized by multiple mother tongues, hard-to-reach populations, and low literacy levels. These en- vironments present a variety of demands and challenges for the design of IRI programs and for plans for sustainability (see Bosch 2004). What Are the Critical Factors for Success? Vast differences exist in how IRI programs have been designed and in the political, social, and economic environments in which they hope to be sustained. Programs have been implemented in centralized and decentralized education systems, in formal schools and in out-of-school learning environments, and for a variety of subjects, audiences, and purposes. Some IRI programs have been institutionalized and sustained; others have been abandoned. Given this disparity of experiences and context, what can be said about the critical factors for success? IRI projects are subject to many of the same risks and opportunities that educa- tional initiatives in developing countries generally face. New projects are often not institutionalized sufficiently. They are affected by changes in administration and policy, and they are not able to cover the costs and responsibilities of continued operation. Creating a successful IRI program and sustaining its benefits is a complicated and dynamic business. To capture the varied experiences of IRI programs and understand which factors are most important for success, the authors conducted a survey to gather and consolidate some of the unique experiences and perspectives of people who have been involved in substantial IRI projects in countries around the world. This survey of IRI implementa- tion specialists and donors found that respondents saw quality control of IRI programs at the design, production, and broadcast stages as a crucial success factor, second only to the strength and quality of technical assistance. Choice of subject matter, uniformity of language of instruction, and the local history of radio education were considered less critical for success. Essentially, a quality IRI program will generate a following or con- stituency that expects the programs to continue. Improvements in learning outcomes provide the justification for continuing a program. Another key characteristic of sus- 29 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION tainable IRI programs is their responsiveness, in both the initial and later phases, to needs emerging from the local context. Other important elements include political will, identification and availability of local champions, relevance to education policy, and donor support. An important part of making IRI projects sustainable involves ensuring that IRI is introduced into an environment that is responsive to change and that there is a plan for shared ownership and shared costs. Enabling policies, strong and active community participation, and local champions are elements in such an environment. These factors have been present to varying degrees, depending on the positioning of the IRI system, but none by itself determines the success of the IRI program. Pilot programs in unwelcoming environments may actually lead to larger, sustainable programs. For ex- ample, in Haiti, IRI programs for literacy and mathematics were purposefully introduced in private rather than public schools due to the turbulent political situation. The use of IRI has spread considerably among these schools, and there is a large shadow audience in other schools. Constituencies within the religious communities and other local insti- tutions have helped sustain the programs. Where programs are large and successful, continued attention has to be paid to keep- ing them dynamic and lively. Leadership and advocates can help learning and delivery systems adapt to changing social, economic, and political conditions. 30 Chapter 2 Feasibility of IRI Main Points s Educational goals. Most IRI programs are intended to improve instruc- tion in primary schools, and their focus is on student achievement. IRI programs actively involve teachers and help standardize instructional qual- ity. They entail annual evaluations of student progress. s Audience. Successful IRI programs identify audience needs, abilities, and interests through research and formative evaluation--that is, early testing of materials as they are used. One-size-fits-all distance education such as IRI has generally worked best with younger audiences. Informal learning on the part of unintended shadow audiences such as parents and out-of- school youths can add to the returns on IRI investments. s Content. IRI has proved versatile in teaching most academic primary school subjects; it has not been as successful with subjects requiring a strong visual demonstration component or face-to-face guidance. IRI could, how- ever, assist in hands-on subjects such as science and life skills, including health practices. IRI produces the best results when designed as a single subject taught in a daily, year-long program, and this is its most common form. Although most IRI programs have been developed for the early primary school years, the method can work for later primary school grades. Adaptation of existing IRI programs for other audiences can simplify de- velopment and save money and time, but a number of constraints have to be overcome. No research has determined how much of the school cur- riculum can be handled by IRI or for how long a period students can be effectively engaged. 31 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION s Program development. Specific personnel should handle program de- sign and management, curriculum design, scriptwriting and support materials, radio production, research and evaluation, and marketing. Successful IRI programs are led by an institution with authority, techni- cal competence, and a commitment to nurturing effective alliances with stakeholders. Planners must decide on technical assistance and capacity building, the most cost-effective approach to delivery of the IRI pro- gram, and the scope and duration of pilot projects. Adequate training should be provided early in the process to ensure quality in design, pro- duction, and delivery. s Cost and finance. Because of economies of scale, IRI offers a relatively low-cost means of improving educational quality in primary schools. It is important, however, to plan for and contain recurrent costs--of broadcasting, radios and batteries, print materials, and teacher training and support--and to be realistic about the long-term implications of these costs for governmental and nongovernmental funding. IRI pro- grams cover recurrent costs through such means as taxes, funds already budgeted, cost-sharing, fees for service, and marketing of products such as calendars. s Planning for institutionalization and sustainability. Plans to integrate IRI into existing institutions vary according to the degree of decentralization in the ministries of education and communication and in other agencies involved in IRI broadcasts. Planners should consider a role for govern- ment agencies, colleges of education and other tertiary institutions, and NGOs with an interest or institutional investment in an IRI program. Planning for the sustainability of the program should be considered at the outset of the project. Sustainability depends on the educational ben- efits documented, the affordability of recurrent costs, successful institutionalization, and strong leadership in adapting the program and in troubleshooting. Creating appropriate policy frameworks is essential to ensuring sustainability, as are cooperation among relevant agencies and the annual commitment of budgetary funds. 32 FEASIBILITY OF IRI T he effectiveness of IRI depends on a fit between available resources, the policy environment, specific educational needs and goals, and the advantages that IRI has to offer. Once key decision-makers understand the general characteristics of IRI and its pros and cons, it is important to determine the feasibility of using IRI as an educational tool in a particular context. The feasibility of an IRI intervention depends on a good match between: s A country's specific conditions, needs, plans, and resources, and s The requirements for developing and implementing IRI programs, as sug- gested by experience from other countries. This chapter, building on the general description of IRI in chapter 1, focuses on the main concerns in determining the potential of IRI to improve the quality of primary school education in a specific location. The discussion will assist planners in sketching a rough plan for an IRI intervention. Such a blueprint can provide a basis for dialogue with stakeholders on whether resources should be committed to designing the interven- tion. The blueprint can take the form of an outline, a concept paper, or a more in-depth analysis of some of the key issues in the particular context. Typically, one or two quali- fied education specialists would produce such a document, with significant input from an IRI specialist. Each of the sections that follow begins with a checklist of conditions that should exist or must be achieved in order to develop a successful IRI program in a specific location. Educational Goals Checklist: Educational Goals ® Clearly stated goals, with an explicit commitment to improving educa- tional quality in regular primary school classrooms ® Primary focus on improving student achievement, with a secondary fo- cus on other indicators of educational quality ® Assessment of stakeholder receptivity to a lead role for IRI and to pro- viding training and support for teachers ® Willingness to conduct annual evaluations of student progress 33 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION The feasibility of an IRI program depends in part on how well the program addresses the specific goals of national educational reforms or educational development plans. The checklist criteria that should be considered are explained in detail below. ® Clearly stated goals, with an explicit commitment to improving educational quality in regular primary school classrooms The use of radio--a low-cost medium of mass communication--as a tool to promote quality, rather than access, may seem counterintuitive to many people. Most IRI pro- grams, however, have been targeted to improve the quality of instruction in primary school subjects and have been undertaken within the context of conventional primary school classrooms. Even in the two cases in which IRI has been used to promote access to learning opportunities for students unable to attend regular schools (Zambia and the Dominican Republic), quality has been a fundamental issue because it is crucial to the legitimacy of the program. This is not to say that IRI would not be an effective means of promoting access to primary school education where it is not possible for children to attend school physi- cally. There is less empirical evidence on the effectiveness of IRI in enhancing access than on its impact on quality in the classroom, but given the anecdotal reports of the success of IRI programs among shadow audiences, there would seem to be potential for providing services to out-of-school children and children with physical disabilities that keep them from getting to school. ® Primary focus on improving student achievement, with a secondary focus on other indicators of educational quality The principal focus of nearly all IRI projects undertaken at the primary school level has been to improve student achievement. Few attempts have been made to examine how IRI might affect educational quality indicators such as repetition, persistence, dropout, attendance, and attitude toward school subjects. Although there is anecdotal evidence that IRI use has, for example, helped improve student attendance, the costs and time frames for evaluating IRI and other education programs have made studies of the effects of IRI on outcomes other than achievement difficult. There is no reason to believe that the use of IRI cannot bring about other outcomes related to educational quality. If, however, planners are considering targeting such out- comes, they should use an experimental approach, supported by the necessary monitoring and evaluation tools. 34 FEASIBILITY OF IRI ® Assessment of stakeholder receptivity to a lead role for IRI and to providing training and support for teachers In determining the feasibility of IRI, an assessment needs to be made of how receptive stakeholders in the education system are to giving IRI a significant role in organizing and delivering instruction and to implementing training and support for the teachers in an IRI system. IRI's direct role. One attraction of IRI is that it is a form of direct instruction. IRI applications go directly to the heart of the teaching/learning process. The use of prere- corded lessons and broadcasting means that students in participating schools receive more or less the same lessons. Consequently, IRI provides a degree of quality standard- ization that is not seen with other educational interventions. An advantage of intervening directly in the teaching/learning process is the potential for a rapid effect on student learning. Other kinds of interventions for educational qual- ity, such as curriculum development projects and teacher training, typically require longer periods of time to demonstrate an effect. An example is the basic education project ED 2004 in Haiti, which included an IRI component. IRI use began to show an impact on student achievement in mathematics and language arts from the start. Schools in the project that did not receive IRI but did receive teacher training and other educational services showed no improvement in achievement when assessed at the end of a three- year period. New instructional role for teachers. Teachers in IRI classrooms are partners with the radio program. Their role in a lesson may include taking part in whole-group activities that are part of the radio broadcast, working with individual students during the broad- cast, and leading follow-up activities afterward. Training, of varying duration, is usually provided to teachers so that they can become familiar with their roles in IRI. Such training often focuses not only on mechanics and administrative aspects but also, and more importantly, on the new role of the teacher as a facilitator of learning in a learner- centered environment, as distinct from the traditional teacher-centered approach. As part of the formative evaluation process, teachers are often asked to provide feedback to program developers on how to improve IRI delivery and impact. Some recent applications of IRI in Guinea, Nepal, and South Africa have attempted to go beyond programs of direct instruction by incorporating explicit teacher-training objectives into IRI lessons. This kind of programming, which may, for example, model the use of various teaching techniques, is likely to become more common in the future. The effectiveness of a teacher-training component within an IRI program has not so far 35 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION been evaluated, but evidence from programs that have used IRI to support early child- hood development does indicate a positive impact for caregiver training so provided. Evaluations of teacher impact and performance show that the IRI programs led to an increase in caregivers' ability to engage children in educational activities (Colletta 1996; Bates 1998). ® Willingness to conduct annual evaluations of student progress Most IRI applications are intended to improve student performance in subjects nor- mally taught during the school day. Consequently, evaluations of IRI typically measure success by achievement as demonstrated by paper-and-pencil tests. The most frequent use of IRI to date has been for the early primary grades, but few countries have stan- dardized tests for measuring achievement at these levels. IRI program evaluations have therefore found it necessary to develop their own measures and testing schemes. This has often been accomplished through an arrangement with a local or external university or specialized body for developing and administering tests. The tests usually assess mastery of the content specified in the prescribed curriculum for a particular grade level. Evalu- ation designs have generally measured learning gains achieved during the course of a year's instruction and have included comparisons of students using IRI with those in control groups. Most evaluations have been driven by the schedules and funding mecha- nism requirements of donor support for IRI. Annual assessments of the impact of IRI on achievement, even if they have been limited to the time frames of donor-assisted projects, have provided important evidence to both stakeholders and program designers that IRI is worthy of continuing investment. Audiences Checklist: Audience ® Clear specification of a primary audience ® Empirical investigation of the proposed audience's needs, interests, and abilities ® Identification of potential shadow audiences 36 FEASIBILITY OF IRI One of the interesting challenges in IRI program development is the cultural and ethnic diversity of a nationwide audience. Because IRI most often reaches its audience through broadcasts, the feedback loop has a very different dynamic from that of standard classroom teaching. The content reaches a broad audience with differing levels of interest and comprehension, and the challenge is to maintain listener interest and involvement in the program. It is useful to look at what IRI understands its audience to be and what is done to monitor reaction to the program and evolve in response. An important step in determining the feasibility of an IRI program is careful delinea- tion of the audience for the program. ® Clear specification of a primary audience Clear specification of the primary target audience is important for determining how an IRI application will be designed and whether it will be able to achieve the desired goals. If the audience includes out-of-school youths or adults, program feasibility may de- 37 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION pend on a combination of IRI development activities and community participation activities. ® Empirical investigation of the proposed audience's needs, interests, and abilities Too often, little attention is paid to whether learners have the necessary background knowledge, comprehension skills, or language abilities to understand and learn from the materials. Communicators are more likely than educators to treat the characteristics of their audiences as something to be discovered rather than as a given. Successful IRI programs assess an audience's needs, abilities, and interests through two means of in- vestigation: audience research, and formative evaluation of programs. 1. Audience research. Confidence in the feasibility of an IRI program is greatly en- hanced through a simple preliminary investigation of the audience. Such a study looks at the intended audience by asking (a) "who they are" (demographic/family background, interests, and so on); (b) "where they are" with respect to the pre- scribed curriculum and prior knowledge of the subject; and (c) what their language capabilities in the language of instruction are, including their vocabulary. Some countries that have embarked on IRI have commissioned audience research studies by profes- sional research organizations, but this work is more commonly done by a program or project development team. The latter option has the advantage of giving the team greater firsthand knowledge of the audience and its knowledge of the subject mat- ter--which can be a revelation. Many project teams are made up of educators who have an idealized sense of learners' experiences and interests and who tend to treat them as adults rather than children. Some have an idealized sense of the subject matter based on where the students "should be" in the official curriculum rather than where they actually are. And some have an idealized sense of the language that teachers and students use or are able to understand. Planners should regard the char- acteristics of the audience as relevant to the feasibility of the program and ensure that the audience profile is based on empirical evidence, rather than accept it as something that is already known or that can wait to be ascertained until well into the design stage. (See chapter 3 for a further discussion.) 2. Formative evaluation. Although the role of formative evaluation (see box 1, "Key IRI Terms") in an IRI program is treated as an aspect of program design, early test- ing of existing or prototype materials can provide a great deal of information on the intended audience and the potential of IRI to address the educational goals under 38 FEASIBILITY OF IRI consideration. Moreover, anticipat- Box 5. Shadow ing an organization's willingness Audiences in and capacity to make use of for- Guinea's IRI Program mative evaluation in an IRI program gives an indication of the program's potential for success. Some of the shadow audiences identified Inability or unwillingness to com- to date are field workers and taxi drivers who want to learn French, out-of-school mit the staff, funds, or transport children who come to classroom windows necessary to conduct ongoing for- to listen to broadcasts, and refugee children mative evaluation of IRI programs from neighboring Sierra Leone and Liberia is a constraint that planners must (where English is the official language) who tune in to learn French. deal with in project design, as dis- cussed in chapter 3. Although the primary audience may be defined as students at a particular level, it is important to remember that IRI also usually strives to improve the quality of teaching by helping teachers acquire better teaching techniques. Teachers must therefore be viewed as an audience to be taken into account in all phases of program design. ® Identification of potential shadow audiences Most IRI programs reach a wider audience than the intended one. The shadow audience of an IRI program includes parents, out-of-school children and youths, and adults in the community (see box 5). It is difficult to plan for, identify, and enumerate the shadow audiences for an IRI program, but in many countries the additional educational returns to investments in IRI through informal learning by such audiences are said to be an important by-product of the program. Content Given the relatively limited time available for introducing content during a radio broad- cast, its choice can be a challenge. Program developers often find that the amount of content originally planned is too ambitious and that if the lessons cover less content and focus on better mastery, IRI can become a catalyst for encouraging listeners to explore and learn at other times. 39 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Checklist: Content ® Match between the subjects designated by a country for quality im- provement and the subjects that IRI can teach effectively ® Commitment to comprehensive grade-level curriculum coverage ® Realistic limitation of the number of school subjects and grade levels to be covered by IRI ® Clarity as to whether IRI subject content will follow the existing official curriculum or will develop a new curriculum ® Consideration of the cost and practical requirements of IRI programs in local languages ® Willingness of national decision-makers to consider adapting IRI les- sons developed in another country ® Evidence of potential to overcome major obstacles to successful adap- tation of IRI programs What Subjects and Grades Can Be Taught with IRI? IRI has proved to be a versatile method for teaching most of the subjects included in the primary school curriculum. These include mathematics, language arts in the mother tongue, a second language, science, civics, environmental studies, and health. The sub- jects vary from country to country, but in most cases the programs are broadcast through the course of a year, following the pace of the regular curriculum. The skills related to mastering the subject matter are introduced, distributed, and reinforced over time. ® Match between the subjects designated by a country for quality improvement and the subjects that IRI can teach effectively Although IRI seems best suited to the teaching of academic subjects, IRI programs need not have, and should not have, a narrowly academic character. IRI offers scope for participatory activity and active learning methods. Programs have included, for ex- ample, hands-on scientific experiments, project-based learning, and environmental field trips. Some subject matter, however, may not lend itself to radio instruction. 40 FEASIBILITY OF IRI IRI has proved effective for teaching many subjects in the primary school curriculum. Content related to essential life skills, such as good health practices, can be included in IRI programs. The importance of life skills can be highlighted through dramatic situ- ations and music. The program in Zambia incorporates a strong element of life skills related to HIV/AIDS. ® Commitment to comprehensive grade-level curriculum coverage IRI programs are most commonly devoted to a single subject and are designed to cover the entire grade-level curriculum for that subject for the year, using one 20-to-30-minute radio program a day. The daily, year-long, full curriculum coverage used in the first IRI series, in Nicaragua, was meant to inaugurate a break with traditional classroom use of radio as a backup element that supplemented conventional instruction only in a minor way. Several recent IRI programs have departed from the general pattern. Guinea has integrated some mathematics into its French-language instruction; Zambia has included mathematics and life skills in its English-language programs; and Haiti has made an effort to include civics in its Creole-language programs. Countries have tended to pro- vide one lesson a day in a single subject, but Guinea's IRI lessons are broadcast three 41 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION times a week rather than daily. The program covers all six grades of primary school. Haiti uses IRI for two subjects and three grade levels; language arts lessons are broad- cast three times a week, and mathematics lessons, four times a week. Practical considerations have dictated these variations. Early evaluations of the impact of the programs in Guinea and Haiti, although posi- tive, show modest results. The experience in Haiti also shows that teachers are not always clear about what to do during the days of the week with no radio broadcast. This leads both to omissions and to overlap in instruction. Most IRI applications so far have been developed for the early primary school grades (1 to 4). There is no evidence to suggest that IRI will not be as successful for the higher grades of primary school, although it may be necessary to provide a more extensive range of supporting print materials for those grades. Another consideration is that stu- dent ability levels in the higher grades are likely to vary more widely, which would require different programming strategies. ® Realistic limitation of the number of school subjects and grade levels to be covered by IRI No empirical data exist to indicate how much of the school curriculum can be effec- tively provided through IRI or how much time during the day students can be effectively engaged through radio. These issues are largely decided on the basis of the costs of developing programs, the amount of airtime that is available for broadcasting, and the number of radios required in each school to accommodate multisubject, mul- tigrade broadcasting. There is also a potential limit on how long during the school day students are likely to attend actively to radio instruction. Current applications of IRI tend to follow multiple or multichannel approaches that seek variety in learning channels, educational methods, and the kinds of stimu- A well-developed IRI program will lation present in the classroom. help teachers and students be more Effective use of supplementary ma- comfortable with the official curriculum. 42 FEASIBILITY OF IRI terials such as worksheets, posters, wall friezes, and activities planned outside the broad- cast time can motivate learners and teachers, improve achievement levels and retention, and maintain interest. IRI programming has also proved particularly useful in helping teachers manage large or multigrade classrooms. How Does the Content of IRI Programs Relate to an Existing Curriculum? Most countries have invested both money and pride in developing primary school cur- ricula for their schools. Adherence to a national curriculum may be legally required. Vested financial interests (textbook authors and publishers) may be associated with the curriculum to such an extent that the suggestion of any change may provoke resistance. An essential strategic aspect of developing an IRI program, therefore, is to define its relationship to the existing curriculum. A well-thought-out policy framework, devel- oped in collaboration with all stakeholders, can facilitate dealing with many of these issues from the very outset of the initiative. ® Clarity as to whether IRI subject content will follow the existing official curriculum or will develop a new curriculum Where the program calls for new content or for coverage of a new subject in the curricu- lum, the relationship to the existing courses may not be a problem. An issue does arise where the program becomes part of an existing curriculum. Many IRI applications have responded through agreements with key stakeholders that the new program will not replace or circumvent what may be an officially sanctioned curriculum. This often means conceptualizing the IRI program as an alternative mode of delivery for the existing curriculum. An important operational step in developing an IRI application is to map the exist- ing curriculum (in which objectives and content topics may not be stated with great specificity or be carefully sequenced) onto a design document (or master plan) for the IRI program. This operation, as described in chapter 3, should involve people with subject matter expertise and those responsible for the national curriculum. Has a Decision Been Made on the Language of Instruction to Be Used in the IRI Program? The choice of language of instruction for an IRI program is generally influenced by the language of instruction in the current curriculum and by the existing language policy. 43 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION IRI works well in multilingual situations, as students and teachers can furnish local-language translations of terms used by the radio speaker. ® Consideration of the cost and practical requirements of IRI programs in local languages The choice of the language of instruction for IRI has rarely been controversial and is usually dictated by practical considerations. IRI programs have most frequently been taught in the language of instruction designated for the specific grade level--usually, the national language. This is the situation for most of the Latin American IRI programs, which have been developed in Spanish. Some programs have been translated into indig- enous languages for young children who do not speak Spanish. In some countries, one of the curriculum objectives supported by IRI is the develop- ment of proficiency in a second language such as English or French. In this group are Guinea, Kenya, Lesotho, Nepal, South Africa, and Zambia. In most of these cases, pro- gramming techniques call for the classroom teacher to provide support to students in the mother tongue, which may vary by region and by school. In Haiti, IRI has been devel- oped in Creole, the mother tongue of young Haitians. This decision is consistent with national education policy, although many educators and parents support the use of French. In assessing the feasibility of using IRI for primary school instruction, planners should bear in mind the costs and practical requirements of developing, producing, and broad- 44 FEASIBILITY OF IRI casting programs. In countries where many local languages are spoken, it would be difficult or impossible to provide instruction in every child's mother tongue. This, how- ever, is likely to be an issue the country is already facing in developing its national curriculum. Can IRI Content Be Adapted from Another Country? The feasibility of adapting IRI content from another country depends on decision- makers' willingness to take this approach and on whether certain obstacles can be over- come. ® Willingness of national decision-makers to consider adapting IRI lessons developed in another country Strong evidence exists that IRI programs developed in one country can be successfully adapted for use elsewhere. The advantages of adapting a program rather than develop- ing one from scratch are lower costs, less complexity, and less time required for initial program development. The best example is the adaptation of the Nicaraguan math- ematics program for IRI programs in other countries. Scripts from Nicaragua were adapted, to varying degrees, for use in Bolivia, the Dominican Republic, Honduras, Thailand, Venezuela, and, currently, Haiti. The math lessons were also tested (but not further developed) in Costa Rica and Ecuador. The Kenya Radio Language Arts (En- glish) lessons were adapted for use in Lesotho and then in Honduras. South Africa analyzed the Kenyan series but, for philosophical and pedagogical reasons, opted to develop its own model for teaching English. More recently, there has been evidence of increased regional cooperation in Africa, with Malawi, Nigeria, Rwanda, and Sudan reviewing and utilizing materials produced in South Africa. This will greatly enhance the capacities of these countries to implement IRI, since up-front costs will be less. South Africa has also conducted orientation and training workshops for IRI in some of these locations. Adaptation of lessons has meant that the scripts produced in one country are rewrit- ten (in some cases, translated) and rerecorded for use elsewhere. The original lessons have not been simply rebroadcast in other countries. If lessons have been well conceived and there has been systematic formative evaluation in the originating country, the use of instructional sequences that have been found to work there can lead to great savings in time and in curriculum and media expertise. The international experience with IRI has so far provided no rules of thumb that permit quantitative estimates of the savings in money or effort when adapting programs from one country for use in another. 45 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION ® Evidence of potential to overcome major obstacles to successful adaptation of IRI programs There are several hindrances to adapting lessons from one country for use in another: s Despite great similarities in primary school curricula, differences in curricu- lar objectives and sequencing exist and must be allowed for if an IRI application is truly to reflect the national curriculum. Willingness and expertise to un- dertake a thorough analysis of the existing curriculum, as well as the IRI curriculum to be adapted, are required. s Differences in the length of the academic calendar and in the time allocated to subjects during the school day can lead to considerable variation in what is appropriate for students at the same grade level, even in neighboring countries. s Identifying and locating scripts from previous IRI projects (some of which date back more than 25 years) can be difficult. In some cases, intellectual property issues may impede access to IRI programming. Program Development An IRI program involves creative and dynamic interaction among several subsystems that must all work toward agreed-on targets. Constructing a well-organized, high-per- formance organization is a critical step in a successful IRI program. Checklist: Program Development ® Availability of resources for all functional elements required for devel- oping and delivering an IRI program ® Identification of a strong institution to lead development and expan- sion of an IRI program ® Commitment to establishing effective working relationships with key stakeholders ® Congruence with emerging plans for decentralization ® Assessment of the amount of capacity building needed for functional tasks 46 FEASIBILITY OF IRI ® Decision on the kinds of technical assistance needed to build capacity ® Consideration of the options available in a country for delivering IRI programs to schools ® Identification of at least one delivery option that helps reach the in- tended schools with the available resources ® Agreement on developing and testing IRI in a sample of schools be- fore introducing it more widely Are the Key Inputs Needed to Develop an IRI Initiative Available? As with any pedagogical innovation, the development process for IRI is both deliberate and creative and involves continuous learning. The necessary inputs must be available at the right times so that the content and quality of program development reflect the best efforts of those called to participate in the process. ® Availability of resources for all functional elements required for developing and delivering an IRI program The inputs needed for developing and implementing an IRI program show a great deal of consistency across countries. Although they may be different from country to coun- try, the functional areas described below need to be operational during an IRI project. Design and management. Responsibility for the day-to-day management of all aspects of the project, including the delivery systems and linkages among other teams. Physical resources Personnel Office facilities Trained manager/director Computer with: Secretary · Bookkeeping software Clerk(s) · Word-processing software Driver · Modem connection · E-mail connection Printer Copying machine Transport Telephones 47 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Curriculum design. Responsibility for content and curriculum design, including the se- quence in which content is taught. Physical resources Personnel Computers, accessories, and Executive producer supplies Education specialist Office space Content specialist Scriptwriting and support materials. Responsibility for designing and writing all educa- tional materials required for the radio programs, according to the design document. Physical resources Personnel Computers Content specialist Delivery mechanism (possibly Executive producer through management team) Trained writers Radio production. Responsibility for all aspects of recording, storing, and preparing ready-to-broadcast programs. These responsibilities can be assigned to a contractor. Physical resources Personnel Technical equipment for: Trained actors · Recording Trained studio producer · Editing Musicians · Duplicating Technicians · Dispatching · Storing Research and evaluation. Responsibility for (a) determining the current knowledge, at- titudes, and practices of the audience in relation to the subject to be taught; (b) establishing the formative evaluation system and arranging, monitoring, and reporting on class- room observations; and (c) designing and carrying out a summative evaluation. Physical resources Personnel Transport Specialists involved in the Computers and accessories curriculum, scriptwriting, Reference materials and production phases Evaluation test materials 48 FEASIBILITY OF IRI Marketing. Responsibility for "selling" the package to clients (schools, teachers, and educational organizations). The marketing team should be involved in the entire pro- cess. These duties could be contracted out. Physical resources Personnel Computer with graphics Marketing expert The above functions are often performed by teams. Figure 3 shows a typical system. What Kind of Institution Should Lead the Development of an IRI Program? A successful IRI program requires strong leadership to ensure (a) performance of the technical functions needed to develop and deliver the program, (b) credibility and ac- ceptance of the program, and (c) provision of a platform for eventual institutionalization and expansion of the program. Strong leadership involves having both the necessary authority and the technical competence to implement the program. The choice of insti- tution depends greatly on the local context. Figure 3 Typical IRI Learning System and Interaction of Functions Scripting Curriculum and design support materials Delivery Production Design and management system Research and Marketing evaluation 49 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION ® Identification of a strong institution to lead development and expansion of an IRI program IRI programs undertaken to date have followed one of three institutional arrangements: s Centralized unit within the ministry of education s Decentralized unit within the ministry of education s Partnership between an NGO and the ministry of education. What Relationships Are Needed with Other Stakeholders? In addition to making institutional arrangements for developing and implementing an IRI program, the lead institution has to have the vision and resources to build alliances with those who command the authority, expertise, and support needed to operate the program and win public acceptance for it. ® Commitment to establishing effective working relationships with key stakeholders Effective working relationships with partners--government bodies, donor and lending organizations, NGOs, teachers' unions and other professional organizations, universi- ties and other educational institutions, and national and local media--are essential at different points in the program development process. They are necessary for securing: s Government support of the program. This is especially important when the lead institution is an NGO. Government awareness of the program often has to be pursued at different levels of authority (national, provincial, district, and community) and in the face of staff turnover resulting from elections and civil service rotation. Government support is necessary for sanctioning the IRI curriculum, introducing and supporting the use of IRI in schools, ensuring access to schools for formative evaluation and monitoring, obtain- ing public financial support for operation of the IRI program, coordinating broadcasting arrangements with other government ministries, providing for training public school teachers in the use of IRI, and obtaining support for provision of print materials to schools. s Donor awareness of the program. Donors and lending organizations that support education in the given country need to be kept up-to-date concern- ing the goals and implementation of the program. This helps avoid duplication 50 FEASIBILITY OF IRI Box 6. Local Community Involvement in IRI in Zambia Zambia's program is designed to be community based. It depends on volunteer mentors who serve as learning facilitators and on the provision by parents and the local community of a learning space and minimal resources--radio, batteries, a chalkboard, and local resource materials. According to the Mentor's Guide for the program, "The administration of the Taonga Learning Centres illustrates most clearly how the partnership between the Ministry of Educa- tion, communities, volunteer mentors and learners must operate if children are to learn effectively. Each partner has a vital role to play, but it is the communities who must ultimately manage their centers." Mentors are trained by the Ministry of Education and are supplied with mentors' guides. As described in the guides for grades 1, 2, and 3, "The mentor is the link between the radio and the children. The radio alone won't teach effectively. We rely on the mentor to set up the center before the lesson, to lead every lesson by guiding the learners and managing the activities, and to follow up after the radio lesson has concluded." of or conflicts with other educational initiatives supported by donors. In some cases support from other donors has helped continue the development of IRI programs after pilot project funding from the original donors had ended. s Awareness and support on the part of other educational organizations. Or- ganizations with interests in curriculum, educational materials, teaching, teacher training, and educational research should be enlisted as partners or supporters whenever possible. They are important sources of expertise for program development and are potential supporters for institutionalizing and expanding an IRI program. s Support at the local level. Community support is usually important at the school level (see box 6). Assistance from community leaders, NGOs, com- munity-based organizations (CBOs), local businesses, and parent-teacher associations can help schools support teachers, obtain or repair radios, buy batteries, etc. Parental attitudes toward the songs and bits of knowledge that children bring home from school are also important. s Support by teachers. Finally, the attitudes of the various stakeholders will have little effect at the classroom level if there is not broad support by individual teachers. The IRI program critically affects how learning takes place and how the classroom is managed. If the teachers find the program 51 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION onerous even at the outset, chances for long-term success are minimal. One of the best ways to gain teachers' support is to make active use of the feedback loop that is generally part of the periodic cluster meetings or other exchanges that are designed primarily to consolidate and reinforce the teacher-training content of the IRI program. Teacher support for IRI can also be enhanced by including IRI training in the pre-service teacher- training curriculum. ® Congruence with emerging plans for decentralization Many countries are decentralizing their education systems. The development of a new IRI program must fit into the existing education structure in such a way that IRI insti- tutional arrangements will be compatible with changes in authority and functions as decentralization proceeds. Even in countries where decentralization efforts have yet to be accompanied by real devolution of authority and resources, IRI programs typically perform better when implementation takes into account the leadership and resources at subnational levels. Most IRI projects have operated as national projects that are designed to develop and implement a national curriculum throughout the country. In large countries such as Nigeria and India, IRI projects in states are on the scale of national programs in other countries. Increasingly, education offices at subnational levels are playing a role in orga- nizing teacher training, school selection and support, and monitoring. Usually, matters relating to curriculum remain at a national or state/provincial level. In one country, as a result of decentralization plans, all curricula were deemed to be "local." This led to the abandonment of a successful national IRI mathematics program. Planners need to address decentralization plans and realities in considering the feasi- bility of an IRI intervention. Experience has shown that even in highly centralized systems, the involvement of district and local officials can be important to program implementa- tion. There are many examples in which municipal and community leaders have served as a "megaphone," transmitting up the chain of authority community and school de- mand for programs and for results. In many cases local officials have supported schools and learning centers with their own discretionary resources. Broadcasting of IRI programs is becoming an increasingly multitiered affair. When national channels are not a good option because of the limited reach of the signal or the unavailability of time slots, broadcast time is used from international sources, private stations, and, in some cases, community radio stations. Table 3 indicates how IRI program development functions have played out in the past, according to level of decentralization. 52 FEASIBILITY OF IRI Table 3 Responsibility for IRI in the Context of Decentralization, by Level of Authority State/provincial/ Function National prefectural District/municipal Authority Resources Authority Resources Authority Resources Curriculum U U S S R R Broadcasting U U S S S R Materials production U U S S R R Teacher training U U S S S S Program monitoring U U U U S S School support U U U U U U Evaluation U U S R R R Note: U = usually; S = sometimes; R = rarely. "Authority" refers to responsibility shared with the level above; "resources" indicates both discretionary funds and nonmonetary contributions. What Level of Technical Assistance Is Required? A successful IRI program requires systematic application of national capacity to develop and distribute IRI in a way that effectively internalizes and applies quality control. Ma- ture and technically complex systems for curriculum planning, scriptwriting, development of support materials, audience research, formative evaluation, radio production, broad- cast monitoring, teacher training, school support, and stakeholder and public awareness have to be built up. ® Assessment of the amount of capacity building needed for functional tasks There is often a tendency to underestimate the capacity required to develop, test, and deliver good IRI programs. Well-developed programs flow smoothly from beginning to end. The skill required to develop such programs is often not apparent, particularly to those with no background in the particular methodologies being used. In many cases, even where personnel have experience in developing and producing radio programs, the 53 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION use of more interactive techniques may be entirely unfamiliar. Similarly, the ability to organize and manage a complex operation capable of developing an effective IRI series is often seriously underestimated. An assessment of the training needed to ensure the availability of skilled personnel to develop the IRI program must take into account the likelihood that personnel may be lost to the project in the course of program development. New personnel may have to be trained periodically throughout the project, sometimes on short notice. ® Decision on the kinds of technical assistance needed to build capacity The capacity to develop and implement a successful IRI program is built up through a mixture of formal and on-the-job training. Training is typically provided through tech- nical assistance, which can vary in magnitude and form. Some countries, such as Thailand and Venezuela, have developed IRI programs with little systematic technical assistance. Others have undertaken capacity building through a donor-funded contract with a tech- nical assistance provider: examples are Bolivia, Costa Rica, the Dominican Republic, the Arab Republic of Egypt, El Salvador, Ethiopia, Guatemala, Guinea, Haiti, Hondu- ras, Kenya, Lesotho, Nepal, Nicaragua, Nigeria, Pakistan, Papua New Guinea, South Africa, Tanzania, and Zambia. Contracting technical assistance through a specialized organization offers the advantage of greater prospects for continuity and consistency. Organizational contracts, however, tend to be more expensive because of overhead costs. When donors or lenders opt to use individual consultants, they should be clear about who will play the role of general contractor and orchestrate the various capacity-build- ing inputs into the development of a smoothly operating system. Most of the technical assistance provided to national applications of IRI has come through contracts, grants, and cooperative agreements funded by USAID. These agree- ments, which are awarded competitively, typically cover assistance--usually during periods of three to five years--for building national capacity to develop IRI lessons, evaluate them, and get them on the air. It is assumed that the countries will "own" all aspects of the programs and that there will not be a continuing financial or technical relationship between the country and the service providers in order to keep the program going. For the service providers, the success of a national program contributes to its reputation and the likelihood of being asked to provide technical assistance in other countries. As shown in appendix A, the largest share of contracts, cooperative agreements, and grants has gone to a few U.S. organizations that have specialized in IRI. This is chang- ing. In some instances, organizations such as OLSET in South Africa are providing assistance in other countries. Similarly, individuals from countries that have success- fully developed IRI programs often lead technical assistance efforts in programs in other 54 FEASIBILITY OF IRI countries--currently, in Ethiopia, Guinea, Guyana, Honduras, India, Nigeria, Tanza- nia, Zambia, and the lusophone (PALOP) countries in Africa. In some countries, especially where an IRI program is being adapted or developed for a new audience (e.g., Hondu- ras and Haiti), funders such as USAID are providing grants, contracts, or subagreements to national organizations to lead the development process, often in partnership with an international provider. Should Programs Be Broadcast, or Delivered on Cassettes or CDs? A major determinant of the feasibility of an IRI program is the availability and effective- ness of broadcast channels for delivering IRI programs to the intended audience. ® Consideration of the options available in a country for delivering IRI programs to schools IRI program planners may have many options for delivering programs, including: s National (public or privately owned) radio stations that cover the entire country s A network of regional or community radio stations, private or public s Channels--including those that provide digital signals and that have broad national or regional coverage--available from a satellite provider s Channels (sometimes shortwave) that are available from private companies and broadcast from other countries s Broadcasting to schools through the Internet--a possibility that is likely to become more feasible in the future s Delivery of IRI programs on audiocassettes or CDs. ® Identification of at least one delivery option that helps reach the intended schools with the available resources Planners need to determine the availability of broadcast slots at appropriate times of the day for school audiences. They also need to know whether a particular broadcast strat- egy covers the intended audience equitably. FM broadcasts, for example, have markedly weaker coverage the farther the listener is from the broadcast relay. If the FM network coverage favors children in urban settings, where transmitters are located, an FM strat- egy would not be appropriate for nationwide coverage. Shortwave broadcasts are sensitive to solar interference, and frequencies must be carefully chosen depending on the time of day of the broadcast. 55 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION The radio may be placed strategically to enhance reception. Providing equitable broadcast reception will likely be less complex, given the num- ber of broadcast options, than the provision of materials and training that accompany a typical IRI program. If, for example, current textbook distribution strategies are less successful in isolated rural communities than in large towns, similar problems will likely arise for the various nonbroadcast components of a multichannel IRI program. Com- munity involvement, linking of teacher training to materials distribution, and strong monitoring systems are some strategies for ensuring that listeners from isolated rural communities receive IRI services comparable to those for urban students. The annual costs of broadcasting an IRI series will differ from country to country. Some ministries of education have been able to arrange broadcasting of programs free of charge, but in many places competition for such slots is growing, and the global trend toward the privatization of broadcasting means that the likelihood of obtaining broad- casting slots gratis is diminishing. The annual cost of broadcasting an IRI series is estimated, on the basis of existing data, at $70,000.1 The relationship of broadcasting costs to total costs and cost per student is addressed in chapter 3. 1All dollar amounts are U.S. dollars, unless otherwise noted. 56 FEASIBILITY OF IRI In some cases, IRI programs can be delivered to schools on audiocassettes or CDs. That option is more expensive and more cumbersome than broadcasting, but it does offer schools greater flexibility in using IRI programs. Guay (2000) made a comparative estimate of the costs of using broadcasting, CDs, and audiocassettes in Haiti. He found that over a three-year period, the cost of distributing a program on CDs would be more than double the cost of broadcasting the program, and the cost of delivering the same program on audiocassettes would be double that of delivering it on CDs. Guay summa- rized the advantages and disadvantages of using CDs rather than radio broadcasting as follows: Advantages s CDs stay in the hands of the teacher; there is no need to repeat broadcasts each year and no further costs for broadcasting in subsequent years. s Sound quality is good and consistent. s Distribution is easy, and a network is already in place. s Teachers can adjust schedules to their needs. Disadvantages s Teachers, if pressed for time, may skip lessons. s The cost of CD players is high, and production cost for CDs is slightly higher than for AM/FM broadcasting. s Running CD players on batteries becomes expensive over the years. s CD players are sometimes fragile and unreliable in dusty conditions. s People outside the school cannot listen in to the lessons. s Total usage costs are not competitive when compared with AM/FM broadcasting. What Kind of Pilot Project Is Required to Test IRI? In most countries, implementation of a pilot project has preceded the adoption of IRI. Planners should consider what kind of pilot is most appropriate for testing their IRI program. ® Agreement on developing and testing IRI in a sample of schools before introducing it more widely Pilot projects have varied in scope and duration. In general, however, they have been designed to demonstrate the new methodology under controlled conditions, observe how IRI will work under the specific conditions encountered in the piloting country, 57 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION seek feedback from stakeholders, and build the piloting institution's or agency's capac- ity to undertake all aspects of its IRI development and dissemination. Given the consistent data from many countries showing the effectiveness of IRI in providing primary school instruction, it is not likely that pilot projects will be under- taken to test the potential effectiveness of the methodology itself. Rather, the focus is on giving countries hands-on experience in developing and using IRI so they can decide whether the methodology is suitable for them and whether they are prepared to em- brace the challenges of IRI program development. The capacity-building aspect of a pilot project is important. A pilot project makes it possible to put in place the embryonic subsystems required for an operational IRI sys- tem. It is essential that capacity building not be short-circuited by overly ambitious program production schedules and targets. During the pilot, staff of the lead institution receive training in scriptwriting and develop and test draft audio programs. Time is allocated for audience research and formative evaluation. Production and broadcasting arrangements are negotiated and, often, tested. Two-phase pilots are common. The first phase concentrates on capacity and aware- ness building using a small number of lessons. The second phase often focuses on developing and testing IRI for a longer period, up to one school year. Evaluation of results after the program has run for a year usually provides sufficient information on whether additional investment in developing a full IRI application is justified. For ex- ample, in Malawi and Sudan small pilot projects involving a limited number of schools were conducted to provide an opportunity for policymakers, planners, teachers, and community leaders to become familiar with the methodology and to assess the appro- priateness of IRI for their circumstances before making a full commitment. Cost and Finance For education systems evolving in resource-lean environments, the structure of IRI pro- gram costs must be clearly understood, and the overall financial commitment must be realistic. Because IRI must understand its audience in order to end up with a product that will "sell," considerable effort and resources must be directed to calibrating the product to the needs of the clients. Fortunately, the lower recurrent costs when the program has gone to scale offset the higher initial costs. 58 FEASIBILITY OF IRI Checklist: Cost and Finance ® Good understanding of the costs of developing and implementing an IRI program compared with those of other technologies or educational interventions ® Consideration of cost containment mechanisms ® Consistency between the actual costs of developing an IRI program and what the country can afford ® Realistic appraisal of the possibilities for financing the development and implementation of an IRI program What Does It Cost to Develop and Deliver IRI? The extensive international interest in using IRI has come about in part because IRI has offered what has been perceived as a relatively low-cost means for improving educa- tional quality in primary schools. Planners need to determine how much it will cost to develop and deliver IRI and whether it is affordable. ® Good understanding of the costs of developing and implementing an IRI program compared with those of other technologies or educational interventions The following issues have to be addressed in assessing the costs of IRI: 1. Relationship between fixed and variable costs. As in other forms of distance educa- tion, the proportion of fixed costs to variable costs is higher for IRI than for more conventional educational applications. Fixed costs are the costs required for devel- oping and operating a learning system, irrespective of the number of learners served. They include the investment costs of developing and distributing lessons and broad- casts and the administrative costs of the institution that organizes the learning. Variable costs depend directly on the number of learners in the system and include teachers' salaries, facilities, books, and other materials. According to Adkins (1999: 39), the annualized fixed investment costs associated with an IRI program are about two- thirds of the annual costs of a small-scale program (with about 100,000 students) and about one-third of the annual costs of a large-scale program (with about 1 mil- 59 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION lion students). This relationship, which helps reduce the burden of recurrent costs for governments, has been attractive to donors and lenders. 2. Economies of scale. IRI applications tend to benefit from economies of scale, as described by Adkins (1999: 38): To serve 1 million students the cost per student is only 40 percent of what it costs to serve 100,000 students. To serve 4 million students, the cost per student contin- ues to decline, although not as quickly. The scale economies are mainly in investment costs. Although some scale economies in recurrent costs exist, per-student recur- rent cost is only 22 percent less for 100,000 students than for 1 million. Changes in total costs (which include investment costs) and recurrent costs as econo- mies of scale materialize are shown in figure 4. 3. The magnitude of add-on costs. The use of IRI to improve primary school quality will not lead to a reduction in the unit costs of educating a student. Unlike distance education systems that Figure 4 are designed to increase access, appli- IRI Per-Student Cost at Various Program Scales cations that use technology to improve and for One Subject the quality of conventional educational 16 programs usually result in increased 14 per-student costs for providing educa- tion at a given level. The reason is that 12 applications take place in regular dollars 10 in schools and involve no reductions in cost 8 teaching time. The use of IRI is never- Unit 6 theless efficient from an economic point of view to the extent that it offers a 4 less costly means of achieving the same 2 results as other quality-improving tools 0 or that it provides better quality at the 50,000 100,000 150,000 250,000 400,000 650,000 same price as other tools. 1,000,000 1,500,000 2,000,000 3,000,000 4,000,000 Number of beneficiaries/audie Costs for IRI in primary schools are Recurrent Total likely to be in the range of $3 to $8 per Source: Dock and Helwig (1999: 38). year per pupil reached, depending on the 60 FEASIBILITY OF IRI size of the program (Adkins 1999). These numbers include the per-pupil annualized investment costs of developing the radio lessons.2 Adkins found that when the les- sons have been developed and the system is in place, the annual recurrent costs are $2.32 for a large-scale program and $2.97 for a small-scale one. The front-end investment for development and production can be significantly reduced through the adoption or adaptation of existing IRI materials. In the early stages of program planning, appropriate officials should not only undertake a re- view and analysis of scripts and print materials but should also consult widely with counterparts in other countries to gain a full understanding of the systems currently in place, especially those in Africa. There is significant potential for regional coop- eration and collaboration and a wealth of experience that can be drawn on, allowing planners to make informed decisions regarding implementation of IRI. How Do IRI Costs Compare with Those of Other Technologies? A study by Murphy et al. (2002) compared the cost of using IRI with the costs of using other media to support education. The cost of using television is usually higher than that for radio--typically, more than 10 times as high per student reached. Studies report wide variation in annual per-student costs of computers in developing countries: $18­ $63 (Perraton and Creed 2000); $78­$104 (Potashnik and Adkins 1996); and $84 (Osin 1998). Cawthera (2000) estimates dramatically lower costs (under $20) when certain assumptions apply. These contrast with the results of a World Bank study that found the per-student cost of computers in Barbados to be more than $600 (Bakia 2000). Although disparities exist among studies, even low estimates are in the double digits. Table 4 shows the results of another World Bank study of the costs of alternative tech- nologies. How Can Recurrent Costs Be Contained? The most important consideration in determining the feasibility of IRI from a cost per- spective is the size of the annual recurrent costs that the government must bear to provide the program to schools. ® Consideration of cost containment mechanisms 2Earlier estimates of per-student costs were as low as $1.51 for Bolivia and $0.91 for Lesotho, but those studies have been criticized for either overestimating the numbers of students that a program might reach or underestimating the recurrent costs. 61 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Table 4 Comparative Per-Student Costs of Selected Educational Technologies, 1998 Scale Per-capita cost Technology Application (number of learners) (U.S. dollars) Radio (large program) Primary basic skills 1,000,000 3.26 Radio (small program) Primary basic skills 100,000 8.12 Computer/Internet (large school) Secondary laboratory 600 72.00 Computer/Internet (small rural school) Secondary laboratory 150 98.00 Source: World Bank (1998). As discussed above, per-student costs benefit to a limited extent from the economies of scale associated with a larger number of students. Aside from scale, four areas are im- portant for planners to address as a means of containing recurrent costs: s Broadcasting. A considerable range exists in what countries pay to broad- cast IRI programs. These costs can be contained if governments are open to exploring the increasing number of alternatives now available and proactive in developing policies and infrastructure that will make airtime available to education at favorable rates. s Radios and batteries. The provision of radios and batteries to schools can be a large component of recurrent costs. Arrangements that shift responsibility for these costs to communities and schools can reduce the overall costs of the IRI program. Savings would depend on the comparative advantage that local communities have in managing the use of radios and batteries at the community level. Use of wind-up and solar-powered radios can further re- duce recurrent costs (see box 7). s Print materials. The cost benefits of using a medium of mass communication such as radio can vanish quickly if program designers fail to keep provision of print materials to a realistic level. s Teacher training and support. The training of teachers to become proficient in using IRI is another area that can inflate recurrent costs. IRI training may be seen as an opportunity to include other needed training. Some form of monitoring of and support for teachers is likely to be essential to new IRI 62 FEASIBILITY OF IRI programs, but it necessitates Box 7. Pros and Cons of considerable effort and travel. Wind-up Radios Containing recurrent costs re- quires that programs be designed to identify what is es- The main advantage of using wind-up ra- sential rather than what is ideal. dios is the reduction of recurrent costs that are borne by either ministries of education ® Consistency between the or communities. The quality of locally avail- able batteries is often so poor that their use actual costs of developing an becomes prohibitively expensive due to the IRI program and what the frequent need for replacement. The main country can afford recurrent cost specific to wind-up radios is for repairs if the wind-up lever breaks. This cost will likely decrease with time as teach- There is no set rule on what proportion ers become more attuned as to how to avoid of a country's educational invest- damaging radios when winding. Local en- ment and recurrent budget should be trepreneurs will often respond to the allocated to an IRI program. When con- demand for radio repair, and as expertise increases, repair costs will decline. It is im- sidering the recurrent cost issue, portant to note that non-wind-up radios also however, it is important to understand need repairs, especially in dusty climates. the current allocations for nonsalary in- puts such as textbooks, materials, supplies, and other didactic materials. A country may already have, for ex- ample, a policy of increasing these nonsalary inputs as part of a push to improve educational quality. Alternatively, a country may choose IRI as a feasible strategy in a context where pedagogical materials are poorly distributed and other in-service teacher-training programs are dysfunctional. Whatever the cost context, it is likely that an IRI solution exists to fit it, as long as the investment and recurrent costs are well understood, properly contained, and supported by policymakers and other stakeholders (see box 8). How Are Costs Distributed and Covered? During the development stages of an IRI project, the overall burden of costs is higher than at later stages. As the products and systems are developed, investment or develop- ment costs are reduced but do not disappear entirely. The total cost of using IRI eventually approaches annual recurrent costs. IRI programs have typically been developed through a considerable investment of donor funds. Donor funds have often been channeled through pilot projects which pro- 63 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION duce IRI programs that continue to be Box 8. Understanding the used after the pilot is over. Donors have Costs of IRI had a good record in maintaining com- mitment and financial support through the time frames needed to conduct pi- The underlying factors that determine the lot projects. Only one IRI application costs of developing an IRI program can be has ever been abandoned because a do- summarized as follows: nor withdrew financial support during s High fixed costs are generally required the course of the project. Generally, to develop an IRI program. These costs there is no clear-cut distinction between are usually absorbed through a grant the period of donor investment and the or loan. time when countries take on the recur- s IRI benefits from economies of scale, rent cost of the programs. Negotiations but the benefits accrue mostly to in- for cost sharing with government usu- vestment in program development ally begin during the pilot stage, so that rather than to payment of recurrent costs. recurrent costs such as broadcast fees are covered by government from the s Annual per-student costs for IRI are sub- beginning. In some cases, donor sup- stantially lower than those for other technologies. port has gone on long enough for projects to evolve beyond a limited pi- s The annual recurrent costs for deliver- lot phase. In other cases, as in Bolivia ing IRI are between $2 and $3 per pupil. and Honduras, donor support contin- ued but was directed toward IRI programs aimed at a different educational goal and audience. Depending on the local institution spearheading the effort, financial support has sometimes become diversified among local sources and donors. OLSET in South Africa was successful in attracting support from various donors for scaling-up its English- language program. But in most cases it is unrealistic to expect one donor to fund expansion of a program initiated by another. It is therefore imperative to develop a realistic sce- nario that anticipates a smooth transition between a development phase and wider implementation. The programming of donor assistance to stop at a point when results have been "demonstrated" is shortsighted. Most countries using IRI have needed donor support to get into an expanded implementation phase and to build momentum with schools, parents, and leaders. Table 5 provides an example of the distribution of costs in an actual IRI project in Africa. The costs are collapsed into the categories used by Adkins (1999) to analyze IRI costs. Adkins's analysis of IRI in six countries found that investment costs made up 65 percent of total costs and recurrent costs were 36 percent. The IRI application featured 64 FEASIBILITY OF IRI Table 5 Distribution of Costs in an IRI Project in Africa (percent) Percentage of Cost item Year 1 Year 2 Year 3 Year 4 total costs Investment costs Curriculum and scripts 4.4 5.7 5.9 5.1 21.2 Radio program production 1.6 1.9 2.0 1.2 6.7 Preparation of printed materials 0.7 0.8 0.9 0.9 3.4 Start-up costs, including upfront training 8.5 6.9 6.9 7.0 29.3 Investment cost as percentage of total four-year budget 15.3 15.3 15.8 14.2 60.5 Recurrent costs Air time, other broadcasting 0.0 0.7 1.7 0.0 2.4 Radios (three-year life) Donated by a foundation 0.0 Production of printed materials and cassettes 0.0 1.3 1.4 0.0 2.7 Distribution of materials 0.0 0.0 0.0 0.1 0.1 Training/training supplies 0.1 1.3 1.3 0.1 2.7 Continuing program development 0.0 1.2 3.1 2.4 6.7 Administrative expense 4.8 6.3 5.7 6.0 22.8 Other fixed expense 2.1 0.0 0.0 0.0 2.1 Recurrent cost as percentage of total four-year budget 6.9 10.8 13.3 8.5 39.5 Total project cost per year 22.2 26.2 29.0 22.6 100.0 Source: Education Development Center. in the table shows a similar picture, with investment costs making up 60 percent of total costs and recurrent costs accounting for 40 percent of the total. Investment costs were spread evenly across the four years of the project; recurrent costs peaked in the middle two years. The project described involved only incidental costs for radios and broad- casting, since the former were donated and the latter was provided at no cost to the project. 65 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION How projects cover the costs of running an IRI learning system after the pilot stage is completed varies widely. Many projects have continued to receive some funding from donors to cover broadcast fees; others have met recurrent costs through a tax or through existing funds within current budgets; and still others have relied on continued relation- ships between ministries of education and communication that include broadcasting arrangements, obviating costs for the IRI program. Other cost-sharing measures have been explored. Educational programs for non­ primary school students have used fee-for-service arrangements or have marketed products such as calendars to pay for the recurrent costs of tapes, batteries, training, or broad- casting. These options have not been explored at the primary level, and costs have generally not been distributed beyond the various agencies within the government and the donor community. What Sources May Be Used to Finance an IRI Program? Determining the feasibility of an IRI program depends as much on understanding how the program will be financed as it does on understanding what it will cost. ® Realistic appraisal of the possibilities for financing the development and implementation of an IRI program The main division of responsibility for paying for the development and implementa- tion of an IRI program has been between a donor or lender and the government. The donor or lender has typically covered program development costs, and the govern- ment has been responsible for recurrent costs. Many IRI programs have succeeded in obtaining funding for parts of the program from other sources, including local gov- ernments, communities, parents, teachers, and public or private radio stations, as shown in table 6. It is important to be realistic when examining two issues: s Long-term recurrent cost implications for the government. The implications of various levels of program magnitude must be carefully assessed. Although expanding a program--say, from 100,000 to 1 million students--has posi- tive implications for per-student costs, the projected costs must still be within the range of what the government budget will allow for nonsalary discre- tionary spending for improving educational quality. 66 FEASIBILITY OF IRI Table 6 Sources of Funding: Five IRI Programs National Local NGOs, ministry of govern- Public Private donations, education ment radio radio com- Country Year budget budgets entity stations munities Parents Teachers Donors Bolivia 1987+ X x x x Dominican Republic 1990+ X x x Honduras 1987+ x x Lesotho 1988+ x x x x x South Africa 1992+ x x x x Note: The plus (+) sign indicates succeeding program years, as well as the year indicated. Source: Education Development Center data, 2003. s Expectations of finance from nongovernmental sources. There have been good examples of getting communities, schools, parents, and other sources to absorb some of the costs of an IRI program. This may take the form of paying for print materials, radios, or batteries. Schools, local governments, or NGOs sometimes help with teacher-training costs. The feasibility of mo- bilizing these sources of funding will depend greatly on conditions in a particular country. Planners should seek evidence of other instances in which this kind of funding has been forthcoming. Planning for Institutionalization and Sustainability Medium- and long-term success in IRI depends on careful planning at the outset con- cerning (a) how the new program will fit into the landscape of education programs and policies and (b) the program's financial and technical sustainability. The need for care- ful, rigorous project planning from the beginning cannot be overstated. Adequate time and resources expended at this stage will provide a solid platform for successful IRI implementation. 67 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Checklist: Planning for Institutionalization and Sustainability ® Compatibility of the plan for institutionalization of IRI with the level of decentralization of the education system ® Clear understanding of the relationship between broadcasts, teacher training, curriculum development, and the delivery of print materials ® A plan for radio broadcasts after the initial IRI development stage that is compatible with the radio policy environment What Is the Plan for Institutionalization? Plans for institutionalization vary according to the degree of decentralization of the ministries of education and communication and of the agencies that are involved in the broadcasting. IRI units have often been created within the ministry of education, and institutionalization has been measured by their activity and survival. In other cases, a national radio and television station houses IRI activities and responds to requests for collaboration from other agencies. This is more often the case where the content is directed toward students who are not in school, as in early childhood development, nonformal education, and health programs. In countries where education systems have been decentralized, institutionalization may be required at the district or municipal level and may depend on relationships among many local departments of education and local radio stations. This situation requires more continuous attention to institutionalization throughout the development stages. Anecdotal evidence suggests that these local relationships tend to last longer, as they are more closely related to the feedback and ownership of communities and fami- lies and less tied to changes of government. Appendix B describes how Lesotho dealt with institutionalization. ® Compatibility of the plan for institutionalization of IRI with the level of decentralization of the education system In the plan for institutionalization, planners should consider including a role for gov- ernment agencies, colleges of education and other tertiary institutions, and NGOs that have an interest or an institutional investment in the program. For example, long after 68 FEASIBILITY OF IRI the Bolivian programs were discontinued as part of the national curriculum, the NGO Fe y Alegria maintained in private schools staff who had been involved in testing and broadcasting IRI math programs. What Is the Plan for Sustainability? Plans for the sustainability of the IRI program should be considered at the outset of a project. Sustainability relates in large measure to the educational benefits that are docu- mented, the affordability of long-term recurrent costs that emerge at the end of the project, and the measures taken to ensure institutionalization. A well-planned IRI ini- tiative will ensure not only that the recurrent funding needs are acceptable but also that an institution which sees continuation of IRI to be in its own interest will be responsible for mobilizing the funding. Recent research shows that sustainability also depends on good leadership. Good leaders tend to be those who can troubleshoot throughout the program development cycle and can think through ways to adapt the IRI learning system to accommodate changes that emerge during implementation (Dock and Helwig 1999). The plan for Widespread student interest in IRI is a key factor in supporting sustainability. 69 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION sustainability must therefore go beyond the need to find an institutional home for IRI. It should focus on developing personal and institutional leadership that can advocate for the continuation and (when warranted) the expansion of IRI and can guide neces- sary adaptations of the program to make it more useful in changing environments. The latter will involve anticipating needed modifications to content, presentation for- mats, teacher training, use of print materials, and so on. It would be a mistake to assume that the IRI program in existence at the end of the initial phase is one that can or will remain in the same form year after year. When teacher-training specialists view an existing IRI program as a potential vehicle for introducing a new technique or concept, they become another constituency that supports the enhancement and con- tinuation of IRI. ® Clear understanding of the relationship between broadcasts, teacher training, curriculum development, and the delivery of print materials Attention to training and development of teachers and program facilitators over the full course of the program has been shown to contribute to sustainability. According to Dock and Helwig (1999), although training new teachers in their IRI role is important, it is equally important to integrate ongoing monitoring, support, and encouragement into in-service training. Teachers also need to know that they are part of a larger system of ministry officials, administrators, production staff and other teachers. The down- stream and spin-off benefits from this investment can be a valuable resource for the country's education system. Furthermore, these educators provide useful feedback on the program and are a vital link to the local community in terms of garnering moral support and resources. Programs such as Teachers in Action, a series of 12 radio broad- casts complemented by Web-based teachers' notes that was developed by BBC World Education Services and OLSET, can assist in motivating teachers in best practices through the examples of other teachers around the world. ® A plan for radio broadcasts after the initial IRI development stage that is compatible with the radio policy environment A final, important consideration for sustainability is the policy environment for radio broadcasts. In countries with few broadcast options for reaching the whole target audi- ence, there may be little opportunity for comparing costs and quality for different service providers. When an IRI program is in its development phase and is receiving support from donors, the challenge of ensuring reliable, quality broadcasting services may be 70 FEASIBILITY OF IRI less of a problem than in later years, after the novelty of the new program decreases. In some cases, the only viable alternative to a public radio network may be a foreign sat- ellite system. Would the policy environment allow for a public entity to purchase airtime from a foreign provider if the national public radio network became overloaded, too expensive, or otherwise inadequate? IRI programs around the world have proved to be flexible in fitting into existing radio policy environments, but thinking through the is- sues at the outset will improve the potential for sustainability. 71 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION 72 Chapter 3 Designing IRI A s was mentioned in chapter 1, the authors conducted a survey for this study of donors, IRI specialists, and local representatives involved in IRI programs around the world. The factor that these respondents rated most critical to the success of IRI, after the level of technical assistance provided, was quality control. This chapter covers the design and evaluation activities that contribute to a quality IRI program. Main Points s Project development entails a number of steps, including (a) determining the profile of the learners--linguistic levels, existing knowledge, interests, local games, local tensions, and role models--and identifying the overarching educational objectives; (b) creating a design document to guide program development and map the learning process of students and teachers; (c) initiating scriptwriting to bring life to the educational activities through radio characters and settings; and (d) conducting formative evaluation to determine whether individual IRI programs, and the entire IRI series, are achieving the desired learning goals. s Production includes local activities that turn learning objectives into scripts into audio programs; reproduction of CDs or tapes for distribution; and perhaps adaptation for different regions of the country. Revisions of pro- grams through formative evaluation involve as much as 50­75 percent more in production costs, although digital editing may reduce this cost. Options for delivery include radio broadcast, CDs, audiocassette tapes, or a combination of these methods. Production of some print materials helps support lessons and training but does not need to be extensive. 73 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION s Marketing is an essential aspect of IRI programs. Possibilities include ra- dio spots, promotion of songs and content featured in the IRI program, development of ancillary materials that support community involvement, and games and competition among schools to motivate students and build program visibility. s Summative evaluation, usually conducted by an external agency, measures the ability of the IRI program to reach its goals and often has a strong qualitative component. Design can begin after it has been determined that the characteristics of IRI are suitable for the country's specific educational conditions and goals and after a general project outline has been developed. During the feasibility stage, decisions will have been made on the language or languages of instruction; subject matter and learning objec- tives (often derived from approved curriculum documents); grade level; learner profile; outline of the components of the learning system (audio, print, training, and so on); primary delivery system (broadcast, CD, audiocassette tapes, or face-to-face); institu- tional home; and potential partners. Project Development The project development phase of IRI is the most technically complex, as it usually entails innovation using a technology that is relatively unfamiliar in the local context. Organizing the process, aligning resources, and working toward targets must be done in a way that encourages creative thinking, allows for new approaches, and, at the same time, produces results on time. IRI program development takes place in a complex en- vironment and demands a highly motivated technical team. Why Is It Important to Understand the Development Process? The development of IRI programs involves a consistent pattern of activity and a fairly consistent set of decisions. Knowledge of these stages is essential in planning an IRI 74 DESIGNING IRI project so that education planners understand how an IRI program develops and what costs are associated with each category of activity (see figure 5). Although a systematic planning process is indispensable, it is important to note that there is no "one-size-fits- all" model. Each country, while drawing on others' experiences, has found it desirable and necessary to customize an IRI solution for local circumstances. Each category of activity involves a specialized team of people with specific talents and skills who work together toward defined goals. Particular decisions and costs are associated with each task, as explained in detail below. What Does Audience Research Include? The first step in IRI development involves reaching a clear understanding of target learners and connecting this knowledge to the educational objectives that the IRI program will serve. Research on the prospective audience identifies a host of characteristics, includ- ing educational and linguistic levels, exposure to other potentially complementary learning channels, prerequisite knowledge in particular subject matter, behaviors related to so- cial inequities, interests, local games that can be built on, the resources available in the classroom and beyond, role models, and typical class sizes and classroom environments. It is the first step in a long, participatory design process that incorporates the feedback Figure 5 Development of an IRI Program Production Scriptwriting Delivery/broadcast Preparation of design document Formative evaluation Audience research 75 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Audience research is key to making programs responsive to the cultural and social contexts of students and teachers. and opinions of local teachers and students and forces people to begin thinking about the classroom as a place where teaching and learning emerge from activities that are both enjoyable and educational. Because most IRI programs target teachers as well as students, the same information is needed for both populations. Audience research is important to IRI development for a variety of reasons. An edu- cational resource that builds on local knowledge and that appropriately targets language, educational levels, and social values and aspirations stands a better chance of being ef- fective. Statistics from school exams are not enough. Because one of the goals of IRI is to create an environment for active learning that goes well beyond the teaching and learn- ing styles prevalent in schools in poorer developing countries, some of the most important information to be gathered may be considered cultural or entertainment-related rather than educational. Yet it is exactly such elements that make IRI relevant and engaging, and in order to identify them and ensure their inclusion in the radio programs, audience research has to be initiated before the structure and objectives of the project are defined. 76 DESIGNING IRI Because IRI builds on local stories, games, and music as source materials for educa- tional activities and aims to reorganize the conduct of teaching and learning, it is important to gather information from students and teachers themselves about what their lives are like and what activities they engage in when they are interacting in nonschool as well as school activities. Generally speaking, information about student likes and dislikes that is provided secondhand by adults, no matter how sincere, misses key connection points for the students. Audience research seeks to observe, listen to, and document what stu- dents and teachers do under normal conditions and to allow them to disclose dreams and interests that can be woven into stories and other activities. The duration and formality of audience research is not set in stone. Usually, audience research gathers key information in the early stages of the development phase and then continues through the formative evaluation process, thereby opening up channels for ongoing feedback and input from communities that will use the IRI programs when they are broadcast. This feedback loop is invaluable in making the programs responsive to the cultural and social contexts of the students and teachers. Finally, most often an IRI series is designed to reach a broad and diverse audience of hundreds of thousands of students. One of the challenges of the audience research phase is to discover ways to reach this large and diverse audience without perpetuating negative stereotypes, while achieving a feeling that the radio program is local. This constraint necessitates an audience research and formative evaluation design that gathers information from many representative environments and children and forces the IRI design team to make important decisions about where to target the language and skill level, how to create attractive and engaging characters that can be used as role models, and how to integrate social and cultural connection points in ways that appeal to the diverse audience. While it might seem irrelevant at first, this aspect of the IRI develop- ment process avoids mistakes that could make the programs less effective and less sustainable. Sample questions in audience research might include: s Who are the heroes for boys in this age group? For girls? (Let the children answer.) s How much of the language of instruction can the students actually under- stand? What about the teachers? s What texts or other resources will accompany the audio programs? What local resources are available that can be used as educational materials (for example, rocks, empty boxes, local songs or cultural activities)? s What are the local accents? Are there speakers who can be understood by many students? If there is no consistent answer, or if certain voices contain stereo- 77 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION types that would devalue the program, is it possible to produce different ver- sions of the program for different parts of the country, with different actors? s What social activities exist that include both boys and girls? If there are few, what activities exist for each gender, and how can they be recreated to be educational? What Are the Educational Objectives? The preliminary research on educational objectives involves formulating the overarching goals of the program in a concrete way. These objectives may appear to be unrelated to other aspects of audience research, but they really are relevant. For example, if an objective of the programs is to increase participation by girls or to offer support to children who have undergone trauma, these goals should be explicitly factored into the development of the design document. As part of the research, information on how to handle these educational objectives is gathered and is included in a participatory de- sign process. Sample questions for educational objectives include the following: s Does the IRI curriculum match/emerge from/complement the national cur- riculum for a particular subject matter? s Do the teachers follow the national curriculum, or do they follow other text- books or local processes that are out of synch with the national curriculum? s Are there other objectives that it is hoped the IRI programs will achieve (e.g., reaching out-of-school youths, increasing participation by girls, improving oral comprehension of language)? s Is the IRI program intended to reach a large population or to focus on the specific educational objectives of a small one? What Should the Design Document Contain? The second stage of project development involves creating the scope and sequence of the programs and individual master plans, integrated within a comprehensive design document. The design document guides every aspect of program development and maps out the learning process for the students and teachers. It plots the curriculum, the char- acteristics of the programs that are designed to engage participants, the evaluation process, the connection of the IRI program to other instructional materials, and teacher training. The period when the design document is being developed is also referred to as the in- 78 DESIGNING IRI structional design phase, when critical decisions have to be made. It is an ideal time to involve various stakeholders in the planning process. The design document provides a blueprint or detailed plan of activities, tasks, budgets, time lines, and deliverables for guiding the IRI team through the development, production, and implementation phases. All of the components of the design document require attention and time throughout the stages of planning, development, implementation, and final evaluation, and none should be neglected. Each component requires a significant amount of training, much of which may be accomplished through learning-by-doing. Successful training will signifi- cantly enhance capacity in the institutions involved. Design documents are generally divided into three parts: 1. Information about overarching learning goals and how they will be met through patterned instructional strategies and formats, including: s Educational objectives for each series and program s Instructional strategies s Audience profile s Program structure s Formative evaluation sites s Print materials needed s Training and teacher development strategies 2. Information that makes the programs engaging and culturally relevant: s Radio characters s Radio setting s Radio formats s Stories, songs, poems, rhymes, and local activities for adaptation s Print support format 3. Learning objectives and activities in each program: s Learning objectives for each program s Formulas for activities and games s Script content. Most design documents also include, for all programs in the series, a chart that tracks when certain types of activities that are important for learning a particular subject matter are introduced. This chart allows the designers to distribute across the 79 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION series activities that reinforce and build on previous knowledge. The design document also indicates how a formative evaluator would be able to observe that the students are engaged in the activities that support each learning objective. On the basis of all this information, a scriptwriter should know exactly what is expected in each script, and observers should know how to validate whether the designated activities are un- der way. Personnel. The people involved in creating the design document--the design team-- include IRI specialists, subject matter experts and instructional designers, producers, community members, scriptwriters, evaluators, and producers. The amount of time des- ignated for each person's contribution depends on the project. Activities. The stage of creating the design document generally involves international IRI specialists and includes initial workshops, field visits, coordination with local cur- riculum specialists, and negotiations concerning the key components of the program. The task can take several months, and it guides the whole process of IRI script produc- tion. A new design document is created for each subject and each level of instruction. Issues s Is the series an adaptation of a previous series from another country? If so, the pedagogical basis of the design document may already be established, but the local characteristics must still be outlined. s How many programs will be broadcast per week, and for how many weeks? This will determine how many episodes are in the design document; the de- cision has implications for cost. What Is Involved in Scriptwriting? The task of scriptwriting is both technical and creative. If the design document is detailed, the job of the scriptwriter is to bring the educational activities to life by using characters and settings that enliven the instructional strategies. The scriptwriting team writes the first draft and submits it to the IRI specialists and subject matter specialists for comment. Personnel. A team of trained local scriptwriters writes or adapts the IRI scripts. It is useful for scriptwriters to have classroom experience so that they can see the connec- tions between the entertaining aspects and the pedagogical aspects of writing radio scripts. As for how many writers are needed, on average a scriptwriter can write one or 80 DESIGNING IRI two scripts per week. This estimate takes into account revision of scripts after they have been reviewed and tested in the classroom, as well as visits to classrooms. In addition, an IRI scriptwriting consultant is usually hired to review the scripts to ensure they are sound. This is often the responsibility of another staff member or of an external consult- ant involved in the design of the program. Activities. Activities generally include several one- or two-week workshops on scriptwriting for each level of instruction and the support of an IRI scriptwriting specialist to provide feedback on scripts. Issues s What should be the composition of the scriptwriting team in terms of local expertise and skills? s How quickly do scripts need to be produced? (A scriptwriting team can be- gin a new level after one level is completed.) s What level of scriptwriting support is desirable? (More is better. Scriptwriters have the power to make or break the success of the series.) What Is Formative Evaluation? Formative evaluation is an investigative system that determines whether individual IRI programs, and the entire IRI series, are achieving the desired learning goals (see figure 6). The intent of the system is to identify and make needed script revisions rather than Figure 6 Formative Evaluation IRI Team FORMATIVE EVALUATION FEEDBACK Classroom 1 Classroom 2 Classroom 3 Classroom 4 Classroom 5 81 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION to evaluate project success. A researcher guides formative evaluation, but it is important that scriptwriters and others also have the opportunity to observe programs in their intended setting. Observers are sometimes also scriptwriters or teachers. Formative evaluation looks at the following: s Technical quality of scripts s Student learning gains s Attainment of educational goals s Demonstration of good classroom practice by the teacher or facilitator. Personnel. Formative evaluation is usually managed internally by project staff (from either the ministry of education or a technical assistance agency) and is intended to inform the development of the program. It includes a key researcher who supervises a group of observers/researchers and local evaluators. Sometimes key partnership organizations or district offices of the ministry of edu- cation manage aspects of formative evaluation in places that are far away from the project development site. These organizations may use their local staffs to conduct observations and testing in local schools. This arrangement is often more economical than bringing in outside staff, and it can provide a more thorough understanding of program success. Another model for formative evaluation is a contract with a local research group that works with the IRI team. This too can work well as long as the goals for formative evaluation are not confused with those for the evaluation of the whole project. Activities. Formative evaluation activities include initial training, regular observations in classrooms, and assessment of individual learning gains. Issues s What model of formative evaluation is desirable--completely in-house, shared among stakeholders, or contracted out? s How will baseline data be collected or pre-testing conducted, and by whom? s How much formative evaluation will take place--for every program, for every five or so programs, or for every program in the beginning, decreasing to fewer? s What types of assessment of learning gains will be administered, and how will evaluators check for progress on other educational objectives such as gender equity and development of other skills? 82 DESIGNING IRI Further information on formative evaluation may be found in Bosch and Miranda (2001). Appendix B contains a description of how the formative evaluation process contributed to IRI development in several Portuguese-speaking African countries. Production This is the stage at which the content and strategies developed by program developers and scriptwriters are transformed into a draft version of the product that could eventually be delivered to the learners. It is a highly iterative process, especially in the beginning. What Are the Options for Audio Production? IRI audio programs are always produced locally. Tasks include the audio production of the scripts, including songs and sound effects; selection and coordination of the actors (with others involved in the IRI project); and reproduction of CDs or tapes for distribu- tion. A local production house is often contracted to coordinate all these activities. Experience with using a government studio for production and management of produc- tion activities has generally been disappointing. Poor audio quality can negatively affect the learning experience and cause frustration for the teacher.3 Production may also involve adaptations for different regions of the country. In these cases, a character, songs, and even an entire segment of each program might be replaced by locally more relevant equivalents. Although this adds a level of complexity, it is one solution to the problems entailed in scaling-up IRI programs to a national audience. When projecting the financing for production, it should be kept in mind that forma- tive evaluation leads to the production of 50 to 100 percent more tapes than would otherwise be required, because of revisions of programs after they have been tested. The revisions result in additional production costs--sometimes 50 to 75 percent more. These costs can be reduced through the use of digital editing, which makes revisions much easier for the producer. 3 One of the greatest challenges in the production phase is balancing the desire to include interesting content with the need for simplicity in the transmission of that content. It is very easy for an IRI program to err on the side of the complexity of the sounds being transmitted into the classroom, where there are already competing sounds. 83 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Personnel. A key producer--usually, the manager of the production company that is contracted to do the work--is needed to coordinate all production activities. He or she oversees the presenters, musicians, singers, actors, editors, and mixers who work on the production of the radio programs. All these costs are generally rolled into the costs of the contract and of production. If an adaptation is needed for a certain part of the country, a representative from that region should oversee the work. Activities. Activities include IRI production training to enable the producers to under- stand the characteristics of sound effects, pauses, and so on. If a new digital studio is established, training in the use of digital editing equipment may be needed. Issues s Will the production use a digital or analog studio? (It is often more economi- cal to include a digital studio in the budget than to pay for the added costs of revising programs through analog editing.) s Will the budget pay by program or by series? s If CDs or tapes are used as one means of delivery, is reproduction of CDs and audiocassettes included in the design document? s If local adaptations are required, do the budget and project plan reflect these costs? What Are the Options for Print Production? Supplementary print materials--sometimes simply the national textbooks and teacher- training supplements--support the lessons and the training that takes place, so that students and teachers can make sense of and optimize the programs. Written materials can be helpful but do not have to be extensive. Personnel. If the materials include content that goes beyond that found in national textbooks, the guides and materials will have to be written, laid out, and printed. These steps follow traditional patterns of printing and distribution. The same people who oversee the creation of the IRI program usually create the guides, which follow the design document. Activities. Activities include writing, layout, and printing. The written materials are evaluated along with the audio programs. 84 DESIGNING IRI Issues s Is it possible for scriptwriters to create written materials concurrently with the audio programs so that when the audio program is completed, a member of the IRI team can review and compile the written segments? s What printing options are available locally? s Will the written materials be delivered with national textbooks or as part of national teacher training? What Are the Delivery Options? Delivery of IRI programs takes place through radio broadcasts, audiocassette tapes, CDs, or a combination of these. Radio broadcasting can be carried out through a na- tional broadcast using relay systems, over shortwave, or over FM radio in local areas. Each option comes with recurrent costs, some of which can be programmed into a continuing ministry budget. The alternatives need to be carefully analyzed in each coun- try, as no two countries will have the same set of advantages and obstacles. Regional digital broadcast is just becoming possible in Africa. In many countries, private radio stations are relatively new and have not yet developed into networks, but they may soon offer a viable alternative to public radio or foreign-based dissemination. Liberalization of national radio policy usually leads to more broadcast options. Issues s What broadcast capacity exists? What does it cost? What is its quality? s Is there a government policy on educational broadcasting? s Are there inexpensive options, free channels, or blocks of time for educa- tional or social programs? s How prevalent are radios themselves across the country? s How will the recurrent costs of broadcasting and batteries be covered? Marketing Education planners often overlook the importance of marketing. Yet thought needs to be given to how best to attract potential audience members and foster effective partici- 85 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Whatever the broadcast strategy, children in the most remote parts of the country must be able to hear lessons clearly. pation. It is, of course, preferable for people to be drawn to the program rather than be obliged to participate. Creative market- ing can maximize willing participation. Marketing can make use of radio spots, promotion of the songs and content featured in the programs, development of ancil- lary materials that support community involvement, and games and competition among schools to motivate students and Marketing should inspire enthusiasm build program visibility. These ef- among potential supporters, including forts can be conducive to parents and government officials. informing and inspiring enthusi- 86 DESIGNING IRI asm among potential supporters in the community and among parents, community- based organizations, funding agencies, government officials, and shadow participants. Issues s How will IRI programs be marketed? s What organizations will be involved in marketing? s How will market research be conducted to determine whether programs are attractive? Summative Evaluation External evaluation is a critical step in understanding the cost-effectiveness of an IRI program, and it helps education decision-makers decide whether to continue, expand, or curtail a given program. Summative evaluation measures the ability of a project to reach its goals. It is con- ducted by an external agency that is generally contracted at the beginning of the project and that conducts pre- and post-tests of student achievement and other factors. Use of an external entity usually produces a more balanced result which provides helpful in- sights into aspects of the program that could be improved. The summative evaluation often has a strong qualitative component. Personnel. Summative evaluation is carried out by an external research organization or set of consultants. Activities. Activities generally take place at set times during the project to determine the impact of the project on students. Designing for Sustainability Once the design document takes shape, it is useful to review how well it will contribute to sustainability. As noted in chapters 1 and 2, sustainability does not just mean the 87 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION ability of an IRI program to continue to function after the end of the initial development phase. It is therefore important to focus on the medium-term sustainability of an IRI program. For example, will the program be able to make the transition from an experi- mental phase supported by technical assistance to a stage in which the IRI program is simply part of the regular education program? To what extent can this transition be enhanced by planning for it in the development phase? Another way of looking at the issue of medium-term sustainability is to imagine the situation at the end of a successful IRI program development process and to review the structures that are to be responsible for various aspects of the ongoing IRI program, using the following criteria: s All critical responsibilities are assigned to a structure. s The structures see their ongoing support for IRI as being in their interest. s The resources required for ongoing support are available. As is generally the case in the development process, the best way to be sure that a criterion is satisfied is to observe whether a task has been completed, rather than simply to assess the ability of the structure to carry out a task. This approach leads to the inclu- sion in the design document of "exit strategy" actions which ensure that the system is ready for, or has successfully completed, the transition. Following are some ideas on medium- term sustainability, organized according to the components of the design document: s Determination of the profile of the learners. Since IRI improves the quality of education, a successful program may enhance the linguistic achievement and existing knowledge that children bring to the classroom. Initially, the IRI programs are calibrated to the needs of children who have not been exposed to such programs. As the development phase draws to a close, most children in the upper grades of a primary school IRI program will have suc- cessfully completed several years of IRI. It will then be necessary to develop a new profile. s Creation of the design document. The plan can explicitly set targets for transfer of critical responsibilities to "ongoing phase" status and can include moni- toring of the transfer. A transfer is not complete until no support from technical expertise or funding remains that will be terminated when the development phase is finished. s Scriptwriting. Because IRI programs evolve in response to the need for new content, improvements in student achievement, and better understanding of 88 DESIGNING IRI specific unmet needs, there will be a need for program revisions. Scriptwriting capabilities will be required in the ongoing phase. s Formative evaluation. This function informs the ongoing evolution of the IRI program, and it must be successfully transferred before the end of the development phase. s Production. There are several options for production in the ongoing phase, depending on the capacity of existing structures to participate in IRI pro- gram development. The initial feasibility analysis should indicate to what extent outsourcing is appropriate for a given context. In the ongoing phase, it is advisable to continue with whatever strategy worked well in the pro- gram development phase. For example, a shift from in-house ministry production heavily dependent on technical assistance to outsourcing to the private sector involves considerable risk. If private sector outsourcing is the vision for the ongoing phase, that approach should be built into the design document for the development phase. Outsourcing requires contracts, and the contracting mechanism in the ongoing phase is usually different, and perhaps more cumbersome, than in the program development phase. It is important to test the new mechanism before the end of the development phase. s Marketing. Marketing will remain important in the ongoing phase, particu- larly for new teachers and students. One of the most effective indirect marketing outcomes during the ongoing phase is when the children's behavior outside the classroom begins to reflect their participation in an IRI program (singing songs, telling stories, repeating IRI health and hygiene counsel to siblings and parents, and so on). s Summative evaluation. The need for evaluation does not end with the devel- opment phase; evaluation should be repeated periodically. As with production, summative evaluation often involves outsourcing. If, during the development phase, an outsourcing mechanism is used, it is important to ensure that an equally effective mechanism is available in the ongoing phase. After reviewing the design document to ensure that it includes a good exit strategy, it is helpful to review the statement of work, or the terms of reference, for any external entity providing technical assistance to the IRI program development process. For ex- ample, is the statement of work couched in terms of activities, or of results? Sustainability is enhanced when entities providing technical assistance are asked to produce a result, as opposed to simply ensuring that an activity occurs. The activity-based approach looks at what goes into a process (training, broadcasts, supplies, and the like), whereas the 89 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION results-based approach looks more at behavioral changes stemming from the implemen- tation of certain activities, as indicated by student achievement, teaching practices, parental perceptions, and so on. When the expected results are clearly stated, a firm that is bid- ding on a technical assistance package can show how its plan will lead to those results. The firm will also need to identify any critical assumptions that are part of its plan. A Final Note The information provided in this toolkit is intended to give the reader the necessary grounding to understand, explore, and eventually launch development of an IRI initia- tive. When considering the potential of an IRI program, it is important to keep in mind that the need for IRI is specific to each context and that there is no single package of inputs that is appropriate to every situation. The toolkit, however, does lay out a gen- eral path of inquiry, as well as the critical intermediate results that must be produced for an IRI program to be successful. 90 Appendix the A haps es. page) (ECD). m. national per, and next with eforr to junctur on occur policy key developed developed and an education). pilot in Present dissemination not at needs. developed to radio costs omfr (continued model compatible model educational does changes both rent model onmental 1974 to points Original (health). Strategy educational Original Uses audiocassettes. Decentralized matches Recur issue. New (envir ransitionT ogram pr due administration Key s s s s s s s s Africa, and Asia, Evaluation UNICEF Caribbean, echnicalT assistance EDC EDC EDC EDC the and Local institution PER PER PER/EDC America , Latin in Funding source USAID USAID, UNICEF Plan International USAID 1­5 3­4 4­5 centers, Programs eschools IRI Grade/ audience Grades Grades ECD pr Grades by CARIBBEAN THE childhood onmental Covered Subject AND Math Health Early development (ECD). Envir education Subjects A­1 AMERICA Rica year ableT TIN Country and LA Bolivia 1998 1992 1994 Costa 1991 91 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION for n- radio year), e cost- oadcast. per sustained and gover entir advan- students br with done. be ess to be eachedr been pr airtime. take developed to developed 8,000 eady developed 14,000 ogramming). has ecreater pr needs. between than alr found private sustain to being cost-sharing. model compatible with to not model math). population ogram years. and rather work continues oximately pr of model 16 eements points fective. Piloted and Original (integrated Small (appr but for Strategy educational Agr ment stations Attempted series, tage Strategy ef Pilot. New eading.r Original (mental Attempted Key s s s s s s s s s s s Evaluation University University echnicalT assistance EDC Local institution AED FONHEP ANCEVA Funding source USAID USAID USAID USAID USAID USAID 1­6 1­2 2­4 1­3 grades ee 220 Grade/ audience Grades 128,000+ Grades Thr in primary schools Grades Grades for formal math civics, for ograms Subject English Integrated pr nonformal; math Math Spanish, Reading, math Math (continued) 1981, 1992, 1990, A­1 year 1996, ableT Salvador Country and 1997 Dominican Republic El Guatemala Haiti Honduras 1987 92 APPENDIX A: SUMMARY TABLE OF IRI PROGRAMS page) has next eements with egionalr basic ganiza- expand million on agr or 3 local adult to improve countries. ogram developed 1992 technical to and developed. pr of and than of education). needs ogram. other the e pr means given (continued model compatible ten in basic government, TODOS a model mor nationwide. stations, as gar . 1995, en amount points Original (adult Strategy educational among radio tions. Continuation EDUCA education Used kinder quality Original Adapted Since eachedr childr Small assistance institution. Key s s s s s s s s s d Evaluation Christian Children' Fund Stanfor University d echnicalT assistance EDC EDC Stanfor University None Short-term international consultant Local institution ANCEVA EDUCA- TODOS FEREMA CENAMEC BRAC Bank orld Funding source USAID USAID USAID USAID W (partial) None 1­6 7­9 and garten 1­4 1­3 en in Grade/ audience Grades Grades Kinder childr teachers Grades Grades Girls secondary school school basic garten Subject Adult education Middle Kinder Math Math English (continued) 1974, 1991, A­1 year ableT Country and 1992 1999 2002 Nicaragua enezuelaV ASIA Bangladesh 93 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION series than on e ECD ent educa- training first by inten- (formal mor multigrade teacher with ecurr(r hill-tribe adapted Bolivia audiocas- teacher ogram on eachr emphasis of e to developed time changed on identified pr the developed; will 2005. of mor nutrition. air methods first model nonformal in for spots focused compatible with and developed. model to oadcast series focuses d br ooms; needs. audience points har woT million that in teachers; tionally classr 4 Adaptation mode, gender Hands-on training Pilot. Original (science). Strategy tional Payment costs). Main schools schools). Distribution (radio sette). Key s s s s s s s s s ganiza- Evaluation Local Indian or tion CERID Unknown University echnicalT assistance EDC EDC EDC AED EDC Short-term international consultant Nepal for Local institution Radio Distance Education Center Center Educational echnologyT Bank orld Funding source USAID UNICEF UNICEF USAID USAID W and 1­2 3­5 4­6 1­2 4­5 and grade d Grade/ audience Grades and (multigrade) Early childhood development eachersT thir fifth students Grades Grades Grades and training, health, solving training science, studies, egiver English Subject eacherT oblem math, social English Car training, nutrition, pr eacherT in math English Science Math (continued) A­1 1992, 1980, year 2004, New 1986, ableT Country and India Nepal 1996 1999 Pakistan Papua Guinea Thailand 94 APPENDIX A: SUMMARY TABLE OF IRI PROGRAMS ent de, page...) 10,000 singles educa- ecurrr erV next all with for on . countries. Cape Mozambique. pay eachr country' developed. Principe, and institutionalized. to other eachr country over levies model in compatible to and (continued fort in institutionalized. ef station. needs. yet oméT points oadcast Pilot schools Nationwide, Br radio Original Adapted Strategy tional Innovative costs. Not Intended São Guinea-Bissau, Key s s s s s s s s s Center early Applied early Evaluation n.a. (in stages) U.S. for Linguistics n.a. (in stages) Unknown and the echnicalT assistance AED EDC EDC AED AED EDC UNESCO government of Netherlands of Local institution INRAP Ministry Education and the Funding source USAID USAID USAID USAID USAID USAID UNESCO government of Netherlands 1 1­6 1­3 1­3 2­4 1­4 Grade/ audience Grade Somali efugeesr Grades Levels Grades Grades Grades subjects math, literacy ench, Subject English Primary Fr others English English Math, Math, Portuguese (continued) A­1 1999, 1980, 1987, 2001, 1992, year ableT Country and AFRICA Ethiopia 1999 2002 Guinea Kenya Lesotho Nigeria Portuguese­ speaking countries 95 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION to Educa- South ebuildr students school. of . orphans for e Educational, English with tot to l'Enseignement southern be labor for AIDS Centr de for in will victims ch Nations dramatically ef been 1 is for child ger of Resear Haïtienne United system never grade lar adapted compatible ms it needs.s a audience eated audience CERID, cr for points oft have math. Fondation, UNESCO, communities. Model make Africa' Par education Sudan; who Series worst Developed and Primary in Education; Radio; Key s s s s s FONHEP por hematics Mat Educación; Educación la and de Evaluation University of Witwaters- rand Consultant para ma . Science ogra Real Pr Fundación PER, and orld Upgrading echnicalT oductions ust;rT Development assistance EDC W Pr EDC EDC EDC for FEREMA, national All; of Center and for Education Inter for Systems Local institution OLSET Educational Broadcasting Service Ministry Education CENAMEC, Education Agency ning U.S. Lear and Didáctica; TODOS, Dept de Open, USAID, Norway Labor EDUCA Funding source USAID government of USAID U.S. of USAID OLSET Fund;s Nacional o Center; en' 1­3 1­3 1­4 1­7 Centr Childr en) Pédagogique; Nations CENADI, Development Grade/ audience Levels Grades Grades (30,000 childr Grades d'Action et United, 2003. Education a che data, as language subjects English math, Development; EDC, UNICEF Recher Center de Subject English second Integrated primary Swahili, English; integrated Educational for Development; Organization; National (continued) , and Development 2002, Cultural A­1 Africa 2005, 1999, Academy Institut, and applicable. year Not AED,: Innovation INRAP Education: ableT Country and South 1992 Sudan anzaniaT Zambia n.a. Note tional Privé; Scientific, Source 96 Appendix B IRI Vignettes from Sub-Saharan Africa T he brief narratives collected in this appendix provide a "snapshot" of experi- ences with IRI programs in Sub-Saharan Africa. They are not intended to be exhaustive or comprehensive; rather, they represent an overview from the per- spective of teachers, administrators, coordinators, and others in selected countries. Some of the vignettes describe IRI programs that have been in operation for over a decade; others tell how interest is emerging in countries that have had no previous experience with IRI and how some countries are preparing to scale up their operations. Finally, the vignettes illustrate how governments are adopting IRI to address educational challenges in their countries and some of the lessons learned. Ethiopia Remote Jigjiga Boasts the Finest Digital Recording Studio in Ethiopia Radio is a common medium for communication throughout the Horn of Africa, and many families own radios. To overcome the obstacles of educating pastoralist children who are often on the move, Edu- cation Development Center (EDC) launched in August 2001 the Interactive Radio Instruction for Somalis Jigjiga in eastern Ethiopia 97 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION (IRIS) Project. The IRIS Project, funded by USAID, grew out of a desire to use the medium of radio to address the problems of access to and quality of basic education. The IRIS Project's goal is to build the capacity of Somali educators and technicians to produce and broadcast interactive radio instruction (IRI) programs that deliver curriculum in reading, basic math, and life skills to primary-school-age children. This will take place in Qur'anic or formal schoolrooms, Actors recording for nonformal basic education centers, refugee camps, interactive radio instruction. or any setting that includes a teacher, a group of stu- dents, and a radio. Teachers are trained to facilitate the IRI broadcasts. Somali poetry will be contributed to increase children's interest and connect them with their culture's language and art form. In addition, after staff failed to find any existing Somali books for children, two basic reading books have been written to enhance the program. New State of the Art Digital Recording Studio The center of commerce for Somalis in eastern Ethiopia is a remote town called Jigjiga. Ironically, in its isolation, Jigjiga now boasts the finest digital recording studio in Ethio- pia. Built to support the IRIS Program, the studio is housed in the government's Capacity Building Bureau of the Regional Educational Bureau building. The head producer is Halimo Hassan. The program's success has come to rely heavily on the newly acquired expertise of local employees like Halimo. Halimo recently commented, Halimo Hassan. Ethiopia's first "I thought I was going to be a secretary my whole digital recording producer. life. But now as the first digital producer in Ethio- pia, I'm able to help the children of my region." This optimism speaks for the promise of the IRIS Program and people like Halimo to significantly raise the educational standards of the children in eastern Ethiopia. ©2003 Education Development Center, Inc. All rights reserved. 98 APPENDIX B: IRI VIGNETTES FROM SUB-SAHARAN AFRICA Guinea Sous le Fromager (Under the Kapok Tree) encourages, motivates, and provides guid- ance for educators. It promotes an atmosphere where children feel encouraged and challenged . . . The fourth graders in Mrs. Koria Kouyaté's class at the small Petel Lelouma Pri- mary School in the Fouta Djallon highlands have come to school very early this morning. So have the first, second, and third graders. Why are all the kids in school [so early] today? "Because it's a radio show day," says Kouyaté, "and the kids don't want to miss it." For an hour and a half each week, children in grades 1­6 throughout Guinea listen to a USAID-sponsored Interactive Radio Show called "Under the Kapok Tree," given wide praise by school directors, parents, teachers, and, most impor- tantly, by the kids themselves. Says Kouyaté, "The radio program is so effective because children learn by playing. They don't even realize it, but they are learning a lot." Developed by nongovernmental organization Education Development Center (EDC) in close collaboration with INRAP, the Guinean government's National Institute for Pedagogical Research and Action, the radio show uses a "multichan- nel approach" in which audio, visual, and tactile-kinesthetic strategies are used to teach children language, mathematics, science, community health, early child- hood development, and other pertinent subjects. During the broadcast, teachers typically follow the instructions of the "radio teachers," guiding children to sing songs, answer questions, manipulate objects, draw pictures, move physically, and work in small groups to solve problems. The broadcasts, done entirely in French, began during the 1999­2000 school year for first and second graders. As of the 2000­01 school year, the broadcasts have been expanded to cover grades 1­6. Guinean elementary school students nationwide now receive 90 minutes of broadcasts per week during 22 weeks of the school year. An estimated 20,000 teachers and 900,000 students are benefit- ing from the program nationally. "Parents were skeptical at first," says PTA President El Hadj Diallo in the small town of Djountou, "but now we see how the children appreciate it, and they're learning a lot. Our kids are anxious to get to school to listen to the radio show." Says Nandiba Koba, Djountou Regional Education Inspector. "The method 99 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION of learning through play is great--we really like it. Before the program, we had a lot of absenteeism. Now all the kids are in school--they're motivated, their par- ents are motivated, and it's going really well. Even businessmen are forming learning groups--they set hours to get together to listen to the show." Says EDC Teacher Training Advisor Mark Lynd, "We always saw the Interac- tive Radio Instruction program as a teacher training tool, not only to introduce teachers to new pedagogical approaches, but also to teach content--good French, grammar, mathematics, etc. Guiding the children through these activities helps teachers understand the subject matter better themselves." Says Sow Abdoul Salam, principal of the grade school in the village of Tountouroun, "Teachers are learn- ing to be more active, and not just stand in front of the class, which was what they did before. They're getting new ideas. With this project, each teacher is learning to become a true artisan of his own training." Although teachers are benefiting from the program, the biggest fans continue to be Guinean children. Says Thelma Khelghati, EDC Chief of Party, "The Pro- gram is like the Sesame Street of Guinea. We all know the Sesame Street song, and you know it was the first program of its kind in our culture. Sous le Fromager is like that." Simply watching young Guinean children come to school early, and seeing them clapping and singing in Guinean classrooms on "radio show days" is a testimony in and of itself. Project staff report that there are several groups of secondary learners: (a) a group of field workers at a monastery in Coyah that take their radios to the field with them every morning so they can learn French from the IRI programs while they work in their field; (b) taxi drivers in Conakry that have told project members they always turn the radio on and tune in to the IRI programs every morning to learn French while driving about the city; (c) a school in a refugee camp in the southeast that rigged its own antenna to be able to listen to the programs and teach French to Anglophone refugees. Source: Excerpted from USAID Website, http://www.usaid.gov/gn/education/news/010701_interactiveradio/ index.htm. 100 APPENDIX B: IRI VIGNETTES FROM SUB-SAHARAN AFRICA Lesotho All schooling after third grade in Lesotho is in English. Prior to the mid-1980s, many children did not continue past fourth grade, dropped out while struggling to complete the grade, or failed the grade because they had not learned enough English. After being exposed to the Radio Language Arts project in Kenya, local educators from Lesotho's Ministry of Education decided to implement a similar program in Lesotho. Let's Learn English was developed in Lesotho between 1985 and 1991 and went on the air in 1987. It consists of 391 half-hour lessons broadcast to all first, second, and third grade classrooms in the country. It was financially and technically supported by USAID and managed by the Instructional Materials Resource Center (now part of the National Curriculum Development Center). The Lesotho Distance Teaching Center pro- duced the radio broadcasts. An estimated 20,000 learners listen to the program each year. Although many people raise the issue of sustainability, Lesotho has succeeded in having a long-running program. Several factors have contributed to this longevity. First, the government has a clear policy in support of open and distance learning for this primarily rural and impoverished country, and there is political will and commitment. The country is small and stable, with well-established, although resource-lean, institu- tions in the education sector. IRI is well integrated into the national curriculum, and the program is firmly entrenched in government institutions. Through cooperation with Radio Lesotho, the national broadcasting authority, broadcasts are aired daily, with generally good coverage and reception. The government has also imposed a tax to assist in meeting recurrent costs. Discussions have recently taken place concerning two key areas: the updating and revision of the scripts, and further training and support of teachers to enhance their role in IRI. Source: Adapted from Dock and Helwig (1999). 101 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Portuguese-Speaking African Countries (PALOP) The IRI-PALOP project "Quality Improvement of Basic Education through the Use of Radio in the PALOP Countries," was designed for five Portuguese-speaking countries in Africa: Angola, Cape Verde, Guinea-Bissau, Mozambique, and São Tomé and Principe. The aim of the program was to provide instruction in mathematics and Portuguese to in-school primary children. The program was financially supported by and managed from the UNESCO Regional Office in Harare; during the implementation period, June 1997­October 2001, it was coordinated from a project office in Cape Verde. Materials were designed to be generic. Country teams were involved in the planning stage, when the master plans for producing radio lessons were developed. The lessons produced were pilot-tested in each of the five countries, and feedback was provided to the subre- gional team in Cape Verde for use in making revisions. The program had a strong evaluation component: valuable lessons were learned, and necessary adjustments were made to the materials, radio scripts, and training. IRI/PALOP was initiated in response to three common problems among primary school teachers in the beneficiary countries: (a) poor command of Portuguese, the lan- guage of instruction, leading to generally poor learning outcomes in basic education; (b) inadequate knowledge of mathematics subject matter and poor mastery of relevant teaching skills, resulting in failure of students to acquire basic mathematics skills; and (c) lack, for the most part, of teaching styles conducive to active learning, resulting in overem- phasis of memorized knowledge as opposed to attainment of usable skills. The project was accordingly intended to improve the quality and effectiveness of basic education in the two core subjects of the general curriculum of the beneficiary countries: Portuguese as a second language, and mathematics. It was planned in such a way that the development of radio lessons for Portuguese as a second language for grades 1 to 4 would follow after some experience had been gained by producing radio lessons in mathematics. The overall development strategy included a stage of concentration on research, de- velopment testing, and validation of interactive radio instruction programs for use in all five PALOP countries. This was to be followed by a national implementation stage during which the radio instruction material produced and validated was to be used in each of the five countries. A midterm evaluation of the IRI/PALOP project that was undertaken in April 2000 confirmed the validity of the objectives of the project within the context of the educa- 102 APPENDIX B: IRI VIGNETTES FROM SUB-SAHARAN AFRICA tion systems of the five beneficiary countries, given the major problems faced. The gen- eral findings from the evaluation were as follows: s The educational conditions of beneficiary countries remained similar to the findings at the time the project document was developed. The improvement of the quality of education was still the objective of ministries of education in all five countries. The long-term and specific objectives of the project were therefore still relevant. s The general objective of developing Portuguese-language and mathematics series was still valid, given the educational needs of the PALOP countries. s All five countries were concerned with the quality of education. In most cases, the education systems performed poorly, and in some instances the problem was exacerbated by poor working conditions and the predominance of untrained teachers. Alternative strategies such as pre-service and in-ser- vice teacher training were found to be very costly. Given the circumstances, IRI was viewed as a viable and valid tool. In addition, IRI promotes politi- cal, social, and educational motivation and results in higher learning performance, as shown by the testing already carried out. Expansion costs in each of the five countries are expected to be significantly lower once a qual- ity product has been developed. s Capacity-building activities were mainly concentrated in Cape Verde and were carried out through the use of consultancies. Capacity-building activi- ties for teams in the other countries were found to be limited. Training activities provided to the team in Cape Verde and the teams in the other countries needed to be strengthened. s The training of teachers and observers was found to be inadequate for pro- moting creative and constructive interaction between teachers, pupils, and the radio characters. Observation showed that teachers remain passive and do not take advantage of the methodological richness that IRI offers. s National teams were established in the five countries and participated actively in the activities of the project. The teams, however, were isolated and did not interact among themselves to exchange information and evaluate programs. s There was a significant delay in the implementation of the project, particu- larly in the production of the Portuguese and mathematics series. In some cases, as in Guinea-Bissau, delays in testing were due to factors beyond the control of the project. In other cases, delays were attributable to inadequate planning, given the specific conditions of the countries. 103 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION s In general, people seemed to be happy with the program, particularly where it was broadcast. Parents and ministry of education officials who were interviewed expressed their interest and that of the communities involved in the program. s There were reported cases of individual and group listening to the program, which was said to be attractive. Pupils interviewed clearly indicated that they liked the program. s There was a feeling that a satisfactory compromise for all the countries with respect to curriculum changes introduced by the project can be reached through more consistent involvement of technicians from all five countries in instruc- tional design and production activities. The main issue that became apparent was the difficulty of sustaining the IRI pro- gram in all five countries after UNESCO funding ended. Even though ministries of education and national institutions were involved from the beginning, it seems that broadcasting costs were making it difficult for the countries to continue the program. Source: Adapted from the IRI/PALOP Project Narrative Report, UNESCO Harare, July 2002, courtesy of T. Dos Santos. South Africa The South African Radio Learning Programme (RLP) is the main project of the Open Learning Systems Education Trust (OLSET), an education NGO. OLSET's three series of core curriculum IRI programs, English in Action (EIA), were developed as an open and distance learning strategy for supporting education transformation in the new de- mocracy. Of the country's 11 official languages, English is widely considered the lingua franca and remains the predominant medium of instruction and examination. The RLP was part of a strategy for enhancing educational quality and equity after the democratic transformation in 1994. The program was particularly geared to underresourced classrooms. The overriding purpose of the EIA programs for junior primary grades was to provide sustained teacher support for the new pedagogical em- phasis on learner participation in lessons and for the creation of more democratic, fun learning environments for children that would accommodate learners with varied abili- ties. The IRI programs served as a vehicle for curriculum change and the shift to a broadly constructivist pedagogy. This warranted a de-emphasis on content-driven teaching 104 APPENDIX B: IRI VIGNETTES FROM SUB-SAHARAN AFRICA and a greater focus on competencies in all learning areas, as well as critical thinking, life orientation and skills, health education, and environmental awareness, among other curriculum outcomes. These changes necessitated great sensitivity to how teachers would perceive their role. EIA consists of three series of between 130 and 180 lessons for daily classroom use for the teaching of English as a second language. Each lesson lasts 30 minutes and is supported by integrated print and visual media, including learner activity books, teach- ers' guides/manuals, posters, alphabet friezes, comic books, and readers. The radio lessons and learner and teacher support materials are produced by OLSET. Five public broad- cast stations air the EIA programs daily, offering near-national access to schools. Where broadcast reception is inadequate because of terrain or other obstacles, OLSET pro- vides audiotapes to schools. The program is currently working with an estimated 48,000 teachers and 1,600,000 learners in eight provinces, primarily in the historically disad- vantaged communities. Teachers in participating schools attend workshops conducted jointly with provin- cial government education curriculum specialists. This partnership with government education services and public broadcasters is critical to program sustainability and to longer-term IRI mainstreaming objectives. The IRI programs remain dependent on sup- port from international donors, particularly the DfID. Initial support for the development, implementation, and evaluation of the programs was provided by USAID and NORAD. Eventually, the source of funding will need to switch to provincial government budgets. Letsatsing Primary School caters to the children of local miners in the Gauteng re- gion. Veliswa Nqarana, a grade 3 teacher, states: My learners are able to do projects working as teams. The program encourages them to enquire about their environment, and as a result, grade 3 learners con- duct their own research on various topics--for example, careers, languages spoken within their communities, sharing information both inside and outside the class- room, bringing into the classroom interesting books, materials, and articles. The lessons are well integrated, addressing language across the curriculum. The regional coordinator of Gauteng B region, Teboho Thebe-Moleko, describes some of the disadvantaged schools in the rural areas: During the rainy season accessibility to this school is difficult for the majority of learners, most of whom live on the other side of the river. . . . Power supply to Monaghan Farm School is not very reliable . . . this school caters for grades one to ten learners from the surrounding informal settlements on the borders of Pretoria 105 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION and Johannesburg. . . . The building in which the foundation's phase (primary level) is hosted has no power supply. Teachers thus have to use long extension cords plugged into the nearest powered block to the classes, a very involved pro- cess which does not allow the teachers to implement the program regularly. Thobeka Mirrian Sima, a regional coordinator in Port Shepstone, comments: You teach ten learners in one class but end up teaching hundreds that are outside the school in the community. I am proud of being an OLSET coordinator because I am able to reach the most neglected schools in my area, the poorest of the poor. English in Action is the best program that changes the lives of learners. All of the learners in rural areas where there is no English language speaking are now able to communicate in English because of the OLSET program. Sources: Adapted from Naidoo (2004) and Potter and Silva (2002). Zambia I am excited I am so happy, Taonga It's your chance to learn Taonga Market It's your chance to learn Ta-Ta-Ta Taonga Market It's your chance to learn! --Taonga Market Song Learning at Taonga Market, produced by the Educational Broadcasting Services of Zambia's Ministry of Education, was launched in 2000. The program, supported by USAID with technical assistance from the Education Development Center, is aimed at primary-age children in communities without access to regular schooling. It teaches children literacy, numeracy, and life skills, with an emphasis on HIV/AIDS. There are 100 broadcast lessons of 30 minutes duration each at the grade 1, 2, and 3 levels. Appropriate policy frameworks were put in place to support the initiative and ensure its sustainability. A considerable number of secondary learners from the community also 106 APPENDIX B: IRI VIGNETTES FROM SUB-SAHARAN AFRICA benefit from the program as they gather to listen to the radio lessons, which are often held in an "under a tree" school. The program is designed to be community based. It depends on volunteer mentors, who serve as learning facilitators, and on the provision by parents and the local commu- nity of a learning space and minimal resources--radio, batteries, chalkboard, and local resource materials. According to the Mentor's Guide, "The administration of the Taonga Learning Centres illustrates most clearly how the partnership between the Ministry of Education, communities, volunteer mentors and learners must operate if children are to learn effectively. Each partner has a vital role to play, but it is the communities who must ultimately manage their centers." Mentors are trained by the Ministry of Education and are supplied with mentor's guides. As described in the guides for grades 1, 2, and 3, "The mentor is the link be- tween the radio and the children. The radio alone won't teach effectively. We rely on the mentor to set up the center before the lesson, to lead every lesson by guiding the learners and managing the activities, and to follow up after the radio lesson has concluded." S. M. Kasanda, permanent secretary in the Ministry of Education, writes, "Government's commitment to achieving Education for All (EFA) cannot, however, be realized even by 2015 if it is to be delivered only though the classroom mode. It is for this reason that the present Interactive Radio Instruction (IRI) launched in 2000 has been tailored to provide a serious alternative to classroom teaching and learning. . . . It is gratifying to note that IRI is not just about literacy and numeracy but also skills (including life skills), values, and attitude formation." Learning time is over We are going home Goodbye, Goodbye, Goodbye Teacher We are going home! Source: Education Development Center (2001). 107 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION 108 Resources Bibliography The word processed describes informally reproduced works that may not be commonly available through libraries. Adkins, D. 1999. "Cost and Finance." In Alan Dock and John Helwig, eds., "Interac- tive Radio Instruction: Impact, Sustainability, and Future Directions." Education and Technology Technical Notes Series 4 (1). World Bank, Washington, D.C. Bakia, Marianne. 2000. "The Cost of Computers in Classrooms: Data from Developing Countries." World Bank, Washington, D.C. Processed. Bates, Jeffrey. 1998. "An Impact Evaluation of Interactive Radio Instruction for Early Childhood Development in Nepal: Bhanyang Chouthari." United Nations Children's Fund, New York. Bosch, Andrea. 1997. "Interactive Radio Instruction: Twenty-three Years of Improving Educational Quality." Education and Technology Technical Notes Series 2 (1). World Bank, Washington, D.C. ------2004. "Sustainability and Interactive Radio Instruction: Why Some Programs Last." In David W. Chapman and Lars O. Mählck, eds., Adapting Technology for School Improvement: A Global Perspective. Paris: International Institute for Educa- tion Planning. United Nations Educational, Scientific, and Cultural Organization (UNESCO). Bosch, Andrea, and Cecilia Crespo Jugando en el Pidi. 1995. "Active Learning, Early Child Development, and Interactive Radio Instruction Supporting Caregivers, Par- ents, and Young Children." LearnTech Case Study 5. Education Development Center and U.S. Agency for International Development, Washington, D.C. 109 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Bosch, Andrea, and Jessica Miranda. 2001. "Formative Evaluation for Interactive Ra- dio Instruction: A Designer's Handbook." Education Development Center, Washington, D.C. Cawthera, Andy. 2000. "Computers in Secondary Schools in Developing Countries: An Analysis of Costs." World Links for Development, World Bank Institute, Washing- ton, D.C. Cobbe, James. 1995. "The Economics of Interactive Instruction: The Case of South Africa." LearnTech Case Study 7. Education Development Center and U.S. Agency for International Development, Washington, D.C. Colletta, Nancy. 1996. "Study of Improvements in the Quality of the Learning Environ- ment for Children 3­6 Years in Bolivia through IRI." LearnTech Project. U.S. Agency for International Development, Washington, D.C. Processed. Corrales, Carleton. 1995. "Adult Basic Education in Honduras: Managing Multiple Channels." LearnTech Case Study 9. Education Development Center and U.S. Agency for International Development, Washington, D.C. Creative Associates International, Inc. 2002. "An Evaluation of the FQEL Education Program in Guinea." U.S. Agency for International Development, Washington, D.C. De Fossard, Esta, Andrea Bosch, and others. 1995. "Interactive Radio Instruction: What It Is, How It Works, and What Is Needed to Get It on the Air." LearnTech Project. Education Development Center and U.S. Agency for International Development, Wash- ington, D.C. Dock, Alan. 1999. "Success and Sustainability." In Alan Dock and John Helwig, eds., "Interactive Radio Instruction: Impact, Sustainability, and Future Directions." Edu- cation and Technology Technical Notes Series 4 (1). World Bank, Washington, D.C. Dock, Alan, and John Helwig, eds. 1999. "Interactive Radio Instruction: Impact, Sustainability, and Future Directions." Education and Technology Technical Notes Series 4 (1). World Bank, Washington, D.C. Education Development Center. 2001. "USAID Summative Evaluation Report: Taonga Market Interactive Radio Instruction Project." Washington, D.C. Figueredo, Vivian, and Stephen Anzalone. 2002. "Alternative Models for Secondary Education in Developing Countries: Rationale and Realities." USAID Improving Edu- cational Quality Project. U.S. Agency for International Development, Washington, D.C. Fryer, Michelle L. 1995. "Ministry, Union, the Church, and Teachers: Bolivia's Partners in Innovation." LearnTech Case Study 6. Education Development Center and U.S. Agency for International Development, Washington, D.C. Goldstein, E., and Altagracia Diaz de De Jesús. 1995. "Dominican Republic: From the Margins to the Mainstream." In Steve Anzalone, ed., "Multichannel Learning: Con- 110 RESOURCES necting All to Education." U.S. Agency for International Development, Washington, D.C. Guay, Dominique. 2000. "An Analysis of Radio Broadcasting and Reception Options in Haiti." Education Development Center, Washington, D.C. Hartenberger, Lisa, and Andrea Bosch. 1996. "Making Interactive Radio Instruction Even Better for Girls: The Data, the Scripts, and the Potential." U.S. Agency for International Development, Washington, D.C. Helwig, John. 2004. Personal interview. Leigh, Stuart. 1995. "Changing Times in South Africa: Remodeling Interactive Learn- ing." LearnTech Case Study 8. Education Development Center and U.S. Agency for International Development, Washington, D.C. Leigh, Stuart, and Francis Cash. 1999. "Effectiveness and Methodology of Interactive Radio Instruction." In Alan Dock and John Helwig, eds., "Interactive Radio Instruc- tion: Impact, Sustainability, and Future Directions." Education and Technology Technical Notes Series 4 (1). World Bank, Washington, D.C. Lockheed, Marlaine, and Eric Hanushek. 1988. "Improving Educational Efficiency in Developing Countries: What Do We Know?" Compare 18 (1): 21­37. Miranda, Jessica. 2002. "Haiti ED2004 Trip Report for the Dominican Republic." January. Education Development Center, Washington, D.C. Processed. Moulton, Jeanne. 1994. "Interactive Radio Instruction: Broadening the Definition." LearnTech Case Study 1. Education Development Center and U.S. Agency for Inter- national Development, Washington, D.C. Murphy, Paud, Stephen Anzalone, Andrea Bosch, and Jeanne Moulton. 2002. "En- hancing Learning Opportunities in Africa: Distance Education and Information and Communication Technologies for Learning." World Bank, Washington, D.C. Naidoo, Gordon. 2004. "ICTs in Education." Open Learning Systems Education Trust (OLSET), Johannesburg. Olsson, Micael. 1994. "Institutionalizing Radio Science in Papua New Guinea: A Re- sponse to Teacher Demand for Interactive Radio Instruction." LearnTech Case Study 2. Education Development Center and U.S. Agency for International Development, Washington, D.C. Osin, Luis. 1998. "Computers in Education in Developing Countries: Why and How?" Education and Technology Technical Notes Series 3 (1). World Bank, Washington, D.C. Perraton, Hillary, and Charlotte Creed. 2000. "Applying New Technologies and Cost- Effective Delivery Systems in Basic Education." World Education Forum, Education for All 2000 Assessment. United Nations Educational, Scientific, and Cultural Orga- nization (UNESCO). 111 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Potashnik, Michael, and Douglas Adkins. 1996. "Cost Analysis of Information Tech- nology Projects in Education: Experiences from Developing Countries." Education and Technology Technical Notes Series 1 (3). World Bank, Washington, D.C. Potter, Charles, and Adilia S. F. Silva, eds. 2002. "Teachers in Action: Case Studies of Radio Learning in South African Primary Schools." Open Learning Systems Educa- tion Trust (OLSET), Johannesburg. Royer, James M. 2003. "Literacy Enhancement Assistance Program: 2002­2003 Stu- dent Assessment Final Report." University of Massachusetts, Amherst. Searle, Barbara, and Klaus Galda. 1980. "Measurement of the Effect of Radio Math- ematics Lessons on Student Achievement. The Radio Mathematics Project: Sample Lesson Materials." Institute for Mathematical Studies in the Social Sciences, Stanford, Calif. Tilson, T., D. Jamison, M. Fryer, D. Edgerton, P. Godoy-Kain, M. Imhoof, P. Christensen, and T. Roy. 1991. "Sustainability in Four Interactive Radio Projects: Bolivia, Hon- duras, Lesotho, and Papua New Guinea." In Marlaine E. Lockheed, John Middleton, and Greta S. Nettleton, eds., "Educational Technology: Sustainable and Effective Use." PHREE 91/32. Population and Human Resources Department, World Bank, Washington, D.C. UNICEF (United Nations Children's Fund). 2000. "Defining Quality in Education." Working Paper. UNICEF/PD/ED/00/02. Programme Division, Education, UNICEF, New York. van Steenwyk, Ned, et al. 2002. "Los graduados del sexto grado de EDUCATODOS: 1999­2001." Secretaria de Educacion, EDUCATODOS, and U.S. Agency for Inter- national Development, Tegucigalpa. Vargas, German. 1995. "Econauts: Mission Nature. Radio Drama for Environmental Education in Costa Rica." LearnTech Case Study 4. Education Development Center and U.S. Agency for International Development, Washington, D.C. World Bank. 1998. "Latin America and the Caribbean: Education and Technology at the Crossroads. A Discussion Paper." World Bank, Human Development Network, Education and Technology Team, Washington, D.C. Yasin, Katharine, and Yvrose Lubérisse. 1998. "Meeting the Needs of a New Democ- racy: Multichannel Learning and Interactive Radio Instruction in Haiti." Education Development Center, Washington, D.C. Zambia, Ministry of Education. 2004. "Evaluation of Interactive Radio Instruction." Draft. Lusaka. Available on the CD-ROM that accompanies this publication. 112 RESOURCES Online Resources Child Survival Technical Support+ Website, http://www.childsurvival.com/features/book- marks/interactive-radio.cfm. Education Development Center, Inc. Website, http://www.edc.org. Freeplay Foundation Website, http://www.freeplayfoundation.org. "From Skepticism to Enthusiasm: Six Years of Education Reform in Guinea," Interna- tional Education Systems Website, http://ies.edc.org/projects/guineaFQEL.htm. "Interactive Radio--The Biggest Fans Are the Kids," U.S. Agency for International Development Website, .usaid.gov/gn/education/news/010701_interactiveradio. "Interactive Radio Instruction in Haiti: An Enriching Experience," United Nations Educational, Scientific and Cultural Organization (UNESCO) Website, http:// www.unesco.org/education/efa/know_sharing/grassroots_stories/haiti.shtml. "Interactive Radio Instruction in Portuguese-speaking Countries in Africa," United Nations Educational, Cultural and Scientific Organization (UNESCO) Website, http:// www.unesco.co.zw/webpages/html/programmes/specprojects/iripalop.htm. "Interactive Radio: RADECO, Dominican Republic," Literacy on Line Website, http:// www.literacyonline.org/explorer/rad_back.html. Open Learning Systems Education Trust (OLSET) Website, http://www.olset.org.za "South Africa," American Jewish World Service Website, http://www.ajws.org/ sa_projpg.html. 113 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION 114 Index Ability levels, 42 Accents. See Language (of instruction) Access to education, 2, 16, 35 Achievement, assessment of, 15­17, 18, 19, 35, 36­37, 42. See also Educational quality Active learning pedagogy, 1, 2, 3, 8, 27, 41, 74 Actors in IRI programs, 48, 75, 80, 81 Adaptation of IRI programs: for local regions and languages, 71, 81; from other coun- tries, 7, 9, 23, 31, 44­46, 60, 77 Add-on costs, 22, 60 Adkins, Douglas, 24, 59, 60, 61, 64 Adult basic education as subject matter of IRI, 7, 10, 15, 18, 27, 28 Africa. See Sub-Saharan Africa; names of individual countries Age groups and grades and IRI, 1, 7, 15, 18, 32, 41, 42 Airtime. See Radio broadcasting AM radio. See Radio broadcasting Analog recording and editing, 4, 81 Angola, 7 Asia, 7, 26. See also names of individual countries Asian Development Bank, 23 Audience for IRI, 1, 7, 31, 37, 38; dual, 11, 13, 27; hard-to-reach, 1, 9­10, 29; out-of- school, 1, 7, 9, 16, 18, 31, 35, 38, 40; primary, 38, 39; profile of, 37­39, 71, 76, 85; research on, 31, 39­40, 58, 73­75; shadow, 24, 30, 31, 35, 40, 84; size of, 1, 76. See also Going to scale Audiocassette tapes: as delivery medium, 21, 55, 71, 80, 81, 82; costs of, 56 Audio programs and production, 58, 71, 80­81 Australia, 2 115 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Bakia, Marianne, 25, 61 Barbados, 61 Baseline data, 18, 80 Bates, Jeffrey, 36 BBC World Education Services, 69 Bolivia, 15, 17, 18, 19, 20, 25, 27, 28, 45, 54, 60n, 64, 66, 68 Bosch, Andrea, 18, 29, 80 Broadcasting. See Radio broadcasting Budgetary allocations as method of paying for IRI, 32, 62, 64, 65, 66, 82 Canada, 2 Capacity building, 32, 53, 54, 57­58. See also Training Cape Verde, 7 Caregivers as IRI audience, 7, 36 Cash, Francis, 15 Cassette tapes, as delivery medium. See Audiocassette tapes Cawthera, Andy, 61 CDs: as delivery medium, 21, 55, 56­57, 71, 72, 80, 81, 82; costs of, 56 Child-centered approach. See Learner-centered approach Child laborers as IRI audience, 10 Civics as subject matter of IRI, 41 Classroom activities. See Learning activities Classroom materials. See Instructional materials Cobbe, James, 24 Colletta, Nancy, 36 Communications media and learning, 4, 50 Community-based organizations (CBOs) as supporters of IRI. See Local (community) support for IRI Computers as alternative learning investment, 25, 61 Containment of recurrent costs, 61­62 Content of IRI, 14, 31, 37, 40­46, 47, 48, 82 Contracting out. See Outsourcing and contracts Corrales, Carleton, 15 Cost of IRI, 1, 22­24, 58­66; add-on, 22, 60; administrative, 59, 65; and of alternative delivery media, 55­57, 61; of broadcasting, 56, 57, 61, 62, 64, 65, 83; containment of, 61­62; and costs of alternative investments, 20, 22, 24, 25, 61; distribution of, 22­23, 59, 63, 64, 65; economies of scale of, 1, 3, 20, 24, 32, 58, 59, 60, 63; and education budget, 22, 62; fixed, 9, 22, 59, 63; front-loading of, 22­23; initial (start- 116 INDEX up), 22, 23, 45, 60, 61, 65; investment, 59, 60, 63, 64, 65; of language versions, 44; nonsalary, 63, 66; per-student, 23, 24, 59, 60, 60n, 61, 63, 66; and print materials, 24, 62, 65; production, 71, 81; and radios and batteries, 62, 65; recurrent, 9, 23, 24, 27, 28, 32, 58­59, 60, 61­62, 63, 64, 65, 66, 69, 82, 83; and teacher training, 24, 62, 66; total, 63, 64, 65; variable (recurrent), 59 Costa Rica, 13, 27, 45, 54 Cost-effectiveness of IRI, 2, 22, 24, 32, 60, 84 Cost-sharing as method of financing IRI, 25, 32, 63, 64 Creative Associates International, 16, 18 Creed, Charlotte, 61 Creole as language of instruction, 41, 44 Curriculum, national, 7, 12, 22, 42, 43, 45, 52, 75 Curriculum design, 7, 8, 32, 41, 42, 43, 44, 45, 47, 65, 69, 75, 76 Decentralization, 21, 25, 26, 27, 32, 49, 51, 52, 53, 67, 68 de De Jesús, Altagracia Diaz, 18 De Fossard, Esta, 4 Delivery options for IRI, 55, 56, 71, 72, 81, 82­83 Department for International Development (DfID) (United Kingdom), 23 Design document, 43, 47, 71, 75, 76­78, 85­87 Development phase of IRI, 63, 64, 70, 72­73, 74, 85­86 Digital radio broadcasting, 55, 82 Digital recording and editing, 4, 71, 81 Distance education, 1, 4, 31; traditional compared with IRI, 2, 8, 60 Distributed learning, 8, 41 Dock, Alan, 21, 60, 69 Dominican Republic, 10, 18, 22, 27, 28, 35, 45, 54, 66 Donor support for IRI, 1, 9, 30, 37, 50, 54, 59, 63, 64, 65, 66 Drills. See Learning activities Dual audience approach, 11, 13, 27 Early childhood development (ECD) as subject matter of IRI, 7, 25, 27, 36, 67 Economies of scale of IRI, 1, 3, 20, 24, 32, 58, 59, 60, 63. See also Going to scale Ecuador, 45 ED 2004 basic education project (Haiti), 36 Education for All, 10, 13 Educational goals and objectives of IRI, 31, 34­36, 64, 71, 73, 75, 76, 77, 79, 80 Educational organizations and IRI, 32, 50­51, 68 117 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Educational quality, 1, 2­3, 6, 10, 13­14, 23, 29, 32, 34­35, 39, 41, 60; indicators of, 15­17, 20; standardization of, 31, 36 EDUCATODOS (Honduras), 22 Effectiveness of IRI, 14­17, 19, 33, 57 Egypt, Arab Republic of, 54 El Salvador, 27, 54 English as subject matter of IRI, 7, 11, 12, 15, 18, 19, 24, 27, 28, 41, 44, 45, 64 English in Action (South Africa), 12, 19, 24, 28 Enrollment, 15­16, 22. See also Access to education Environmental studies as subject matter of IRI, 7, 13, 27, 41 Equity gaps (gender, rural-urban): impact of IRI on, 1, 2, 10, 15­16, 17­18 Ethiopia, 9, 10, 27, 54 Evaluation, 32, 52, 76; annual, 31, 36­37; design of, 37, 48, 76; external, 37, 72, 84; formative, 3, 8, 31, 36, 39, 48, 50, 53, 58, 71, 74, 76, 78­80, 81, 86; summative, 4, 48, 72, 84, 86 Examination Council of Zambia (ECZ), 16 Exercises. See Learning activities Exit strategy in design document, 85, 86 Expansion of IRI. See Going to scale Experiments. See Learning activities External assistance. See Donor support for IRI External evaluation, 37, 72, 84 Facilitators, 2, 8, 10, 36, 69, 79. See also Teachers Feasibility of IRI, 31­34 Fee for service as method of financing IRI, 32, 64 Feedback in IRI, 4, 8, 36, 37, 51, 57, 68, 69, 73, 74, 79 Fe y Alegria (Bolivia), 68 Field trips. See Learning activities Financing options for IRI. See Funding for IRI Fixed cost, 9, 22, 59, 63. See also Cost of IRI FM radio, 56, 57, 82. See also Radio broadcasting Formative evaluation, 3, 8, 31, 36, 39, 48, 50, 53, 58, 71, 74, 76, 78­80, 81, 86 French as subject matter of IRI, 7, 12, 18, 27, 41, 44 Fundamental Quality and Equity Levels (FQEL) (Guinea), 17­18 Funding for IRI, 58, 59, 63­64, 65, 66, 81; through cost-sharing, 25, 32, 63, 64; by donors, 1, 9, 30, 50, 54, 59, 63, 64, 65; by government, 50, 65, 66; by local commu- 118 INDEX nities, 10, 25, 28, 30, 62, 64, 66, 67­68; by nongovernmental organizations, 25, 32, 49, 50, 51, 66, 68; through taxes or fees, 25, 28, 32, 64 Games. See Learning activities; Marketing Gender equity gaps, 1, 15­16, 18 Ghana, 2 Girls, educational attainment of, 15­16, 18 Going to scale (scaling up), 20­21, 81 Goldstein, E., 18 Government support for IRI, 50, 65, 66 Group work. See Learning activities Guatemala, 27, 54 Guay, Dominique, 56 Guinea, 1, 10, 12, 15­16, 17­18, 20, 27, 36, 41, 42, 44, 54 Guinea-Bissau, 7 Guyana, 54 Haiti, 27, 30, 36, 41, 42, 44, 45, 54, 56 Hanushek, Eric, 24 Hard-to-reach audiences and IRI, 1, 9­10, 29 Hartenberger, Lisa, 18 Health as subject matter of IRI, 7, 14, 27, 31, 41, 67 Helwig, John, 28, 60, 69 HIV/AIDS, 7, 10, 13; as subject matter of IRI, 14, 41 Holistic learning, 4, 14 Honduras, 7, 10, 15, 18, 22, 25, 27, 28, 45, 54, 64, 66 Income generation schemes, 25 India, 2, 27, 52, 54; Karnataka State, 13 Indigenous and local knowledge, 1, 4, 8, 71, 74 Inputs for IRI, 47­49, 63 Institutionalization of IRI, 32, 49, 67­68 Instructional activities. See Learning activities Instructional design phase of IRI planning, 76 Instructional materials, 3, 6, 42, 63, 82 Instructional role of teachers, 1, 2, 4, 5, 8, 10­12, 14, 31, 36, 39, 44 Intellectual property issues and adaptation of IRI programs, 46 Interactive learning, 3, 6, 11. See also Active learning pedagogy 119 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Interactive radio instruction (IRI): adaptation of country programs for, 7, 9, 23, 31, 44­ 46, 60, 77; adaptation for regions of, 71, 81; content of, 14, 31, 37, 40­46, 47, 48, 82; definition of, 1; delivery options for, 55, 56, 71, 81, 82­83; design document for, 43, 47, 71, 75, 76­78, 85­87; development phase of, 63, 64, 70, 72­73, 74, 85­86; effectiveness of, 14­17, 19, 33, 57; effect over time of, 1, 19; equity gaps and, 1, 2, 10, 15­16, 17­18; factors in success of, 9, 29­30; feasibility of, 31­34; inputs for, 47­49, 3; lead role of, 35­36; leadership of, 21, 30, 32, 49, 69; as learning system, 2, 8, 14; management of, 21, 32, 47; marketing of, 48, 72, 83­84, 86; ongoing phase of, 85­86; pedagogy of, 2, 3, 8; pilot projects for, 16, 20, 23, 26, 27, 28, 30, 32, 57­ 58, 63; in primary schools, 1, 7, 31, 34­35, 41, 42, 57, 60, 85; program develop- ment for, 9, 32, 45, 46­58, 65, 71, 72, 73; responsiveness of 30; scaling-up of, 20­21, 81; teachers' role in, 1, 2, 4, 5, 8, 10­12, 14, 31, 36, 39, 44; and traditional instruc- tional methods, 7. See also Audience for IRI; Cost of IRI; Cost-effectiveness of IRI; Evaluation; Institutionalization of IRI; Scripts for IRI lessons; Sustainability of IRI; names of individual countries and subjects Inter-American Development Bank, 23 International Education Systems, 16 Internet, as broadcast method, 55 Investment cost. See Cost of IRI IRI. See Interactive radio instruction (IRI) IRI Math: in Bolivia, 19; in Nicaragua, 2, 8, 11 Kenya, 27, 44, 45, 54 Language: indigenous, 43; of instruction, 39, 43­44, 72, 75; as subject matter of IRI, 8, 18, 24, 36, 41, 44. See also specific languages Language arts. See Language Latin America and the Caribbean, 1, 7, 25, 28, 29, 43. See also names of individual countries Leadership of IRI, 21, 30, 32, 49, 69 Learner-centered approach, 11, 12, 14, 36 Learning activities, 1, 2, 3, 5, 6, 8, 11, 41, 42, 74, 77 Learning environment, 4, 7, 14, 29, 36 Learning system, 4; IRI as, 2, 8, 14 Leigh, Stuart, 15, 19 Lenders. See Donor support for IRI Lesotho, 20, 25, 27, 28, 44, 45, 54, 60n, 66 Life skills as subject matter of IRI, 10, 14, 31, 41 120 INDEX Literacy and IRI, 10, 14, 16, 27, 30 Local (community) support for IRI, 10, 25, 28, 30, 51, 62, 63, 64, 66, 67­68 Local-language versions of programs, 43­44 Lockheed, Marlaine, 24 Malawi, 45, 58 Management of IRI, 21, 32, 47 Marketing, 48, 72, 83­84, 86; of IRI ancillary materials, 32, 72, 84 Master plan for IRI. See Design document Mathematics as subject matter of IRI, 2, 7, 8, 11, 12, 13, 15, 16, 17, 18, 19, 24, 27, 28, 30, 36, 41, 45 Ministry of communication and IRI, 25, 32, 64, 67 Ministry of education and IRI, 7, 10, 13, 16, 17, 25, 32, 49, 56, 64, 66, 67, 79 Miranda, Jessica, 28, 80 Mother tongue. See Language (of instruction) Mozambique, 7 Multichannel learning, 4, 6, 42, 56 Multicountry use of IRI, 7, 9. See also Regional cooperation Multigrade teaching, 4, 12­13, 27, 42 Murphy, Paud, 4, 15, 61 Music and songs in IRI programs, 6, 41, 48, 72, 76, 77, 80, 81, 84, 86, 80 National curriculum, 7, 12, 22, 42, 43, 45, 52, 75 Nepal, 10, 11, 27, 36, 44, 54 Nicaragua, 2, 8, 11, 15, 27, 41, 45, 54 Nigeria, 10, 12, 16, 27, 45, 52, 54 Nongovernmental organizations (NGOs) as supporters of IRI, 25, 32, 49, 50, 51, 66, 68 Norwegian Agency for Development Cooperation (NORAD), 23 Numeracy and IRI, 10, 14, 27 Ongoing phase of IRI, 85­86 Open Learning Systems Education Trust (OLSET) (South Africa), 12, 54, 64, 69 Osin, Luis, 61 Out-of-school audiences and IRI, 1, 7, 9, 16, 18, 31, 35, 38, 40 Outsourcing and contracts, 48, 54, 79, 80, 81, 84, 86 Pakistan, 54 PALOP (Portuguese-speaking African) countries, 7, 54 121 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Papua New Guinea, 15, 18, 20, 27, 28, 54 Parents and IRI, 31, 40, 51, 66 Partnerships in support of IRI, 25, 49, 50, 79 Pedagogical approach of IRI, 2, 3, 8 Perraton, Hillary, 61 Personnel, numbers and skills of, 32, 47, 48, 53, 73, 77, 78, 81, 82 Per-student cost of IRI, 23, 24, 59, 60, 60n, 61, 63, 66 Physical resources for IRI development. See Inputs for IRI Pilot projects for IRI, 16, 20, 23, 26, 27, 28, 30, 32, 57­58, 63 Political support for IRI, 21, 30 Portuguese as subject matter of IRI, 7 Postconflict environment, 7 Posters. See Instructional materials Post-testing, 15­16, 18, 84 Potashnik, Michael, 61 PREPI preschool program (Honduras), 22 Preschool children and IRI, 7, 22, 27 Pre-testing, 80, 84 Primary schools and IRI, 1, 7, 31, 34­35, 41, 42, 57, 60, 85 Print materials: cost of, 24, 62, 65; production of, 42, 50, 65, 66, 69, 71, 76, 77, 82 Private schools and IRI, 30, 68 Privatization, 26­27, 56 Producer for IRI programs, 47, 48, 77, 81 Production for IRI, 48, 52, 58, 60, 65, 71, 80­82, 86 Program development for IRI, 9, 32, 45, 46­58, 65, 71, 72, 73 Project development phase of IRI, 71, 72­80 Quality control, 29, 53, 71 Quality of education. See Educational quality Qur'anic schools and IRI, 10 RADECO (Dominican Republic), 10, 18, 22 Radio broadcasting, 1, 2, 3, 20, 24, 35, 48, 50, 52, 53, 78, 82­83; alternatives to, 55, 56, 57, 72, 82­83; competition in, 82; costs of, 56, 57, 61, 62, 64, 65, 83; coverage and reception quality of, 21, 53; by foreign providers, 70; by Internet, 55; options for (AM/FM, digital, community, local, national, private, shortwave), 28, 53, 55, 56, 57, 70, 71, 82­83; by satellite, 9, 55, 70 Radio Language Arts (Kenya), 45 122 INDEX Radio production, 48, 58, 60, 71, 80­81, 86 Radio sets, cost of, 24, 62, 65 Radio stations, support for IRI by, 66, 67 Reading as subject matter of IRI, 7, 18 Recurrent cost of IRI, 9, 23, 24, 27, 28, 32, 58­59, 60, 61­62, 64, 65, 66, 69, 82, 83. See also Cost of IRI Refugees as audience for IRI, 7, 10, 27 Regional cooperation, 9, 45, 60 Reinforcement of lessons, 8, 41 Research: on audience for IRI, 31, 39­40, 48, 58, 73­75; on educational objectives, 75­ 76; on market for IRI programs, 84 Results-based approach to terms of reference, 87 Royer, James M., 16 Rural areas, educational attainment in, 1, 2, 9, 15, 17­18, 56 Rwanda, 45 São Tomé and Principe, 7 Scaling up IRI. See Economies of scale; Going to scale Science as subject matter of IRI, 7, 13, 18, 27, 28, 31, 41 Scripts for IRI lessons: adaptation of, 45­46; design, development, and revision of, 3, 8, 47­48, 58, 60, 65, 71, 77, 78, 79, 82, 86; example of, 5 Scriptwriting. See Scripts for IRI lessons Secondary audience. See Shadow audience Secondary education as subject of IRI, 10, 27, 61 Shadow audience, 24, 30, 31, 35, 40, 84 Shortwave radio, 55, 56, 82 Skills, mastery of, 8, 41 Somali speakers in Ethiopia, 9, 10, 27 Songs in IRI programs. See Learning activities; Music and songs in IRI programs South Africa, 1, 12, 14, 15, 17, 18, 19, 20, 24, 27, 28, 36, 44, 45, 54, 64, 66 Spanish: as language of instruction, 43; as subject matter of IRI, 7, 15 Stakeholders in IRI, 32, 34, 35­36, 42, 43, 50­51, 53, 57, 63, 76 Stanford University, 2, 8 Stories. See Learning activities Structured learning, 8 Student-centered (learner-centered) approach, 11, 12, 14, 36 Subject matter for IRI, 1, 7, 15, 31, 41, 42, 43. See also names of individual subjects 123 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION Sub-Saharan Africa, 7, 10, 13, 15, 18, 19, 20, 25, 29, 45, 60, 64, 65, 82. See also names of individual countries Sudan, 7, 10, 45, 58 Summative evaluation, 4, 48, 72, 84, 86 Support for IRI. See Donor support for IRI; Government support for IRI; Local (com- munity) support for IRI; Nongovernmental organizations (NGOs) as supporters of IRI; Teachers (support for IRI by) Survey for IRI report, 29, 71 Sustainability of IRI, 21, 25­30, 32, 67, 68­79, 85­86 Synergies between IRI and other educational methods and activities, 7, 12 Tanzania, 10, 22, 27, 54 Taonga Market program (Zambia), 18 Tax as method of financing IRI, 25, 28, 32, 64 Teachers: role of in IRI, 1, 2, 4, 5, 8, 10­12, 14, 31, 36, 39, 44; support for IRI by, 21, 50, 51, 66 Teachers in Action, 69 Teacher training, 8, 11­12, 13, 21, 24, 35, 36, 51, 52, 56, 62, 66, 69, 76, 82 Team activities. See Learning activities Technical assistance, 23, 29, 32, 53, 54, 85, 86­87 Television as alternative learning investment, 61 Testing: of achievement, 15­16, 17­18, 37, 79, 84; of program materials, 31, 39, 45, 53, 57­58, 68, 78, 81 Textbooks as alternative investment, 7, 22, 24. See also Instructional materials Thailand, 17, 27, 45, 54 Tilson, T., 7, 15, 19 Training: for IRI personnel, 32, 53, 54, 58, 65, 76, 81; for teachers, 8, 11­12, 13, 21, 24, 35, 36, 51, 52, 56, 62, 66, 69, 76, 82 United Nations Children's Fund (UNICEF), 11, 14, 23 United Nations Convention on the Rights of the Child, 13­14 United Nations Educational, Scientific and Cultural Organization (UNESCO), 7 Universal Primary Education, 13 University of Zambia, 16 Urban-rural equity gaps, 2, 9, 15­16, 17­18, 56 U.S. Agency for International Development (USAID), 2, 13, 15, 21, 23, 28, 54 U.S. Department of Labor, 23 124 INDEX van Steenwyk, Ned, 7 Variable cost, 59. See also Cost of IRI Venezuela, 20, 27, 45, 54 Wall friezes. See Instructional materials Workbooks. See Instructional materials World Bank, 21, 23, 25, 61 Writing as subject matter for IRI, 18 Zambia, 2, 10, 16, 17, 18, 22, 27, 35, 41, 44, 51, 54 125 IMPROVING EDUCATIONAL QUALITY THROUGH INTERACTIVE RADIO INSTRUCTION 126 (from previous page) No. 77 Notes on the Impact of HIPC Initiative on Public Expenditures in Education and Health in African No. 56 Santé et Pauvreté au Rwanda: reconstruire et réformer les countries (June 2004) services de santé dans la perspective des objectifs du mil- lénaire pour le développement (OMD) * (March 2005) No. 78 Open and Distance Learning for Teacher Education in Africa: Innovation and Change (Forthcoming) No. 58 Etat du Système Educatif Camerounais * (May 2004) No. 79 Proceedings from the First Regional Conference on No. 59 Management and Disparities of Education (Forthcoming) Secondary Education in Africa (SEIA)--Also available in French (September 2004) No. 60 Accelerating Progress Toward the MDGs (Forthcoming) No. 80 L' Education Secondaire en Afrique : Des stratégies de renouvellement (forthcoming) No. 61 Nigeria Institutional Capacity Assessment (April 2004) No. 81 Trends in secondary education in industrialized countries : No. 62 Public and Private Funding in Zambia Basic Education Are they relevant for African Countries? (forthcoming) (April 2004) No. 82 Targeting Vulnerable Groups in National HIV/AIDS No. 63 Nourrir, éduquer et soigner les Nigériens: La démographie Programs: The Case of Men Who Have Sex With Men, en perspective (March 2004) Senegal, Burkina Faso, The Gambia No. 64 Crafting Institutional Responses to HIV/AIDS: Guidelines No. 83 Disease and Mortality in Sub Saharan Africa--a sub and Resources for Tertiary Institutions in Sub-Saharan series Africa (March 2004) No. 84 Namibia: Human capital and Knowledge Developemnt for No. 65 Higher Education Development for Ethiopia: Pursuing the Economic Growth with Equity (February 2005) Vision (March 2004) No. 85 The Determinants of Schooling Investment Among No. 66 Improving Tertiary Education in Sub-Saharan Africa: Primary-School-Aged Children in Ethiopia (November Things that Work (March 2004) 2004) No. 67 Integration of Gender Issues in Selected HIV/AIDS No. 86 Education in Ethiopia: Strengthening the Foundation for Projects in the Africa Region: A baseline assessment Sustainable Progress * (in progress) (March 2004) No. 87 Utilisation des Services de Santé au Mali (Forthcoming) No. 68 Le systéme éducatif de la République démocratique du Congo: Priorités et alternatives * (January 2005) No. 88 Ethiopia: A Country Status Report on Health and Poverty* (forthcoming) No. 69 Primary Health Care in Mozambique: Service Delivery in Complex Hierarchy (April 2004) No. 89 Improving Health Outcomes for the Poor in Uganda: cur- rent status and implication for health sector development * No. 70 Decentralized Delivery of Primary Health Services in (forthcoming) Nigeria (June 2004) No. 90 Le Système Educatif Guinéen: Diagnostic et Perspectives No. 71 Amélioration de l'Enseignement Supérieur en Afrique au pour la Politique Educative dans le Contexte de Sud du Sahara (April 2004) Contraintes Macro-économiques Fortes et de Réduction de la Pauvreté * (forthcoming) No. 72 Managing Public-Private Partnership ­ Lessons from Senegal Literacy Education in Senegal (May 2004) No. 91 The Human Resources for Health Situation in Mozambique (forthcoming) No. 73 Girls' Education in Africa: What Do We Know about Strategies That Work? (May 2004) No. 74 Resources Allocation and Purchasing in Africa (August 2004) No. 75 The State of Workforce in Sub-Saharan Africa (September 2004) No. 76 Efficience et Equité des Formations Sanitaires Malgaches (forthcoming) * Country Status Report THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. Telephone: 202 477 1234 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: afrhdseries@worldbank.org This toolkit, Improving Educational Quality through Interactive Radio Instruc- tion, is part of the ongoing effort by the World Bank's Africa Region to deepen understanding of how distance education as well as information com- munication technologies (ICTs) can support education in Africa. The toolkit builds on the strategy paper "Enhancing Learning Opportunities in Africa" (Murphy et al. 2002), which reviews past experience with distance educa- tion in Africa and reaffirms its strong potential, with judicious integration of ICTs. Interactive radio instruction (IRI) has emerged as an important option for the improvement of educational quality in primary school classrooms in developing countries around the world. Since IRI's introduction nearly 30 years ago, studies indicate its effective- ness in improving educational quality as well as retention rates at the pri- mary school level. Studies also demonstrate IRI's value as an effective mechanism to introduce new pedagogy and curriculum. This toolkit is intended for African policymakers, education planners, and pedagogical specialists who may be considering the feasibility of using IRI in their education systems. It is designed to helps assess the potential of IRI to improve educational quality in primary school classrooms, as well as to guide the design and development of IRI programs, once a decision is made to implement a program. The toolkit offers reference information on IRI expe- rience worldwide, including Africa and provides step-by-step details that will be valuable to both planners and technical specialists alike.