WPS6079 Policy Research Working Paper 6079 The Jordan Education Initiative A Multi-Stakeholder Partnership Model to Support Education Reform Haif Bannayan Juliana Guaqueta Osama Obeidat Harry Anthony Patrinos Emilio Porta The World Bank Human Development Network Education Team June 2012 Policy Research Working Paper 6079 Abstract The Jordan Education Initiative, launched in 2003 under develop critical knowledge economy skills crucial for the umbrella of the World Economic Forum, is a public- competitiveness and economic growth. The Initiative private partnership, or multi-stakeholder partnership, also seeks to build the capacity of the local information that integrates information and communication technology industry for the development of innovative technologies into the education process as a tool for learning solutions, and to build a sustainable model of teaching and learning in grades 1–12. This initiative fits reform supported by the private sector that could be within the ongoing reform of the education system in scaled nationally and replicated in other developing Jordan that began in the 1990s. The Jordan Education countries. Initiative’s main objective is to help Jordanian students This paper is a product of the Education Team, Human Development Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at hpatrinos@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team The Jordan Education Initiative: A Multi-Stakeholder Partnership Model to Support Education Reform1 Haif Bannayan Juliana Guaqueta Osama Obeidat Harry Anthony Patrinos Emilio Porta JEL Codes: I21, I25 Keywords: Jordan, education, public-private partnership, e-learning Sector Board: Education 1 The views expressed here are those of the authors and should not be attributed to the authors’ respective organizations. We thank Alex Wong for his encouragement and support, as well as the World Bank and the Government of Jordan. All errors are those of the authors. Authors’ affiliations are: Bannayan: Jordan Education Initiative; Guaqueta: International Finance Corporation; Obeidat: Jordan Education Initiative; Patrinos: World Bank; Porta: World Bank. 1 Introduction The Jordan Education Initiative (JEI) was launched in 2003 under the umbrella of the World Economic Forum (WEF). It is a local-global, public-private (or multistakeholder) partnership for education that integrates information and communication technologies (ICT) into grades 1–12 as a tool for teaching and learning. The Jordan Education Initiative’s main objective is to help Jordanian students develop the critical knowledge economy skills needed for national competitiveness and economic growth. JEI also seeks to build the capacity of the local information technology industry to develop innovative learning solutions. In addition, the initiative seeks to build a sustainable model of reform supported by the private sector that can be scaled up nationally and replicated in other developing countries. Formally, JEI has four main objectives:  Improving the delivery of education to Jordan’s citizens through a public-private partnership.  Unleashing the innovation of teachers and students through the effective use of ICT.  Building the capacity of the local information technology industry.  Creating a model of reform that can be used by other countries. JEI was developed at a time when Jordan was in the midst of implementing a very important, comprehensive reform of the whole education system, known as the Education Reform for Knowledge Economy Project (ERfKE). The objectives of ERfKE are derived from the Government of Jordan Vision, which was developed at the September 2002 Vision Forum and strongly supported and promoted by His Majesty King Abdullah II Bin Al Hussein. The strategy of ERfKE is based on structuring the educational system to ensure lifelong learning, ensuring the responsiveness of the educational system to the economy, accessing and utilizing ICT to support both effective learning and management of the education system, and ensuring quality learning experiences and environments. One set of ERfKE reforms includes teacher development. Another includes updating and revising the national curriculum in order to move beyond content knowledge towards supporting the development of intellectual and personal competencies among Jordanian students. The rationale behind the reforms is that the education sector must be responsive to the labor market demands of key industries. JEI attempts to help Jordanian students develop knowledge economy skills, which encompass both academic-related competencies (i.e., communication and thinking) and personal management skills (i.e., responsibility, adaptability, and teamwork). Reflecting on these goals, JEI set out to improve the delivery of education through an efficient public-private model, one that had the potential to accelerate education reform by means of the effective use of ICT. Background JEI was launched in 2003 when a group of nonprofit, information technology (IT) and telecommunications industry leaders at the World Economic Forum (WEF) agreed to join forces to improve education in a specific developing country. Jordan was selected because the project was supported by the top leadership, as represented by his Majesty King Abdullah II. Jordan had 2 also already established an enabling and supporting environment for education, including a National Broadband Learning and Research Network. In early 2002, the WEF organized a kick-off meeting for the initiative in London, inviting organizations from around the world to participate. JEI was officially conceived at the extraordinary IT and Telecom Governance meeting of the World Economic Forum in January 2003 and launched in June 2003 at the WEF Jordan meeting at the Dead Sea. JEI was also a response to the call of the Arab Human Development Report 2002 (UNDP and Arab Fund 2002) for public and private collaboration to promote effective change in the region. Initial investment in the first phase of JEI operations was $25 million (Table 1). There are 46 partners in the initiative, including: the British Council, Cisco Systems, Computer Associates, Corning Cable Systems, Dell, Fastlink, France Telecom, Hewlett Packard, Intel, the Krach Family Foundation, Microsoft, the Jordanian Ministry of Education (MoE), the Jordanian Ministry of ICT (MOICT), RazorView, the U.S. Agency for International Development (USAID), the U.S. Middle East Partnership Initiative, and the WEF. During this first phase, the initiative mainly provided infrastructure—that is, it equipped schools with computers, laptops, and other technologies—and trained teachers in various ICT literacy programs. The second phase of the initiative, currently underway, focuses on pedagogy and developing student skills. Table 1: Contributors to JEI Phase I Operations ($ million) Contributor Investment Jordanian and international partners 19 Ministry sources 6 Total 25 Source: USAID 2008 The program is managed by the JEI Program Management Office (PMO), which reports to a local board and operates under the umbrella of an executive committee. Members of the executive committee include the Minister of Education, the Minister of ICT, and the Minister of Planning and International Cooperation, as well as representatives of Cisco, Microsoft, USAID, Fastlink, the WEF, and Intel. The Executive Committee is chaired by the Minister of ICT, under the patronage of Her Majesty Queen Rania Al Abdullah. JEI moved quickly from an externally supported public-private partnership project to become nationally owned. The WEF umbrella was limited to three years and most partners' support for content development and provision of ICT inputs were completed by early 2007. This period represented the first phase of JEI. In order to both continue and expand its activities in support of Jordanian education, a decision was made in early 2008 by all local and global stakeholders in the initiative to transform JEI from a PMO to a not-for-profit company. The new organization would have an expanded national role: to incorporate innovative research, monitoring, and evaluation into its activities, in addition to project management. In 2008, JEI was registered with the Ministry of Industry and Trade as a not-for-profit limited liability company working to improve education. As was true of its previous phase, JEI is supported by Her Majesty Queen Rania Al Abdullah. As an independent organization, JEI has the autonomy and ability to pursue objectives that are wider in scope. No longer simply a PMO that implements a pre-set strategy, JEI can now develop its own strategy based on local needs. It can also evolve into an organization dedicated to research and innovation in education, performing monitoring and evaluation at several levels, in addition to being a strong implementer 3 of projects in Jordanian schools. The new entity also leverages years of experience in integrating technology into the education process to provide global consulting services. Figure 1 shows the current programming ―tracks‖ of JEI’s second phase of operations. Figure 1: Programming Tracks of JEI Phase II Operations Phase II tracks Discovery Expansion Global Outreach Pilots new, Provides services Rolls out innovative solutions to local and global projects & creates private sector firms through Madrasati new partnerships and governments Source: Authors The original project management office maintains the vision of JEI, tracks progress and project implementation, and holds partners accountable for their commitments. In addition, it has put standard technical and maintenance procedures in place to improve the initiative’s ability to monitor and maintain ICT infrastructure in Jordanian schools. The PMO also conducts site surveys to identify when and where assistance is needed in schools and its staff facilitates communications between the MoE and the various partners that develop content. The PMO sits in between the public and private entities—it is neither inside any of the key ministries involved, nor attached to any of the private companies. This neutrality offers the PMO flexibility in its operations, allowing it to experiment and pilot new activities, and the independence needed to facilitate and support complex discussions between all stakeholders. The PMO is accordingly perceived as the interface between the initiative’s global, local, private, and public sector partners. It manages projects to achieve the results desired by the Ministry of Education by continuously coordinating, facilitating, and following up on matters related to projects that seek to achieve education reform through technology projects. JEI therefore engages various partners in a dynamic relationship and provides guidance and feedback to stakeholders. Its continuous contact and interaction with global partners, moreover, keep it abreast of the latest developments in education, which help JEI operate as an agent of innovation and research. 4 JEI’s relevance at the national level has been strengthened by its development of a model for the creation of e-content. The model promotes the creation of Jordanian educational software companies and e-content aligned with the national curriculum. The process is a collaboration between the public sector and global and local companies, as well as the global public sector, represented by foreign government agencies and international NGOs—all managed under the umbrella of JEI. The MoE and JEI first established a list of key content areas to target. The PMO then started recruiting global entities to become key sponsors of this content, although Jordanian companies have begun to assume this sponsorship role. Sponsors were then paired with a local company, which would develop the e-materials, using the expertise and resources of the MoE and JEI. Once a partnership was established, the MoE provided a team of curriculum developers to work with the content production team. The sponsor provided funding, advice, and expertise to the developer and the MoE curriculum team. Coverage To support the process of reform at the Ministry of Education, JEI established a test bed for the introduction of ICT and e-content resources into classrooms that could support innovative practices. The test bed is comprised of 100 public schools in Amman called ―Discovery Schools.‖ Schools in Amman were selected because the construction of the National Broadband Learning and Research Network (NBN) had started there in 2003. The selected schools were divided equally between boys’ and girls’ schools at different levels of education from grades 1 through 12. The only prerequisite was that each school should have at least one extra room that could be designated a computer lab. Table 2 provides basic data on the Discovery Schools. It should be noted that these schools were selected to create a model that could be replicated in other schools in Jordan; it was never JEI’s intention to increase the number of Discovery Schools or assume responsibility for rolling out the model. 5 Table 2: Characteristics of the 100 Discovery Schools School variable Number Gender Male 42 Female 44 Co-ed school (up to 4th grade only) 14 Level Primary: grades 1–9 24 Secondary: grades 10–12 76 Size 1–499 21 (number 500–999 43 of 1,000–1,499 34 students) 1,500–2,000 2 Source: Authors’ calculations. The program has succeeded in introducing ICT equipment and software into 100 schools, providing, on average, 6 to 8 laptop computers to each school and creating at least one (in some cases, up to six) computer lab per school. The initiative has also trained 3,000 teachers and promoted the development of six packages of educational software and corresponding teacher training modules (Table 3). In fact, initial investments made during the first phase of operations resulted in products ready for export. JEI also introduced the concept of ―in-classroom‖ technology and advocated the teaching of IT to students based on practical, hands-on use of technology as a tool for learning. Table 3: Core Data on Discovery School Beneficiaries Beneficiary Number Comments Schools 100 Discovery Schools (DS) Administrators 200 Principals and assistant principals Teachers 3,000 Users of e-content in DS Students 80,000 Users of e-content in DS Ministry experts 120 Writers and reviewers Lab technicians 1000 Jobs created as an outcome of JEI Youth 150 New college graduates offered internship opportunities Local companies 5 Worked on developing e-content for the first time Source: Authors’ calculations Performance Consistent with its objectives, JEI has been effective in: (i) establishing a test bed for innovation in the realm of ICT and education in Jordan; (ii) helping develop the ICT and education industry in Jordan; and (iii) developing and testing a model of education reform based on partnerships that can be applied by other countries. 6 Impact on learning As used here, ―impact on learning‖ refers to improving the delivery of education in Jordan through the effective use of ICT, with the ultimate goal of supporting the development of a knowledge economy and producing sustainable national economic growth. To date, certain external reviews of the program have documented positive indications of satisfaction from teachers and students who are using the e-curriculum and technology of JEI (USAID 2008). Recent changes in the Jordanian education system have created a suitable environment for piloting innovations, given that the country’s main education objectives of access and transition rates to secondary school are well on their way to being achieved. For example, Jordan has established significant positive momentum in student performance in recent years. Net enrollment in basic education increased from 89 percent in 2000 to 96.5 percent in 2005–2006; transition rates to secondary education increased from 63 to 79 percent over the same period. Jordan has also made significant gains in international assessments of student achievement, with a particularly impressive gain of almost 30 points on the science portion of the 2007 Trends in International Mathematics and Science Study (Mullis, Martin, and Foy 2008). Jordan’s progress in international assessments actually dates to the early 1990s. In the 1991 International Assessment of Educational Progress, which was conducted in 20 countries, Jordan finished ahead only of Brazil and Mozambique in the mathematics and science tests for 13-year-olds. By the late 1990s, marked change had occurred. In the 1999 Third International Mathematics and Science Study (TIMSS), in which 38 countries participated, Jordan finished ahead of six (Iran, Indonesia, Chile, Philippines, Morocco, and South Africa) in mathematics and ahead of eight (Iran, Indonesia, Turkey, Tunisia, Chile, Philippines, Morocco, and South Africa) in science. However, the country’s progress did not stop there. In 2003, Jordan improved its TIMSS science score to 475, from 450 in 1999—an increase of 25 points, or 0.25 standard deviations. This was a significant increase, equivalent to about an entire year of learning. In 2007, Jordan continued to improve its TIMSS student scores, surpassing several countries that had had a similar or slightly higher performance in 1999. In fact, between 1999 and 2007, no other country has improved as greatly in assessments of the science knowledge of its students. Discovery Schools, moreover, appear academically superior to other public schools (see Figure 2). Students attending these schools had, on average, higher scores in all three subject areas of the PISA 2006 assessment than did those in regular public schools. 7 Figure 2: 2006 PISA Scores of Jordanian Students, by Type of School Science Non-Discovery Reading Discovery Math 0 100 200 300 400 500 Source: Authors’ calculations Data from the National Assessment for Knowledge Economy Skills (NAfKE)—which measures Jordanian student achievement in the subjects of math, science, and the Arabic language—sheds further light on the impact of JEI, particularly as the NAfKE assessment complements achievement data with surveys of participating students, teachers and principals. The NAfKE data set offers information on student achievement at two points in time, 2006 and 2008, in five types of schools in Jordan: public non-Discovery schools, public Discovery Schools, Directorate of Military Culture schools, schools of the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), and private schools. The analysis in this chapter concentrates on a sample of students attending public Discovery and non-Discovery schools in the 5th, 9th, and 11th grades in urban areas in Amman who participated in NAfKE in 2006 and 2008. A simple analysis of the characteristics of this sample shows differences in average characteristics between students in public Discovery and public non-Discovery schools (Table 4). When data is grouped by student, family, and school characteristics, it is notable that a larger proportion of students in Discovery Schools attended preschool; however, this difference becomes much smaller in the cohort of students tested in 2008. On the other hand, the difference in the proportion of students that repeated classes was almost negligible between Discovery and non-Discovery schools, favoring non-Discovery students in the 5th and 9th grades in 2006 and those in 5th grade in 2008. 8 Table 4: Mean Characteristics of Students Participating in NAfKE in Urban Areas of Amman, 2006 and 2008 Grade 5 Grade 9 Grade 11 Discove Non- Discove Non- Discove Non- 2006 ry Discovery Total ry Discovery Total ry Discovery Total Attended preschool 77% 64% 74% 66% 54% 64% 67% 56% 64% Have repeated classes 12% 14% 12% 6% 8% 7% 3% 3% 3% Number of people living at home 6.7 7.6 6.9 6.9 8.1 7.2 6.4 8.2 6.8 Mother with secondary education 36% 32% 35% 46% 42% 45% 45% 44% 45% Mother with college education 21% 14% 20% 18% 16% 17% 20% 15% 19% Mother with first grade of 21% 10% 19% 13% 5% 12% 13% 5% 11% university education Mother beyond first-grade 0% 0% 0% 0% 0% 0% 0% 0% 0% university Parents prefer academic track 77% 81% 78% 89% 84% 88% 97% 94% 96% Principal has master’s degree 24% 20% 23% 33% 20% 30% 15% 36% 20% Principal undergoes training in 65% 50% 62% 64% 76% 66% 64% 39% 58% administration Number of principals in the last 5 2.3 15.1 4.8 5.4 6.4 5.6 3.9 6.9 4.7 years Parents meet with principal to 73% 64% 71% 77% 76% 77% 82% 74% 80% discuss academics Corporal punishment 46% 68% 51% 41% 61% 45% 23% 24% 23% Teacher with teaching experience 12.8 11.4 12.5 12.3 11.9 12.2 13.8 9.4 12.7 Teacher with college degree 18% 20% 19% 5% 6% 6% 0% 0% 0% Teacher with bachelor’s degree 67% 77% 69% 73% 86% 76% 74% 84% 76% Teacher with bachelor’s degree 11% 3% 9% 13% 9% 12% 18% 12% 17% plus education diploma Teacher with master’s degree 4% 1% 3% 7% 0% 6% 7% 3% 6% Teacher with Ph.D. 0% 0% 0% 0% 0% 0% 0% 2% 0% Teacher with Arabic major 64% 42% 60% 60% 51% 58% 59% 45% 55% Teacher in the field Arabic 19% 37% 24% 13% 22% 15% 8% 1% 6% language Teacher with math major 62% 57% 61% 47% 72% 52% 70% 47% 63% Teacher in the field of math 30% 13% 25% 13% 3% 11% 9% 9% 9% Educational software available 34% 13% 29% 42% 19% 38% 32% 14% 28% Computer available for classes 48% 11% 41% 53% 30% 48% 53% 41% 50% Teacher has International 64% 34% 58% 71% 54% 67% 70% 65% 69% Computer Driving License Teacher has Intel certification 22% 10% 19% 36% 19% 32% 33% 27% 32% Source: Authors’ calculations, based on NAfKE 2006 and 2008 surveys 9 Within the group of variables that refer to family characteristics, a larger proportion of students who attended Discovery Schools reported having mothers with a college education and a university degree relative to students in public schools. The differences applied to students in all grades in years 2006 and 2008. For instance, in 2006, 21 percent of students in Discovery Schools reported having mothers with a university degree, as compared to only 10 percent of students in non-Discovery schools. As for parental interest and involvement in their children’s education, there was no clear difference between Discovery and non-Discovery schools. On one hand, parental preferences for an academic track were higher among Discovery School students in the 9th and 11th grade in 2006 and for 5th grade students in 2008; however, these differences decreased or disappeared for 9th and 11th grade students in the latter year. Modest differences were also reported in the level of parental involvement in school, as measured by the proportion of parents that met with the principal to discuss the academic development of their children. A higher proportion of Discovery School parents of children in 5th and 11th grade in 2006, and in 9th grade in 2008, reported meeting with the principal to discuss academic matters. Within the range of school-level variables, teachers in Discovery Schools reported higher levels of educational attainment relative to teachers in non-Discovery schools. In all grades and for all years, the proportion of teachers who reported having a bachelor’s degree plus an education diploma or master’s degree was higher in Discovery Schools, with the exception of 11th grade in 2008. Variables that indicate the availability of ICT elements—characteristic of the JEI intervention—show that a higher proportion of Discovery Schools reported having educational software available, computers available in classes, as well as teachers having an international computer certification (i.e., the ―International Computer Driving License‖) and Intel certification. For instance, 43 percent of Discovery Schools reported having educational software available for classes in 2008, relative to 13 percent in non-Discovery schools. Similarly, 62 percent of Discovery Schools had computers available in classes, compared to only 46 percent of non-Discovery schools. Do student, school, or family background characteristics, and the prevalence of ICT components in school, explain the small differences in achievement between Discovery and non- Discovery school students? To answer this question, a linear regression analysis was conducted that focused on the same sample of students (i.e., students who participated in NAfKE in 5th, 9th, and 11th grades in urban areas in public Discovery and non-Discovery schools in Amman). According to the regression analysis, Discovery School students performed better than students in other public schools without controlling for student, family background, or school characteristics. All of these positive differences have significant coefficients with four exceptions: 5th grade math in 2006, 11th grade math in 2008, 5th grade science in 2008, and 11th grade Arabic in 2008 (Table 5). These exceptions do not, however, indicate any particular pattern. Nonetheless, after controlling for family, student, and school characteristics, the findings become mixed, as the advantages of Discovery Schools drops in some cases and holds in others. On one hand, the academic advantage of Discovery School students in science decreases, for example, in grades 9 and 11 in 2006. On the other hand, the advantage increases by a small amount for math test scores of grade 11 students in 2006, and for grade 5 students in science in the same year (Table 6). 10 Table 5: Mean Differences in NAfKE Student Scores at Different Schools, 2006 and 2008 Difference: Discovery Schools Non-Discovery Schools Discovery – Non-Discovery Year Subject Grade Mean SD SE Mean SD SE Mean Sig. (2-tailed) 2006 Math 5 27.90 13.19 0.69 29.26 11.12 1.19 -1.37 0.37 2006 Math 9 39.37 15.40 0.85 29.78 12.83 1.35 9.59* 0.00 2006 Math 11 27.02 13.35 0.67 22.85 10.93 1.01 4.17* 0.00 2006 Science 5 52.06 18.73 0.99 44.99 17.81 1.90 7.07* 0.00 2006 Science 9 41.53 16.46 0.88 35.29 16.85 1.97 6.24* 0.00 2006 Science 11 42.19 18.53 0.95 37.28 20.61 1.75 4.91* 0.01 2006 Arabic 5 49.57 20.93 1.07 45.11 20.39 2.03 4.45* 0.05 2006 Arabic 9 48.14 15.20 0.84 41.89 15.73 1.70 6.24* 0.00 2006 Arabic 11 57.90 15.02 0.77 52.45 15.33 1.38 5.45* 0.00 2008 Math 5 30.52 13.49 0.76 26.67 9.64 0.95 3.85* 0.00 2008 Math 9 39.49 14.86 0.83 31.84 12.81 1.38 7.65* 0.00 2008 Math 11 29.90 14.79 0.79 29.67 14.18 1.24 0.23 0.88 2008 Science 5 51.47 17.03 0.97 48.70 19.76 1.97 2.77 0.17 2008 Science 9 46.88 18.43 1.03 36.87 17.21 1.89 10.01* 0.00 2008 Science 11 44.49 22.62 1.20 42.52 23.60 2.07 1.97* 0.40 2008 Arabic 5 54.38 22.76 1.28 47.51 22.89 2.18 6.88* 0.01 2008 Arabic 9 56.66 19.84 1.07 48.47 17.58 1.86 8.20* 0.00 2008 Arabic 11 63.95 14.58 0.76 62.52 17.55 1.53 1.44 0.36 Source: Authors’ calculations based on NAfKE 2006 and 2008 data Note: SD=standard deviation; SE=standard error The overall trend is that the academic advantage of Discovery Schools decreases once the data are controlled for background characteristics, however, some advantage remains for grade 9 students in most subjects in both years (with the exception of science in 2006). In sum, some evidence indicates that, on average, Discovery School students achieve higher scores in math and science relative to non-Discovery school students. Results become mixed after controlling for background characteristics, which could be due to non-Discovery school students improving over time. Table 6: Mean Differences with Controls, NAfKE Student Scores at Different Schools Difference: Discovery – Non-Discovery Without controls Controls for family background Year Subject Grade Mean Sig. (2-tailed) Mean Sig. (2-tailed) 2006 Math 5 -1.37 0.37 2.53 2.53 2006 Math 9 9.59* 0.00 7.58* 0.02 2006 Math 11 4.17* 0.00 4.74* 0.01 2006 Science 5 7.07* 0.00 10.83* 0.04 2006 Science 9 6.24* 0.00 0.88 0.78 2006 Science 11 4.91* 0.01 1.34 0.65 2006 Arabic 5 4.45* 0.05 5.62 0.33 2006 Arabic 9 6.24 0.00 6.60* 0.00 2006 Arabic 11 5.45 0.00 3.86* 0.05 2008 Math 5 3.85 0.00 2.96 0.17 2008 Math 9 7.65 0.00 6.74* 0.00 2008 Math 11 0.23 0.88 -1.20 0.45 2008 Science 5 2.77 0.17 -4.25* 0.09 2008 Science 9 10.01* 0.00 7.11* 0.00 2008 Science 11 1.97 0.40 1.91 0.43 2008 Arabic 5 6.88* 0.01 -3.62 0.28 2008 Arabic 9 8.20* 0.00 6.00* 0.01 2008 Arabic 11 1.44 0.36 1.69 0.27 Source: Authors’ calculations based on NAfKE 2006 and 2008 surveys Note: * Significant with at least a 10 percent confidence level 11 Development of the local ICT industry A second intended outcome of JEI was the development of a national ICT industry in Jordan by encouraging partnerships with global firms, transferring and building capacity, and generating export opportunities for products and services. Five local companies have benefited from close working relationships with global partners in the initiative. For example, Rubicon, a local e-content developer, partnered with Cisco Systems and the Cisco Learning Institute to produce the math e-curriculum; and Menhaj worked closely with Microsoft to develop an ICT e- curriculum. It is estimated that approximately $3.7 million has been transferred from global partners to local companies as a direct result of JEI programs. Indirect benefits have also accrued to these companies, as JEI has encouraged global companies to take a closer look at Jordan and created a favorable environment for investment in the country. For example, in August 2004 Cisco Systems formed a strategic partnership with Estarta to develop a Cisco Technical Support Center that will serve enterprise customers in Europe and the Middle East. JEI also helped build the capacity of the local ICT industry and its applications for the education sector by fostering partnerships between international ICT corporations and local businesses to develop teacher training programs. The fact that these products are being commercialized and applied at the regional level has contributed to the larger goal of the sustainability and scalability of the JEI model. Developing and testing a replicable model of reform Since its inception, one of the strategic goals of JEI has been to create a model of educational reform and partnerships that can be promoted in other countries around the world. JEI therefore actively participates in regional and international conferences and forums to promote the JEI model and the efforts of its partners. The initiative is continuously approached by local and global private sector actors because it has a strategy and its partners recognize its commitment; as a result, they are willing to continue their partnerships. Although ICT is the program’s market niche, the focus of JEI is education. Based on this focus, JEI created a consultancy arm that can manage and implement projects in several countries around the globe in such areas as scoping and managing the implementation of projects, researching and piloting innovative educational solutions, conducting technical evaluations of technology products, carrying out monitoring and evaluation, and building public- private partnerships. The experience of JEI has also informed other partnerships supported by the World Economic Forum in Palestine, Egypt, and Rajasthan. JEI has also provided valuable support to the WEF endeavors in Rwanda by helping develop an ―ICT in Education‖ policy for that nation’s government under the leadership of the latter’s Ministry of Education and the United Kingdom’s Department for International Development. It also helped design the coordination unit that will implement that policy. How Sustainable is the Jordan Education Initiative? There are two dimensions of the initiative’s sustainability. The first has to do with continuity, monitoring, and support of ongoing interventions in the existing 100 Discovery Schools in Amman. The second concerns the financial flows needed to maintain and scale up ICT in education initiatives in Jordanian schools. 12 The establishment of JEI as a permanent not-for-profit company is a positive outcome for the initiative’s sustainability, as it provides institutional continuity for the existing intervention. The fact that JEI receives support and direction from Her Majesty Queen Rania Al Abdullah, as well as maintains strong ties with four ministries (those of Education, Finance, ICT, and Planning and International Cooperation) demonstrates that the initiative, although initially supported by external resources, has successfully transitioned to become a locally owned organization embedded in the overall structure of the education system. JEI has also become an implementing partner in Madrasati, an initiative launched and led by Her Majesty Queen Rania, where it is in charge of the implementation of all ICT-in-education interventions in public schools adopted by Madrasati. At the same time, JEI is positioned as a local institution that channels funds from external partners, including USAID (which remains a major sponsor of JEI) and major international IT companies, such as Microsoft, Cisco, and Intel. JEI’s strongest ties are with the Ministry of Education, which is the primary beneficiary of its activities, which include sparking the interest of global partners, introducing modern innovations into the education system, and attracting in-kind contributions for projects. Other JEI activities that benefit MoE include scoping and implementing educational projects in Discovery and Madrasati schools; monitoring and evaluating projects, providing advice on policy issues at upper levels of government (i.e., to Ministers, Secretary Generals, and Directors), conducting regular site surveys at JEI schools, and providing feedback to decision makers in the Ministry of Education. The second dimension of JEI’s sustainability relates to financial support and the cost of maintaining its investments in the Discovery Schools, as well as the investments needed for possible expansion. JEI is still operating according to its original principles as a public-private partnership. In this model, the national government finances the administration of JEI and partners fund individual projects. One of the most valuable services JEI performs for the Government of Jordan is, therefore, the continuous research needed to find new solutions, promoting Jordan's quest to improve its education system, and harnessing large contributions from its partners for particular value-added services. The main sources of financing for JEI activities are the Government of Jordan and the initiative’s private and public sector partners. JEI is allocated an annual budget by the Ministry of Finance, through the Local Currency Program created by USAID and the government (represented by the Ministry of Planning and International Cooperation). JEI receives these funds in return for the services that it provides to the Ministry of Education. Under the Local Currency Program, funds donated to the government by USAID go to the Ministry of Finance, which disperses the funds to ministries and other entities, such as JEI, after submitting a plan to the Ministry of Planning and International Cooperation. The donations are governed by the Law of Not for Profit Companies, which specifies that all donations must be approved by the Prime Minister’s Office and the receiving entity must submit annual reports on its use of the funds. The receiving entity must also show that it adheres to its organizational objectives and purposes. Scaling up ICT and e-learning to reach a larger number of schools in Jordan will depend on the extent to which external sources of funding are available through partnerships. Financial support from external sources would, however, probably only cover the initial cost of investment, rather than the recurrent costs needed to provide technical support to teachers and schools and maintain ICT equipment (USAID 2008). In fact, most of the support provided by external partners is in the form of in-kind contributions, which highlights the responsibility of the government for the recurrent costs of JEI, as well as its management and administrative needs. 13 Moreover, projections for JEI’s financial sustainability should include the cost of additional components that are likely to contribute to its effectiveness and potential impact. These costs include those associated with antivirus software and the technical support, training, and maintenance of ICT equipment needed to improve the usability of this equipment and educational software. Available data on the operating costs of JEI suggest that the full cost of implementing JEI's learning approach would be, on an annual basis, $27,743 per year per school, or $28 per student (USAID 2008). However, the public sector cost would be less, given that JEI is now doing so much to generate its own resources through projects with global partners outside of Jordan. Operating model moving forward In establishing JEI as a not-for-profit organization, its role was expanded beyond a project management office to include a research and innovation function (i.e., researching and attracting new partners, as well as finding the latest innovations in education) and a monitoring and evaluation function (i.e., measuring the impact and success of JEI efforts at the project, strategy, and program level, and eventually, offering these services to other programs on a fee basis). For this purpose, JEI has adopted a holistic approach (Figure 3) to the planning and implementation of education projects. The cycle begins with researching current innovations in education in general and ICT in education in particular. The next step is to select a certain number of the Discovery Schools in which to pilot the innovation. The number of selected schools depends on the idea, as well as funding support, the cost, and the readiness of schools to adopt the innovation. During this phase JEI assigns a project manager, who designs the project charter and project plan, as well as identifies anticipated risks. The implementation process then begins and a monitoring plan is established, involving frequent progress reports. Figure 3: The JEI Project Planning and Implementation Cycle Source: JEI The next step is the evaluation of the short- and long-term impact of the project. The pilot concludes with a final report that is shared with all stakeholders, including the Ministry of Education, the private sector, the JEI board of directors, and, of course, the direct beneficiaries (see Table 7 for examples of its partnerships for the development of educational resources). If the 14 program is found to have a positive impact, it may then be recommended for roll out to all Discovery Schools and a new cycle of research and innovation would begin. Table 7: Stakeholder Participation in JEI e-content Participating JEI partners e-Content area Developers Sponsors e-Arabic (K–12) e-Dimension France Telecom/Jordan Telecom e-Science (K–12) Rubicon Fastlink (since acquired by Zain) e-Math (K–12) Rubicon CISCO Systems e-ICT (K–10) Menhaj Technologies Microsoft e-English (7–12) Rubicon Middle East Partnership Initiative (MEPI) e-Civics Jordan Center for Civic MEPI Education Source: JEI The future of the JEI business model Currently, the partners of JEI support it with cash and in-kind contributions, with the latter reaching $1 million in 2008. In addition, at any given time, JEI manages a pool of over 20 projects valued between $1.5 to $2 million. Apart from the government, JEI also considers its partners from the private sector as clients and seeks to engage with them in projects inside and outside of Jordan. The organization has put in place a plan to become self-sustaining within five years, the success of which mainly depends on earning service fees from the Government of Jordan as well as consultancy fees from other governments and international bodies. JEI has also redefined its relationship with its original partners, such as Cisco, Microsoft, Intel, and the WEF, to whom it now offers such services as consultancy projects, workshops, representation in international forums, and implementing tasks globally on their behalf. Conclusion The Jordan Education Initiative represents a successful model of an international public- private partnership in the education sector. From the very start, JEI was able to meet its objectives of establishing such a partnership with considerable external donor and private sector participation, contributing to the creation of a new partnership model. Also from the very beginning, it created a common vision for its partners to work together to achieve national goals in Jordan. JEI has been able to provide classrooms in the country with state-of-the-art ICT and software. Students have accordingly been exposed to the latest technology and teachers have received significant ICT training. JEI has also been able to mobilize its network of partners to provide its test bed, the Discovery Schools in Amman, a broad range of ICT resources, including hardware and software. Despite ongoing challenges to providing these schools stable Internet connectivity, the data show that teachers are able to use the ICT resources they have. Discovery Schools, which are now fully equipped according to JEI’s technology model, serve as both platforms for experimentation and venues for attracting and piloting further innovations in education. JEI now conducts research in the area of educational innovation and seeks to create new international and local partnerships and projects. Each new partnership in Jordan becomes a project that is implemented in the Discovery Schools. To make sure that its projects achieve desired outcomes, JEI identifies a set of key performance indicators and then monitors and 15 assesses the project throughout its implementation. Moreover, JEI has established permanent links to the Ministry of Education. As a result, the organization has become an entry point for innovation in education in Jordan. All indications are that the JEI model will be sustainable going forward. As noted earlier, JEI has consistently sought to create a model of educational reform and partnerships that can be promoted in other countries. Recent partners of the organization include, among others, Harvard University, Hitachi, IBM, Intel, and Microsoft (Table 8). The chief executive officer of JEI has, moreover, named a member of Microsoft's Partners in Learning initiative and of the WEF Technical Advisory Group. JEI’s early success led to its transformation into a national body, firmly embedding it in the national education sector. The organization quickly became an exporter of knowledge, as seen in the examples of Rajasthan, Palestine, and Egypt (which the WEF helped spearhead by spreading the JEI model). It has also achieved success in its own right, playing a key role in creating a space for the ICT sector to help formulate Rwanda’s Education for All Fast Track Initiative plans. Table 8: Current Projects and Partners of JEI Project Category Partner(s) e-Content Deployment and Training JEI pedagogy and training – Increased ICT adoption and changing Teaching for Understanding Harvard University pedagogy English as a Foreign Language Technology model Strategy Interactive White Boards Innovation, plus local and international IT SMART, Hitachi, support KETAB, Promethean Reading Companion Pedagogy and after-school program IBM Teacher Class Training Pedagogical approaches Menhaj Towards New Schools Attitude Driving change Microsoft, CADER Classmate PC Innovation Intel JEI Proprietary Training Research Innovative Teachers Network Harvesting innovations from the field Microsoft Neurolanguage Pedagogy and skills; after-school program E-Educational Regional Company Try Science Pedagogy and training IBM Source: JEI JEI is an important element of Jordan’s ongoing educational reform program. It is critical, therefore, that program impacts be estimated. Unfortunately, the design of the program does not allow for a more rigorous assessment of impact. Given its importance for both Jordan and the international community, the program would have benefited all stakeholders enormously if evaluation had been incorporated into program design from the beginning (see McKinsey & Company 2005 and USAID 2008). Moreover, since it will require many years before students reap the full benefits of the program in grades 1 through 12, and before the program’s impact on 16 the workforce and economy is evident, a longitudinal, randomized study that followed up on students would have been preferred. It is not too late, however, to initiate a follow-up study of students in the Discovery Schools. As JEI goes forward with its second phase of operations, randomized trials will be the best choice, as there is no substitute for baseline data and control groups. Evaluating the various technologies being implemented in the Discovery Schools, together with detailed information on the costs of these technologies—both initial and recurrent—would provide rigorous, valid estimates of the cost effectiveness of alternative approaches to introducing technology into the classroom. JEI is already testing various technology programs; reliable cost-effectiveness information can be produced by ensuring that the pilot phase of technologies is conducted using randomized allocation of each technology to treatment and control groups of schools. In conclusion, more attention needs to go to rigorous impact evaluation, not only to document overall program effectiveness, but also to provide JEI an assessment of what technologies, processes, and models work best for which students. Beyond implementing ICT in the classroom, a goal that JEI has achieved, the second phase develops international models for introducing 21st-century teaching and learning to Jordan. Given their role as a test bed for innovative technologies, the Discovery Schools could provide important information on what technologies work under what circumstances for which groups of students. Rigorous evaluations of these new technologies would provide information of value not only to Jordan’s public schools, but also to the international education community at large. Technology in the classroom is no longer a question; rather, the most important question of the day is: Which technologies work best? 17 References McKinsey & Company. 2005. ―Building Effective Public Private Partnerships: Lessons Learnt from the Jordan Education Initiative; An Initiative of the World Economic Forum and Government of Indonesia.‖ McKinsey & Company. https://members.weforum.org/pdf/JEI/JEIreport.pdf. Mullis, Ina V. S., Michael O. Martin, and Pierre Foy. 2008. TIMSS 2007 International Mathematics Report: Findings from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. With John F. Olson, Ebru Erberber, Corinna Preuschoff, Alka Arora, and Joseph Galia. Chestnut Hill, MA: TIMSS and PIRLS International Study Center, Boston College. UNDP (United Nations Development Programme) and Arab Fund for Economic and Social Development. 2002. Arab Human Development Report 2002: Creating Opportunities for Future Generations. New York: UNDP. USAID (U.S. Agency for International Development). 2008. ―Evaluation of the Jordan Education Initiative: Synthesis Report; Overview and Recommendations for the Jordan Education Initiative.‖ Prepared for USAID by Education Development Center, Inc., with RTI International. 18