4 In Indonesia, a The Project replication of a Growth Between October 2016 and May 2017, the World Mindset intervention Bank, with support from the Australian Department of Foreign Affairs and Trade (DFAT), conducted the had significant impacts adapted Growth Mindset intervention across 2,404 public junior secondary schools, divided evenly and on student academic randomly between those receiving the program and performance– particularly those serving as the control group. Nearly 160,000 students received the intervention, making this one in schools located of the largest Growth Mindset applications to date. The intervention designed a set of four comic books to in regions that could make the messages as simple, attractive, and context- benefit the most. relevant as possible. It featured key messages about Growth Mindset, self-belief, and perseverance, adapted for Indonesian students attending Grade 9, the last year of junior secondary school. In the comic books, key messages were repeated several times in large, bold text, including “otakmu seperti otot” (“Your brain is like a muscle”) and “belajar dari bertanya” (“Learning by asking others”). The books used relatable characters designed in the style of modern-day cartoons and included storylines that put the characters in classroom situations that would be familiar to adolescent Student learning outcomes in Indonesia have students in Indonesia. remained lower than their regional neighbors and other middle-income countries, as suggested by The design also included classroom discussions to the results of the Program for International Student emphasize key messages and simplified delivery for Assessment – PISA. Aside from socioeconomic teachers, including a lesson plan that incorporated conditions and access to educational resources, visual, auditory, and kinesthetic sessions, to ensure that student motivation may be another relevant reason students were able to absorb the information. To avoid overloading the students with new materials, the delivery behind the lag, particularly among low-income was split into two 40-minute sessions, also matching students.1 Data from PISA indicates that the majority the length of classroom periods in Indonesian academic of 15-year-old Indonesian students do not believe contexts. that abilities and intelligence can be developed.2 To allow teachers to deliver the sessions with minimal In prior work in Peru, eMBeD and World Bank preparation, the instruction manuals included guiding colleagues had demonstrated the positive impact of details like scripts and timestamps. In addition, all a Growth Mindset intervention to improve student materials were printed and shipped directly to schools. effort and, subsequently, test scores.3 Groundbreaking As part of the evaluation, the team conducted research, developed by Carol Dweck at Stanford compliance checks on the materials, wrote letters to the University, has shown that people with “fixed mindsets” headmasters and teachers, and conducted phone calls (who believe that intelligence is static) put less effort and emails to track the delivery and ensure that activities into learning than those with “Growth Mindsets” (who were carried out. The team also asked principals and believe that intelligence can be improved with effort). teachers to send pictures of students holding up the In Indonesia, the Education, Poverty & Equity, and comics at the completion of the sessions as a form eMBeD teams from the World Bank, developed the of evidence of successful implementation. In order to “Semua Bisa Pintar!” (Everyone Can Be Smart!) project, assess its impact, data on student performance from a Growth Mindset classroom program adapted to the the national standardized exam (Ujian Nasional) was Indonesian context, with the goal of replicating the analyzed, comparing students from the schools that success in Peru. received the intervention with those that did not. The Results Overwhelmingly, schools complied Students in Sumatra benefited the most. with the treatment. The program was implemented in the two main From the total of 1,202 schools selected to receive islands of Indonesia, Java and Sumatra, which the program materials, only 198 (16.5%) did not together represent 72% of the population of the send a message confirming that they effectively country. However, the provinces from Java tend delivered the sessions. Additionally, we verified that to have higher scores of the national exam than the these non-complying schools did not have different ones from Sumatra. With this intervention, the performance than the rest of participating schools impact on student performance was higher in the before the application of the program. schools of Sumatra, where test scores increased for all subjects covered by the national exam. In schools from Sumatra that effectively implemented the sessions, test scores increased by 0.08 sd for Language and 0.18 sd for Math, while scores in English and Science increased by 0.14 sd. The intervention contributed to an overall increase on test scores. Schools that were selected to receive the intervention showed an increase of 0.06 standard deviations (sd) on the aggregate score of the national standardized exam and of 0.08 sd on the A high impact at a low cost. Math score, conducted at the end of the academic As with the experience from Peru, the “Semua Bisa year. Furthermore, schools that effectively complied Pintar!” program was highly cost-effective, providing with the activities of the intervention presented substantial impacts on academic performance at an additional impacts on Science and English scores of average cost of 25 cents of US dollar per student, at 0.06 sd. the moment of implementation. Policy Implications As in Peru and later, South Africa, the Growth Mindset The results from Indonesia also corroborate the intervention in Indonesia demonstrates the promising important takeaway that we’ve seen in other impact of teaching this skill to students to improve country contexts: while Growth Mindset can have effort and, subsequently, test an impact on all students, it seems scores and student performance Successful interventions need particularly impactful on students over time. Additionally, it shows from distinctive backgrounds. In the potential for doing it at scale not be a “one size fits all” the case of Indonesia, students in a cost-effective manner. approach but rather adapted from schools in Sumatra, with and contextualized. lower average test scores than The results also emphasize those from Java, experienced the importance of adaptation to local contexts. By more substantial impacts, suggesting that the simplifying the materials for students and instructions intervention shifts the beliefs about learning for for teachers, providing humorous content, and those who can make the most of it. introducing novel, country-specific visuals, the team was able to modify what has worked elsewhere for Following these promising initial results, the maximum impact. This demonstrates that successful Indonesia Ministry of Education and Culture has interventions need not be a “one size fits all” approach engaged the World Bank for further development of but rather adapted and contextualized. this successful Growth Mindset module. About eMBeD The Mind, Behavior, and Development Unit (eMBeD), the World Bank’s behavioral science team in the Poverty and Equity Global Practice, works closely with project teams, governments, and other partners to diagnose, design, and evaluate behaviorally informed interventions. By collaborating with a worldwide network of scientists and practitioners, the eMBeD team provides answers to important economic and social questions, and contributes to the global effort to eliminate poverty and enhance equity. Stay Connected eMBeD@worldbank.org #embed_wb worldbank.org/embed bit.ly/eMBeDNews 1. See Blackwell et Al. (2007), Paunesku et Al. (2015), and Yeager et Al. (2014) 2. According to the latest results from PISA (2018), 29% of Indonesian students agree that intelligence can be changed over time, compared to an average of 63% for OECD countries. 3. For further information, see the results brief of our project in Peru following this link and the results of our South Africa work at this link. 4. This work was supported by the Australian Department of Foreign Affairs and Trade (DFAT) through the Partnership for Knowledge-Based Poverty Reduction Trust Fund (WB Trust Fund N° TF071514, AusAID N° 56043). Last Update: December 13th, 2019