RESULTS-BASED FINANCING FOR PRODUCING SUPPLEMENTAL READING MATERIALS IN BANGLADESH DESCRIPTIVE NOTE Contract #7186365 Contact: Joel Turner, Technical Advisor – jturner@irex.org September 2019 Revised: November 2019 Executive Summary This descriptive note outlines conclusions and policy implications of the Onuprerona Book Challenge in Bangladesh. It will provide analysis and recommendations for the design and costing of similar challenges in other contexts. Onuprerona is an 18-month project that aims to open a pathway for aspiring authors of underserved languages to create quality supplementary reading materials for early grade learners in the Khagrachari division of Bangladesh. The project is implemented by IREX with support from its in-country partners, Zabarang and Save the Children in Bangladesh. The project was executed through three streams of activity: First, IREX and partners conducted scoping work and an attitudinal study to understand the publishing landscape and stakeholder attitudes toward Mother Tongue-Based Multi-Lingual Education in the target division of Khagrachari. IREX and partners also conducted various awareness raising activities in over 100 communities in the Khagrachari division, ensuring parents, teachers, and students are aware of the Government of Bangladesh’s plans to roll out a new MTB-MLE policy. Second, IREX designed and conducted a multi-round book creation competition, attracting aspiring authors to create multiple titles in four different languages. Competitors attended an orientation to understand contest rules, national content and quality standards for supplementary reading materials, and took part in a required training in how to use Bloom, the self-publishing software. IREX and its partners then distributed print copies of the books to 100 communities and submitted digital copies to a2i, to be published through their Teachers’ Portal. Third, IREX and partners conducted the second phase of the attitudinal study to track changes in attitudes toward MTB-MLE and the value of a competitive book challenge to increase the availability and use of MTB-MLE Supplementary Reading Materials (SRMs) in Bangladesh. By the conclusion of the project, IREX and its partners published 61 unique Grade 1 SRMs, distributed 30,000 SRMs to primary schools and Para-Kendras (Pre-K learning centers) throughout the Khagrachari division, and ensured that all 61 titles were loaded to the a2i Teachers’ Portal. IREX is awaiting response from a2i on the latest download data for these titles and will happily share with the World Bank Group once these data have been furnished. The Onuprerona project focuses its Results-Based Financing intervention at the content creation level, providing incentives to authors to create content quickly that meets national curriculum standards. Authors that adhere to the submission guidelines and produce a winning title receive a cash prize, and recognition as a published author with content that has been approved by the regional tribal language institutes and the Government of Bangladesh. IREX was interested to understand if its efforts to raise awareness of the importance of MTB-MLE in the Khagrachari division resulted in attitudinal shifts from parents and the author community. Through a two-phase attitudinal study conducted over a 14-month period, IREX found mixed changes to attitudes among parents and authors. The study examined attitudinal shifts through three lenses: Cognitive, Affective, and Behavioral. IREX observed positive cognitive changes, as both parents and authors 2 Contract #7186365 – Descriptive Note (Revised November 2019) indicated that, irrespective of available resources or programs on offer, if a child can learn to read in their mother tongue, they are better positioned to excel in future scholarly pursuits. Respondents’ feelings or emotions toward MTB-MLE in their communities (Affective Component) did not change significantly over the life of the project. Considering that this component sought opinions about the publishing sector, the cost of purchasing books, and attitudes toward policy changes in MTB-MLE, we do not expect that attitudes surrounding these topics will change in such a short period of time. Lastly, the study looked at respondents’ interest or ability to play an active role in MTB-MLE efforts in their own communities (Behavioral Component). Both parents and authors expressed more interest and confidence during the second phase of data collection, citing greater confidence in either generating materials (authors) or helping their children access MTB-MLE content (parents). This suggests that both the workshops for authors and the parent awareness sessions offered some value to the respondents. While the policy implications for this activity in Bangladesh are limited, there are a few recommendations and conclusions worth noting. First, it is important to point out that this project was designed to lay a partial foundation for an MTB-MLE roll-out in select divisions of Bangladesh. IREX recognized that without a supply of content and without providing incentives to authors to create content in underserved languages, many students in the target division will struggle to get access to content at a critical stage in their primary education. While this activity might have limited potential for national scale, given the vast majority of Bangladeshis speak Bangla, there are still pockets of indigenous groups that lack reading materials in their mother tongue. If the Government of Bangladesh remains keen on operationalizing its MTB-MLE policy, these other divisions could benefit from a similar project model. Second, when examining the incentive structures of the Onuprerona project, it is very important to challenge prevailing assumptions about what might motivate a particular stakeholder group. While this project and the attitudinal study did not investigate broader applications beyond Bangladesh, we feel confident that similar human-centered approaches can serve complimentary RBF approaches well in different contexts. Lastly, Onuprerona’s RBF strategy has focused on the content creation piece, but the project’s engagement with a2i underscores the importance of at least understanding incentives of government stakeholders. Future efforts to engage government should consider a multi-prong engagement strategy, reaching out to multiple departments and ministries, keeping a broader swath of stakeholders accountable. 3 Contract #7186365 – Descriptive Note (Revised November 2019) Contents List of Acronyms .............................................................................................................................. 5 Introduction and Project Overview ................................................................................................ 6 Project Objectives ..................................................................................................................................... 6 Activities .................................................................................................................................................... 7 Theory of Change ...................................................................................................................................... 7 Use of Results-Based Financing ................................................................................................................ 8 Review of Key Activities .................................................................................................................. 9 Activity 1 ................................................................................................................................................... 9 Activity 2 ................................................................................................................................................. 13 Activity 3 ................................................................................................................................................. 18 Onuprerona Cost Analysis............................................................................................................. 22 Review of RBF Approach ............................................................................................................... 25 Recommendations and Conclusions ............................................................................................. 26 Opportunities for replication in Bangladesh........................................................................................... 26 Competition Model holds scaling potential ............................................................................................ 26 Benefits and Limitations of Bloom .......................................................................................................... 27 RBF Conclusions ...................................................................................................................................... 27 Appendices.................................................................................................................................... 28 Appendix A: End line Study tables .......................................................................................................... 28 Appendix B: Survey Instruments............................................................................................................. 30 4 Contract #7186365 – Descriptive Note (Revised November 2019) List of Acronyms a2i Access to Information Program CHT Chattogram Hill Tracts CHTDB Chattogram Hill Tracts Development Board CHTRC Chattogram Hill Tracts Regional Council DPE Directorate of Primary Education HDC Hill District Council KHDC Khagrachari Hill District Council MOCHTA Ministry of Chattogram Hill Tracts Affairs MoU Memorandum of Understanding MTB-MLE Mother Tongue-Based - Multi-Lingual Education NCTB National Curriculum and Textbook Board NGOAB NGO Affairs Bureau RBF Results-Based Financing SCI Save the Children SRM Supplementary Reading Materials TCI Tribal Cultural Institute ZKS Zabarang Kalyan Samity 5 Contract #7186365 – Descriptive Note (Revised November 2019) Introduction and Project Overview The Onuprerona Book Challenge is a competitive book creation activity that was carried out in the Khagrachari Division of Bangladesh. The Challenge uses Bloom software to streamline community-level production and distribution of supplementary reading materials for primary school-aged children. This activity seeks to re-envision the way mother tongue-based, multi- lingual education (MTB-MLE) supplementary reading materials (SRMs) enter the book supply chain, and in turn, communities, by adopting a results-based financing (RBF) model that offers unique incentives to content creators (authors, teachers, other community members), with the hope of sparking a new approach to creating SRMs in languages that children speak at home. This activity is borne out of the need for a new process that addresses bottlenecks in the current book supply chain. IREX and partners have identified three problems in Bangladesh: 1. A lack of SRMs for early grades and a lack of local content creators with tools to create those materials. This is particularly pronounced in regions where the minority mother tongue languages are spoken at home, including Bangla. 2. Overly complex government-driven book production, procurement, and distribution system 3. Reluctance within the traditional publishing industry in Bangladesh to publish underserved mother tongue materials of perceived limited demand, despite a recent policy by the Government of Bangladesh to progressively introduce MTB-MLE materials in underserved languages. This project sought to create a pathway for the Directorate of Primary Education (DPE) to deliver on its policy promise of expanding Mother tongue-based Multi-lingual education (MTB- MLE) to Grade 1 classrooms in Bangladesh. IREX forged this pathway by training authors, including teachers and tribal language experts, in the use of Bloom software. Participation was fueled by a mix of incentives—cash prizes to winning authors that enter the competition as well as recognition within the community as an award-winning author recognized by the Government of Bangladesh. The winning titles have been submitted to the a2i Teachers’ Portal, an online repository of educational resources for teachers, freely available for download and use. Project Objectives 1. Mobilize a cadre of teachers, librarians, and community resource representatives by offering capacity building opportunities, training on effective use of the Bloom Software and support on how to map new primary grade content to national quality standards 2. Incentivize primary grade book/title creation by awarding prizes to content creators who complete the software and capacity building training 3. Make available 60 new primary grade titles on the Teachers’ Portal, a digital repository access by more than 70,000 teachers. 6 Contract #7186365 – Descriptive Note (Revised November 2019) Activities This project facilitates the growth of a new cadre of local authors in Bangladesh, equipping them with the tools and skills to create high quality supplementary reading materials in underserved languages. The resulting content will enhance the supply of primary grade books in underserved languages and be made readily available for digital distribution through a2i’s Teachers’ Portal. The following activities drive this intervention: Activity 1: Conduct a longitudinal, attitudinal survey to access regional/national readiness for a results- based competitive approach to content creation; confirm divisions/mother-tongue languages/RBF activities/incentives; conduct “Awareness Raising Campaign”, marketing the Book Challenge to key local stakeholders Activity 2: Implement the Onuprerona Book Challenge, train teachers, librarians, and other stakeholders/content creators on the use of Bloom software and national-level quality standards Activity 3: Conduct end-line and satisfaction surveys and cost analysis on the use of a results-based competitive book challenge in Bangladesh; produce summary/analytical/descriptive notes Theory of Change IREX’s theory of change is structured to introduce inputs that improve cost-effectiveness as well as quality for the book supply chain by laying the foundation for a volume of production that meets local demand. IF existing and aspiring local language content creators are incentivized and given the capacity through a book challenge to create new high-quality content in selected mother tongues; AND existing government programs, like the a2i program, under the authority of the Prime Minister’s Office, have the opportunity and tools to test a cost-effective book creation challenge rooted in results-based financing; THEN high-quality reading materials will be more readily available to support MTB-MLE in Bangladesh, and the Directorate of Primary Education will have an evidence base for a cost- effective model to create supplementary reading materials in underserved languages where there is not a market for book publishers in Bangladesh to create those materials. This Theory of Change also addresses barriers outlined above by bypassing constraints within existing book production channels. It provides clear incentives for content creators to efficiently 7 Contract #7186365 – Descriptive Note (Revised November 2019) create diverse, high quality books, in addition to expediting the approval of these books through the Teachers’ Portal. Use of Results-Based Financing This project created incentives for communities to meet the need for diverse, but locally relevant materials in mother tongue languages, all the while aligning the process with a2i’s mandate to make SRMs more readily available to teachers and schools. Through these regional competitions, authors were given the opportunity to create new titles, increase their visibility in the local publishing community, and win prizes for their efforts. These incentives fueled a cadre of teachers, librarians, and community members to produce almost 300 submissions for consideration. By incentivizing local content creation in underserved languages, the project is filling a gap left by traditional publishing houses, who struggle to justify the cost of executing small print runs for languages other than Bangla. This descriptive note will provide conclusions and recommendations for the costing and potential replication challenges in other contexts. The note will also look at how RBF was applied in Bangladesh and how it could be replicated to other similarly structured activities. The descriptive note is organized as follows: 1. Review of Key Activities over the life of the project 2. Presentation and analysis of results of the second phase attitudinal survey 3. Cost analysis 4. Review of RBF approach 5. Recommendations and Conclusions 8 Contract #7186365 – Descriptive Note (Revised November 2019) Review of Key Activities Table 1 Process Indicators Key Process Indicators To Date Target Notes # Content creators trained in Bloom 101 100 # Titles Finalized 61 60 Disaggregated next row by language Bangla 16 15 Marma 15 15 Kokborok 15 15 Chakma 15 15 # Titles uploaded to a2i Teachers’ Portal 61 60 # Reading Festivals 105 100 # Parent Awareness Sessions 92 100 # Copies distributed to primary schools 28,250 N/A See p. 5 (book distribution) for # Copies distributed to Para-Kendras (Pre-K 1750 N/A a print run breakdown by programs in CHT) language Activity 1 January-February 2018: Project Scoping and Assessment IREX conducted an assessment in Khagrachari, the focal administrative division for the Onuprerona Book Challenge. The purpose of the assessment was to understand the status and availability of children’s books, particularly in areas where underserved languages are dominant. This is also a division that has been targeted by the Directorate of Primary Education (DPE) in the roll-out of the new MTB-MLE policy in Bangladesh. During the assessment, IREX visited Chakma and Kokborok-speaking areas, including Panchari and Matiranga. Key Takeaways from this assessment • Public express appetite for more minority language materials appropriate for children. There have been several small-scale initiatives, but the costs of publishing widely have been prohibitive. The Bloom software offers an effective solution worth testing. • With so few people literate in minority languages, it will be important to start from beginner levels. Books created through Onuprerona must meet the needs of very early readers and their parents. • The pause in full implementation of the MTB-MLE policy raises the issue of distribution of books in underserved languages. Even teachers who are native Chakma and Kokborok speakers have not used the government-produced materials, so additional materials are not a school priority right now. However, as the policy is pending, some experience in meeting the need for supplementary reading materials will 9 Contract #7186365 – Descriptive Note (Revised November 2019) be extremely valuable. The Onuprerona program will need to focus on promotion through reading festivals and local activities once books are selected and published. • Overall, there remains a lack of appropriate reading materials for children learning to read, across all languages. Stakeholders saw the value in mechanisms for addressing this deficiency. This suggests a much wider audience for Onuprerona beyond minority languages in Bangladesh. The second stage of the contests will invite participants in Bangla as well. June 2018 – Baseline Attitudinal Survey IREX and Save the Children designed a prospective cohort survey that tracked the attitudes and opinions of key stakeholders in the book creation space. The study is intended to give voice to stakeholders and to ensure that the findings are grounded in respondents’ experiences and attitudes toward the production and supply of early grade supplementary reading materials in mother tongue languages. The baseline survey has confirmed several assumptions that our team made in the lead up to this project, but it has also highlighted some challenges. First, while there is broad agreement about the value of creating new materials in mother tongue, there is a lack of consensus on how this should be administered. Second, the results underscore the importance of sustaining engagement with key stakeholders throughout the process. Parents, teachers, and authors all acknowledge that there is value in providing materials to children in their mother tongue; what remains is to test new approaches that circumvent the traditional book production process and see if the base support among stakeholders can blossom into long-term engagement to help sustain a new method of producing local language reading content. Confirm Divisions/Mother Tongue Languages In consultation with partners, IREX selected Chakma, Kokborok, Marma and Bangla as focus languages for the competition. Chakma and Kokborok were selected for the first set of workshops in July 2018, as these languages are severely underserved and anticipated to be among the first languages instituted in the formal school system when the MTB-MLE policy is implemented. Bangla and Marma were selected for the second round of the competition, which was carried out in February 2019. Table 2 Competition structure Competition Round Language Date Participants Books Winning Submitted Titles Chakma July 3-5, 2018 25 15 Round 1 121 Kokborok July 6-8, 2018 26 15 Marma February 4-6, 2019 25 15 Round 2 131 Bangla February 7-9, 2019 25 16 Total 101 252 60 10 Contract #7186365 – Descriptive Note (Revised November 2019) Awareness Raising Campaign Acting upon the information contained in the initial project assessment in February 2018, IREX has continued to prioritize raising the profile of the project and bringing MTB-MLE issues to the fore at both the national and regional level. In 2018, IREX and Save the Children held a stakeholders’ forum, convening representatives from various government agencies, the publishing sector, NGOs, and indigenous language expert advocacy groups. This, combined with media coverage of the Onuprerona competition and outreach, the National Curriculum and Textbook Board (NCTB) contacted the Onuprerona team with a request that the project team deliver professional development training to NCTB staff on how to use Bloom software to create SRMs. While this fell outside the immediate scope of this activity, it was an incredibly valuable opportunity to engage this critically important government partner, using the training event as a springboard for more substantive conversations about how this model can be folded into NCTB planning efforts around MTB-MLE materials creation. At the regional level, the Onuprerona project team continued to engage key stakeholders, to sustain interest in the Onuprerona model and to ensure that regional stakeholders are maintaining their interest and focus on the new MTB-MLE policy. The project team held several discussions with the Chattogram Hill Tracts Development Board (CHTDB), which supports an active network of Para-Kendra (pre-school) facilities throughout the Khagrachari district. The CHTDB committed to promoting the Onuprerona model and took steps to build reading programs using SRMs created through the Onuprerona competitions in their Para-Kendras. To support this process, the Onuprerona project distributed 1750 copies of the winning titles to the CHTDB, which will distribute the books to Para-Kendras throughout Khagrachari. IREX and partners continued local-level engagement in 2019, hosting reading festivals and parent awareness sessions in rural communities throughout Khagrachari. As of August 2019, the project team conducted 92 training events with parents and 105 reading festivals, held at local primary schools. The parent awareness sessions continue to be an effective way of educating parents about MTB-MLE policy, so that they have the information they need to discuss with teachers and school administrators. It is also an opportunity to showcase the Onuprerona project, share sample books, and to pick up some techniques on how to engage their children in reading in their mother tongue. The training is sensitive to the fact that many parents themselves are illiterate, so it focuses on reading engagement strategies that do not require that parents know how to read. 11 Contract #7186365 – Descriptive Note (Revised November 2019) Parent Awareness Training Session, Khagrachari, 6 November 2018 The reading festivals take place at the local primary schools throughout the Khagrachari district. The reading festival is a celebrated and well-established model at the local level. Given the significant overlap between the Onuprerona outreach effort and the typical structure of a community-level reading festival, the project team felt that this was a natural fit. After hosting reading festivals in over 100 communities, the Onuprerona project team organized a district level reading festival, which served as a culmination of top performing students representing 105 communities. Together with their parents and their teachers, these students competed at the district level. During the festival, the project team gave a presentation on Onuprerona. In addition to teachers, parents, and students, participants included: • The Chairman of the Khagrachari Hill District Council • The District Assistant Primary Education Officer of Khagrachari • The Deputy Director of the Cultural Institute of Khagrachari • Upazila Education Officers, Assistant Upazila Education Officers from every Upazila in the district 12 Contract #7186365 – Descriptive Note (Revised November 2019) Upazila reading festival 29 July 2019 Activity 2 Onuprerona Book Challenge The Onuprerona Book Challenge two place in two rounds. The first round took place in July 2018 for the Chakma and Kokborok language groups. The second round took place in February 2019 for Marma and Bangla. Each competition followed the same structure: Step 1: Open Call for participation. Participants complete application to participate in Onuprerona Book Competition. Step 2: Onuprerona Workshop. Four workshops (Chakma, Marma, Kokborok, Bangla), 2 days each, approx. 25 participants per workshop. At the workshop, participants are trained in use of bloom, team up with illustrators, and create a minimum of four books, one from each category (Alphabet; Picture Dictionary; Rhyming Stories; Storybook). Step 3: Initial Review Workshop. Two-day activity. Project team conducts initial pass of the submissions and weeds out submissions that do not meet basic submission requirements. Step 4: Final Review and Selection Workshop. Project partners invite language experts, district education officials, and members of the tribal cultural institute to review the shortlisted titles and select the winning titles (approximately 15 titles per language). In many cases, even the winning titles require additional edits, so the review team provides comment on the winning titles). 13 Contract #7186365 – Descriptive Note (Revised November 2019) Step 5: Author feedback workshop. Project partners meet with winning authors to review, update, and finalize their books). Step 6: Award Ceremony. Winning authors are recognized for their hard work and are awarded a trophy and cash prize. Representatives from DPE, Khagrachari Hill District Council, Tribal Cultural Institute, and the Chattogram Hill Tract Regional Council all to attend the award ceremonies. Round 1 (June-November 2018) IREX and partners conducted Round 1 of the Onuprerona Book Challenge in two successive blocks, divided by language groups. The first block took place July 3-5 (Chakma language) and the second block took place July 6-8 (Kokborok language). IREX received 101 applications to participate in the workshops. SCI and ZKS reviewed the submissions and selected 51 participants (25 competing in the Chakma track and 26 competing in the Kokborok track). The Book Challenge Workshop consisted of an introduction to Bloom software, training on the book development process, an overview of the competition guidelines, and time to draft mockups of books, enter stories into Bloom, and finalize submissions. Reflecting on the experience of conducting Round 1 of the workshops, the team did identify some challenges that would eventually be addressed in the second round. The structure of the workshop required participants to generate books in each of the four categories. As a result, quality suffered on some of the books. For Round 2 of the competition, competitors will submit only as many books as they believe they can complete that meet quality standards – there will be no expectation of a complete set. IREX and partners anticipate this will result in fewer total submissions, but those submitted of higher quality. Round 2 (February-June 2019) Building from lessons learned in the first round of the competition (July 2018), IREX and partners adjusted the training curriculum, facilitation materials, and some parameters of the competition to generate more interest and to optimize quality output of titles. During the first round, participants were required to submit at least one title from each of the four book categories. This forced some authors to divert time and attention away from categories that best aligned with their strength and interests. As a result, we received a number of low-quality submissions for some of the categories. For the second round, there was no requirement that each author submit at least one title for each category. IREX assumed this would result in fewer, but higher quality submissions. Both the output and the quality increased (131 books created in Round 2 vs 121 in Round 1). The second round of the competition focused on Bangla and Marma languages. The original project work plan did not include Bangla; however, in considering the overarching goal of the activity—to elevate the profile of local content creation and to promote a new incentive 14 Contract #7186365 – Descriptive Note (Revised November 2019) structure for generating content quickly and at low cost —IREX recognized that including Bangla, the national language of Bangladesh, was a critical and strategically significant adjustment. Illustrator at the Marma Onuprerona competition, 4 February 2019, Chattogram, Bangladesh In early January 2019, IREX and partners published announcements in the national newspaper the Daily Prothom-Alo, announcing the second round of the Book Challenge. Within two weeks, the project team had received 131 applications to participate. Of these, 50 participants were selected to take part in the competition. Like Round 1 workshops, Round 2 was split into two groups, on the basis of language. 25 authors convened for the Marma language workshop in Chattogram, Bangladesh, February 4-6, 2019. 25 authors participated in the Bangla workshop immediately following the Marma workshop, February 7-9, 2019. While we set a target for 64 titles (from which the selection committee would select the top 30), authors from the Marma and Bangla workshop groups produced 131 titles. 15 Contract #7186365 – Descriptive Note (Revised November 2019) Title review, selection, recognition Following the same format as Round 1, IREX and partners executed a multi-stage process to review titles, winnow down the pool of titles, submit feedback to the remaining authors, and ultimately select the winning titles. IREX and partners Save the Children in Bangladesh and ZKS carried out these activities between February and April 2019. IREX and SCI co-hosted award ceremonies for the Marma and Bangla competition groups. The ceremonies were held June 17 and June 26, 2019, respectively. Following the same format as Round 1 award ceremonies, these events were open to the public and intended to showcase a new cadre of trained children’s book authors. The winning authors were awarded plaques and prize money as recognition. Book distribution The winning titles are distributed through two channels: physical print distribution and digital distribution through a2i. The physical distribution, while relatively small in scale, is a critically important component for raising the profile and generating demand for the books among teachers and school officials at the district level. As of August 2019, the Onuprerona project team has printed and distributed 30,000 copies of 61 titles, distributing these to over 100 schools and the Chattogram Hill Tracts Development Board (CHTDB). Table 3 Book distribution by language Language Distribution Chakma 7,800 Tripura/Kokborok 7,800 Marma 7,400 Bangla 7,000 Total 30,000 Shortly after selecting the 31 winning titles from Round 2, SCI submitted digital copies to a2i to be uploaded to the Teachers’ Portal, bringing the total number of titles to 61. In January 2018, a2i officially launched the MTB-MLE book repository on the Teachers’ Portal. At that time, a2i had 30 of the eventual 61 titles. Unfortunately, a few months after going live, IREX observed issues with the site, and was unable to download titles. IREX instructed SCI to contact a2i and request details related to the matter. a2i has since resolved the issue and the books are once again available to teachers and the general public. For more information on the partnership with a2i, see “MOU with a2i” below. 16 Contract #7186365 – Descriptive Note (Revised November 2019) MOU with a2i On March 13, 2019, the Onuprerona project team and a2i participated in a formal MOU signing ceremony, concluding months of negotiating mutually agreeable terms that will outline a partnership to ensure the content created through the Onuprerona competition model will have institutional backing from the government of Bangladesh. The agreed upon responsibilities of the different parties is included below. Table 4 MOU responsibilities by party Onuprerona Team (IREX, SCI) Commit to developing at least 60 supplementary reading materials using Bloom software, targeting early grade learners in different mother tongue languages spoken in the Chattogram Hill Tract (CHT) area. Submit content to a2i for upload to the Teachers’ Portal. Commit to marketing the content through professional networks and social channels to promote the new materials and to make sure that teachers, parents, and students are aware of the content. Access portal to collect information on the usage of the supplementary reading materials. Provide guidance and instructions to develop books using Bloom software. Ensure that books will be maintained on the a2i Teachers’ Portal. Any cost related to updating the platform will be covered by a2i. Provide feedback on text and image quality of supplementary reading materials, as needed. Leverage a2i communications and advocacy networks to promote these books with authorities at various levels in the Government. a2i Provide written guidance and instructions to teachers on how to develop books using Bloom software. Provide instructions on how to download the free software. Ensure that books will be maintained on a2i Teachers’ Portal. Any cost related to ensuring performance of the portal remains a2i’s sole responsibility. Provide feedback on text and image quality of supplementary reading materials, as needed. Leverage a2i communications and advocacy networks to promote these books at different levels of authorities. The MOU signing ceremony was delayed due to national elections the last week of December 2018. The subsequent government transition in early 2019 further delayed the signing ceremony. Despite this delay, a2i staff maintained close contact and assured the Onuprerona project team that they were moving forward with setting up the Teachers’ Portal in anticipation of a signed MOU. a2i received the first batch of titles in late 2018 and in January 2019, a2i announced that the Teachers’ Portal had gone live with the first batch of 30 titles. 17 Contract #7186365 – Descriptive Note (Revised November 2019) In early May 2019, IREX observed some technical issues with the Teachers’ Portal. Specifically, the digital story files were not loading properly. IREX immediately flagged this issue and with the support of partner SCI, began troubleshooting with a2i. a2i immediately responded and confirmed that there was a system wide issue affecting the whole digital reading hub, not just the a2i -Onuprerona portal. In August 2019, a2i confirmed that a2i the issue cannot be resolved without first re-building the platform. Anticipating that it would take an indeterminate amount of time to re-launch the fully functioning portal, the project team negotiated a temporary workaround with a2i−embedding a link to an external repository of the story files, so that teachers and the public can still access the content while the Teachers’ Portal is being rebuilt. In early September 2019, a2i shared the following link, which includes access to all 61 of the winning titles: https://bit.ly/2kF1HY5 IREX submitted requests to a2i for download data. In October 2019, the current project manager of the Teachers’ Portal informed IREX that the legacy system, which was in place for most of 2019, did not have a system to track visits or downloads. a2i is currently rebuilding the Teachers’ Portal and intends to track downloads, page visits, among other forms of site engagement. IREX has requested that a2i provide monthly analytics and a2i has agreed. Activity 3 End line Attitudinal Study This study is the second of two phases of a prospective cohort study, measuring changes in the attitudes, beliefs, and feelings of community stakeholders that are active participants in children’s learning in the Khagrachari District of Bangladesh. The specific objectives of t his study are as follows: • Understand and measure changes in people’s stated beliefs and feelings of early reading materials in mother-tongue languages • Ascertain the perceived value of a competition model to generate early reading materials in mother tongue languages. • Understand the challenges and barriers in the production of early reading materials in mother tongue languages. Approach/Methodology The study followed a panel analysis to identify changes in attitudes among respondents. The study examines changes in respondents’ beliefs, feelings, and confidence as it pertains to MTB- MLE policy, the Onuprerona competition model, and the perceived value of providing children with mother tongue supplementary reading materials. The enumerators engaged the same respondent groups from the first phase of the study, which was conducted in June 2018: 18 Contract #7186365 – Descriptive Note (Revised November 2019) • Parents of Grade 1-3 students in the Khagrachari district (Parents). Total Respondents from this group: 56 • Participants in the Onuprerona Book Challenge (Authors). Total respondents from this group: 33 Cognitive Component Affective Component Behavioral Component Respondents’ knowledge and Respondents’ feelings or Respondents’ intentions and beliefs about MTB-MLE emotions toward MTB-MLE in assessment of their interest/ability their community and how it to play an active role in supporting might affect Grade 1-3 or challenging the roll-out of MTB- MLE in their communities students The study was designed around a set of statements intended to understand respondents’ attitudes toward MTB-MLE for Grade 1-3 students in the Khagrachari division. Statements were measured using a 5-point Likert scale, ranging from (1) strongly agree to (5) strongly disagree. While questions directed at authors attempted gauge their attitudes toward the publishing industry and unique challenges faced by authors of mother tongue content, the study did not describe RBF specifically and as such, Authors did not comment explicitly on their motivations to participate in the competition. In addition to these statements, enumerators administered a semi-structured interview tool to glean deeper insights into respondents’ attitudes toward the Onuprerona competition as well the general state of publishing mother tongue content for children. Data were analyzed using a data scoring framework and applying a weighted mean to participants’ opinions about their attitude toward the competition model for generating supplementary reading content. Table 5 Data scoring framework Responses Weight for positive Weight for negative statement statement Strongly Agree 1 5 Agree 2 4 Neutral 3 3 Disagree 4 2 Strongly Disagree 5 1 19 Contract #7186365 – Descriptive Note (Revised November 2019) Table 6 Weighing attitudinal components Weighted Mean Cognitive Component Affective Component Behavioral Component < 1.5 Strong positivity Highly positive feeling High confidence 1.6 - 2.5 Fairly good positivity Fairly good feeling Fairly good confidence 2.6 – 3.5 Mildly low positivity Mildly low feeling Mildly low confidence 3.6 – 4.5 Moderately low positivity Moderately low feeling Moderately low confidence > 4.5 Severely low positivity Severely low feeling Severely low confidence Summary of findings Cognitive Component - Changes in belief structures IREX observed positive cognitive changes, as both parents and authors indicated that, irrespective of available resources or programs on offer, if a child can learn to read in their mother tongue, they are better positioned to excel in future scholarly pursuits. Authors selected to participate in the competition uniformly expressed strong belief that learning how to read in one’s mother tongue is a necessity. All respondents agreed or strongly agreed that reading for pleasure is a critical component of a child’s learning process. As stated in the baseline, authors continue to believe that learning to read in one’s mother tongue is a right. Parents indicated a strong belief that learning in one’s mother tongue is a necessity for future educational success. Results also show that most parents strongly agreed or agreed that children must first learn to read in their mother tongue before learning to read in Bangla. Here, we found limited change between baseline and end line responses, with a slight trend toward stronger beliefs in the importance of MTB-MLE provision. Affective - Changes in emotions toward MTB-MLE efforts The affective component attempted to understand changes in respondents’ feelings toward MTB-MLE efforts, the current state of the publishing sector, and opinions related to whether MTB-MLE instruction should be a child’s right. Attitudes did not change significantly over the life of the project. Considering that this component sought opinions about the publishing sector, the cost of purchasing books, and attitudes toward policy changes in MTB-MLE, we do not expect that attitudes surrounding these topics will change in such a short period of time. Authors had mixed opinions about the cost of producing books (12% believe the cost is too high while 18% believe that costs are reasonable). More than half of the authors believed that there is little to no market demand for SRMs in minority languages, but most still believe that efforts 20 Contract #7186365 – Descriptive Note (Revised November 2019) such as Onuprerona play an important role in filling a gap that publishing houses are unwilling or unequipped to fill. Most parents expressed optimism and eagerness surrounding their child’s ability to learn how to read in their mother tongue. Most agreed that there remains a lack of local authors and interested publishers to generate a sufficient supply of content to meet a growing demand. Parents mostly agree that learning to read in one’s mother tongue is a right. This finding remains consistent with baseline results. Behavioral - Changes in Respondents’ interest/ability to play an active role in MTB-MLE efforts in their community and professional networks The study looked at respondents’ interest or ability to play an active role in MTB -MLE efforts in their own communities. Both parents and authors expressed more interest and confidence during the second phase of data collection, citing greater confidence in either generating materials (authors) or helping their children access MTB-MLE content (parents). This suggests that both the workshops for authors and the parent awareness sessions offered some value to the respondents. Authors expressed strong support for the Onuprerona Challenge model citing high levels of satisfaction with the Bloom software, pointing to its user-friendly interface and ability to accommodate many languages and scripts. We asked authors to comment on how the Onuprerona Challenge model differs from traditional publishing methods. All respondents agreed that the challenge provides more concrete pathways to publishing and most have increased confidence in their ability to generate new mother tongue-based stories in the future. While most parents were confident that the Onuprerona Challenge encouraged them to play a more active role in their child’s reading experience, some were less confident that there is sufficient access to supplementary reading materials. The general concern surrounds volume of content. When compared to English or Bangla language content, content in minority languages, by comparison, remains hard to come by. Several parents sampled took part in Onuprerona Parent Awareness sessions (see p. 6), but some raise concerns that parental support for connecting children to mother tongue supplementary content remains inconsistent. Finally, parents also expressed concern over the feasibility of exposing children to supplementary reading materials through the smartphone, presumably a reference to the a2i Teachers’ Portal. Study Conclusions The study provided useful comparative analysis of both authors and parents of Grade 1-3 students in Khagrachari district. At the time when IREX designed the baseline, IREX had not finalized the list of languages for the competition. Based on an assessment conducted in 21 Contract #7186365 – Descriptive Note (Revised November 2019) February 2018, IREX determined it would likely engage Chakma and Tripura communities in the first round of the competition, and tentatively planned to engage Bangla and Marma language communities for the second round of the competition. When designing the baseline study, IREX chose to limit the sampling to Chakma and Tripura communities, as it had not finalized the language groups for the second round of the competition. As such, the attitudinal study does not include parents or authors from the Bangla and Marma language communities. The end line study generated few surprises. The study showed modest gains across all three components - affective, cognitive, behavioral. While the study did not surface any issues that warrant significant course-correction in future RBF-based projects, there remain a few critical takeaways: • Future book development competitions that take place in resource-constrained environments should be intentional about pairing competitors to promote collaboration and to optimize creative flow. During the second round of competitions, IREX and project partners adjusted the parameters of the competition to invite authors to collaborate to generate more content more quickly. What resulted was a significant increase in book submissions. Allowing for more collaboration allowed authors with different skillsets to team up to streamline activities. For example, one author with limited computer skills but with a pool of story ideas could partner with another author that can quickly convert story ideas into text using the Bloom software. • To the extent that resources allow, consider higher quality printing options for the final products. At present, Bloom books are intentionally configured to be printed using basic desk jet or inkjet printers. This is to ensure that anyone with access to a basic printer can print a Bloom title. But the project team recognizes that the visual appeal of SRMs is a powerful component that cannot be ignored. Children are drawn to books not only for the quality of the story but the vibrancy of the illustrations. At present Bloom titles are only available in black and white. We recommend working with the makers of Bloom (SIL-Lead) to explore ways to generate full color illustrations to strengthen the appeal of the titles. Onuprerona Cost Analysis As an addendum to the attitudinal study, IREX commissioned a comparative cost analysis of the Onuprerona competition model. The purpose of the analysis was to compare the cost of generating titles under the Onuprerona Book Challenge model to traditional publishing methods in Bangladesh. The study looked at three different types of content creation for early grade SRMs. 1. The Onuprerona Project 22 Contract #7186365 – Descriptive Note (Revised November 2019) 2. Traditional publishing sector 3. International NGO The cost analysis attempted to control for significant variance in cost assumptions for the three different publishing approaches. For example, International NGO publishing efforts in Bangladesh, typically combine SRM print runs with textbook or primary reader procurements, which could result in volume-based cost savings. NGO publishing activities are also subject to strict procurement policies, both imposed by their donor (in the case of this analysis, USAID) and internal policies to ensure cost-reasonable procurements. The Onuprerona project sees significant cost savings in manuscript development, due primarily to cost savings in avoiding royalties and marketing expenditures. Bloom software, the platform used to create and format the titles, is free to download and competing authors provided their own laptops to generate content. The major costs for manuscript development under the Onuprerona project were operational—the prize money and other incentives formed a small portion of the total cost to develop a title. Table 7 Comparative cost analysis for manuscript development Publishing Manuscript Printing # of Total Cost Cost per Method Development (Variable copies Copy (Fixed cost) Cost) Onuprerona $181.87 $0.11 500 $242.88 $0.48 Project Private sector $266.15 $0.11 500 $324.22 $0.65 book publisher International $585.70 $0.11 500 $643.60 $1.29 NGO To evaluate the cost-effectiveness of the Onuprerona project, we compared Onuprerona’s inputs with the standard practice of private sector book publishers and the practice of other organizations that develop books for Grade 1-3 children that follow a similar process and projected cost for manual printing at local level by individual users. The cost to develop or otherwise source a manuscript and then prepare a printing plate are considered fixed costs for the purpose of this analysis. The cost of printing of the content pages and cover page and charge of binding are treated as variable costs. This analysis is based on assumption that single story will have a print run of 500 copies. The study also considers quality control —review, editing of manuscripts, the extent to which content adheres to recognized learning standards, age-appropriateness of the content, and the quality (durability) of the printed books. 23 Contract #7186365 – Descriptive Note (Revised November 2019) Manuscript Development Costs - Onuprerona In the process of developing 61 stories, the Onuprerona project incurred a total cost of BDT 930,239.13 (~11,000 USD) or BDT 15,250.00 (~182 USD) per title. This cost includes all operational expenses to organize training for competition participants, development of training materials, and project staff time, among other costs. For printing 500 copies, the project incurred a cost approximately BDT 10 per copy, which is consistent with the industry standard. In consideration of these expenses and factoring in all other inputs, the cost per copy comes to ~.48 USD. Manuscript Development Costs - Private Publishers Private publishing houses generally buy the rights to a manuscript from the authors. While the market rate varies widely depending on genre or reputation of the author, SRMs manuscripts in Bangladesh for Grade 1 students typically range between BDT 15,000 to BDT 30,000 (~175-350 USD). For the sake of this analysis, we took the average of BDT 22,500 (266 USD) as the base cost per manuscript. Assuming a print run of 500 copies, and a privately published story will cost approximately BDT 56 (~.64 USD) Manuscript Development Costs - International NGO Other organizations that publish SRMs spend approximately BDT 50,000 (585 USD) per manuscript. Assuming the same variable printing cost of BDT 10 per copy, International NGO- published SRMs cost BDT 110 (1.29 USD) per copy. Like Onuprerona Project the organizations can impose similar level of authority to ensure style, quality, focus, and context appropriateness of the stories. Limits of this analysis This analysis controls print volume for the purposes of comparing publishing methods. That said, it is true that larger print runs will typically result in cost savings. While it is true that a private book publisher or the International NGO can lower their unit cost by securing large volume orders, it is important to note that this cost analysis did not explore this variable. What this analysis can confirm is that, when controlling for print run volume, manuscript development for SRM content is significantly less per title under Onuprerona when compared to private and bilateral-funded publishing efforts. While this cost analysis provides a useful basis of comparison, the comparison is limited to data within Bangladesh and does not necessarily represent pricing in other countries or regions. Also, the data was provided by a limited number of sources so this analysis cannot state that the data shared by private publishers and NGO publishers are truly representative of the broader market in Bangladesh. Furthermore, this analysis does not account for variation in quality of published product, which certainly affects the cost of manuscript development. Bloom books are designed to be printed using any available printer, and as such, do not meet the printing standards of a traditional publishing house. 24 Contract #7186365 – Descriptive Note (Revised November 2019) Review of RBF Approach To date, Bangla continues to dominate the book market in Bangladesh. Existing incentive structures push authors toward creating content in Bangla, challenging efforts to generate reading content in underserved languages. This is especially pronounced in the education space, where the book supply chain prioritizes Bangla language content and is primarily focused on textbooks. While there is a new Department of Primary Education (DPE) policy that mandates MTB-MLE instruction for Grade 1 students and mandates that SRMs be made available to all Grade 1 children, district level education offices are still awaiting guidance on how to execute the new policy. It is under this backdrop that IREX proposed the Onuprerona Book Challenge model, with a specific focus on generating incentives for local authors to stimulate the creation of content so that communities are better poised to respond to planned directives from education officials. By hosting four separate book creation competitions, IREX and its partners have given teachers and authors legitimacy, visibility, and a platform to create and promote new content. By producing printed copies that have been distributed to 100 communities, combined with submitting 61 unique titles to a2i, authors that represent underserved languages now have training on and access to a platform that can address content gaps for SRMs. The project provides two streams of incentives: financial and reputational. The financial incentives came in the form of cash prizes to winning authors. The reputational came in the form of public recognition, both through public events and through print and digital distribution of the authors’ winning titles. The use of the a2i Teachers’ Portal has added a level of prestige to the process, that, in the absence of a commitment from an established publisher, is difficult to replicate by other means. Many teachers that we interviewed, as well as authors that took part in the competition were aware of a2i and were energized by the prospect of having their content featured on the Teachers’ Portal. By incentivizing local content creation and linking this content to a2i’s digital distribution mechanism, the Onuprerona project has provided a blueprint to mobilize authors of underserved languages and to provide a sustainable pipeline of new SRM content that meets DPE and NCTB quality standards. During the field assessment in early 2018, IREX learned that financial incentives alone might not be the sole motivator for authors in the CHT region to fully participate in the competition. Stakeholders cited cultural preservation and a duty to preserve underserved languages as being critical motivating factors in participating in the competition. While it is possible that focusing purely on financial incentives could have resulted in greater initial participation/expression of interest, we expect that there would be greater variance in the qualifications of participants and in turn, greater inconsistencies in the quality of the submitted titles. By diversifying the 25 Contract #7186365 – Descriptive Note (Revised November 2019) incentive structure to include both financial and reputational incentives, the project team is confident that it achieved a productive balance of authors that were energized by the effort (i.e. generating new SRM content in underserved languages), motivated to compete with their peers for prize money and public recognition, but also positioned to generate content when the government is ready to operationalize its MTB-MLE policy. Recommendations and Conclusions Opportunities for replication in Bangladesh It is important to note that the Onuprerona project, from the very beginning, focused on a region of Bangladesh that is ethnically diverse, especially in comparison to most other regions. The original impetus for developing the Onuprerona Challenge was to establish a model that will help one component of a larger MTB-MLE policy roll-out, in this case, the provision of SRMs in underserved languages. On a national level, Bangladesh can be considered a mono-linguistic country. 98% of Bangladeshis are fluent in Bangla and consider it their first language.1 This certainly helps explain why national publishers struggle to see the value in publishing content in indigenous languages. From a technical programming perspective, there is certainly opportunity to scale this model in other regions of Bangladesh, but the need for such a model diminishes significantly when focusing on Bangla language content creation. It is a market with a healthy bench of authors and publishers that are positioned to respond to public and government demand. That said, there is still a place for this model in Bangladesh, expanding to more districts in the Chattogram division and other divisions where indigenous groups struggle to source mother tongue reading content for children, including Rajshahi, and Sylhet Divisions. The challenge will be generating a groundswell of political will from the national government. With this project, small as it may be, we were able to secure commitments from a2i to publish and distribute the titles, but the next, critical step is to formalize a plan with DPE. Competition Model holds scaling potential Each round of the Onuprerona competition followed a similar format. The first three hours is focused on orienting participants to the rules of the competition and taking part in a training on how to use Bloom software. The remaining 1.5 days are dedicated to content creation. 1 Faquire, A.B.M Razaul Karim (December 2010). “Language Situation in Bangladesh”. The Dhaka University Studies. 67: 63-77 26 Contract #7186365 – Descriptive Note (Revised November 2019) Compared to other training activities, this is a highly efficient process, and it speaks to how easy Bloom is to use, even for those with limited digital literacy skills. Another way to frame the efficiency of the Onuprerona competition model is to compare training hours to manuscript output. In total, the Onuprerona competition logged 300 person hours of training, resulting in 241 manuscript submissions, which in turn resulted in 61 winning titles. Compared to other training programs, this is a relatively productive return on investment, especially considering that not a single participant entered the competition with any working knowledge of Bloom software. Furthermore, now that these participants are equipped with the software and know how to use it, they are positioned to generate more titles, should new opportunities arise. This suggests that the model works and, with some adjustments, could be scaled to other divisions in Bangladesh and certainly other countries that are grappling with a shortage of MTB-MLE content for children. Benefits and Limitations of Bloom As noted above, one of the benefits of Bloom is the relative ease by which first time users can master its core functions. Even those with limited digital literacy skills have lauded the software’s user-friendly interface and intuitive functions. IREX has trained authors in the use of bloom throughout Asia and believes it is the best free tool for generating MTB-MLE materials quickly. That said, Bloom does have limitations. Bloom cannot match the print quality of a professional publishing house. Bloom is intended to be printed on a personal printer, so the final product is not as durable as a book produced at a traditional printing house. RBF Conclusions When examining this effort, focusing on the content creation aspect alone, what we have learned is that it is very important to challenge prevailing assumptions about what might motivate a particular stakeholder group. While this project and the attitudinal study did not investigate broader applications beyond Bangladesh, we feel confident that similar human- centered approaches can serve complimentary RBF approaches well in different contexts— engaging stakeholders early before prescribing a specific RBF approach. Onuprerona’s RBF strategy has focused on the content creators, but the project’s engagement with a2i underscores the importance of at least understanding incentives of government stakeholders. In the case of a2i, IREX lacked sufficient leverage to guarantee distribution of digital titles. Given the small scale of this activity, we did not feel equipped to consider an RBF approach to securing intended results with our government partners; rather, we relied on the scale and scope of a2i’s mission as a driver to ensure their ability to provide this vital service to the public. For future engagements, one might consider a more aggressive incentive structure or an agreement with binding terms to ensure expected performance. 27 Contract #7186365 – Descriptive Note (Revised November 2019) Appendices Appendix A: End line Study tables Table A1. Changes in attitude (Authors - gender) Gender Cognitive Affective Behavioral Baseline End line Baseline End line Baseline End line Male Mean 2.15 1.61 2.23 2.43 2.45 2.20 N 30 20 30 20 30 20 Std. 0.26 0.35 0.21 0.24 0.37 0.42 Dev. Female Mean 2.07 1.40 2.25 2.26 2.67 1.96 N 10 13 10 13 10 13 Std. 0.26 0.18 0.27 0.26 0.44 0.54 Dev. Total Mean 2.13 1.53 2.23 2.36 2.51 2.11 N 40 33 40 33 40 33 Std. 0.26 0.31 0.22 0.26 0.39 0.48 Dev. Table A2. Changes in Attitude (Authors – language) Mother Tongue Cognitive Affective Behavioral Baseline End line Baseline End line Baseline End line Chakma Mean 2.22 1.55 2.27 2.41 2.55 2.17 N 20 18 20 18 20 18 Std. 0.27 0.31 0.23 0.25 0.43 0.48 Dev. Tripura Mean 2.05 1.50 2.20 2.30 2.47 2.03 N 20 15 20 15 20 15 Std. 0.22 0.31 0.21 0.26 0.36 0.48 Dev. Total Mean 2.13 1.53 2.23 2.36 2.51 2.11 N 40 33 40 33 40 33 28 Contract #7186365 – Descriptive Note (Revised November 2019) Std. 0.26 0.31 0.22 0.26 0.39 0.48 Dev. Table A3. Changes in Attitude (Parents – gender) Gender Cognitive Affective Behavioral Baseline End line Baseline End line Baseline End line Male Mean 1.88 1.72 2.47 2.34 3.03 2.59 N 22 19 22 19 22 19 Std. 0.22 0.27 0.25 0.21 0.19 0.51 Dev. Female Mean 1.88 1.72 2.38 2.30 3.05 2.41 N 34 37 34 37 34 37 Std. 0.30 0.19 0.26 0.31 0.17 0.59 Dev. Total Mean 1.88 1.72 2.42 2.31 3.04 2.47 N 56 56 56 56 56 56 Std. 0.27 0.22 0.26 0.28 0.17 0.57 Dev. Table A4. Changes in attitude (Parents – language) Mother Tongue Cognitive Affective Behavioral Baseline End line Baseline End line Baseline End line Chakma Mean 1.87 1.67 2.50 2.31 3.07 2.51 N 29 33 29 33 29 33 Std. 0.30 0.23 0.24 0.32 0.16 0.63 Dev. Tripura Mean 1.89 1.78 2.33 2.33 3.02 2.42 N 27 23 27 23 27 23 Std. 0.25 0.18 0.24 0.21 0.19 0.48 Dev. Total Mean 1.88 1.72 2.41 2.31 3.04 2.47 N 56 56 56 56 56 56 Std. 0.27 0.22 0.26 0.28 0.17 0.57 Dev. 29 Contract #7186365 – Descriptive Note (Revised November 2019) Table A5. Significance test - Attitudes of Authors Mean Std. Std. 95% Confidence t Df Sig. (2- Deviation Error Interval of the tailed) Mean Difference Lower Upper Cognitive 0.79221 0.31952 0.05562 0.90550 0.67891 14.243 32 .000 Affective 0.13609 0.28016 0.04877 0.03675 0.23543 2.790 32 .009 Behavioral 0.39600 0.58049 0.10105 0.60183 0.19016 3.919 32 .000 All attitudinal components have improved irrespective of gender and mother tongue. Table A6. Significance test - Attitudes of Parents Mean Std. Std. 95% Confidence t Df Sig. (2- Deviation Error Interval of the tailed) Mean Difference Lower Upper Cognitive 0.15995 0.34248 0.04577 0.25167 0.06823 3.495 55 .001 Affective 0.10076 0.39380 0.05262 0.20622 0.00470 1.915 55 .061 Behavioral 0.57105 0.57002 0.07617 0.72371 0.41840 7.497 55 .000 Appendix B: Survey Instruments Table B1: Cognitive – Authors Sl. Statement Strongly Agree Neutral Disagree Strongly Agree Disagree 1. I believe that pleasure n 23 10 - - - reading is important for children’s educational % 69.7 30.3 - - - success. 2. Every child has a right to n 29 3 - 1 - learn mother tongue at first. % 87.9 9.1 - 3.0 - 3. I believe supplementary n 25 7 1 - - reading materials for children should be available % 75.8 21.2 3.0 - - in mother tongue. 4. I believe that children are n 24 9 - - - mostly interested in reading 30 Contract #7186365 – Descriptive Note (Revised November 2019) stories that are published in % 72.7 27.3 - - - mother tongue. 5. I believe that children are n 25 8 - - - mostly interest in rhymes that are published in mother % 75.8 24.2 - - - tongue. 6. I believe that children are n 15 18 - - - mostly interest in reading comics that are published in % 45.5 54.5 - - - mother tongue. 7. I believe that parents and n 26 6 - 1 - teachers play roles in % 78.8 18.2 - 3.0 - reading habit formation of children. 8. I believe that children prefer n 24 8 - - 1 books with mother tongue language because it is % 72.7 24.2 - - 3.0 interesting, easy to read, pronounce and understand. 9. I believed that the language n 18 9 1 1 4 problem is not only hurting kids in our education, it is a % 54.5 27.3 3.0 3.0 12.1 barrier to preserving our culture. 10. I believe that a strong n 21 11 - - 1 foundation of educating in mother tongue provides a good bridge to learning a % 63.6% 33.3% - - 3.0 second language. 11. I believe that students n 23 10 - - - acquire mother tongue language quickly if they have support to access reading % 69.7 30.3 - - materials in their mother tongue. 12. I believe all children are n 20 12 1 - - interested in learning how to % 60.6 36.4 3.0 - - read in their mother tongue. 13. I believe that mother tongue n - - - 22 11 is not important to teach 31 Contract #7186365 – Descriptive Note (Revised November 2019) children as they won’t be % - - - 66.7 33.3 able to get a job in their native language. 14. I believe that without n 1 10 3 10 9 learning the Bangla language in early stage, children cannot communicate with % 3.0 30.3 9.1 30.3 27.3 the mainstream population of Bangladesh effectively. Table B2: Affective - Authors Sl. Statement Strongly Agree Neutral Disagree Strongly Agree Disagree 1. Parents are happy that their n 11 17 2 3 - children can learn to read, write and speak in mother % 33.3 51.5 6.1 9.1 - tongue. 2. All languages and scripts are n 13 17 1 1 1 available in Bangla for % 39.4 51.5 3.0 3.0 3.0 supplementary reading materials, and an entire generation is growing up without a sense of their own cultural history and identity. 3. Children’s reading materials n 4 7 2 14 6 and production costs are high. % 12.1 21.2 6.1 42.4 18.2 4. There is no market demand n 6 12 4 10 1 of children’s supplementary reading materials in mother tongue at the local level. % 18.2 36.4 12.1 30.3 3.0 5. Price is a factor that n 3 17 4 6 3 contributes to the unavailability of children % 9.1 51.5 12.1 18.2 9.1 reading materials in local market. 6. There is a lack of skilled local n 10 11 2 5 5 writers for children reading materials in mother tongue. % 30.3 33.3 6.1 15.2 15.2 7. n 11 7 1 9 4 32 Contract #7186365 – Descriptive Note (Revised November 2019) There is a lack of skilled local % 34.4 21.9 3.1 28.1 12.5 illustrators for children reading materials in mother tongue. 8. People do not want to n 3 - 1 14 15 recognize local writers/ illustrators/ publishers who % 9.1 - 3.0 42.4 45.5 are involved in reading materials of mother tongue. 9. There is a lack of n 13 17 1 1 1 motivational/ encouragement to promote % 39.4 51.5 3.0 3.0 3.0 writers in mother tongue. 10. Children should have access n 21 11 1 - - to supplementary reading materials in their mother tongue because it is their % 63.6 33.3 3.0 - - right. 11. Children’s supplementary n 17 16 - - - reading materials in mother tongue are not available in the local market. % 51.5 48.5 - - 12. Children are losing interest n - 3 2 18 10 in learning their own mother % - 9.1 6.1 54.5 30.3 tongues due to the lack of relevant content. 13. If children grow up learning n 24 8 1 - - to speak and write with their mother tongue and % 72.7 24.2 3.0 - - developing a sense of confidence about their culture and language, they will become a good citizen in the future. 14. Parents are not interested in n 4 12 3 12 2 buying children supplementary reading materials in mother tongue. % 12.1 36.4 9.1 36.4 6.1 15. Parents still don’t n 10 18 4 1 10 understand or perceive the 33 Contract #7186365 – Descriptive Note (Revised November 2019) value of supplementary % 30.3 54.5 12.1 3.0 30.3 reading materials for children in their mother tongue. 16. Very few skilled authors and n 6 22 3 1 1 educators reside in local communities who can promote children’s % 18.2 66.7 9.1 3.0 3.0 supplementary reading materials in mother tongue. 17. There is a lack of sufficient n 6 16 3 5 2 resources to produce a sufficient quantity of printed % 18.8 50.0 9.4 15.6 6.2 supplementary reading materials in mother tongue. 18. I don’t know where to find n 8 10 3 8 2 supplementary reading materials in mother tongue. % 25.8 32.3 9.7 25.8 6.5 19. I am not familiar with Bloom n 10 13 3 6 1 to create supplementary % 30.3 39.4 9.1 18.2 3.0 reading materials. 20. I am not aware of the n 12 9 2 9 1 “Teachers’ Portal/National Reading Hub” where digital supplementary reading % 36.4 27.3 6.1 27.3 3.0 materials are available for download. 21. The Onuprerona project n 17 14 2 - - promotes a love for reading. % 51.5 42.4 6.1 - - Table B3: Behavioral – Authors Sl. Statement Strongly Agree Neutral Disagree Strongly Agree Disagree 1. The Onuprerona project n 15 18 - - - encouraged me to write supplementary reading % 45.5 54.5 - - - materials in mother tongue in a way that is easier and appropriate for children. 2. The Onuprerona project n 17 15 1 - - attempts to provide a % 51.5 45.5 3.0 - - creative solution to promote 34 Contract #7186365 – Descriptive Note (Revised November 2019) mother tongue language practice for children. 3. I have confidence in writing n 17 15 1 - - mother tongue supplementary reading % 51.5 45.5 3.0 - - materials. 4. I can write and produce n 10 20 - 3 - mother tongue reading materials for children easily. % 30.3 60.6 - 9.1 - 5. I feel confident in my ability n 15 15 - 2 1 to write supplementary reading materials for % 45.5 45.5 - 6.1 3.0 children in mother tongue. 6. I can produce mother n 8 9 2 11 2 tongue reading materials for children without any cost. % 25.0 28.1 6.2 34.4 6.2 7. I can produce n 12 8 6 6 - supplementary reading % 37.5 25.0 18.8 18.8 - materials for children in mother tongue which are available in school. 8. Teachers are able to easily n 6 9 4 10 4 access supplementary reading materials in mother % 18.2 27.3 12.1 30.3 12.1 tongue in schools. 9. I produced supplementary n 2 4 6 15 6 reading materials in mother tongue which are available % 6.1 12.1 18.2 45.5 18.2 in the local market and everyone can access these books easily, even from remote places. 10. Onuprerona has helped n 16 14 3 - - children to practice reading in their own language. % 48.5 42.4 9.1 - - 11. I adapted Bloom to meet n 14 13 5 1 - quality standards of 35 Contract #7186365 – Descriptive Note (Revised November 2019) supplementary reading % 42.4 39.4 15.2 3.0 - materials in mother tongue and can create more titles easily. 12. Bloom helps to create local n 14 17 2 - - writers and build skills to % 42.4 51.5 6.1 - - create child-friendly supplementary reading materials in mother tongue. 13. As a tool, Bloom helps to n 9 20 1 3 - reducing bureaucratic delays, ensuring supply chain % 27.3 60.6 3.0 9.1 - quickly and at a reduced cost. 14. As a writer/ publisher/ n 5 16 5 4 2 illustrator of supplementary reading materials in mother in tongue, I am honored by % 15.6 50.0 15.6 12.5 6.2 the community. 15. I am connected with n 7 20 2 3 - different platforms so that I can produce books in % 21.9 62.5 6.2 9.4 - mother tongue easily. 16. I am getting support from n 2 15 6 6 4 the community to produce books in mother tongue. % 6.1 45.5 18.2 18.2 12.1 17. I am motivated to promote n 10 20 1 2 - the use of Bloom software to generate books in mother % 30.3 60.6 3.0 6.1 - tongue Table B4: Cognitive - Parents Sl. Statement Strongly Agree Neutral Disagree Strongly Agree Disagree 1. I believe that reading for n 32 24 - - - pleasure is important for children’s educational % 57.1 42.9 - - - success. 2. Every child has right to learn n 39 17 - - - how to read first in their % 69.6 30.4 - - - mother tongue. 36 Contract #7186365 – Descriptive Note (Revised November 2019) 3. I believe supplementary n 17 39 - - - reading materials for children should be offered in % 30.4 69.6 - - - a child’s mother tongue. 4. I believe that children are n 32 24 - - - mostly interested in reading stories that are published in mother tongue. % 57.1 42.9 - - - 5. I believe that children are n 19 37 - - - mostly interested in rhymes that are published in mother % 33.9 66.1 - - - tongue. 6. I believe that children are n 20 36 - - - mostly interested in reading comics that are published in mother tongue. % 35.7 64.3 - - - 7. I believe that parents and n 33 23 - - - teachers play roles in % 58.9 41.1 - - - reading habit formation of children. 8. I believe that children prefer n 19 37 - - - books in their own language because it is interesting, % 33.9 66.1 - - - easy to read, pronounce and understand. 9. I believed that the language n 14 39 - 2 1 problem is not only hurting kids in our education, it is a % 25.0 69.6 - 3.6 1.8 barrier to preserving our culture. 10. I believe that a strong n 20 36 - - - foundation of literacy education based in mother tongue provides a good % 35.7 64.3 - - - bridge to learning a second language. 11. I believe that students n 20 25 - 9 2 acquire mother tongue 37 Contract #7186365 – Descriptive Note (Revised November 2019) language quickly if they have % 35.7 44.6 - 16.1 3.6 support to read reading materials in their own language at school or in the home. 12. I believe all children have n 31 24 1 - - interest to learn mother % 55.4 42.9 1.8 - - tongue. 13. I believe that mother tongue n 3 - - 28 25 instruction is not an important skill to children, as % 5.4 - - 50.0 44.6 they won’t be able to get a job in their native language. 14. I believe that without n 4 25 2 15 10 learning the Bangla language at the early stages, children cannot communicate with % 7.1 44.6 3.6 26.8 17.9 the mainstream population of Bangladesh effectively. Table B5: Affective - Parents Sl. Statement Strongly Agree Neutral Disagree Strongly Agree Disagree 1. I am happy that my children n 28 21 - 3 4 can learn to read, write and speak in mother tongue. % 50.0 37.5 - 5.4 7.1 2. All languages and scripts are n 20 29 - 4 3 available in Bangla for % 35.7 51.8 - 7.1 5.4 supplementary reading materials, and an entire generation is growing up without a sense of our own cultural history and identity 3. The cost of children’s n 1 17 9 20 9 supplementary reading materials is too high. % 1.8 30.4 16.1 35.7 16.1 4. There is no market demand n 6 28 1 11 10 for children’s supplementary reading materials in mother tongue at the local level. % 10.7 50.0 1.8 19.6 17.9 5. n 28 7 - 15 6 38 Contract #7186365 – Descriptive Note (Revised November 2019) Price is a factor that affects % 50.0 12.5 - 26.8 10.7 the availability of children’s reading materials in market. 6. There is a lack of skilled local n 26 22 1 4 3 writers for children’s reading materials in mother tongue % 46.4 39.3 1.8 7.1 5.4 7. There is a lack of skilled local n 20 27 - 7 2 illustrators for children’s % 35.7 48.2 - 12.5 3.6 reading materials in mother tongue 8. People don’t recognize local n 6 33 2 6 9 writers/ illustrators /publishers who are involved % 10.7 58.9 3.6 10.7 16.1 in reading materials of mother tongue. 9. There is a lack of motivation/ n 8 42 1 3 2 encouragement to promote writers in mother tongue % 14.3 75.0 1.8 5.4 3.6 10. Children’s supplementary n 27 27 1 - 1 reading materials should be offered in their mother tongue because it is a child’s % 48.2 48.2 1.8 - 1.8 right to learn in their mother tongue. 11. Children’s supplementary n 13 37 3 - 3 reading materials in mother tongue are not available in the local market. % 23.2 66.1 5.4 - 5.4 12. Children are losing interest n 27 25 - 3 1 in learning their own mother % 48.2 44.6 - 5.4 1.8 tongues due to lack of mother tongue content. 13. If the children grow up n 30 25 - 3 1 learning to speak and write with their mother tongue % 48.2 44.6 - 5.4 1.8 and developing a sense of confidence about their culture and language, they will become a good citizen in the future. 39 Contract #7186365 – Descriptive Note (Revised November 2019) 14. I am not interested in buying n 4 14 1 24 13 supplementary reading materials in mother tongue. % 7.1 25.0 1.8 42.9 23.2 15. Parents don’t understand or n 15 35 - 6 - perceive the value of children’s supplementary % 26.8 62.5 - 10.7 - reading materials in mother tongue language. 16. I am not aware of where I n 2 46 3 4 1 can find supplementary reading materials in mother tongue. % 3.6 82.1 5.4 7.1 1.8 17. I think Onuprerona helps n 18 35 1 2 18 children develop a joy for reading % 32.1 62.5 1.8 3.6 32.1 Table B6: Behavioral – Parents Sl. Statement Strongly Agree Neutral Disagree Strongly Agree Disagree 1. The Onuprerona project n 14 40 2 - - encouraged me to support children to read % 25.0 71.4 3.6 - - supplementary reading materials in mother tongue in a way that is easier and appropriate for children. 2. The Onuprerona project is n 16 38 2 - - an attempt to provide a % 28.6 67.9 3.6 - - creative solution to promote mother tongue language practice for children. 3. I am confident in my ability n 16 31 1 5 3 to support children to read supplementary reading % 28.6 55.4 1.8 8.9 5.4 materials in mother tongue. 4. n 6 18 7 21 4 40 Contract #7186365 – Descriptive Note (Revised November 2019) I can easily access mother % 10.7 32.1 12.5 37.5 7.1 tongue reading materials for children. 5. I collect supplementary n 4 16 1 23 12 reading materials for children in mother tongue. % 7.1 28.6 1.8 41.1 21.4 6. I am happy to have mother n 16 26 8 4 2 tongue reading materials for children without any cost. % 28.6 46.4 14.3 7.1 3.6 7. I can access supplementary n 5 22 6 16 7 reading materials in mother % 8.9 39.3 10.7 28.6 12.5 tongue without issue. 8. My children are happy to n 28 18 4 6 - have supplementary reading materials in their mother % 50.0 32.1 7.1 10.7 - tongue at home. 9. Onuprerona helped children n 15 29 8 4 - learn how to practice reading their own language. % 26.8 51.8 14.3 7.1 - 10. I am better able to access n 1 12 11 25 7 books using my smartphone. % 1.8 21.4 19.6 44.6 12.5 11. My children are growing up n 13 32 2 6 3 by learning with their mother tongue and developing a sense of % 23.2 57.2 3.6 10.7 5.4 confidence about their culture and language. 12. I am getting support from n 3 20 4 20 9 the parents/ community to % 5.4 35.7 7.1 35.7 16.1 use the books in mother tongue. 13. I encouraged myself to n 9 28 8 9 2 promote mother tongue language by using books. % 16.1 50.0 14.3 16.1 3.6 41 Contract #7186365 – Descriptive Note (Revised November 2019)