By Dr He Sun, Jieying Loh and Dr Adam Charles Roberts
Animation in electronic books (eBooks) can significantly improve a child’s bilingual Mandarin language learning as shown in a study conducted on preschoolers by the National Institute of Education.
Singapore is a multilingual society with three main ethnic groups and four official languages: English, Mandarin, Malay, and Tamil. English is the official language for education, government, and commerce while the other three languages, or Mother tongue languages, are for ethnic identity and heritage maintenance. Although children are encouraged to develop English and Mother tongue languages simultaneously (Ministry of Education (MOE), 2013), recent years have witnessed a discrepancy between English and Mother tongue languages in both learning outcomes and environment.
By following 805 Singaporean preschoolers, aged 4 to 5 years, Sun, Yin, Amsah, and O’Brien (2018) found that children’s Mother tongue language vocabulary size was substantially smaller than their English vocabulary size across all three ethnic groups. Their input quantity and quality of ethnic language were significantly lower than that for English learning as well.
As Cavallaro and Ng described (2014), “English is increasingly becoming the Mother tongue for more and more Singaporeans, and their ethnic languages are technically more like second languages” (p. 36). Against this social backdrop, research on animated e-storybooks is especially relevant because of their promising effects on young second language learners’ emergent literacy skills.
According to the Singapore National Library Board, the number of electronic books borrowed has reached 11 million in 2015, increasing almost four times since 2009. Such a reading format is favoured by children because of its entertaining elements, e.g., sound and interactive games (Hio, 2015). Despite the changing landscape of children’s reading format, little is known how these eBooks may influence bilingual language learning in Singapore and whether children can ultimately benefit from such exposure.
To strengthen Singapore children’s Mandarin language development as emphasised by the MOE, our investigation into approaches that motivate our children to better master the Mandarin language seems more essential than ever. In our study, we explored the efficacy of animated e-storybooks on preschoolers’ Mandarin language development and acquisition.
Storybook reading is one of the most effective approaches for children to acquire novel words and grammar in a meaningful context (Weizman & Snow, 2001). However, children with limited language knowledge, such as emergent bilinguals, may benefit less from reading activities due to the gap between their skills and those needed for processing the story. They may fail to derive the meaning of new words from the verbal context and consequently face difficulties in figuring out the story plots (Verhallen & Bus, 2010).
Well-designed animated eBooks hold great promise for children’s emerging literacy as they can stimulate readers' visual, auditory, and even kinesthetic senses to comprehend and digest a story by matching between the animated features (motion pictures, hotspot, and sound) and the read-aloud (De Jong & Bus, 2002, 2004; Neuman, 1997; Verhallen, Bus, & de Jong, 2006). Features like highlights or zoom-in effects can further direct children’s attention to essential details in the story and provide them with richer sensory information to process and retain the story, and ultimately, enhance their language acquisition through repetitive readings (Bus, Takacs, & Kegel, 2015; Verhallen & Bus, 2009).
Animated eBooks equipped with multimedia features (e.g., video, sound, and music) have been found to support early language acquisition (Verhallen et al., 2006). The mechanism of such effects may lie in the presentation of the information via multiple channels (Paivio, 1986). Rather than distracting from literacy, multimedia may expose children to an additional set of processing tools to interpret events, allowing them to benefit from the “redundancy effect” (Neuman, 2008), where information has been delivered through multiple channels. By providing rich verbal and nonverbal information, animated eBooks can optimise temporal congruity of narration and pictures, thus facilitating children’s selection of content for story processing and strengthening their story recall thereafter (Bus et al., 2015; Mayer, 2001).
In our study, we compared children’s vocabulary acquisition and reading comprehension in four conditions: (a) animated stories with motion and sound; (b) corresponding static stories, with sound and softcopy printed illustrations; (c) corresponding static stories with soft-copy illustration only, and; (d) a control condition (no reading exposure).
We also explored the relation between features of animated e-storybooks, i.e., motion and sound, and preschoolers’ visual attention with eye trackers. Eye-tracking has become increasingly common for investigating visual attention as eye movement is a direct measure of overt visual attention (Kulke, Atkinson, & Braddick, 2016). It has been used to examine novel word learning (Mather & Plunkett, 2012), where gaze duration is increased when novel objects are paired with novel names, and eBook reading in Germanic languages (Takacs & Bus, 2016).
By measuring gaze direction and saccadic eye movements using infrared light reflected off the surface of the eye, this non-invasive technology can be easily administered to children of all ages. In the context of eBook reading, eye-tracking allows the possibility of analysing whether attention is allocated to the presented material or other distracting stimuli in the environment through fixation duration (Wass, Smith, & Johnson, 2012).
Our study confirmed previous findings that animated eBooks could facilitate children’s productive vocabulary learning and attract better attention from them via listening to the stories. We extended the beneficial scope from Germanic languages to Mandarin Chinese. Illustrations, by adding motion, could enhance the congruity of the auditory reading of the story and the visual illustrations. This could direct children’s attention to the rich details of the story, thereby enhancing story comprehension and word learning.
The finding is consistent with the multimedia learning hypothesis that children could use dual channel resources to effectively process the input as long as the verbal and nonverbal information is coherently designed.
Read the full paper Motion and sound in animated storybooks for preschoolers’ visual attention and mandarin language learning: An eye-tracking study with bilingual children.
