Investigating the Relationship Between Visual Attention, Story Comprehension, and Vocabulary Skills in Malaysian Prereaders: An Eye-Tracking Study

Investigating the Relationship Between Visual Attention, Story Comprehension, and Vocabulary Skills in Malaysian Prereaders: An Eye-Tracking Study

Hoo Keat Wong, Siew Ming Thang, Chee Hao Sue, Rosalind A. K. Ahju, Fung Lan Loo
DOI: 10.4018/IJCALLT.332878
Article PDF Download
Open access articles are freely available for download

Abstract

Based on the cross-channel connections between auditory and pictorial representations, it has been proposed that the presentation of coherent narration along with the picture and text content may enhance children's story comprehension and vocabulary learning. The authors tested 40 four- to five-year-old Malaysian prereaders (17 Malays, 23 Chinese) for story comprehension while observing their eye movements to determine the degree to which the presence of pictures and/or text aids understanding of the narration and influences looking patterns. Both Malay and Chinese prereaders showed no interest in the printed text that was presented alongside the picture on the same page, which is consistent with earlier findings. This suggests that ethnic origins have little influence on how prereaders direct their visual attention to the relevant information for story comprehension. When there was no narration, they fixated longer on the text and less on the image, indicating that a significant amount of mental effort was required to process the words without verbal information. Regardless of stimulus congruency, storytelling performance affected how much children focused on target objects and keywords. More intriguingly, it was found that in Malay prereaders, there was a correlation between story comprehension and vocabulary skills across tasks. Additionally, Malay prereaders who performed well looked at the displayed stimuli longer than Chinese prereaders who performed well, especially when a narrator was presented alongside the visual stimulus. These novel findings are discussed along with their implications for multimedia learning and future research directions.
Article Preview
Top

Cognitive Processes In Multimedia Learning

Multimedia learning involves using visual and auditory stimuli to convey information to learners. In recent years, there has been a growing interest in understanding the cognitive processes involved in multimedia learning. Several empirical studies have investigated how the young brain processes information presented in different multimedia formats, such as videos, animations, and interactive simulations. Some of these studies have examined the effects of multimedia on attention, working memory, and comprehension. For example, Moreno and Mayer (2007) investigated the effect of multimedia on attention and comprehension in college students. The researchers found that students who received multimedia instruction showed greater attention and comprehension compared to those who received text-only instruction. Another study by Mayer, Heiser, and Lonn (2001) examined the effect of multimedia on working memory, revealing that multimedia instruction resulted in better performance on tasks that required working memory.

Sharing picture storybooks with young children is crucial for developing literacy skills. Many children’s storybooks are illustrated via visual and/or oral narratives. When children read or listen to a story, both visual (pictorial) and auditory (verbal) processing channels are employed. Several cognitive theories and models have been proposed and further adopted by educators and modern researchers in empirical studies to understand the cognitive processes underlying a child’s story comprehension and multimedia learning. According to Paivio’s (1986) Dual Channel Model, humans possess two channels to process visual and auditory information. The first is the verbal channel, which processes spoken words heard through the ears; the other is the visual channel, which processes images seen through the eyes (Paivio, 2007). Each channel processes and stores information differently, with the visual channel being specialised in processing and storing visual information and the verbal channel being specialised in processing and storing verbal information. The Dual Channel Model suggests that multimedia learning is most effective when it utilises both channels, as this creates a stronger memory trace and facilitates better comprehension of the information. This idea is particularly relevant when considering multimedia learning in young children, as the use of both visual and verbal information can help them better understand and retain the information being presented.

The cognitive theory of multimedia learning (CTML; Mayer, 2001) further explains how children process information when reading or hearing a story via multimedia. That is, the information in both visual and verbal form contributes to the comprehension of the story and increases the learner’s visual attention. CTML proposes a model that comprises three memory stores: sensory memory, working memory, and long-term memory (see Figure 1). In working memory, the auditory information is organised to construct a verbal model; meanwhile, the images will be processed to create pictorial coding to build a visual model. When these two processes occur concurrently, integrating the auditory and visual channels will increase the efficiency of the information processing. This effect, in turn, facilitates the comprehension of the learners (Sorden, 2013).

Figure 1.

Mayer’s (2001) cognitive theory of multimedia learning

IJCALLT.332878.f01
Note. Adapted from “Applying the Science of Learning: Evidence-Based Principles for the Design of Multimedia Instruction” by Mayer, R. E., 2008, The American Psychologist, 63(8), p. 762. Copyright 2008 by the American Psychological Association.

Complete Article List

Search this Journal:
Reset
Volume 14: 1 Issue (2024): Forthcoming, Available for Pre-Order
Volume 13: 1 Issue (2023)
Volume 12: 5 Issues (2022)
Volume 11: 4 Issues (2021)
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing