Should learners use their hands for learning? Results from an eye‐tracking study

• Published in: Journal of computer assisted learning Vol. 36; no. 1; pp. 102 - 113
• Main Authors: Korbach, Andreas, Ginns, Paul, Brünken, Roland, Park, Babette
• Format: Journal Article
• Language: English
• Published: Oxford Wiley-Blackwell 01.02.2020; Wiley Subscription Services, Inc
• Subjects: Attention; Cognition; Cognitive Ability; cognitive load theory; Cognitive Processes; Control Groups; Data processing; Design of experiments; Difficulty Level; embodied cognition; Eye Movements; eye‐tracking; Handheld Devices; Hands; Human Body; Information processing; Interactive computer systems; Learning; Multimedia; Multimedia Instruction; Nonverbal Communication; Research Design; Touch screens; Tracing; tracing gestures; Tracking
• ISSN: 0266-4909; 1365-2729
• DOI: 10.1111/jcal.12396

Given the widespread use of touch screen devices, the effect of the users' fingers on information processing and learning is of growing interest. The present study drew on cognitive load theory and embodied cognition perspectives to investigate the effects of pointing and tracing gestures on the surface of a multimedia learning instruction. Learning performance, cognitive load and visual attention were examined in a one‐factorial experimental design with the between‐subject factor pointing and tracing gestures. The pointing and tracing group were instructed to use their fingers during the learning phase to make connections between corresponding text and picture information, whereas the control group was instructed not to use their hands for learning. The results showed a beneficial effect of pointing and tracing gestures on learning performance, a significant shift in visual attention and deeper processing of information by the pointing and tracing group, but no effect on subjective ratings of cognitive load. Implications for future research and practice are discussed. Lay Description What is currently known about the subject matter? Pointing and tracing gestures enhance learning with multimedia The effect is true for paper based and digital learning instructions Less is known about the cause of this effect with regard to complex learning Different theoretical explanations are currently discussed What our paper adds to this A detailed analysis of the effect using eye‐tracking and touch screen data Evidence that the effect of pointing and tracing is mediated by eye‐movements Relevant information concerning the discussion about theoretical explanations The implications of study findings for practitioners Pointing and tracing gestures can guide the learners' visual focus of attention Pointing and tracing gestures can increase the learners' visual attention Pointing and tracing gestures foster information processing when learning with multimedia Learners can or even should use pointing and tracing gestures to support learning