Exploring the Impact of Multimodal Long Conversations in VR on Attitudes Toward Behavior Change, Memory Retention, and Cognitive Load

• Published in: Computer animation and virtual worlds Vol. 36; no. 3
• Main Authors: Vankit, Sagar A., Zamanifard, Samaneh, Diaz, Deyrel, Mousas, Christos, Richardson, Kelly, Duchowski, Andrew T., Volonte, Matias
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.05.2025; Wiley Subscription Services, Inc
• Subjects: Attitudes; embodied conversational agents; Immersive virtual reality; multimodal conversations; virtual reality
• ISSN: 1546-4261; 1546-427X
• DOI: 10.1002/cav.70023

ABSTRACT This study examines how multimodal communication strategies (subtitles, visualizations, and their combination), affect memory retention, attitudes toward behavior change, and cognitive load during long conversations (+20 min) in immersive virtual reality (VR). Using embodied conversational agents to educate participants on diabetes and healthy eating, we found that all conditions effectively improved memory retention and behavior change attitudes. However, the combination of multimodal strategies increased cognitive load, suggesting a trade‐off between engagement and cognitive demands. These findings highlight the potential of long VR conversations for healthcare education, while emphasizing the importance of balancing cognitive demands and exploring personalization for diverse users. This study investigates how multimodal design features like subtitles and visualizations affect user attention, perceptions, gaze behavior and motivation in virtual health conversations with Embodied Conversational Agents (ECAs). Conducted in immersive VR, participants interacted with a virtual health coach delivering guidance on healthy eating. Results from path modeling revealed that visual attention to the ECA was linked to stronger working alliance and behavior change intentions, while excessive visual input introduced trade‐offs in engagement and persuasion.