Holographic sign language avatar interpreter: A user interaction study in a mixed reality classroom

• Published in: Computer animation and virtual worlds Vol. 33; no. 3-4
• Main Authors: Yang, Fu‐Chia, Mousas, Christos, Adamo, Nicoletta
• Format: Journal Article
• Language: English
• Published: Chichester Wiley Subscription Services, Inc 01.06.2022
• Subjects: Avatars; Classrooms; holographic avatar; Holography; Interpreters; Mixed reality; Qualitative analysis; sign language animation; user interaction
• ISSN: 1546-4261; 1546-427X
• DOI: 10.1002/cav.2082

We explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters. We recruited eight participants and conducted a usability test focused on avatar framing (full‐body vs. half‐body) and avatar manipulation (fixed position, scale, and orientation vs. user‐adjustable position, scale, and orientation) in the MR classroom. We used a mixed‐method approach to analyze quantitative and qualitative data through recordings, surveys, and interviews. The results show user preferences toward viewing holographic signing avatars in the MR environment and user acceptability toward such applications. We explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters.