The SmARtphone Controller

Current interaction modalities for mobile Augmented Reality (AR) are tedious and lack expressiveness. To overcome these prevalent limitations, we developed and evaluated a multimodal interaction concept by pairing a smartphone as an input and output modality for mobile AR. In a user study (n = 24),...

Full description

Saved in:
Bibliographic Details
Published inI-com Vol. 20; no. 1; pp. 49 - 61
Main Authors Knierim, Pascal, Hein, Dimitri, Schmidt, Albrecht, Kosch, Thomas
Format Journal Article
LanguageEnglish
Published De Gruyter Oldenbourg 27.04.2021
Subjects
Augmented Reality
Head-Mounted Display
Multi-Display Interaction
Object Manipulation
Smartphone
Online AccessGet full text

Cover

More Information
Summary:Current interaction modalities for mobile Augmented Reality (AR) are tedious and lack expressiveness. To overcome these prevalent limitations, we developed and evaluated a multimodal interaction concept by pairing a smartphone as an input and output modality for mobile AR. In a user study (n = 24), we investigated the effects on interaction speed, accuracy, and task load for (1) virtual object manipulation as well as (2) interaction with established graphical user interfaces (GUIs). Our findings show that a smartphone-based AR controller results in significantly faster and more accurate object manipulation with reduced task load than state-of-art mid-air gestures. Our results also indicate a significant enhancement for using the physical touchscreen as a modality compared to mid-air gestures for GUI interaction. We conclude that interaction in mobile AR environments can be improved by utilizing a smartphone as an omnipresent controller. Additionally, we discuss how future AR systems can benefit from tangible touchscreens as an additional and complementary interaction modality.
ISSN:1618-162X
2196-6826
DOI:10.1515/icom-2021-0003