Questionnaires embedded in virtual environments: reliability and positioning of rating scales in virtual environments

• Published in: Quality and user experience Vol. 4; no. 1
• Main Authors: Regal, Georg, Voigt-Antons, Jan-Niklas, Schmidt, Steven, Schrammel, Johann, Kojić, Tanja, Tscheligi, Manfred, Möller, Sebastian
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.12.2019
• Subjects: Behavioral Sciences; Cognitive Psychology; Communications Engineering; Engineering; Media Research; Networks; Research Article; Signal,Image and Speech Processing; User Interfaces and Human Computer Interaction; Subjective rating; Test method; Virtual reality
• ISSN: 2366-0139; 2366-0147
• DOI: 10.1007/s41233-019-0029-1

Current developments in virtual reality (VR) hardware have made immersive VR experiences more affordable through commercially available head-mounted displays. As more studies are likely to be conducted using these devices, the question arises how to embed questionnaires in virtual environments without impairing the immersive user experience. In this work we investigate two different aspects: (1) if a rating performed in a virtual environment is comparable to a rating obtained via a paper questionnaire and (2) how questionnaires for assessing virtual experiences should be designed and integrated into the virtual environment. For this research, we used our own extended version of VRate—a VR questionnaire asset for Unity. In the first study with 27 participants, we compared ratings assessed within VR with ratings obtained using a paper questionnaire. We found that the ratings gathered in VR are comparable to the ratings gathered in the real world by paper–pencil questionnaires (subscales: global presence, spatial presence, and experience realism). In the second study with 48 participants, we investigated the users’ perceived suitability of the VR questionnaire and the optimal mounting position of the questionnaire (hand-mounted, head-up display or billboard). Moreover, we investigated whether the questionnaire should be answered in the same or in a separate dedicated virtual environment and how the users’ feeling of presence in VR is influenced by this placement. Results indicate a subjective preference for the billboard position, with a significant preference for billboard over hand-mounted and no significant preference between billboard and head-up-display. Regarding the placement of the VR questionnaire (in-scene vs. dedicated virtual environment) we did not find any influence on presence. In the following, we discuss the pros and cons of different placement/mounting options and provide suggestions for designing and implementing questionnaires embedded in virtual environments.