Psychometric evaluation of Simulator Sickness Questionnaire and its variants as a measure of cybersickness in consumer virtual environments

• Published in: Applied ergonomics Vol. 82; p. 102958
• Main Authors: Sevinc, Volkan, Berkman, Mehmet Ilker
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.01.2020
• Subjects: Cybersickness; Psychometric evaluation; Scale; Simulator sickness questionnaire; Virtual reality; Visually induced motion sickness; Psychometric evaluation; Visually induced motion sickness; Scale; Cybersickness; Simulator sickness questionnaire; Virtual reality
• ISSN: 0003-6870; 1872-9126; 1872-9126
• DOI: 10.1016/j.apergo.2019.102958

Cybersickness, i.e. visually induced motion sickness, remains as a negative effect that is detrimental to the user experience of VEs (virtual environments) developed for VR (virtual reality) consumers. As the VR technology evolves, it is rather triggered by application aspects rather than hardware limitations. For this reason, there is still a need for a measurement method to assess and compare VEs for cybersickness effects. SSQ (Simulation Sickness Questionnaire) is used for measuring users’ level of sickness symptoms and is highly appreciated in VR research. However, it is criticized for its psychometric qualities and applicability in VR, as a measure of cybersickness. Recently, two variants of SSQ were offered for measuring cybersickness, CSQ (Cybersickness Questionnaire) and VRSQ (Virtual Reality Sickness Questionnaire). There is also another variant with a different factor structure, which we call FSSQ, that is based on French translation of SSQ. Our study compares SSQ and these variants for their psychometric qualities; construct validity, discriminant validity, internal reliability, test-retest reliability and sensitivity to distinguish application aspects of VEs that are related to cybersickness. Using a within-subjects experiment design, we evaluated 7 different VEs with 32 participants through 9 sessions, resulting with 288 responses to the 16-item SSQ. Results suggested that both VRSQ and CSQ were valid and reliable measures of cybersickness, as well as being sensitive to application aspects such as translational and rotational movements required by users for navigation in VEs. Compared to SSQ and FSSQ; the cybersickness questionnaires, CSQ and VRSQ, revealed better indicators of validity. On the other hand, we assume that the development of the two cybersickness scales had limitations in sample size to represent VR consumers and limitations in stimuli to represent the applications aspects of consumer VEs. We suggest further evaluation of cybersickness symptoms with larger samples and broader range of applications to identify the symptoms and the construct of a subjective measurement tool. •SSQ was not applicable for measuring cybersickness in commercial HMD VR applications, because of its psychometric qualities.•Symptoms did not correlate sufficiently in repeated experiments, due to users’ adaptation or self-induced cybersickness.•VRSQ and CSQ were found to be psychometrically sound measures of cybersickness in terms of validity and reliability.•Although VRSQ and CSQ employ fewer items, these scales were sensitive to application aspects that affect cybersickness.•Development of VRSQ and CSQ have drawbacks due to sparsity in sample size and stimuli diversity, as in this study.