Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games

• Published in: Sensors (Basel, Switzerland) Vol. 19; no. 13; p. 2877
• Main Authors: Bevilacqua, Fernando, Engström, Henrik, Backlund, Per
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 28.06.2019; MDPI
• Subjects: Adult; affective computing; Boredom; Computer vision; Emotions; Emotions - physiology; Female; Games; Humans; human–computer interaction; Interaction Lab (ILAB); Learning; Male; Middle Aged; Photoplethysmography - methods; remote photoplethysmography; Remote Sensing Technology; Surveys and Questionnaires; User behavior; User-Computer Interface; Video Games - psychology; games; remote photoplethysmography; affective computing; human–computer interaction
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s19132877

Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments ( n = 20 and n = 62 ) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games.