Absolute and Relative User Perception of Classification Accuracy in an Affective Video Game

• Published in: Interacting with computers Vol. 29; no. 2; pp. 271 - 286
• Main Authors: McCrea, Sean, Geršak, Gregor, Novak, Domen
• Format: Journal Article
• Language: English
• Published: Oxford University Press 01.03.2017
• Subjects: Affective computing; physiological computing; machine learning; user experience; computer games
• ISSN: 0953-5438; 1873-7951
• DOI: 10.1093/iwc/iww026

Abstract Classification algorithms are used in affective computing to classify the state of the user and adapt the computer's behaviour, but it is unclear how classification accuracy influences the overall user experience. We present a study in which classification accuracy is artificially pre-defined and used to adapt to the difficulty of a video game. Eighty subjects played the game and were told that difficulty would be adapted according to the measured brain activity. They played the game twice, with different classification accuracies, and then reported different aspects of their overall game experience using questionnaires. Classification accuracy was correlated with both in-game fun (r = 0.46) and satisfaction with the difficulty adaptation (r = 0.56). Most subjects could perceive a difference between two classification accuracies that differed by 16.7%. We tentatively posit that, for affective video games, an acceptable classification accuracy is 70–80%. Furthermore, studies that attempt to improve affect classification accuracy should aim for a practically meaningful improvement of 10%.