Absolute and Relative User Perception of Classification Accuracy in an Affective Video Game - McCrea, Sean; Geršak, Gregor; Novak, Domen (Interacting with computers, 2017)
Abstract Classification algorithms are used in affective computing to classify the state of the user and adapt the computer's behaviour, but it is unclear how classification accuracy influences the overall user experience. We present a study in which classification accuracy is artificially pre-defined and used to adapt to the difficulty of a video game. Eighty subjects played the game and were told that difficulty would be adapted according to the measured brain activity. They played the game twice, with different classification accuracies, and then reported different aspects of their overall game experience using questionnaires. Classification accuracy was correlated with both in-game fun (r = 0.46) and satisfaction with the difficulty adaptation (r = 0.56). Most subjects could perceive a difference between two classification accuracies that differed by 16.7%. We tentatively posit that, for affective video games, an acceptable classification accuracy is 70–80%. Furthermore, studies that attempt to improve affect classification accuracy should aim for a practically meaningful improvement of 10%.