The Scream Stream: Multimodal Affect Analysis of Horror Game Spaces

• Published in: 2024 12th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) pp. 128 - 135
• Main Authors: Xylakis, Emmanouil, Liapis, Antonios, Yannakakis, Georgios N.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15.09.2024
• Subjects: Affective computing; Arousal; Cameras; Faces; Facial Expressions Analysis; Fear; Games; Level design; Speech analysis; Streams; Surprise; Survival horror; Video games; Video on demand; Virtual environments; Web sites
• DOI: 10.1109/ACIIW63320.2024.00027

Virtual environments allow us to study the impact of space on the emotional patterns of a user as they navigate through it. Similarly, digital games are capable of eliciting intense emotional responses from their players; moreso when the game is explicitly designed to do this, as in the Horror game genre. A growing body of literature has already explored the relationship between varying virtual space contexts and user emotion manifestation in horror games, often relying on physiological data or self-reports. In this paper, instead, we study players' emotion manifestations within this game genre. Specifically, we analyse facial expressions, voice signals, and verbal narration of YouTube streamers while playing the Horror game Outlast. We document the collection of the Outlast Asylum Affect corpus from in-the-wild videos, and its analysis into three different affect streams based on the streamer's speech and face camera data. These affect streams are juxtaposed with manually labelled gameplay and spatial transitions during the streamer's exploration of the virtual space of the Asylum map of the Outlast game. Results in terms of linear and non-linear relationships between captured emotions and the labelled features demonstrate the importance of a gameplay context when matching affect to level design parameters. This study is the first to leverage state-of-the-art pre-trained models to derive affect from streamers' facial expressions, voice levels, and utterances and opens up exciting avenues for future applications that treat streamers' affect manifestations as in-the-wild affect corpora.