Deep Temporal Analysis for Non-Acted Body Affect Recognition

• Published in: IEEE transactions on affective computing Vol. 13; no. 3; pp. 1366 - 1377
• Main Authors: Avola, Danilo, Cinque, Luigi, Fagioli, Alessio, Foresti, Gian Luca, Massaroni, Cristiano
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D skeleton; Artificial neural networks; automatic emotion recognition; body movement; Datasets; deep learning; Emotion recognition; Emotions; Feature extraction; Games; long short-term memory (LSTM); Machine learning; Non-acted affective computing; Pain; recurrent neural network (RNN); Skeleton; Task analysis; Three-dimensional displays
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2020.3003816

In the field of body affect recognition, the majority of literature is based on experiments performed on datasets where trained actors simulate emotional reactions. These acted and unnatural expressions differ from the more challenging genuine emotions, thus leading to less valuable results. In this article, a solution for basic non-acted emotion recognition based on 3D skeleton and Deep Neural Networks (DNNs) is provided. The proposed work introduces three majors contributions. First, temporal local movements performed by subjects are examined frame-by-frame, unlike the current state-of-the-art in non-acted body affect recognition where only static or global body features are considered. Second, an original set of global and time-dependent features for body movement description is provided. Third, this is one of the first works to use deep learning methods in the current non-acted body affect recognition literature. Due to the novelty of the topic, only the UCLIC dataset is currently considered the benchmark for comparative tests. On the latter, the proposed method outperforms all the competitors.