Quaternion Orthogonal Transformer for Facial Expression Recognition in the Wild

• Published in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 1 - 5
• Main Authors: Zhou, Yu, Guo, Liyuan, Jin, Lianghai
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04.06.2023
• Subjects: Face recognition; Facial expression recognition; Feature extraction; Orthogonal Feature; Quaternion; Quaternions; Redundancy; Signal processing; Speech recognition; Transformer; Transformers
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP49357.2023.10096851

Facial expression recognition (FER) is a challenging topic in artificial intelligence. Recently, many researchers have attempted to introduce Vision Transformer (ViT) to the FER task. However, ViT cannot fully utilize emotional features extracted from raw images and requires a lot of computing resources. To overcome these problems, we propose a quaternion orthogonal transformer (QOT) for FER. Firstly, to reduce redundancy among features extracted from pre-trained ResNet-50, we use the orthogonal loss to decompose and compact these features into three sets of orthogonal sub-features. Secondly, three orthogonal sub-features are integrated into a quaternion matrix, which maintains the correlations between different orthogonal components. Finally, we develop a quaternion vision transformer (Q-ViT) for feature classification. The Q-ViT adopts quaternion operations instead of the original operations in ViT, which improves the final accuracies with fewer parameters. Experimental results on three in-the-wild FER datasets show that the proposed QOT outperforms several state-of-the-art models and reduces the computations.Codes are available at https://github.com/Gabrella/QOT.