Unconstrained Facial Expression Recognition Based on Feature Enhanced CNN and Cross-Layer LSTM

• Published in: IEICE Transactions on Information and Systems Vol. E103.D; no. 11; pp. 2403 - 2406
• Main Authors: LIANG, Ruiyu, CHEN, Rui, TONG, Ying
• Format: Journal Article
• Language: English
• Published: Tokyo The Institute of Electronics, Information and Communication Engineers 01.11.2020; Japan Science and Technology Agency
• Subjects: Computer architecture; facial expression recognition; Feature extraction; Feature recognition; long short-term memory; video sequence
• ISSN: 0916-8532; 1745-1361
• DOI: 10.1587/transinf.2020EDL8065

LSTM network have shown to outperform in facial expression recognition of video sequence. In view of limited representation ability of single-layer LSTM, a hierarchical attention model with enhanced feature branch is proposed. This new network architecture consists of traditional VGG-16-FACE with enhanced feature branch followed by a cross-layer LSTM. The VGG-16-FACE with enhanced branch extracts the spatial features as well as the cross-layer LSTM extracts the temporal relations between different frames in the video. The proposed method is evaluated on the public emotion databases in subject-independent and cross-database tasks and outperforms state-of-the-art methods.