Multi‐view facial action unit detection via deep feature enhancement

• Published in: Electronics letters Vol. 57; no. 25; pp. 970 - 972
• Main Authors: Tang, Chuangao, Lu, Cheng, Zheng, Wenming, Zong, Yuan, Li, Sunan
• Format: Journal Article
• Language: English
• Published: Stevenage John Wiley & Sons, Inc 01.12.2021; Wiley
• Subjects: Bias; Complex variables; Computer vision and image processing techniques; Datasets; Image recognition; Mapping
• ISSN: 0013-5194; 1350-911X
• DOI: 10.1049/ell2.12322

Multi‐view facial action unit (AU) analysis has been a challenging research topic due to multiple disturbing variables, including subject identity biases, variational facial action unit intensities, facial occlusions and non‐frontal head‐poses. A deep feature enhancement (DFE) framework is presented to tackle some of these coupled complex disturbing variables for multi‐view facial action unit detection. The authors' DFE framework is a novel end‐to‐end three‐stage feature learning model with taking subject identity biases, dynamic facial changes and head‐pose into consideration. It contains three feature enhancement modules, including coarse‐grained local and holistic spatial feature learning (LHSF), spatio‐temporal feature learning (STF) and head‐pose feature disentanglement (FD). Experimental results show that the proposed method achieved state‐of‐the‐art recognition performance on the FERA2017 dataset. The code is released at http://aip.seu.edu.cn/cgtang/. A deep feature enhancement (DFE) framework is proposed, in which a novel end‐to‐end three‐stage feature learning model is presented taking subjects identity biases, dynamic facial changes and head‐pose into consideration. Experimental results show that the proposed method achieved state‐of‐the‐art recognition performance on the FERA2017 dataset.