Face Manipulation Detection Based on Supervised Multi-Feature Fusion Attention Network

• Published in: Sensors (Basel, Switzerland) Vol. 21; no. 24; p. 8181
• Main Authors: Cao, Lin, Sheng, Wenjun, Zhang, Fan, Du, Kangning, Fu, Chong, Song, Peiran
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 08.12.2021; MDPI
• Subjects: attention mechanism; capsule network; Deep learning; Eye (anatomy); face manipulation detection; Fourier transforms; Head; Humans; Methods; Neural networks; Neural Networks, Computer; supervised multi-feature fusion attention network; Video; capsule network; face manipulation detection; supervised multi-feature fusion attention network; attention mechanism
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s21248181

Nowadays, faces in videos can be easily replaced with the development of deep learning, and these manipulated videos are realistic and cannot be distinguished by human eyes. Some people maliciously use the technology to attack others, especially celebrities and politicians, causing destructive social impacts. Therefore, it is imperative to design an accurate method for detecting face manipulation. However, most of the existing methods adopt single convolutional neural network as the feature extraction module, causing the extracted features to be inconsistent with the human visual mechanism. Moreover, the rich details and semantic information cannot be reflected with single feature, limiting the detection performance. Therefore, this paper tackles the above problems by proposing a novel face manipulation detection method based on a supervised multi-feature fusion attention network (SMFAN). Specifically, the capsule network is used for face manipulation detection, and the SMFAN is added to the original capsule network to extract details of the fake face image. Further, the focal loss is used to realize hard example mining. Finally, the experimental results on the public dataset FaceForensics++ show that the proposed method has better performance.