Multi-domain awareness for compressed deepfake videos detection over social networks guided by common mechanisms between artifacts

• Published in: Computer vision and image understanding Vol. 247; p. 104072
• Main Authors: Wang, Yan, Sun, Qindong, Rong, Dongzhu, Geng, Rong
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.10.2024
• Subjects: Common mechanism between artifacts; Compressed deepfake video; Deepfake detection; Multi-domain awareness; Multimedia forensics; Deepfake detection; Compressed deepfake video; Multi-domain awareness; Common mechanism between artifacts; Multimedia forensics
• ISSN: 1077-3142
• DOI: 10.1016/j.cviu.2024.104072

The viral spread of massive deepfake videos over social networks has caused serious security problems. Despite the remarkable advancements achieved by existing deepfake detection algorithms, deepfake videos over social networks are inevitably influenced by compression factors. This causes deepfake detection performance to be limited by the following challenging issues: (a) interfering with compression artifacts, (b) loss of feature information, and (c) aliasing of feature distributions. In this paper, we analyze the common mechanism between compression artifacts and deepfake artifacts, revealing the structural similarity between them and providing a reliable theoretical basis for enhancing the robustness of deepfake detection models against compression. Firstly, based on the common mechanism between artifacts, we design a frequency domain adaptive notch filter to eliminate the interference of compression artifacts on specific frequency bands. Secondly, to reduce the sensitivity of deepfake detection models to unknown noise, we propose a spatial residual denoising strategy. Thirdly, to exploit the intrinsic correlation between feature vectors in the frequency domain branch and the spatial domain branch, we enhance deepfake features using an attention-based feature fusion method. Finally, we adopt a multi-task decision approach to enhance the discriminative power of the latent space representation of deepfakes, achieving deepfake detection with robustness against compression. Extensive experiments show that compared with the baseline methods, the detection performance of the proposed algorithm on compressed deepfake videos has been significantly improved. In particular, our model is resistant to various types of noise disturbances and can be easily combined with baseline detection models to improve their robustness. •We analyzed the common mechanisms of compression artifacts and deepfake artifacts.•Based on common mechanisms between artifacts, we designed an anti-compression model.•We designed adaptive notch filter to remove the interference of compression noise.•A multi-task learning strategy was adopted to optimize the detection model.•This method can be integrated with baselines as a plug-and-play model.