A Meta-Learning Approach for Few-Shot Face Forgery Segmentation and Classification

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 7; p. 3647
• Main Authors: Lin, Yih-Kai, Yen, Ting-Yu
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 31.03.2023; MDPI
• Subjects: Analysis; Classification; Deep learning; Detectors; digital forensics; face forgery detection; few-shot learning; Forgery; Medical imaging equipment; meta-learning; Methods; Neural networks; segmentation; Sensors; Technology application; U-Net; digital forensics; segmentation; face forgery detection; U-Net; meta-learning; few-shot learning
• DOI: 10.3390/s23073647
• ISSN: 1424-8220

The technology for detecting forged images is good at detecting known forgery methods. It trains neural networks using many original and corresponding forged images created with known methods. However, when encountering unseen forgery methods, the technology performs poorly. Recently, one suggested approach to tackle this problem is to use a hand-crafted generator of forged images to create a range of fake images, which can then be used to train the neural network. However, the aforementioned method has limited detection performance when encountering unseen forging techniques that the hand-craft generator has not accounted for. To overcome the limitations of existing methods, in this paper, we adopt a meta-learning approach to develop a highly adaptive detector for identifying new forging techniques. The proposed method trains a forged image detector using meta-learning techniques, making it possible to fine-tune the detector with only a few new forged samples. The proposed method inputs a small number of the forged images to the detector and enables the detector to adjust its weights based on the statistical features of the input forged images, allowing the detection of forged images with similar characteristics. The proposed method achieves significant improvement in detecting forgery methods, with IoU improvements ranging from 35.4% to 127.2% and AUC improvements ranging from 2.0% to 48.9%, depending on the forgery method. These results show that the proposed method significantly improves detection performance with only a small number of samples and demonstrates better performance compared to current state-of-the-art methods in most scenarios.