An effective framework based on hybrid learning and kernel principal component analysis for face manipulation detection

• Published in: Signal, image and video processing Vol. 18; no. 5; pp. 4811 - 4820
• Main Authors: Thakur, Rahul, Rohilla, Rajesh
• Format: Journal Article
• Language: English
• Published: London Springer London 01.07.2024; Springer Nature B.V
• Subjects: Blended learning; Computer Imaging; Computer Science; Data augmentation; Datasets; Feature extraction; Image manipulation; Image Processing and Computer Vision; Learning; Multimedia Information Systems; Original Paper; Pattern Recognition and Graphics; Principal components analysis; Signal,Image and Speech Processing; Video; Vision; Deep learning; EfficientNetV2; DeepFake; Hybrid learning; Face manipulation detection
• ISSN: 1863-1703; 1863-1711
• DOI: 10.1007/s11760-024-03117-0

Face manipulation is the process of modifying facial features in videos or images to produce a variety of artistic or deceptive effects. Face manipulation detection looks for altered or falsified visual media in order to differentiate between real and fake facial photographs or videos. The intricacy of the techniques used makes it difficult to detect face manipulation, particularly in the context of technologies like DeepFake. This paper presents an efficient framework based on Hybrid Learning and Kernel Principal Component Analysis (KPCA) to extract more extensive and refined face-manipulating attributes. The proposed method utilizes the EfficientNetV2-L model for feature extraction, topped up with KPCA for feature dimensionality reduction, to distinguish between real and fake facial images. The proposed method is robust to various facial manipulations techniques such as identity swap, expression swap, attribute-based manipulation, and entirely synthesized faces. In this work, data augmentation is used to solve the problem of class imbalance present in the dataset. The proposed method has less execution time while achieving an accuracy of 99.3% and an F1 Score of 0.98 on the Diverse Fake Face Dataset (DFFD).