PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization

• Published in: IEEE transactions on circuits and systems for video technology Vol. 32; no. 11; pp. 7505 - 7517
• Main Authors: Liu, Xiaohong, Liu, Yaojie, Chen, Jun, Liu, Xiaoming
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: attention mechanism; Coders; Correlation; Encoders-Decoders; Estimation; Feature extraction; Forgery; Image manipulation; Image manipulation detection and localization; Localization; Location awareness; Masks; progressive mechanism; Splicing; Task analysis; Weight reduction
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2022.3189545

To defend against manipulation of image content, such as splicing, copy-move, and removal, we develop a Progressive Spatio-Channel Correlation Network (PSCC-Net) to detect and localize image manipulations. PSCC-Net processes the image in a two-path procedure: a top-down path that extracts local and global features and a bottom-up path that detects whether the input image is manipulated, and estimates its manipulation masks at multiple scales, where each mask is conditioned on the previous one. Different from the conventional encoder-decoder and no-pooling structures, PSCC-Net leverages features at different scales with dense cross-connections to produce manipulation masks in a coarse-to-fine fashion. Moreover, a Spatio-Channel Correlation Module (SCCM) captures both spatial and channel-wise correlations in the bottom-up path, which endows features with holistic cues, enabling the network to cope with a wide range of manipulation attacks. Thanks to the light-weight backbone and progressive mechanism, PSCC-Net can process <inline-formula> <tex-math notation="LaTeX">1,080\text{P} </tex-math></inline-formula> images at 50+FPS. Extensive experiments demonstrate the superiority of PSCC-Net over the state-of-the-art methods on both detection and localization. Codes and models are available at https://github.com/proteus1991/PSCC-Net .