Artifact feature purification for cross-domain detection of AI-generated images

• Published in: Computer vision and image understanding Vol. 247; p. 104078
• Main Authors: Meng, Zheling, Peng, Bo, Dong, Jing, Tan, Tieniu, Cheng, Haonan
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.10.2024
• Subjects: AI-generated image detection; Artifact Purification; Cross-domain; Artifact Purification; AI-generated image detection; Cross-domain
• ISSN: 1077-3142
• DOI: 10.1016/j.cviu.2024.104078

In the era of AIGC, the fast development of visual content generation technologies, such as diffusion models, brings potential security risks to our society. Existing generated image detection methods suffer from performance drops when faced with out-of-domain generators and image scenes. To relieve this problem, we propose Artifact Purification Network (APN) to facilitate the artifact extraction from generated images through the explicit and implicit purification processes. For the explicit one, a suspicious frequency-band proposal method and a spatial feature decomposition method are proposed to extract artifact-related features. For the implicit one, a training strategy based on mutual information estimation is proposed to further purify the artifact-related features. The experiments are conducted in two settings. Firstly, we perform a cross-generator evaluation, wherein detectors trained using data from one generator are evaluated on data generated by other generators. Secondly, we conduct a cross-scene evaluation, wherein detectors trained for a specific domain of content (e.g., ImageNet) are assessed on data collected from another domain (e.g., LSUN-Bedroom). Results show that for cross-generator detection, the average accuracy of APN is 5.6%∼16.4% higher than the previous 11 methods on the GenImage dataset and 1.7%∼50.1% on the DiffusionForensics dataset. For cross-scene detection, APN maintains its high performance. Via visualization analysis, we find that the proposed method can extract diverse forgery patterns and condense the forgery information diluted in irrelated features. We also find that the artifact features APN focuses on across generators and scenes are global and diverse. The code will be available at https://github.com/RichardSunnyMeng/APN-official-codes. [Display omitted] •Performance drops across generators and scenes in detecting AI-generated images.•Artifact Purification Network with explicit purification and implicit purification.•At least 1.7% improvement across both domains on two open datasets.