AIRD: Adversarial Learning Framework for Image Repurposing Detection

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 11322 - 11331
• Main Authors: Jaiswal, Ayush, Wu, Yue, AbdAlmageed, Wael, Masi, Iacopo, Natarajan, Premkumar
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2019
• Subjects: Deep Learning; Fake news; Internet; Manuals; Measurement; Metadata; Pattern recognition; Publishing; Semantics; Social networking (online); Training; Vision Applications and Systems
• ISSN: 1063-6919
• DOI: 10.1109/CVPR.2019.01159

Image repurposing is a commonly used method for spreading misinformation on social media and online forums, which involves publishing untampered images with modified metadata to create rumors and further propaganda. While manual verification is possible, given vast amounts of verified knowledge available on the internet, the increasing prevalence and ease of this form of semantic manipulation call for the development of robust automatic ways of assessing the semantic integrity of multimedia data. In this paper, we present a novel method for image repurposing detection that is based on the real-world adversarial interplay between a bad actor who repurposes images with counterfeit metadata and a watchdog who verifies the semantic consistency between images and their accompanying metadata, where both players have access to a reference dataset of verified content, which they can use to achieve their goals. The proposed method exhibits state-of-the-art performance on location-identity, subject-identity and painting-artist verification, showing its efficacy across a diverse set of scenarios.