A Representative Study on Human Detection of Artificially Generated Media Across Countries

• Published in: arXiv.org
• Main Authors: Frank, Joel, Herbert, Franziska, Ricker, Jonas, Schönherr, Lea, Eisenhofer, Thorsten, Fischer, Asja, Dürmuth, Markus, Holz, Thorsten
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 10.12.2023
• Subjects: Deception; Literature reviews; Regression analysis

AI-generated media has become a threat to our digital society as we know it. These forgeries can be created automatically and on a large scale based on publicly available technology. Recognizing this challenge, academics and practitioners have proposed a multitude of automatic detection strategies to detect such artificial media. However, in contrast to these technical advances, the human perception of generated media has not been thoroughly studied yet. In this paper, we aim at closing this research gap. We perform the first comprehensive survey into people's ability to detect generated media, spanning three countries (USA, Germany, and China) with 3,002 participants across audio, image, and text media. Our results indicate that state-of-the-art forgeries are almost indistinguishable from "real" media, with the majority of participants simply guessing when asked to rate them as human- or machine-generated. In addition, AI-generated media receive is voted more human like across all media types and all countries. To further understand which factors influence people's ability to detect generated media, we include personal variables, chosen based on a literature review in the domains of deepfake and fake news research. In a regression analysis, we found that generalized trust, cognitive reflection, and self-reported familiarity with deepfakes significantly influence participant's decision across all media categories.