DALD: Improving Logits-based Detector without Logits from Black-box LLMs

The advent of Large Language Models (LLMs) has revolutionized text generation, producing outputs that closely mimic human writing. This blurring of lines between machine- and human-written text presents new challenges in distinguishing one from the other a task further complicated by the frequent up...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Zeng, Cong, Tang, Shengkun, Yang, Xianjun, Chen, Yuanzhou, Sun, Yiyou, Xu, Zhiqiang, Yao, Li, Chen, Haifeng, Cheng, Wei, Xu, Dongkuan
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 27.10.2024
Subjects
Black boxes
Blurring
Large language models
Misalignment
Performance degradation
Target detection
Online AccessGet full text

Cover

More Information
Summary:The advent of Large Language Models (LLMs) has revolutionized text generation, producing outputs that closely mimic human writing. This blurring of lines between machine- and human-written text presents new challenges in distinguishing one from the other a task further complicated by the frequent updates and closed nature of leading proprietary LLMs. Traditional logits-based detection methods leverage surrogate models for identifying LLM-generated content when the exact logits are unavailable from black-box LLMs. However, these methods grapple with the misalignment between the distributions of the surrogate and the often undisclosed target models, leading to performance degradation, particularly with the introduction of new, closed-source models. Furthermore, while current methodologies are generally effective when the source model is identified, they falter in scenarios where the model version remains unknown, or the test set comprises outputs from various source models. To address these limitations, we present Distribution-Aligned LLMs Detection (DALD), an innovative framework that redefines the state-of-the-art performance in black-box text detection even without logits from source LLMs. DALD is designed to align the surrogate model's distribution with that of unknown target LLMs, ensuring enhanced detection capability and resilience against rapid model iterations with minimal training investment. By leveraging corpus samples from publicly accessible outputs of advanced models such as ChatGPT, GPT-4 and Claude-3, DALD fine-tunes surrogate models to synchronize with unknown source model distributions effectively.
ISSN:2331-8422