Understanding the Collapse of LLMs in Model Editing

• Main Authors: Yang, Wanli, Sun, Fei, Tan, Jiajun, Ma, Xinyu, Su, Du, Yin, Dawei, Shen, Huawei
• Format: Journal Article
• Language: English
• Published: 17.06.2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language

Despite significant progress in model editing methods, their application in real-world scenarios remains challenging as they often cause large language models (LLMs) to collapse. Among them, ROME is particularly concerning, as it could disrupt LLMs with only a single edit. In this paper, we study th...