Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
The locate-then-edit paradigm has shown significant promise for knowledge editing (KE) in Large Language Models (LLMs). While previous methods perform well on single-hop fact recall tasks, they consistently struggle with multi-hop factual recall tasks involving newly edited knowledge. In this paper,...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
08.10.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | The locate-then-edit paradigm has shown significant promise for knowledge
editing (KE) in Large Language Models (LLMs). While previous methods perform
well on single-hop fact recall tasks, they consistently struggle with multi-hop
factual recall tasks involving newly edited knowledge. In this paper,
leveraging tools in mechanistic interpretability, we first identify that in
multi-hop tasks, LLMs tend to retrieve implicit subject knowledge from deeper
MLP layers, unlike single-hop tasks, which rely on earlier layers. This
distinction explains the poor performance of current methods in multi-hop
queries, as they primarily focus on editing shallow layers, leaving deeper
layers unchanged. To address this, we propose IFMET, a novel locate-then-edit
KE approach designed to edit both shallow and deep MLP layers. IFMET employs
multi-hop editing prompts and supplementary sets to locate and modify knowledge
across different reasoning stages. Experimental results demonstrate that IFMET
significantly improves performance on multi-hop factual recall tasks,
effectively overcoming the limitations of previous locate-then-edit methods. |
---|---|
DOI: | 10.48550/arxiv.2410.06331 |