Leveraging Logical Rules in Knowledge Editing: A Cherry on the Top
Multi-hop Question Answering (MQA) under knowledge editing (KE) is a key challenge in Large Language Models (LLMs). While best-performing solutions in this domain use a plan and solve paradigm to split a question into sub-questions followed by response generation, we claim that this approach is sub-...
Saved in:
Main Authors | , , , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
24.05.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Multi-hop Question Answering (MQA) under knowledge editing (KE) is a key
challenge in Large Language Models (LLMs). While best-performing solutions in
this domain use a plan and solve paradigm to split a question into
sub-questions followed by response generation, we claim that this approach is
sub-optimal as it fails for hard to decompose questions, and it does not
explicitly cater to correlated knowledge updates resulting as a consequence of
knowledge edits. This has a detrimental impact on the overall consistency of
the updated knowledge. To address these issues, in this paper, we propose a
novel framework named RULE-KE, i.e., RULE based Knowledge Editing, which is a
cherry on the top for augmenting the performance of all existing MQA methods
under KE. Specifically, RULE-KE leverages rule discovery to discover a set of
logical rules. Then, it uses these discovered rules to update knowledge about
facts highly correlated with the edit. Experimental evaluation using existing
and newly curated datasets (i.e., RKE-EVAL) shows that RULE-KE helps augment
both performances of parameter-based and memory-based solutions up to 92% and
112.9%, respectively. |
---|---|
DOI: | 10.48550/arxiv.2405.15452 |