Leveraging Logical Rules in Knowledge Editing: A Cherry on the Top

Multi-hop Question Answering (MQA) under knowledge editing (KE) is a key challenge in Large Language Models (LLMs). While best-performing solutions in this domain use a plan and solve paradigm to split a question into sub-questions followed by response generation, we claim that this approach is sub-...

Full description

Saved in:
Bibliographic Details
Main Authors Cheng, Keyuan, Ali, Muhammad Asif, Yang, Shu, Lin, Gang, Zhai, Yuxuan, Fei, Haoyang, Xu, Ke, Yu, Lu, Hu, Lijie, Wang, Di
Format Journal Article
LanguageEnglish
Published 24.05.2024
Subjects
Computer Science - Artificial Intelligence
Computer Science - Computation and Language
Computer Science - Learning
Online AccessGet full text

Cover

More Information
Summary:Multi-hop Question Answering (MQA) under knowledge editing (KE) is a key challenge in Large Language Models (LLMs). While best-performing solutions in this domain use a plan and solve paradigm to split a question into sub-questions followed by response generation, we claim that this approach is sub-optimal as it fails for hard to decompose questions, and it does not explicitly cater to correlated knowledge updates resulting as a consequence of knowledge edits. This has a detrimental impact on the overall consistency of the updated knowledge. To address these issues, in this paper, we propose a novel framework named RULE-KE, i.e., RULE based Knowledge Editing, which is a cherry on the top for augmenting the performance of all existing MQA methods under KE. Specifically, RULE-KE leverages rule discovery to discover a set of logical rules. Then, it uses these discovered rules to update knowledge about facts highly correlated with the edit. Experimental evaluation using existing and newly curated datasets (i.e., RKE-EVAL) shows that RULE-KE helps augment both performances of parameter-based and memory-based solutions up to 92% and 112.9%, respectively.
DOI:10.48550/arxiv.2405.15452