Leveraging Logical Rules in Knowledge Editing: A Cherry on the Top

TL;DR

RULE-KE enhances multi-hop question answering under knowledge editing by leveraging logical rules for more consistent and effective knowledge updates, significantly improving existing methods' performance.

Contribution

The paper introduces RULE-KE, a novel framework that uses logical rule discovery to improve knowledge editing in large language models, addressing limitations of existing plan-based approaches.

Findings

RULE-KE improves performance of existing methods by up to 92%.

RULE-KE increases memory-based solutions' performance by 112.9%.

Experimental results demonstrate the effectiveness of logical rules in knowledge editing.

Abstract

Multi-hop Question Answering (MQA) under knowledge editing (KE) is a key challenge in Large Language Models (LLMs). While best-performing solutions in this domain use a plan and solve paradigm to split a question into sub-questions followed by response generation, we claim that this approach is sub-optimal as it fails for hard to decompose questions, and it does not explicitly cater to correlated knowledge updates resulting as a consequence of knowledge edits. This has a detrimental impact on the overall consistency of the updated knowledge. To address these issues, in this paper, we propose a novel framework named RULE-KE, i.e., RULE based Knowledge Editing, which is a cherry on the top for augmenting the performance of all existing MQA methods under KE. Specifically, RULE-KE leverages rule discovery to discover a set of logical rules. Then, it uses these discovered rules to update knowledge about facts highly correlated with the edit. Experimental evaluation using existing and newly curated datasets (i.e., RKE-EVAL) shows that RULE-KE helps augment both performances of parameter-based and memory-based solutions up to 92% and 112.9%, respectively.