EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries

TL;DR

This paper introduces EVEDIT, a novel event-based knowledge editing framework for large language models that improves the reliability and logical consistency of knowledge updates by considering contextual information and deduction anchors.

Contribution

The work proposes a new event-based editing task, a theoretical framework highlighting deduction anchors, and a novel approach Self-Edit that outperforms existing methods in knowledge editing.

Findings

Event-based editing reduces uncertainty in knowledge updates.

Self-Edit achieves 55.6% consistency improvement.

Curated the EvEdit benchmark dataset from CounterFact.

Abstract

The dynamic nature of real-world information necessitates efficient knowledge editing (KE) in large language models (LLMs) for knowledge updating. However, current KE approaches, which typically operate on (subject, relation, object) triples, ignore the contextual information and the relation among different knowledge. Such editing methods could thus encounter an uncertain editing boundary, leaving a lot of relevant knowledge in ambiguity: Queries that could be answered pre-edit cannot be reliably answered afterward. In this work, we analyze this issue by introducing a theoretical framework for KE that highlights an overlooked set of knowledge that remains unchanged and aids in knowledge deduction during editing, which we name as the deduction anchor. We further address this issue by proposing a novel task of event-based knowledge editing that pairs facts with event descriptions. This task manifests not only a closer simulation of real-world editing scenarios but also a more logically sound setting, implicitly defining the deduction anchor to address the issue of indeterminate editing boundaries. We empirically demonstrate the superiority of event-based editing over the existing setting on resolving uncertainty in edited models, and curate a new benchmark dataset EvEdit derived from the CounterFact dataset. Moreover, while we observe that the event-based setting is significantly challenging for existing approaches, we propose a novel approach Self-Edit that showcases stronger performance, achieving 55.6% consistency improvement while maintaining the naturalness of generation.