FAME: Towards Factual Multi-Task Model Editing

TL;DR

FAME introduces a comprehensive dataset and SKEME, a novel model editing method with caching, to improve factual accuracy and practicality of large language models without retraining.

Contribution

The paper presents FAME, a multi-task dataset for realistic model editing, and SKEME, a new editing method with caching to enhance factual correctness in LLMs.

Findings

SKEME outperforms existing methods across multiple tasks.

FAME dataset improves evaluation of practical model editing.

Caching mechanism ensures better synchronization with real-world knowledge.

Abstract

Large language models (LLMs) embed extensive knowledge and utilize it to perform exceptionally well across various tasks. Nevertheless, outdated knowledge or factual errors within LLMs can lead to misleading or incorrect responses, causing significant issues in practical applications. To rectify the fatal flaw without the necessity for costly model retraining, various model editing approaches have been proposed to correct inaccurate knowledge within LLMs in a cost-efficient way. To evaluate these model editing methods, previous work introduced a series of datasets. However, most of the previous datasets only contain fabricated data in a single format, which diverges from real-world model editing scenarios, raising doubts about their usability in practice. To facilitate the application of model editing in real-world scenarios, we propose the challenge of practicality. To resolve such challenges and effectively enhance the capabilities of LLMs, we present FAME, an factual, comprehensive, and multi-task dataset, which is designed to enhance the practicality of model editing. We then propose SKEME, a model editing method that uses a novel caching mechanism to ensure synchronization with the real world. The experiments demonstrate that SKEME performs excellently across various tasks and scenarios, confirming its practicality.