DocMEdit: Towards Document-Level Model Editing

TL;DR

This paper introduces a new task and dataset for document-level model editing in large language models, addressing real-world needs by focusing on complex, multi-fact, document-based corrections.

Contribution

It presents enchmarkname, a novel dataset for document-level editing, along with evaluation metrics and experiments highlighting challenges for current methods.

Findings

Existing model editing methods struggle with document-level tasks.

The enchmarkname dataset enables assessment of complex, multi-fact edits.

Document-level editing presents unique challenges not captured by previous datasets.

Abstract

Model editing aims to correct errors and outdated knowledge in the Large language models (LLMs) with minimal cost. Prior research has proposed a variety of datasets to assess the effectiveness of these model editing methods. However, most existing datasets only require models to output short phrases or sentences, overlooks the widespread existence of document-level tasks in the real world, raising doubts about their practical usability. Aimed at addressing this limitation and promoting the application of model editing in real-world scenarios, we propose the task of document-level model editing. To tackle such challenges and enhance model capabilities in practical settings, we introduce \benchmarkname, a dataset focused on document-level model editing, characterized by document-level inputs and outputs, extrapolative, and multiple facts within a single edit. We propose a series of evaluation metrics and experiments. The results show that the difficulties in document-level model editing pose challenges for existing model editing methods.