StruEdit: Structured Outputs Enable the Fast and Accurate Knowledge Editing for Large Language Models

TL;DR

StruEdit introduces a structured output approach for large language models that enables faster and more accurate knowledge editing by focusing on reasoning triplets, addressing the limitations of unstructured natural language outputs.

Contribution

The paper proposes StruEdit, a novel method that uses structured reasoning triplets to improve the speed and accuracy of knowledge editing in large language models.

Findings

StruEdit achieves higher accuracy than existing methods.

StruEdit has lower latency in knowledge editing tasks.

Structured outputs facilitate more effective knowledge updates.

Abstract

As the modern tool of choice for question answering, large language models (LLMs) are expected to deliver answers with up-to-date knowledge. To achieve such ideal question-answering systems, locating and then editing outdated knowledge in the natural language outputs is a general target of popular knowledge editing methods. However, this target is challenging, as both identifying which tokens to edit in the reasoning steps and ensuring the coherence of the revised reasoning chain are difficult tasks. We argue that these challenges stem from the unstructured nature of natural language outputs. To address the above challenges, we propose $\textbf{Stru}$ctural $\textbf{Edit}$ing ($\textbf{StruEdit}$), an improved baseline for knowledge editing. We first prompt LLMs to produce structured outputs consisting of reasoning triplets. Then, StruEdit removes any potentially outdated knowledge and efficiently refills the structured outputs with up-to-date information in a single step. Experimental results show that StruEdit consistently delivers the highest accuracy with lowest latency compared with other knowledge editing methods.