Knowledge Editing through Chain-of-Thought

TL;DR

EditCoT is a new framework for updating large language models' knowledge efficiently and flexibly across various tasks using chain-of-thought generation and refinement, outperforming existing methods.

Contribution

It introduces EditCoT, a novel, task-agnostic knowledge editing method that improves stability, generalization, and effectiveness without retraining the model.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Demonstrates superior generalization across diverse tasks.

Offers enhanced stability compared to prior methods.

Abstract

Knowledge Editing is a technique that updates large language models (LLMs) with new information to maintain their world knowledge. This approach avoids the need to rebuild the model from scratch, thereby addressing the high costs associated with frequent retraining. Among these, the in-context editing paradigm stands out for its effectiveness in integrating new knowledge while preserving the model's original capabilities. Despite its potential, existing in-context knowledge editing methods are often task-specific, focusing primarily on multi-hop QA tasks using structured knowledge triples. Moreover, their reliance on few-shot prompting for task decomposition makes them unstable and less effective in generalizing across diverse tasks. In response to these limitations, we propose EditCoT, a novel knowledge editing framework that flexibly and efficiently updates LLMs across various tasks without retraining. EditCoT works by generating a chain-of-thought (CoT) for a given input and then iteratively refining this CoT process using a CoT editor based on updated knowledge. We evaluate EditCoT across a diverse range of benchmarks, covering multiple languages and tasks. The results demonstrate that our approach achieves state-of-the-art performance while offering superior generalization, effectiveness, and stability compared to existing methods, marking a significant advancement in the field of knowledge updating. The code and data of EditCoT are available at: https://github.com/bebr2/EditCoT .