BetaEdit: Null-Space Constrained Sequential Model Editing

TL;DR

BetaEdit introduces a novel null-space constrained model editing framework that reduces knowledge leakage and maintains model performance during extensive sequential updates, outperforming previous methods.

Contribution

The paper identifies knowledge leakage issues in existing null-space methods and proposes BetaEdit, which effectively controls leakage and incorporates history-aware updates.

Findings

BetaEdit outperforms prior methods in large-scale sequential editing tasks.

It effectively reduces knowledge leakage during model updates.

BetaEdit maintains model capabilities over long editing sequences.

Abstract

Null-space-based methods have garnered considerable attention in model editing by constraining updates to the null space of the pre-existing knowledge representation, thereby preserving the model's original behavior. However, in practice these methods rely on an approximate null space--leading to knowledge leakage--and further suffer from severe performance degradation during sequential editing. Recent work shows that history-aware editing strategies can empirically mitigate this decline, yet the underlying reason remains unclear. In this paper, we first expose the knowledge leakage inherent in existing null-space approaches and then analyze why history-aware updates effectively preserve both editing performance and general capabilities during long-horizon editing. Building on these insights, we propose BetaEdit, a refined framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm. Extensive experiments on three large language models across two standard benchmarks show that BetaEdit consistently outperforms prior methods in the challenging regime of massive-scale sequential editing. Code is available at: https://github.com/lbq8942/BetaEdit.