Mitigating Negative Interference in Multilingual Sequential Knowledge Editing through Null-Space Constraints

TL;DR

This paper introduces LangEdit, a null-space constrained framework that isolates language-specific knowledge updates in multilingual LLMs, effectively reducing destructive interference and improving update accuracy across languages.

Contribution

LangEdit's novel null-space projection method enables independent, interference-free knowledge updates across multiple languages in large language models.

Findings

LangEdit outperforms existing methods in multilingual knowledge editing.

It effectively mitigates destructive parameter interference.

Demonstrates robustness across different architectures and tasks.

Abstract

Efficiently updating multilingual knowledge in large language models (LLMs), while preserving consistent factual representations across languages, remains a long-standing and unresolved challenge. While deploying separate editing systems for each language might seem viable, this approach incurs substantial costs due to the need to manage multiple models. A more efficient solution involves integrating knowledge updates across all languages into a unified model. However, performing sequential edits across languages often leads to destructive parameter interference, significantly degrading multilingual generalization and the accuracy of injected knowledge. To address this challenge, we propose LangEdit, a novel null-space constrained framework designed to precisely isolate language-specific knowledge updates. The core innovation of LangEdit lies in its ability to project parameter updates for each language onto the orthogonal complement of previous updated subspaces. This approach mathematically guarantees update independence while preserving multilingual generalization capabilities. We conduct a comprehensive evaluation across three model architectures, six languages, and four downstream tasks, demonstrating that LangEdit effectively mitigates parameter interference and outperforms existing state-of-the-art editing methods. Our results highlight its potential for enabling efficient and accurate multilingual knowledge updates in LLMs. The code is available at https://github.com/VRCMF/LangEdit.git.