Modality-Decoupled Online Recursive Editing

TL;DR

M-ORE is a modality-decoupled online recursive editing method for multimodal large language models that improves lifelong adaptation by mitigating cross-modal conflict and interference.

Contribution

It introduces a unified proximal-projection formulation with a closed-form update and Sherman-Morrison recursion for efficient, modality-decoupled online model editing.

Findings

Consistently improves reliability, generality, and locality over strong baselines.

Achieves favorable quality-efficiency scaling.

Effective across multiple MLLM backbones and benchmarks.

Abstract

Online model editing for multimodal large language models (MLLMs) requires assimilating a stream of corrections under tight compute and memory budgets. Yet editors developed for text-only LLMs often degrade on MLLMs: visually dominant activations skew the statistics that shape updates, causing cross-modal conflict, while sequential writes become entangled in a shared edit space and amplify long-horizon interference, causing inter-edit interference. To address these, we propose M-ORE, a modality-decoupled online recursive editor for lifelong MLLM adaptation. M-ORE is derived from a unified proximal-projection formulation and admits a closed-form update with a Sherman-Morrison recursion, yielding constant per-edit overhead. It maintains module-wise locality statistics for the text stack and the visual projector to avoid visually dominated update shaping and performs continual updates in a fixed orthogonal low-rank edit subspace via a Sherman-Morrison recursion to mitigate long-horizon interference. Experiments on multiple MLLM backbones and online editing benchmarks show that our M-ORE method consistently improves reliability, generality, and locality over strong baselines, while achieving favorable quality-efficiency scaling. Our code is publicly available at https://github.com/lab-klc/M-ORE.