BalancEdit: Dynamically Balancing the Generality-Locality Trade-off in Multi-modal Model Editing

TL;DR

BalancEdit is a novel method for multi-modal model editing that dynamically balances the trade-off between generality and locality, improving update effectiveness without compromising model performance.

Contribution

This paper introduces BalancEdit, the first approach explicitly addressing the generality-locality trade-off in multi-modal model editing, using a new dataset and a unique influence scope mechanism.

Findings

BalancEdit achieves minimal trade-offs between generality and locality.

The method effectively updates models without modifying underlying weights.

Experimental results confirm the robustness and efficiency of BalancEdit.

Abstract

Large multi-modal models inevitably decay over time as facts update and previously learned information becomes outdated. Traditional approaches such as fine-tuning are often impractical for updating these models due to their size and complexity. Instead, direct knowledge editing within the models presents a more viable solution. Current model editing techniques, however, typically overlook the unique influence ranges of different facts, leading to compromised model performance in terms of both generality and locality. To address this issue, we introduce the concept of the generality-locality trade-off in multi-modal model editing. We develop a new model editing dataset named OKEDIT, specifically designed to effectively evaluate this trade-off. Building on this foundation, we propose \textbf{BalancEdit}, a novel method for balanced model editing that dynamically achieves an optimal balance between generality and locality. BalancEdit utilizes a unique mechanism that generates both positive and negative samples for each fact to accurately determine its influence scope and incorporates these insights into the model's latent space using a discrete, localized codebook of edits, without modifying the underlying model weights. To our knowledge, this is the first approach explicitly addressing the generality-locality trade-off in multi-modal model editing. Our comprehensive results confirm the effectiveness of BalancEdit, demonstrating minimal trade-offs while maintaining robust editing capabilities. Our code and dataset are available at https://github.com/donglgcn/BalancEdit/tree/MMOKVQA.