Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning

TL;DR

This paper presents a novel method for unlearning specific visual knowledge in Multimodal Large Language Models by using contrastive mechanisms and null space constraints to balance forgetting and retention.

Contribution

It introduces a contrastive visual forgetting mechanism and null space constraints for effective unlearning in MLLMs, extending to continual unlearning scenarios.

Findings

Achieves effective target visual knowledge removal while preserving non-target knowledge.

Significantly reduces degradation of retained knowledge during unlearning.

Demonstrates strong performance across diverse benchmarks.

Abstract

The core challenge of machine unlearning is to strike a balance between target knowledge removal and non-target knowledge retention. In the context of Multimodal Large Language Models (MLLMs), this challenge becomes even more pronounced, as knowledge is further divided into visual and textual modalities that are tightly intertwined. In this paper, we introduce an MLLM unlearning approach that aims to forget target visual knowledge while preserving non-target visual knowledge and all textual knowledge. Specifically, we freeze the LLM backbone and achieve unlearning by fine-tuning the visual module. First, we propose a Contrastive Visual Forgetting (CVF) mechanism to separate target visual knowledge from retained visual knowledge, guiding the representations of target visual concepts toward appropriate regions in the feature space. Second, we identify the null space associated with retained knowledge and constrain the unlearning process within this space, thereby significantly mitigating degradation in knowledge retention. Third, beyond static unlearning scenarios, we extend our approach to continual unlearning, where forgetting requests arrive sequentially. Extensive experiments across diverse benchmarks demonstrate that our approach achieves a strong balance between effective forgetting and robust knowledge retention.