Hierarchical Dual-Subspace Decoupling for Continual Learning in Vision-Language Models

TL;DR

This paper introduces HDSD, a hierarchical dual-subspace decoupling framework that improves continual learning in vision-language models by reducing subspace interference and parameter drift.

Contribution

It proposes a novel lightweight feature modulation module and a hierarchical learning approach to explicitly decompose and constrain parameter updates in high-dimensional spaces.

Findings

HDSD achieves state-of-the-art results on standard benchmarks.

The method effectively reduces catastrophic forgetting.

Experimental results demonstrate improved knowledge retention and transfer.

Abstract

Class-incremental learning aims to continuously acquire new knowledge while preserving previously learned information, thereby mitigating catastrophic forgetting. Existing methods primarily restrict parameter updates but often overlook their structural properties in high-dimensional spaces. From a subspace perspective, updates induced by different tasks tend to lie in multiple overlapping low-rank subspaces, leading to cross-task subspace interference and severe forgetting. To address this issue, we propose HDSD, a Hierarchical Dual-Subspace Decoupling framework for continual learning in vision-language models. Specifically, we introduce a lightweight Feature Modulation Module (FMM) that explicitly decomposes the parameter space into general and task-specific subspaces. Building on this design, we develop two complementary components. First, a General Fusion Module (GFM) evaluates relative parameter changes across tasks and uses an adaptive threshold to capture stable and transferable knowledge. Second, a Hierarchical Learning Module (HLM) performs structured parameter decomposition via Singular Value Decomposition (SVD) and uses a scaling mechanism to constrain updates within distinct subspace scales. Together, these designs reduce subspace interference and parameter drift. Extensive experiments on conventional benchmarks show that HDSD achieves state-of-the-art results.