Language as an Anchor: Preserving Relative Visual Geometry for Domain Incremental Learning

TL;DR

LAVA introduces a novel domain incremental learning framework that uses language-anchored relative visual alignment to preserve semantic relationships across shifting domains, improving knowledge retention and transfer.

Contribution

The paper proposes LAVA, a new method that leverages text-based references to maintain relative visual geometry across domains, addressing interference and fragmentation issues in DIL.

Findings

LAVA outperforms existing methods on standard DIL benchmarks.

The approach effectively preserves semantic relationships across domains.

LAVA demonstrates robust knowledge reuse and reduced forgetting.

Abstract

A key challenge in Domain Incremental Learning (DIL) is to continually learn under shifting distributions while preserving knowledge from previous domains. Existing methods face a fundamental dilemma. On one hand, projecting all domains into a single unified visual space leads to inter-domain interference and semantic distortion, as large shifts may vary with not only visual appearance but also underlying semantics. On the other hand, isolating domain-specific parameters causes knowledge fragmentation, creating "knowledge islands" that hamper knowledge reuse and exacerbate forgetting. To address this issue, we propose LAVA (Language-Anchored Visual Alignment), a novel DIL framework that replaces direct feature alignment with relative alignment driven by a text-based reference anchor. LAVA guides the visual representations of each incoming domain to preserve a consistent relative geometry, which is defined by mirroring the pairwise semantic similarities between the class names. This anchored geometric structure acts as a bridge across domains, enabling the retrieval of class-aware prior knowledge and facilitating robust feature aggregation. Extensive experiments on standard DIL benchmarks demonstrate that LAVA achieves significant performance improvements over state-of-the-arts. Code is available at https://github.com/ShuyiGeng/LAVA.