GR4CIL: Gap-compensated Routing for CLIP-based Class Incremental Learning

TL;DR

GR4CIL is a novel framework for CLIP-based class-incremental learning that improves task discrimination and knowledge routing, reducing interference and bias while maintaining zero-shot capabilities.

Contribution

It introduces task discrimination, knowledge routing, and an orthogonal compensation mechanism to enhance CLIP-based CIL performance.

Findings

Consistently outperforms strong baselines on multiple benchmarks.

Reduces interference across tasks and mitigates modality-gap bias.

Enhances within-task discrimination and score margin between tasks.

Abstract

Class-Incremental Learning (CIL) aims to continuously acquire new categories while preserving previously learned knowledge. Recently, Contrastive Language-Image Pre-trained (CLIP) models have shown strong potential for CIL due to their powerful generalization ability. However, existing methods still face two key challenges: shared-parameter adaptation tends to cause old-knowledge drift, and task-specific knowledge organization often leads to poorly calibrated cross-task responses, making reliable routing difficult. To address these issues, we propose GR4CIL, a framework combining task discrimination and knowledge routing for CLIP-based CIL. GR4CIL preserves task-specific visual knowledge while maintaining an incrementally stable shared textual semantic space, thereby reducing interference across tasks. Moreover, we introduce an orthogonal compensation mechanism to mitigate modality-gap-induced bias, enhance within-task discrimination, and enlarge the score margin between the ground-truth task and competing tasks. As a result, GR4CIL enables more reliable task-aware routing over learned knowledge while retaining the zero-shot generalization capability. Experiments on multiple benchmarks show that GR4CIL consistently outperforms strong baselines.