Integrating Task-Specific and Universal Adapters for Pre-Trained Model-based Class-Incremental Learning

TL;DR

This paper introduces TUNA, a novel approach combining task-specific and universal adapters with an entropy-based selection and fusion strategy to improve class-incremental learning with pre-trained models, achieving state-of-the-art results.

Contribution

The paper proposes integrating task-specific and universal adapters with an entropy-based selection mechanism and fusion strategy for improved class-incremental learning.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Effectively balances task-specific and shared knowledge during inference.

Demonstrates robustness across various incremental learning scenarios.

Abstract

Class-Incremental Learning (CIL) requires a learning system to continually learn new classes without forgetting. Existing pre-trained model-based CIL methods often freeze the pre-trained network and adapt to incremental tasks using additional lightweight modules such as adapters. However, incorrect module selection during inference hurts performance, and task-specific modules often overlook shared general knowledge, leading to errors on distinguishing between similar classes across tasks. To address the aforementioned challenges, we propose integrating Task-Specific and Universal Adapters (TUNA) in this paper. Specifically, we train task-specific adapters to capture the most crucial features relevant to their respective tasks and introduce an entropy-based selection mechanism to choose the most suitable adapter. Furthermore, we leverage an adapter fusion strategy to construct a universal adapter, which encodes the most discriminative features shared across tasks. We combine task-specific and universal adapter predictions to harness both specialized and general knowledge during inference. Extensive experiments on various benchmark datasets demonstrate the state-of-the-art performance of our approach. Code is available at: https://github.com/LAMDA-CL/ICCV2025-TUNA