Dynamic Integration of Task-Specific Adapters for Class Incremental Learning

TL;DR

This paper introduces DIA, a novel framework for class incremental learning that effectively mitigates catastrophic forgetting without storing old data, by integrating task-specific adapters and patch-level model alignment.

Contribution

The paper proposes DIA, combining task-specific adapter integration and patch-level model alignment to improve NECIL performance with low computational costs.

Findings

Significant accuracy improvements on benchmark datasets.

Effective mitigation of catastrophic forgetting.

Maintains low computational complexity.

Abstract

Non-exemplar class Incremental Learning (NECIL) enables models to continuously acquire new classes without retraining from scratch and storing old task exemplars, addressing privacy and storage issues. However, the absence of data from earlier tasks exacerbates the challenge of catastrophic forgetting in NECIL. In this paper, we propose a novel framework called Dynamic Integration of task-specific Adapters (DIA), which comprises two key components: Task-Specific Adapter Integration (TSAI) and Patch-Level Model Alignment. TSAI boosts compositionality through a patch-level adapter integration strategy, which provides a more flexible compositional solution while maintaining low computation costs. Patch-Level Model Alignment maintains feature consistency and accurate decision boundaries via two specialized mechanisms: Patch-Level Distillation Loss (PDL) and Patch-Level Feature Reconstruction method (PFR). Specifically, the PDL preserves feature-level consistency between successive models by implementing a distillation loss based on the contributions of patch tokens to new class learning. The PFR facilitates accurate classifier alignment by reconstructing old class features from previous tasks that adapt to new task knowledge. Extensive experiments validate the effectiveness of our DIA, revealing significant improvements on benchmark datasets in the NECIL setting, maintaining an optimal balance between computational complexity and accuracy.