Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning

TL;DR

ACMap introduces a scalable, exemplar-free class-incremental learning framework that merges adapters into a shared space, maintaining accuracy and constant inference time across multiple tasks.

Contribution

It proposes a novel adapter merging and centroid prototype mapping approach that enhances scalability and reduces inference time without accuracy loss in CIL.

Findings

Achieves state-of-the-art accuracy on five benchmarks.

Maintains constant inference time regardless of task number.

Effectively mitigates catastrophic forgetting.

Abstract

We propose Adapter Merging with Centroid Prototype Mapping (ACMap), an exemplar-free framework for class-incremental learning (CIL) that addresses both catastrophic forgetting and scalability. While existing methods involve a trade-off between inference time and accuracy, ACMap consolidates task-specific adapters into a single adapter, thus achieving constant inference time across tasks without sacrificing accuracy. The framework employs adapter merging to build a shared subspace that aligns task representations and mitigates forgetting, while centroid prototype mapping maintains high accuracy by consistently adapting representations within the shared subspace. To further improve scalability, an early stopping strategy limits adapter merging as tasks increase. Extensive experiments on five benchmark datasets demonstrate that ACMap matches state-of-the-art accuracy while maintaining inference time comparable to the fastest existing methods. The code is available at https://github.com/tf63/ACMap.