Versatile Incremental Learning: Towards Class and Domain-Agnostic Incremental Learning

TL;DR

This paper introduces Versatile Incremental Learning (VIL), a new challenging scenario where models must learn from tasks with unpredictable class or domain changes, and proposes the ICON framework with CAST regularization to effectively handle intra-class and inter-domain confusion.

Contribution

The paper proposes a novel VIL scenario and introduces the ICON framework with CAST regularization and an incremental classifier to address class and domain confusion in unpredictable task sequences.

Findings

Effective in handling random task alterations

Outperforms existing methods on benchmarks

Reduces catastrophic forgetting in VIL scenarios

Abstract

Incremental Learning (IL) aims to accumulate knowledge from sequential input tasks while overcoming catastrophic forgetting. Existing IL methods typically assume that an incoming task has only increments of classes or domains, referred to as Class IL (CIL) or Domain IL (DIL), respectively. In this work, we consider a more challenging and realistic but under-explored IL scenario, named Versatile Incremental Learning (VIL), in which a model has no prior of which of the classes or domains will increase in the next task. In the proposed VIL scenario, the model faces intra-class domain confusion and inter-domain class confusion, which makes the model fail to accumulate new knowledge without interference with learned knowledge. To address these issues, we propose a simple yet effective IL framework, named Incremental Classifier with Adaptation Shift cONtrol (ICON). Based on shifts of learnable modules, we design a novel regularization method called Cluster-based Adaptation Shift conTrol (CAST) to control the model to avoid confusion with the previously learned knowledge and thereby accumulate the new knowledge more effectively. Moreover, we introduce an Incremental Classifier (IC) which expands its output nodes to address the overwriting issue from different domains corresponding to a single class while maintaining the previous knowledge. We conducted extensive experiments on three benchmarks, showcasing the effectiveness of our method across all the scenarios, particularly in cases where the next task can be randomly altered. Our implementation code is available at https://github.com/KHU-AGI/VIL.