Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning

TL;DR

This paper introduces a new stochastic incremental learning scenario called Si-Blurry that mimics real-world data stream challenges and proposes Mask and Visual Prompt Tuning (MVP) to address forgetting and class imbalance, significantly improving performance.

Contribution

The paper presents the Si-Blurry scenario reflecting real-world stochastic properties and introduces MVP with novel masking, contrastive loss, and adaptive scaling to enhance continual learning.

Findings

MVP outperforms existing methods in Si-Blurry scenario.

Proposed techniques effectively reduce forgetting and class imbalance.

Significant performance improvements demonstrated through extensive experiments.

Abstract

Continual learning aims to learn a model from a continuous stream of data, but it mainly assumes a fixed number of data and tasks with clear task boundaries. However, in real-world scenarios, the number of input data and tasks is constantly changing in a statistical way, not a static way. Although recently introduced incremental learning scenarios having blurry task boundaries somewhat address the above issues, they still do not fully reflect the statistical properties of real-world situations because of the fixed ratio of disjoint and blurry samples. In this paper, we propose a new Stochastic incremental Blurry task boundary scenario, called Si-Blurry, which reflects the stochastic properties of the real-world. We find that there are two major challenges in the Si-Blurry scenario: (1) inter- and intra-task forgettings and (2) class imbalance problem. To alleviate them, we introduce Mask and Visual Prompt tuning (MVP). In MVP, to address the inter- and intra-task forgetting issues, we propose a novel instance-wise logit masking and contrastive visual prompt tuning loss. Both of them help our model discern the classes to be learned in the current batch. It results in consolidating the previous knowledge. In addition, to alleviate the class imbalance problem, we introduce a new gradient similarity-based focal loss and adaptive feature scaling to ease overfitting to the major classes and underfitting to the minor classes. Extensive experiments show that our proposed MVP significantly outperforms the existing state-of-the-art methods in our challenging Si-Blurry scenario.