Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning

TL;DR

This paper reveals that incorporating domain shift in class-incremental learning reduces catastrophic forgetting by enhancing feature separation, and introduces DisCo, a contrastive learning-based method that improves existing CIL techniques.

Contribution

The paper introduces DisCo, a contrastive learning-based approach that leverages domain shift to mitigate forgetting in class-incremental learning, compatible with existing methods.

Findings

DisCo improves performance of CIL methods significantly.

Domain shift enhances feature separation across tasks.

DisCo reduces parameter interference during learning.

Abstract

In the realm of class-incremental learning (CIL), alleviating the catastrophic forgetting problem is a pivotal challenge. This paper discovers a counter-intuitive observation: by incorporating domain shift into CIL tasks, the forgetting rate is significantly reduced. Our comprehensive studies demonstrate that incorporating domain shift leads to a clearer separation in the feature distribution across tasks and helps reduce parameter interference during the learning process. Inspired by this observation, we propose a simple yet effective method named DisCo to deal with CIL tasks. DisCo introduces a lightweight prototype pool that utilizes contrastive learning to promote distinct feature distributions for the current task relative to previous ones, effectively mitigating interference across tasks. DisCo can be easily integrated into existing state-of-the-art class-incremental learning methods. Experimental results show that incorporating our method into various CIL methods achieves substantial performance improvements, validating the benefits of our approach in enhancing class-incremental learning by separating feature representation and reducing interference. These findings illustrate that DisCo can serve as a robust fashion for future research in class-incremental learning.