Contrastive Continual Learning with Feature Propagation

TL;DR

This paper introduces a feature-propagation based contrastive continual learning approach that effectively handles multiple scenarios by aligning representations across tasks and mitigating domain and class shifts, achieving superior results on benchmarks.

Contribution

It presents a novel contrastive continual learning method using feature propagation and supervised contrastive loss to improve knowledge transfer across tasks.

Findings

Outperforms existing methods on six benchmarks.

Effectively reduces domain and class shifts.

Enhances continual learning performance.

Abstract

Classical machine learners are designed only to tackle one task without capability of adopting new emerging tasks or classes whereas such capacity is more practical and human-like in the real world. To address this shortcoming, continual machine learners are elaborated to commendably learn a stream of tasks with domain and class shifts among different tasks. In this paper, we propose a general feature-propagation based contrastive continual learning method which is capable of handling multiple continual learning scenarios. Specifically, we align the current and previous representation spaces by means of feature propagation and contrastive representation learning to bridge the domain shifts among distinct tasks. To further mitigate the class-wise shifts of the feature representation, a supervised contrastive loss is exploited to make the example embeddings of the same class closer than those of different classes. The extensive experimental results demonstrate the outstanding performance of the proposed method on six continual learning benchmarks compared to a group of cutting-edge continual learning methods.