Replay-free Online Continual Learning with Self-Supervised MultiPatches

TL;DR

This paper introduces Continual MultiPatches (CMP), a replay-free self-supervised learning method for online continual learning that generates multiple patches from each example to improve learning without privacy-compromising replay samples.

Contribution

CMP is a novel plug-in that enables replay-free online continual learning by using multi-patch generation and shared feature space projection, outperforming replay-based methods.

Findings

CMP surpasses replay strategies in OCL tasks.

CMP effectively avoids privacy issues associated with replay.

CMP improves learning efficiency in non-stationary data streams.

Abstract

Online Continual Learning (OCL) methods train a model on a non-stationary data stream where only a few examples are available at a time, often leveraging replay strategies. However, usage of replay is sometimes forbidden, especially in applications with strict privacy regulations. Therefore, we propose Continual MultiPatches (CMP), an effective plug-in for existing OCL self-supervised learning strategies that avoids the use of replay samples. CMP generates multiple patches from a single example and projects them into a shared feature space, where patches coming from the same example are pushed together without collapsing into a single point. CMP surpasses replay and other SSL-based strategies on OCL streams, challenging the role of replay as a go-to solution for self-supervised OCL.