Contrastive Learning for Online Semi-Supervised General Continual Learning

TL;DR

This paper introduces SemiCon, a contrastive loss for online semi-supervised continual learning, demonstrating high efficiency with minimal labels and outperforming existing methods on benchmark datasets.

Contribution

The paper proposes SemiCon, a novel contrastive loss tailored for partly labeled data in online continual learning, with a memory-based approach that uses an oracle for labeling.

Findings

Outperforms existing semi-supervised methods with few labels

Achieves results comparable to supervised methods using minimal labels

Effective on Split-CIFAR10 and Split-CIFAR100 datasets

Abstract

We study Online Continual Learning with missing labels and propose SemiCon, a new contrastive loss designed for partly labeled data. We demonstrate its efficiency by devising a memory-based method trained on an unlabeled data stream, where every data added to memory is labeled using an oracle. Our approach outperforms existing semi-supervised methods when few labels are available, and obtain similar results to state-of-the-art supervised methods while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on Split-CIFAR100.