CNLL: A Semi-supervised Approach For Continual Noisy Label Learning

TL;DR

This paper introduces CNLL, a simple semi-supervised method for continual learning with noisy labels, which effectively cleans data streams and achieves state-of-the-art results on benchmark datasets.

Contribution

Proposes a cost-effective purification technique combined with semi-supervised fine-tuning for noisy continual learning, improving performance and robustness.

Findings

Achieves 24.8% performance gain on CIFAR10 with 20% noise

Outperforms previous SOTA methods on benchmark datasets

Effective in handling noisy labels in continual learning scenarios

Abstract

The task of continual learning requires careful design of algorithms that can tackle catastrophic forgetting. However, the noisy label, which is inevitable in a real-world scenario, seems to exacerbate the situation. While very few studies have addressed the issue of continual learning under noisy labels, long training time and complicated training schemes limit their applications in most cases. In contrast, we propose a simple purification technique to effectively cleanse the online data stream that is both cost-effective and more accurate. After purification, we perform fine-tuning in a semi-supervised fashion that ensures the participation of all available samples. Training in this fashion helps us learn a better representation that results in state-of-the-art (SOTA) performance. Through extensive experimentation on 3 benchmark datasets, MNIST, CIFAR10 and CIFAR100, we show the effectiveness of our proposed approach. We achieve a 24.8% performance gain for CIFAR10 with 20% noise over previous SOTA methods. Our code is publicly available.