A Framework using Contrastive Learning for Classification with Noisy Labels

TL;DR

This paper introduces a contrastive learning framework for image classification that enhances robustness to noisy labels through pre-training, with empirical evidence showing improved performance on benchmarks and real-world datasets.

Contribution

It demonstrates that contrastive pre-training significantly improves robustness to noisy labels and enhances classification accuracy when combined with various loss functions.

Findings

Contrastive pre-training boosts robustness to noisy labels.

Fine-tuning further improves accuracy but adds complexity.

Empirical results on benchmarks validate the approach.

Abstract

We propose a framework using contrastive learning as a pre-training task to perform image classification in the presence of noisy labels. Recent strategies such as pseudo-labeling, sample selection with Gaussian Mixture models, weighted supervised contrastive learning have been combined into a fine-tuning phase following the pre-training. This paper provides an extensive empirical study showing that a preliminary contrastive learning step brings a significant gain in performance when using different loss functions: non-robust, robust, and early-learning regularized. Our experiments performed on standard benchmarks and real-world datasets demonstrate that: i) the contrastive pre-training increases the robustness of any loss function to noisy labels and ii) the additional fine-tuning phase can further improve accuracy but at the cost of additional complexity.