Multi-Objective Interpolation Training for Robustness to Label Noise

TL;DR

This paper introduces MOIT, a novel training method combining contrastive learning and classification to improve neural network robustness against label noise, along with a noise detection technique and an enhanced version MOIT+.

Contribution

It proposes a multi-objective interpolation training approach that jointly leverages contrastive learning and classification, along with a new label noise detection method and an improved training scheme MOIT+.

Findings

MOIT outperforms existing methods on synthetic and real-world noise benchmarks.

Contrastive learning degrades with noisy labels, but interpolation mitigates this.

MOIT+ further improves performance through fine-tuning on clean samples.

Abstract

Deep neural networks trained with standard cross-entropy loss memorize noisy labels, which degrades their performance. Most research to mitigate this memorization proposes new robust classification loss functions. Conversely, we propose a Multi-Objective Interpolation Training (MOIT) approach that jointly exploits contrastive learning and classification to mutually help each other and boost performance against label noise. We show that standard supervised contrastive learning degrades in the presence of label noise and propose an interpolation training strategy to mitigate this behavior. We further propose a novel label noise detection method that exploits the robust feature representations learned via contrastive learning to estimate per-sample soft-labels whose disagreements with the original labels accurately identify noisy samples. This detection allows treating noisy samples as unlabeled and training a classifier in a semi-supervised manner to prevent noise memorization and improve representation learning. We further propose MOIT+, a refinement of MOIT by fine-tuning on detected clean samples. Hyperparameter and ablation studies verify the key components of our method. Experiments on synthetic and real-world noise benchmarks demonstrate that MOIT/MOIT+ achieves state-of-the-art results. Code is available at https://git.io/JI40X.