Unsupervised Label Noise Modeling and Loss Correction

TL;DR

This paper introduces an unsupervised approach using a beta mixture model to estimate and correct label noise during training of neural networks, improving robustness against noisy labels.

Contribution

It proposes a novel unsupervised label noise modeling method with loss correction and mixup augmentation, outperforming recent state-of-the-art techniques.

Findings

Significantly improves robustness to label noise on CIFAR-10/100 and TinyImageNet.

Effectively estimates mislabelled samples using a beta mixture model.

Enhances training stability and accuracy with loss correction.

Abstract

Despite being robust to small amounts of label noise, convolutional neural networks trained with stochastic gradient methods have been shown to easily fit random labels. When there are a mixture of correct and mislabelled targets, networks tend to fit the former before the latter. This suggests using a suitable two-component mixture model as an unsupervised generative model of sample loss values during training to allow online estimation of the probability that a sample is mislabelled. Specifically, we propose a beta mixture to estimate this probability and correct the loss by relying on the network prediction (the so-called bootstrapping loss). We further adapt mixup augmentation to drive our approach a step further. Experiments on CIFAR-10/100 and TinyImageNet demonstrate a robustness to label noise that substantially outperforms recent state-of-the-art. Source code is available at https://git.io/fjsvE