Learning Deep Networks from Noisy Labels with Dropout Regularization

TL;DR

This paper introduces a dropout-based regularization method for deep neural networks that effectively models and mitigates label noise during training, improving performance on noisy datasets.

Contribution

It proposes a novel dropout regularization technique applied to a noise model integrated with deep networks, enhancing robustness to label noise.

Findings

Outperforms existing methods on noisy CIFAR-10 and MNIST datasets

Effective modeling of label noise improves classification accuracy

Dropout regularization prevents overfitting to noisy labels

Abstract

Large datasets often have unreliable labels-such as those obtained from Amazon's Mechanical Turk or social media platforms-and classifiers trained on mislabeled datasets often exhibit poor performance. We present a simple, effective technique for accounting for label noise when training deep neural networks. We augment a standard deep network with a softmax layer that models the label noise statistics. Then, we train the deep network and noise model jointly via end-to-end stochastic gradient descent on the (perhaps mislabeled) dataset. The augmented model is overdetermined, so in order to encourage the learning of a non-trivial noise model, we apply dropout regularization to the weights of the noise model during training. Numerical experiments on noisy versions of the CIFAR-10 and MNIST datasets show that the proposed dropout technique outperforms state-of-the-art methods.