Error-Bounded Correction of Noisy Labels

TL;DR

This paper provides a theoretical foundation for using noisy classifiers to identify clean labels and introduces a label correction algorithm that improves deep neural network training on noisy datasets.

Contribution

It offers the first theoretical explanation for noisy classifier heuristics and proposes a novel label correction method that enhances model performance under label noise.

Findings

The noisy classifier's predictions reliably indicate label cleanliness.

The proposed correction algorithm aligns labels with the true Bayesian classifier.

Models trained with corrected labels outperform those trained with noisy labels.

Abstract

To collect large scale annotated data, it is inevitable to introduce label noise, i.e., incorrect class labels. To be robust against label noise, many successful methods rely on the noisy classifiers (i.e., models trained on the noisy training data) to determine whether a label is trustworthy. However, it remains unknown why this heuristic works well in practice. In this paper, we provide the first theoretical explanation for these methods. We prove that the prediction of a noisy classifier can indeed be a good indicator of whether the label of a training data is clean. Based on the theoretical result, we propose a novel algorithm that corrects the labels based on the noisy classifier prediction. The corrected labels are consistent with the true Bayesian optimal classifier with high probability. We incorporate our label correction algorithm into the training of deep neural networks and train models that achieve superior testing performance on multiple public datasets.