Training Convolutional Networks with Noisy Labels

TL;DR

This paper investigates training convolutional neural networks with noisy labels by introducing an adaptive noise layer, demonstrating effective learning even with imperfect data, including large-scale ImageNet experiments.

Contribution

It proposes a novel noise adaptation layer integrated into CNNs, enabling robust training on noisy datasets with minimal modifications to existing training procedures.

Findings

Effective noise modeling improves accuracy on noisy datasets

The method scales well to large datasets like ImageNet

Training with noisy labels can approach the performance of clean data training

Abstract

The availability of large labeled datasets has allowed Convolutional Network models to achieve impressive recognition results. However, in many settings manual annotation of the data is impractical; instead our data has noisy labels, i.e. there is some freely available label for each image which may or may not be accurate. In this paper, we explore the performance of discriminatively-trained Convnets when trained on such noisy data. We introduce an extra noise layer into the network which adapts the network outputs to match the noisy label distribution. The parameters of this noise layer can be estimated as part of the training process and involve simple modifications to current training infrastructures for deep networks. We demonstrate the approaches on several datasets, including large scale experiments on the ImageNet classification benchmark.