Learning with Neighbor Consistency for Noisy Labels

TL;DR

This paper introduces a neighbor consistency regularization method for training deep models with noisy labels, improving robustness by encouraging similar predictions among neighboring data points, and demonstrates state-of-the-art results on multiple noisy datasets.

Contribution

It proposes a simple regularization technique leveraging feature space similarities, serving as an inductive extension of label propagation, to effectively handle noisy labels in deep learning.

Findings

Achieves competitive or state-of-the-art accuracy on various noisy datasets.

Effective in both synthetic and real-world noisy label scenarios.

Simplifies training by adding a regularization term without complex multi-model setups.

Abstract

Recent advances in deep learning have relied on large, labelled datasets to train high-capacity models. However, collecting large datasets in a time- and cost-efficient manner often results in label noise. We present a method for learning from noisy labels that leverages similarities between training examples in feature space, encouraging the prediction of each example to be similar to its nearest neighbours. Compared to training algorithms that use multiple models or distinct stages, our approach takes the form of a simple, additional regularization term. It can be interpreted as an inductive version of the classical, transductive label propagation algorithm. We thoroughly evaluate our method on datasets evaluating both synthetic (CIFAR-10, CIFAR-100) and realistic (mini-WebVision, WebVision, Clothing1M, mini-ImageNet-Red) noise, and achieve competitive or state-of-the-art accuracies across all of them.