Transform consistency for learning with noisy labels

TL;DR

This paper introduces a single-network method that leverages transform consistency to identify clean samples in noisy datasets, improving robustness and achieving state-of-the-art results on multiple benchmarks.

Contribution

It proposes a novel transform consistency approach for noisy label learning that does not require training multiple networks, enhancing simplicity and effectiveness.

Findings

Outperforms baselines on CIFAR-10, CIFAR-100, and Clothing1M datasets.

Achieves state-of-the-art performance in noisy label learning.

Effectively distinguishes clean from noisy samples using transform consistency.

Abstract

It is crucial to distinguish mislabeled samples for dealing with noisy labels. Previous methods such as Coteaching and JoCoR introduce two different networks to select clean samples out of the noisy ones and only use these clean ones to train the deep models. Different from these methods which require to train two networks simultaneously, we propose a simple and effective method to identify clean samples only using one single network. We discover that the clean samples prefer to reach consistent predictions for the original images and the transformed images while noisy samples usually suffer from inconsistent predictions. Motivated by this observation, we introduce to constrain the transform consistency between the original images and the transformed images for network training, and then select small-loss samples to update the parameters of the network. Furthermore, in order to mitigate the negative influence of noisy labels, we design a classification loss by using the off-line hard labels and on-line soft labels to provide more reliable supervisions for training a robust model. We conduct comprehensive experiments on CIFAR-10, CIFAR-100 and Clothing1M datasets. Compared with the baselines, we achieve the state-of-the-art performance. Especially, in most cases, our proposed method outperforms the baselines by a large margin.