Towards Harnessing Feature Embedding for Robust Learning with Noisy Labels

TL;DR

This paper introduces LEND, a feature embedding-based method that leverages early-stage robust features to dilute noisy labels, improving deep learning robustness against label noise.

Contribution

The paper proposes a novel label noise mitigation technique using feature embeddings to dilute noisy labels, outperforming existing methods in noisy dataset scenarios.

Findings

LEND effectively reduces the impact of noisy labels in training.

LEND outperforms several robust learning approaches on synthetic and real-world datasets.

Feature embedding robustness enhances label noise correction.

Abstract

The memorization effect of deep neural networks (DNNs) plays a pivotal role in recent label noise learning methods. To exploit this effect, the model prediction-based methods have been widely adopted, which aim to exploit the outputs of DNNs in the early stage of learning to correct noisy labels. However, we observe that the model will make mistakes during label prediction, resulting in unsatisfactory performance. By contrast, the produced features in the early stage of learning show better robustness. Inspired by this observation, in this paper, we propose a novel feature embedding-based method for deep learning with label noise, termed LabEl NoiseDilution (LEND). To be specific, we first compute a similarity matrix based on current embedded features to capture the local structure of training data. Then, the noisy supervision signals carried by mislabeled data are overwhelmed by nearby correctly labeled ones (\textit{i.e.}, label noise dilution), of which the effectiveness is guaranteed by the inherent robustness of feature embedding. Finally, the training data with diluted labels are further used to train a robust classifier. Empirically, we conduct extensive experiments on both synthetic and real-world noisy datasets by comparing our LEND with several representative robust learning approaches. The results verify the effectiveness of our LEND.