Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection

TL;DR

This paper introduces DynaCor, a framework that detects noisy labels by analyzing training dynamics and using label corruption to improve robustness across different noise types and levels.

Contribution

DynaCor uniquely models training dynamics with label corruption to effectively distinguish clean from noisy labels, surpassing existing methods.

Findings

Outperforms state-of-the-art noisy label detection methods

Robust across various noise types and noise rates

Effectively distinguishes clean and noisy instances using training dynamics

Abstract

Label noise, commonly found in real-world datasets, has a detrimental impact on a model's generalization. To effectively detect incorrectly labeled instances, previous works have mostly relied on distinguishable training signals, such as training loss, as indicators to differentiate between clean and noisy labels. However, they have limitations in that the training signals incompletely reveal the model's behavior and are not effectively generalized to various noise types, resulting in limited detection accuracy. In this paper, we propose DynaCor framework that distinguishes incorrectly labeled instances from correctly labeled ones based on the dynamics of the training signals. To cope with the absence of supervision for clean and noisy labels, DynaCor first introduces a label corruption strategy that augments the original dataset with intentionally corrupted labels, enabling indirect simulation of the model's behavior on noisy labels. Then, DynaCor learns to identify clean and noisy instances by inducing two clearly distinguishable clusters from the latent representations of training dynamics. Our comprehensive experiments show that DynaCor outperforms the state-of-the-art competitors and shows strong robustness to various noise types and noise rates.