Regroup Median Loss for Combating Label Noise

TL;DR

This paper introduces Regroup Median Loss (RML), a novel loss function and sample selection strategy that effectively reduces the impact of label noise in training deep models, improving performance on noisy datasets.

Contribution

The paper proposes RML, a new loss and sampling method that better handles label noise, along with a semi-supervised approach, outperforming existing methods.

Findings

RML significantly improves accuracy on noisy datasets.

The semi-supervised RML method enhances model robustness.

Source code is publicly available.

Abstract

The deep model training procedure requires large-scale datasets of annotated data. Due to the difficulty of annotating a large number of samples, label noise caused by incorrect annotations is inevitable, resulting in low model performance and poor model generalization. To combat label noise, current methods usually select clean samples based on the small-loss criterion and use these samples for training. Due to some noisy samples similar to clean ones, these small-loss criterion-based methods are still affected by label noise. To address this issue, in this work, we propose Regroup Median Loss (RML) to reduce the probability of selecting noisy samples and correct losses of noisy samples. RML randomly selects samples with the same label as the training samples based on a new loss processing method. Then, we combine the stable mean loss and the robust median loss through a proposed regrouping strategy to obtain robust loss estimation for noisy samples. To further improve the model performance against label noise, we propose a new sample selection strategy and build a semi-supervised method based on RML. Compared to state-of-the-art methods, for both the traditionally trained and semi-supervised models, RML achieves a significant improvement on synthetic and complex real-world datasets. The source code of the paper has been released.