Adaptive Sample Selection for Robust Learning under Label Noise

TL;DR

This paper introduces an adaptive sample selection method for training deep neural networks that is robust to label noise, requiring no prior noise rate knowledge or extra hyperparameters, and demonstrating effectiveness on benchmark datasets.

Contribution

The proposed method adaptively selects samples based on batch statistics, eliminating the need for noise rate information or additional hyperparameters in noisy label scenarios.

Findings

Effective in handling label noise without prior noise rate knowledge

No additional hyperparameters required for sample selection

Demonstrates superior robustness on benchmark datasets

Abstract

Deep Neural Networks (DNNs) have been shown to be susceptible to memorization or overfitting in the presence of noisily-labelled data. For the problem of robust learning under such noisy data, several algorithms have been proposed. A prominent class of algorithms rely on sample selection strategies wherein, essentially, a fraction of samples with loss values below a certain threshold are selected for training. These algorithms are sensitive to such thresholds, and it is difficult to fix or learn these thresholds. Often, these algorithms also require information such as label noise rates which are typically unavailable in practice. In this paper, we propose an adaptive sample selection strategy that relies only on batch statistics of a given mini-batch to provide robustness against label noise. The algorithm does not have any additional hyperparameters for sample selection, does not need any information on noise rates and does not need access to separate data with clean labels. We empirically demonstrate the effectiveness of our algorithm on benchmark datasets.