Learning with Noisy Labels over Imbalanced Subpopulations

TL;DR

This paper introduces a novel learning method that effectively handles noisy labels and imbalanced subpopulations by estimating sample cleanliness through correlation and applying distributionally robust optimization, improving robustness and accuracy.

Contribution

It proposes a feature-based correlation method for label correction combined with DRO to enhance learning with noisy, imbalanced data, outperforming existing approaches.

Findings

Consistently improves state-of-the-art robustness against noisy labels.

Effectively handles imbalanced subpopulations in various benchmarks.

Enhances generalization performance in real-world noisy, imbalanced datasets.

Abstract

Learning with Noisy Labels (LNL) has attracted significant attention from the research community. Many recent LNL methods rely on the assumption that clean samples tend to have "small loss". However, this assumption always fails to generalize to some real-world cases with imbalanced subpopulations, i.e., training subpopulations varying in sample size or recognition difficulty. Therefore, recent LNL methods face the risk of misclassifying those "informative" samples (e.g., hard samples or samples in the tail subpopulations) into noisy samples, leading to poor generalization performance. To address the above issue, we propose a novel LNL method to simultaneously deal with noisy labels and imbalanced subpopulations. It first leverages sample correlation to estimate samples' clean probabilities for label correction and then utilizes corrected labels for Distributionally Robust Optimization (DRO) to further improve the robustness. Specifically, in contrast to previous works using classification loss as the selection criterion, we introduce a feature-based metric that takes the sample correlation into account for estimating samples' clean probabilities. Then, we refurbish the noisy labels using the estimated clean probabilities and the pseudo-labels from the model's predictions. With refurbished labels, we use DRO to train the model to be robust to subpopulation imbalance. Extensive experiments on a wide range of benchmarks demonstrate that our technique can consistently improve current state-of-the-art robust learning paradigms against noisy labels, especially when encountering imbalanced subpopulations.