Distribution-Aware Robust Learning from Long-Tailed Data with Noisy Labels

TL;DR

This paper introduces DaSC, a robust learning framework that improves classification accuracy on long-tailed, noisy datasets by using distribution-aware centroid estimation and confidence-based contrastive learning.

Contribution

The paper proposes a novel framework combining distribution-aware centroid estimation with confidence-aware contrastive learning for robust training on noisy, long-tailed data.

Findings

DaSC outperforms previous methods on CIFAR and real-world datasets.

Enhanced class centroids improve model robustness against noise.

Confidence-aware strategies effectively balance representation learning.

Abstract

Deep neural networks have demonstrated remarkable advancements in various fields using large, well-annotated datasets. However, real-world data often exhibit long-tailed distributions and label noise, significantly degrading generalization performance. Recent studies addressing these issues have focused on noisy sample selection methods that estimate the centroid of each class based on high-confidence samples within each target class. The performance of these methods is limited because they use only the training samples within each class for class centroid estimation, making the quality of centroids susceptible to long-tailed distributions and noisy labels. In this study, we present a robust training framework called Distribution-aware Sample Selection and Contrastive Learning (DaSC). Specifically, DaSC introduces a Distribution-aware Class Centroid Estimation (DaCC) to generate enhanced class centroids. DaCC performs weighted averaging of the features from all samples, with weights determined based on model predictions. Additionally, we propose a confidence-aware contrastive learning strategy to obtain balanced and robust representations. The training samples are categorized into high-confidence and low-confidence samples. Our method then applies Semi-supervised Balanced Contrastive Loss (SBCL) using high-confidence samples, leveraging reliable label information to mitigate class bias. For the low-confidence samples, our method computes Mixup-enhanced Instance Discrimination Loss (MIDL) to improve their representations in a self-supervised manner. Our experimental results on CIFAR and real-world noisy-label datasets demonstrate the superior performance of the proposed DaSC compared to previous approaches.