Dimensionality-Driven Learning with Noisy Labels

TL;DR

This paper introduces a dimensionality-based approach to improve deep neural network training on datasets with noisy labels by monitoring and adapting the learning process based on the subspace dimensionality.

Contribution

The paper presents a novel dimensionality-driven learning strategy that enhances DNN robustness to noisy labels by dynamically adjusting training based on subspace dimensionality.

Findings

The approach is highly tolerant to large proportions of noisy labels.

It effectively learns low-dimensional subspaces representing data distribution.

The method improves generalization in noisy label scenarios.

Abstract

Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such datasets, by investigating the dimensionality of the deep representation subspace of training samples. We show that from a dimensionality perspective, DNNs exhibit quite distinctive learning styles when trained with clean labels versus when trained with a proportion of noisy labels. Based on this finding, we develop a new dimensionality-driven learning strategy, which monitors the dimensionality of subspaces during training and adapts the loss function accordingly. We empirically demonstrate that our approach is highly tolerant to significant proportions of noisy labels, and can effectively learn low-dimensional local subspaces that capture the data distribution.