Meta Transition Adaptation for Robust Deep Learning with Noisy Labels

TL;DR

This paper introduces a meta transition adaptation method that leverages a small set of clean data to improve robustness in deep learning with noisy labels, avoiding anchor point assumptions and providing statistical guarantees.

Contribution

The proposed meta transition learning strategy jointly estimates transition matrices and classifiers without anchor points, enhancing robustness against label noise.

Findings

More accurate transition matrix estimation

Robust performance under high noise levels

Effective even with no label noise

Abstract

To discover intrinsic inter-class transition probabilities underlying data, learning with noise transition has become an important approach for robust deep learning on corrupted labels. Prior methods attempt to achieve such transition knowledge by pre-assuming strongly confident anchor points with 1-probability belonging to a specific class, generally infeasible in practice, or directly jointly estimating the transition matrix and learning the classifier from the noisy samples, always leading to inaccurate estimation misguided by wrong annotation information especially in large noise cases. To alleviate these issues, this study proposes a new meta-transition-learning strategy for the task. Specifically, through the sound guidance of a small set of meta data with clean labels, the noise transition matrix and the classifier parameters can be mutually ameliorated to avoid being trapped by noisy training samples, and without need of any anchor point assumptions. Besides, we prove our method is with statistical consistency guarantee on correctly estimating the desired transition matrix. Extensive synthetic and real experiments validate that our method can more accurately extract the transition matrix, naturally following its more robust performance than prior arts. Its essential relationship with label distribution learning is also discussed, which explains its fine performance even under no-noise scenarios.