Deep Learning with Label Noise: A Hierarchical Approach

TL;DR

This paper introduces a simple hierarchical method for training deep neural networks that effectively handles label noise without altering the network architecture or optimization process.

Contribution

It proposes a novel hierarchical approach that leverages label hierarchies to improve robustness against label noise in deep learning models.

Findings

Hierarchical approach outperforms standard DNNs on noisy datasets

Combining with pre-trained models achieves state-of-the-art results

Method is effective across simulated and real-world noisy data

Abstract

Deep neural networks are susceptible to label noise. Existing methods to improve robustness, such as meta-learning and regularization, usually require significant change to the network architecture or careful tuning of the optimization procedure. In this work, we propose a simple hierarchical approach that incorporates a label hierarchy when training the deep learning models. Our approach requires no change of the network architecture or the optimization procedure. We investigate our hierarchical network through a wide range of simulated and real datasets and various label noise types. Our hierarchical approach improves upon regular deep neural networks in learning with label noise. Combining our hierarchical approach with pre-trained models achieves state-of-the-art performance in real-world noisy datasets.