Robust Temporal Ensembling for Learning with Noisy Labels

TL;DR

This paper introduces Robust Temporal Ensembling (RTE), a method combining robust loss and semi-supervised regularization to effectively train deep neural networks on datasets with noisy labels, achieving state-of-the-art results.

Contribution

The paper proposes RTE, a novel approach that improves noise-robust learning without label filtering, outperforming existing methods on multiple datasets.

Findings

RTE achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N.

RTE maintains competitive robustness to input noise, with a mean corruption error of 13.50% on CIFAR-10-C.

RTE outperforms standard methods even with 80% label noise.

Abstract

Successful training of deep neural networks with noisy labels is an essential capability as most real-world datasets contain some amount of mislabeled data. Left unmitigated, label noise can sharply degrade typical supervised learning approaches. In this paper, we present robust temporal ensembling (RTE), which combines robust loss with semi-supervised regularization methods to achieve noise-robust learning. We demonstrate that RTE achieves state-of-the-art performance across the CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N datasets, while forgoing the recent trend of label filtering and/or fixing. Finally, we show that RTE also retains competitive corruption robustness to unforeseen input noise using CIFAR-10-C, obtaining a mean corruption error (mCE) of 13.50% even in the presence of an 80% noise ratio, versus 26.9% mCE with standard methods on clean data.