Temporal Calibrated Regularization for Robust Noisy Label Learning

TL;DR

This paper introduces Temporal Calibrated Regularization (TCR), a simple method that leverages previous epoch predictions and original labels to improve deep neural networks' robustness against noisy labels, with minimal computational overhead.

Contribution

The paper proposes a novel TCR method that enhances noisy label robustness by using temporal information from previous epochs, outperforming existing complex approaches.

Findings

TCR consistently improves robustness across various architectures and datasets.

TCR requires little additional computation compared to existing methods.

Experimental results show significant performance gains in noisy label scenarios.

Abstract

Deep neural networks (DNNs) exhibit great success on many tasks with the help of large-scale well annotated datasets. However, labeling large-scale data can be very costly and error-prone so that it is difficult to guarantee the annotation quality (i.e., having noisy labels). Training on these noisy labeled datasets may adversely deteriorate their generalization performance. Existing methods either rely on complex training stage division or bring too much computation for marginal performance improvement. In this paper, we propose a Temporal Calibrated Regularization (TCR), in which we utilize the original labels and the predictions in the previous epoch together to make DNN inherit the simple pattern it has learned with little overhead. We conduct extensive experiments on various neural network architectures and datasets, and find that it consistently enhances the robustness of DNNs to label noise.