A Gift from Label Smoothing: Robust Training with Adaptive Label Smoothing via Auxiliary Classifier under Label Noise

TL;DR

This paper introduces ALASCA, a method that enhances robustness of deep neural networks against noisy labels by combining adaptive label smoothing and auxiliary classifiers, achieving improved performance efficiently.

Contribution

ALASCA is a novel approach that implicitly induces Lipschitz regularization through adaptive label smoothing and uses auxiliary classifiers for practical, efficient robust training under label noise.

Findings

ALASCA improves robustness of feature extractors in noisy label scenarios.

Combining ALASCA with existing methods enhances overall performance.

Experimental results demonstrate efficiency and effectiveness across multiple datasets.

Abstract

As deep neural networks can easily overfit noisy labels, robust training in the presence of noisy labels is becoming an important challenge in modern deep learning. While existing methods address this problem in various directions, they still produce unpredictable sub-optimal results since they rely on the posterior information estimated by the feature extractor corrupted by noisy labels. Lipschitz regularization successfully alleviates this problem by training a robust feature extractor, but it requires longer training time and expensive computations. Motivated by this, we propose a simple yet effective method, called ALASCA, which efficiently provides a robust feature extractor under label noise. ALASCA integrates two key ingredients: (1) adaptive label smoothing based on our theoretical analysis that label smoothing implicitly induces Lipschitz regularization, and (2) auxiliary classifiers that enable practical application of intermediate Lipschitz regularization with negligible computations. We conduct wide-ranging experiments for ALASCA and combine our proposed method with previous noise-robust methods on several synthetic and real-world datasets. Experimental results show that our framework consistently improves the robustness of feature extractors and the performance of existing baselines with efficiency. Our code is available at https://github.com/jongwooko/ALASCA.