Learning to Forget with Information Divergence Reweighted Objectives for Noisy Labels

TL;DR

ANTIDOTE introduces a novel adversarial training-based objective that adaptively reduces the influence of noisy labels, effectively enabling models to forget corrupted data during training, and outperforms existing methods across various noise scenarios.

Contribution

The paper presents ANTIDOTE, a new class of objectives for noisy label learning, reformulated via convex duality as an efficient adversarial training method that reduces label noise impact.

Findings

Outperforms leading loss functions on noisy datasets

Maintains computational cost similar to standard cross-entropy

Effectively reduces influence of noisy labels during training

Abstract

We introduce ANTIDOTE, a new class of objectives for learning under noisy labels which are defined in terms of a relaxation over an information-divergence neighborhood. Using convex duality, we provide a reformulation as an adversarial training method that has similar computational cost to training with standard cross-entropy loss. We show that our approach adaptively reduces the influence of the samples with noisy labels during learning, exhibiting a behavior that is analogous to forgetting those samples. ANTIDOTE is effective in practical environments where label noise is inherent in the training data or where an adversary can alter the training labels. Extensive empirical evaluations on different levels of symmetric, asymmetric, human annotation, and real-world label noise show that ANTIDOTE outperforms leading comparable losses in the field and enjoys a time complexity that is very close to that of the standard cross entropy loss.