Learning to Detect Noisy Labels Using Model-Based Features

TL;DR

This paper introduces SENT, a data-driven method that uses model-based features to effectively identify and handle noisy labels, improving learning performance across various tasks without relying on meta learning.

Contribution

SENT offers a flexible, non-meta learning approach to detect noisy labels by transferring noise distribution and training a model with model-based features.

Findings

SENT outperforms strong baselines in text classification.

SENT improves results in speech recognition tasks.

Effective in self-training and label corruption scenarios.

Abstract

Label noise is ubiquitous in various machine learning scenarios such as self-labeling with model predictions and erroneous data annotation. Many existing approaches are based on heuristics such as sample losses, which might not be flexible enough to achieve optimal solutions. Meta learning based methods address this issue by learning a data selection function, but can be hard to optimize. In light of these pros and cons, we propose Selection-Enhanced Noisy label Training (SENT) that does not rely on meta learning while having the flexibility of being data-driven. SENT transfers the noise distribution to a clean set and trains a model to distinguish noisy labels from clean ones using model-based features. Empirically, on a wide range of tasks including text classification and speech recognition, SENT improves performance over strong baselines under the settings of self-training and label corruption.