Approximating Instance-Dependent Noise via Instance-Confidence Embedding

TL;DR

This paper introduces the instance-confidence embedding (ICE) method, which approximates instance-dependent label noise in classification tasks by using a trainable embedding and a confidence scalar, improving noise robustness and mislabeled data detection.

Contribution

The paper proposes a novel variational approximation for instance-dependent noise using a trainable embedding and a confidence scalar, enhancing label noise modeling in classification.

Findings

ICE performs well under label noise conditions.

ICE effectively detects ambiguous or mislabeled instances.

The method improves robustness in image and text classification tasks.

Abstract

Label noise in multiclass classification is a major obstacle to the deployment of learning systems. However, unlike the widely used class-conditional noise (CCN) assumption that the noisy label is independent of the input feature given the true label, label noise in real-world datasets can be aleatory and heavily dependent on individual instances. In this work, we investigate the instance-dependent noise (IDN) model and propose an efficient approximation of IDN to capture the instance-specific label corruption. Concretely, noting the fact that most columns of the IDN transition matrix have only limited influence on the class-posterior estimation, we propose a variational approximation that uses a single-scalar confidence parameter. To cope with the situation where the mapping from the instance to its confidence value could vary significantly for two adjacent instances, we suggest using instance embedding that assigns a trainable parameter to each instance. The resulting instance-confidence embedding (ICE) method not only performs well under label noise but also can effectively detect ambiguous or mislabeled instances. We validate its utility on various image and text classification tasks.