Learning from Multiple Annotator Noisy Labels via Sample-wise Label Fusion

TL;DR

This paper introduces a novel learning algorithm that effectively handles multiple noisy labels per data sample by modeling annotator and sample-dependent errors, outperforming existing methods on standard datasets.

Contribution

It proposes a new approach that accounts for both annotator and data sample-dependent label errors, improving learning from noisy labels.

Findings

Outperforms state-of-the-art methods on MNIST, CIFAR-100, and ImageNet-100.

Demonstrates the importance of modeling sample-dependent annotator errors.

Provides a scalable solution for learning from noisy multi-annotator labels.

Abstract

Data lies at the core of modern deep learning. The impressive performance of supervised learning is built upon a base of massive accurately labeled data. However, in some real-world applications, accurate labeling might not be viable; instead, multiple noisy labels (instead of one accurate label) are provided by several annotators for each data sample. Learning a classifier on such a noisy training dataset is a challenging task. Previous approaches usually assume that all data samples share the same set of parameters related to annotator errors, while we demonstrate that label error learning should be both annotator and data sample dependent. Motivated by this observation, we propose a novel learning algorithm. The proposed method displays superiority compared with several state-of-the-art baseline methods on MNIST, CIFAR-100, and ImageNet-100. Our code is available at: https://github.com/zhengqigao/Learning-from-Multiple-Annotator-Noisy-Labels.