Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension

TL;DR

Annot-Mix introduces a novel mixup extension that effectively learns from noisy, multi-annotator labels, improving neural network robustness and outperforming existing methods across multiple datasets.

Contribution

It presents a new mixup-based framework that explicitly models multiple annotator labels and their sources, enhancing learning from noisy annotations.

Findings

Outperforms eight state-of-the-art methods on eleven datasets.

Effectively handles noisy labels from both human and simulated annotators.

Improves neural network generalization in noisy label scenarios.

Abstract

Training with noisy class labels impairs neural networks' generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that, typically, multiple annotators, e.g., crowdworkers, provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eight state-of-the-art approaches on eleven datasets with noisy class labels provided either by human or simulated annotators. Our code is publicly available through our repository at https://github.com/ies-research/annot-mix.