Towards the Identifiability in Noisy Label Learning: A Multinomial Mixture Modelling Approach

TL;DR

This paper introduces a multinomial mixture model approach that makes noisy label learning identifiable by generating additional labels through nearest neighbors, enabling accurate clean label estimation without heuristics.

Contribution

It demonstrates that noisy label learning becomes identifiable with at least 2C-1 i.i.d. noisy labels per instance and proposes a method to generate these labels automatically.

Findings

Accurately estimates clean labels across various datasets.

Performs competitively with state-of-the-art methods.

Requires no heuristics or manual annotations.

Abstract

Learning from noisy labels (LNL) is crucial in deep learning, in which one of the approaches is to identify clean-label samples from poorly-annotated datasets. Such an identification is challenging because the conventional LNL problem, which assumes only one noisy label per instance, is non-identifiable, i.e., clean labels cannot be estimated theoretically without additional heuristics. This paper presents a novel data-driven approach that addresses this issue without requiring any heuristics about clean samples. We discover that the LNL problem becomes identifiable if there are at least $2C - 1$ i.i.d. noisy labels per instance, where $C$ is the number of classes. Our finding relies on the assumption of i.i.d. noisy labels and multinomial mixture modelling, making it easier to interpret than previous studies that require full-rank noisy-label transition matrices. To fulfil this condition without additional manual annotations, we propose a method that automatically generates additional i.i.d. noisy labels through nearest neighbours. These noisy labels are then used in the Expectation-Maximisation algorithm to infer clean labels. Our method demonstrably estimates clean labels accurately across various label noise benchmarks, including synthetic, web-controlled, and real-world datasets. Furthermore, the model trained with our method performs competitively with many state-of-the-art methods.