Probabilistic Label Spreading: Efficient and Consistent Estimation of Soft Labels with Epistemic Uncertainty on Graphs

TL;DR

This paper introduces a probabilistic label spreading technique that efficiently estimates label uncertainties on graphs, reducing annotation costs while maintaining high label quality for perception tasks.

Contribution

It proposes a scalable graph-based diffusion method for reliable uncertainty estimation, proven to be consistent even with minimal annotations per data point.

Findings

Reduces annotation effort for high-quality labels

Achieves state-of-the-art results on image classification benchmarks

Provides reliable aleatoric and epistemic uncertainty estimates

Abstract

Safe artificial intelligence for perception tasks remains a major challenge, partly due to the lack of data with high-quality labels. Annotations themselves are subject to aleatoric and epistemic uncertainty, which is typically ignored during annotation and evaluation. While crowdsourcing enables collecting multiple annotations per image to estimate these uncertainties, this approach is impractical at scale due to the required annotation effort. We introduce a probabilistic label spreading method that provides reliable estimates of aleatoric and epistemic uncertainty of labels. Assuming label smoothness over the feature space, we propagate single annotations using a graph-based diffusion method. We prove that label spreading yields consistent probability estimators even when the number of annotations per data point converges to zero. We present and analyze a scalable implementation of our method. Experimental results indicate that, compared to baselines, our approach substantially reduces the annotation budget required to achieve a desired label quality on common image datasets and achieves a new state of the art on the Data-Centric Image Classification benchmark.