Unsupervised Data Uncertainty Learning in Visual Retrieval Systems

TL;DR

This paper presents an unsupervised method to estimate data uncertainty in visual retrieval systems, improving interpretability, robustness, and noise detection in image and video retrieval tasks.

Contribution

It introduces an extension to triplet loss that models heteroscedastic uncertainty, enhancing retrieval performance and interpretability without requiring labeled uncertainty data.

Findings

Improves retrieval accuracy by modeling data uncertainty.

Effectively identifies noisy and confusing data in real-world datasets.

Enables data cleaning through uncertainty estimation.

Abstract

We introduce an unsupervised formulation to estimate heteroscedastic uncertainty in retrieval systems. We propose an extension to triplet loss that models data uncertainty for each input. Besides improving performance, our formulation models local noise in the embedding space. It quantifies input uncertainty and thus enhances interpretability of the system. This helps identify noisy observations in query and search databases. Evaluation on both image and video retrieval applications highlight the utility of our approach. We highlight our efficiency in modeling local noise using two real-world datasets: Clothing1M and Honda Driving datasets. Qualitative results illustrate our ability in identifying confusing scenarios in various domains. Uncertainty learning also enables data cleaning by detecting noisy training labels.