Bayesian Triplet Loss: Uncertainty Quantification in Image Retrieval

TL;DR

This paper introduces a Bayesian approach to image retrieval that models embeddings as stochastic features, providing well-calibrated uncertainty estimates while maintaining high retrieval accuracy.

Contribution

It proposes a novel likelihood and prior framework for uncertainty quantification in image embeddings, along with a variational approximation for efficient computation.

Findings

State-of-the-art uncertainty estimation in image retrieval

Maintains competitive retrieval performance

Provides well-calibrated uncertainty measures

Abstract

Uncertainty quantification in image retrieval is crucial for downstream decisions, yet it remains a challenging and largely unexplored problem. Current methods for estimating uncertainties are poorly calibrated, computationally expensive, or based on heuristics. We present a new method that views image embeddings as stochastic features rather than deterministic features. Our two main contributions are (1) a likelihood that matches the triplet constraint and that evaluates the probability of an anchor being closer to a positive than a negative; and (2) a prior over the feature space that justifies the conventional l2 normalization. To ensure computational efficiency, we derive a variational approximation of the posterior, called the Bayesian triplet loss, that produces state-of-the-art uncertainty estimates and matches the predictive performance of current state-of-the-art methods.