Holistic Uncertainty Estimation For Open-Set Recognition

TL;DR

This paper introduces HolUE, a Bayesian-based holistic uncertainty estimation method for open-set recognition that effectively accounts for both gallery ambiguity and embedding uncertainty, improving error detection across diverse datasets.

Contribution

HolUE is a novel uncertainty estimation approach that considers multiple ambiguity sources, advancing open-set recognition accuracy beyond existing sample quality-based methods.

Findings

HolUE outperforms alternative methods in identifying recognition errors.

It is effective on diverse datasets including IJB-C, VoxBlink, and a new whale/dolphin recognition protocol.

HolUE better captures uncertainty by modeling both gallery and embedding ambiguities.

Abstract

Accurate uncertainty estimation is a critical challenge in open-set recognition, where a probe biometric sample may belong to an unknown identity. It can be addressed through sample quality estimation via probabilistic embeddings. However, the low variance of probabilistic embedding only partly implies a low identification error probability: an embedding of a sample could be close to several classes in a gallery, thus yielding high uncertainty despite high sample quality. We propose HolUE - a holistic uncertainty estimation method based on a Bayesian probabilistic model; it is aware of two sources of ambiguity in the open-set recognition system: (1) the gallery uncertainty caused by overlapping classes and (2) the uncertainty of embeddings. Challenging open-set recognition datasets, such as IJB-C for the image domain and VoxBlink for the audio domain, serve as a testbed for our method. We also provide a new open-set recognition protocol for the identification of whales and dolphins. In all cases, HolUE better identifies recognition errors than alternative uncertainty estimation methods, including those based solely on sample quality.