Probabilistic Contrastive Learning with Explicit Concentration on the Hypersphere

TL;DR

This paper proposes a probabilistic contrastive learning framework on the hypersphere using von Mises-Fisher distribution, explicitly modeling uncertainty and improving out-of-distribution detection and failure analysis.

Contribution

It introduces an unnormalized vMF-based contrastive learning method with a concentration parameter for explicit uncertainty quantification.

Findings

Concentration parameter correlates with data corruption levels.

Enhances out-of-distribution detection accuracy.

Enables failure analysis through uncertainty estimation.

Abstract

Self-supervised contrastive learning has predominantly adopted deterministic methods, which are not suited for environments characterized by uncertainty and noise. This paper introduces a new perspective on incorporating uncertainty into contrastive learning by embedding representations within a spherical space, inspired by the von Mises-Fisher distribution (vMF). We introduce an unnormalized form of vMF and leverage the concentration parameter, kappa, as a direct, interpretable measure to quantify uncertainty explicitly. This approach not only provides a probabilistic interpretation of the embedding space but also offers a method to calibrate model confidence against varying levels of data corruption and characteristics. Our empirical results demonstrate that the estimated concentration parameter correlates strongly with the degree of unforeseen data corruption encountered at test time, enables failure analysis, and enhances existing out-of-distribution detection methods.