Is it all a cluster game? -- Exploring Out-of-Distribution Detection based on Clustering in the Embedding Space

TL;DR

This paper investigates out-of-distribution detection for image classification by analyzing clustering structures in embedding spaces, comparing various training and clustering strategies, and highlighting the dependence on model architecture and datasets.

Contribution

It provides a comprehensive analysis of clustering-based OOD detection methods, comparing contrastive and cross-entropy training, and evaluates their effectiveness across different datasets and models.

Findings

Supervised contrastive learning yields well-separated clusters.

No single clustering approach is universally best; effectiveness varies with architecture and dataset.

Cross-entropy with cosine similarity often outperforms contrastive methods on certain datasets.

Abstract

It is essential for safety-critical applications of deep neural networks to determine when new inputs are significantly different from the training distribution. In this paper, we explore this out-of-distribution (OOD) detection problem for image classification using clusters of semantically similar embeddings of the training data and exploit the differences in distance relationships to these clusters between in- and out-of-distribution data. We study the structure and separation of clusters in the embedding space and find that supervised contrastive learning leads to well-separated clusters while its self-supervised counterpart fails to do so. In our extensive analysis of different training methods, clustering strategies, distance metrics, and thresholding approaches, we observe that there is no clear winner. The optimal approach depends on the model architecture and selected datasets for in- and out-of-distribution. While we could reproduce the outstanding results for contrastive training on CIFAR-10 as in-distribution data, we find standard cross-entropy paired with cosine similarity outperforms all contrastive training methods when training on CIFAR-100 instead. Cross-entropy provides competitive results as compared to expensive contrastive training methods.