SupEuclid: Extremely Simple, High Quality OoD Detection with Supervised Contrastive Learning and Euclidean Distance

TL;DR

SupEuclid shows that a simple ResNet18 trained with supervised contrastive learning and using Euclidean distance can achieve state-of-the-art out-of-distribution detection results without complex models or extensive tuning.

Contribution

Demonstrates that a straightforward supervised contrastive learning approach with Euclidean distance outperforms complex OoD detection methods.

Findings

Achieves state-of-the-art OoD detection with simple ResNet18 and Euclidean distance.

Outperforms more complex methods on standard benchmarks.

Requires no pretraining or extensive hyperparameter tuning.

Abstract

Out-of-Distribution (OoD) detection has developed substantially in the past few years, with available methods approaching, and in a few cases achieving, perfect data separation on standard benchmarks. These results generally involve large or complex models, pretraining, exposure to OoD examples or extra hyperparameter tuning. Remarkably, it is possible to achieve results that can exceed many of these state-of-the-art methods with a very simple method. We demonstrate that ResNet18 trained with Supervised Contrastive Learning (SCL) produces state-of-the-art results out-of-the-box on near and far OoD detection benchmarks using only Euclidean distance as a scoring rule. This may obviate the need in some cases for more sophisticated methods or larger models, and at the very least provides a very strong, easy to use baseline for further experimentation and analysis.