Evaluation of Out-of-Distribution Detection Performance of Self-Supervised Learning in a Controllable Environment

TL;DR

This paper introduces a new framework for evaluating out-of-distribution detection in self-supervised learning, demonstrating consistent performance improvements across various data types and detection algorithms.

Contribution

The paper proposes a novel evaluation framework that adjusts OOD sample distances, providing a more comprehensive assessment of SSL methods' detection capabilities.

Findings

SSL methods outperform baselines in OOD detection across all settings

The framework effectively compares different OOD detection algorithms

Performance gains are consistent across simulated, image, and text data

Abstract

We evaluate the out-of-distribution (OOD) detection performance of self-supervised learning (SSL) techniques with a new evaluation framework. Unlike the previous evaluation methods, the proposed framework adjusts the distance of OOD samples from the in-distribution samples. We evaluate an extensive combination of OOD detection algorithms on three different implementations of the proposed framework using simulated samples, images, and text. SSL methods consistently demonstrated the improved OOD detection performance in all evaluation settings.