Self-Supervised Anomaly Detection by Self-Distillation and Negative Sampling

TL;DR

This paper introduces a self-distillation and negative sampling approach for unsupervised out-of-distribution detection, leveraging negative samples that alter high-level semantics while preserving low-level features, achieving state-of-the-art results.

Contribution

It proposes a novel self-supervised method combining self-distillation with negative sampling strategies for improved OOD detection without labels.

Findings

Negative sampling strategies significantly impact detection performance.

Leveraging negative samples that alter high-level semantics improves OOD detection.

The method sets new benchmarks across diverse visual OOD detection tasks.

Abstract

Detecting whether examples belong to a given in-distribution or are Out-Of-Distribution (OOD) requires identifying features specific to the in-distribution. In the absence of labels, these features can be learned by self-supervised techniques under the generic assumption that the most abstract features are those which are statistically most over-represented in comparison to other distributions from the same domain. In this work, we show that self-distillation of the in-distribution training set together with contrasting against negative examples derived from shifting transformation of auxiliary data strongly improves OOD detection. We find that this improvement depends on how the negative samples are generated. In particular, we observe that by leveraging negative samples, which keep the statistics of low-level features while changing the high-level semantics, higher average detection performance is obtained. Furthermore, good negative sampling strategies can be identified from the sensitivity of the OOD detection score. The efficiency of our approach is demonstrated across a diverse range of OOD detection problems, setting new benchmarks for unsupervised OOD detection in the visual domain.