Understanding normalization in contrastive representation learning and out-of-distribution detection

TL;DR

This paper investigates the role of the $ ext{l}_2$-norm of contrastive features in out-of-distribution detection, proposing a flexible method that leverages out-of-distribution data to improve anomaly detection across diverse datasets.

Contribution

It introduces a simple, adaptable contrastive learning-based approach that incorporates out-of-distribution data, enhancing anomaly detection especially in challenging datasets like aerial and microscopy images.

Findings

Method outperforms existing approaches in various scenarios

Incorporating out-of-distribution data improves detection accuracy

High-quality contrastive features boost out-of-distribution detection performance

Abstract

Contrastive representation learning has emerged as an outstanding approach for anomaly detection. In this work, we explore the $\ell_2$-norm of contrastive features and its applications in out-of-distribution detection. We propose a simple method based on contrastive learning, which incorporates out-of-distribution data by discriminating against normal samples in the contrastive layer space. Our approach can be applied flexibly as an outlier exposure (OE) approach, where the out-of-distribution data is a huge collective of random images, or as a fully self-supervised learning approach, where the out-of-distribution data is self-generated by applying distribution-shifting transformations. The ability to incorporate additional out-of-distribution samples enables a feasible solution for datasets where AD methods based on contrastive learning generally underperform, such as aerial images or microscopy images. Furthermore, the high-quality features learned through contrastive learning consistently enhance performance in OE scenarios, even when the available out-of-distribution dataset is not diverse enough. Our extensive experiments demonstrate the superiority of our proposed method under various scenarios, including unimodal and multimodal settings, with various image datasets.