Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities

TL;DR

This paper reviews the use of self-supervised learning for unsupervised anomaly detection, emphasizing the importance of SSL strategies, hyperparameter tuning, and exploring future research directions including pretraining and data augmentation.

Contribution

It analyzes the impact of SSL strategies on anomaly detection performance and discusses recent advances in model selection, hyperparameter tuning, and future challenges.

Findings

SSL strategies significantly affect AD performance

Recent methods improve hyperparameter tuning for SSL in AD

Pretrained foundation models offer new opportunities for AD

Abstract

Self-supervised learning (SSL) is a growing torrent that has recently transformed machine learning and its many real world applications, by learning on massive amounts of unlabeled data via self-generated supervisory signals. Unsupervised anomaly detection (AD) has also capitalized on SSL, by self-generating pseudo-anomalies through various data augmentation functions or external data exposure. In this vision paper, we first underline the importance of the choice of SSL strategies on AD performance, by presenting evidences and studies from the AD literature. Equipped with the understanding that SSL incurs various hyperparameters (HPs) to carefully tune, we present recent developments on unsupervised model selection and augmentation tuning for SSL-based AD. We then highlight emerging challenges and future opportunities; on designing new pretext tasks and augmentation functions for different data modalities, creating novel model selection solutions for systematically tuning the SSL HPs, as well as on capitalizing on the potential of pretrained foundation models on AD through effective density estimation.