Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection

TL;DR

This paper introduces CutSwap, a saliency-guided augmentation method for self-supervised image anomaly detection that creates realistic negative samples by swapping semantically similar patches, leading to improved detection performance.

Contribution

The paper proposes a novel saliency-guided augmentation technique, CutSwap, that enhances self-supervised learning for anomaly detection by generating more realistic negative samples.

Findings

Achieves state-of-the-art results on benchmark datasets.

Outperforms existing augmentation methods in anomaly detection.

Demonstrates effectiveness of semantic-aware negative sample generation.

Abstract

Anomaly detection (AD) is a fundamental task in computer vision. It aims to identify incorrect image data patterns which deviate from the normal ones. Conventional methods generally address AD by preparing augmented negative samples to enforce self-supervised learning. However, these techniques typically do not consider semantics during augmentation, leading to the generation of unrealistic or invalid negative samples. Consequently, the feature extraction network can be hindered from embedding critical features. In this study, inspired by visual attention learning approaches, we propose CutSwap, which leverages saliency guidance to incorporate semantic cues for augmentation. Specifically, we first employ LayerCAM to extract multilevel image features as saliency maps and then perform clustering to obtain multiple centroids. To fully exploit saliency guidance, on each map, we select a pixel pair from the cluster with the highest centroid saliency to form a patch pair. Such a patch pair includes highly similar context information with dense semantic correlations. The resulting negative sample is created by swapping the locations of the patch pair. Compared to prior augmentation methods, CutSwap generates more subtle yet realistic negative samples to facilitate quality feature learning. Extensive experimental and ablative evaluations demonstrate that our method achieves state-of-the-art AD performance on two mainstream AD benchmark datasets.