Focus the Discrepancy: Intra- and Inter-Correlation Learning for Image Anomaly Detection

TL;DR

This paper introduces FOD, a novel anomaly detection framework using transformer-based intra- and inter-correlation learning to effectively identify anomalies by modeling patch discrepancies and correlations.

Contribution

The paper proposes a transformer renovation with I2Correlation for simultaneous intra- and inter-discrepancy learning, enhancing anomaly detection performance.

Findings

Outperforms existing methods on three benchmarks.

Effectively models patch-wise discrepancies and correlations.

Achieves superior anomaly detection accuracy.

Abstract

Humans recognize anomalies through two aspects: larger patch-wise representation discrepancies and weaker patch-to-normal-patch correlations. However, the previous AD methods didn't sufficiently combine the two complementary aspects to design AD models. To this end, we find that Transformer can ideally satisfy the two aspects as its great power in the unified modeling of patch-wise representations and patch-to-patch correlations. In this paper, we propose a novel AD framework: FOcus-the-Discrepancy (FOD), which can simultaneously spot the patch-wise, intra- and inter-discrepancies of anomalies. The major characteristic of our method is that we renovate the self-attention maps in transformers to Intra-Inter-Correlation (I2Correlation). The I2Correlation contains a two-branch structure to first explicitly establish intra- and inter-image correlations, and then fuses the features of two-branch to spotlight the abnormal patterns. To learn the intra- and inter-correlations adaptively, we propose the RBF-kernel-based target-correlations as learning targets for self-supervised learning. Besides, we introduce an entropy constraint strategy to solve the mode collapse issue in optimization and further amplify the normal-abnormal distinguishability. Extensive experiments on three unsupervised real-world AD benchmarks show the superior performance of our approach. Code will be available at https://github.com/xcyao00/FOD.