Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy

TL;DR

The paper introduces the Anomaly Transformer, a novel model leveraging association discrepancy via self-attention to improve unsupervised time series anomaly detection, achieving state-of-the-art results across multiple benchmarks.

Contribution

It proposes the Anomaly Transformer with Anomaly-Attention mechanism and a minimax strategy to enhance anomaly detection by modeling association discrepancy.

Findings

Achieves state-of-the-art results on six benchmarks

Effectively distinguishes anomalies through association discrepancy

Demonstrates robustness across diverse application domains

Abstract

Unsupervised detection of anomaly points in time series is a challenging problem, which requires the model to derive a distinguishable criterion. Previous methods tackle the problem mainly through learning pointwise representation or pairwise association, however, neither is sufficient to reason about the intricate dynamics. Recently, Transformers have shown great power in unified modeling of pointwise representation and pairwise association, and we find that the self-attention weight distribution of each time point can embody rich association with the whole series. Our key observation is that due to the rarity of anomalies, it is extremely difficult to build nontrivial associations from abnormal points to the whole series, thereby, the anomalies' associations shall mainly concentrate on their adjacent time points. This adjacent-concentration bias implies an association-based criterion inherently distinguishable between normal and abnormal points, which we highlight through the \emph{Association Discrepancy}. Technically, we propose the \emph{Anomaly Transformer} with a new \emph{Anomaly-Attention} mechanism to compute the association discrepancy. A minimax strategy is devised to amplify the normal-abnormal distinguishability of the association discrepancy. The Anomaly Transformer achieves state-of-the-art results on six unsupervised time series anomaly detection benchmarks of three applications: service monitoring, space & earth exploration, and water treatment.