Sub-Adjacent Transformer: Improving Time Series Anomaly Detection with Reconstruction Error from Sub-Adjacent Neighborhoods

TL;DR

The paper introduces the Sub-Adjacent Transformer, a novel attention mechanism that enhances unsupervised time series anomaly detection by focusing on non-adjacent neighborhoods, leading to improved detection of rare anomalies.

Contribution

It proposes a new attention mechanism that restricts focus to sub-adjacent neighborhoods, improving anomaly detection by emphasizing differences from immediate surroundings.

Findings

Achieves state-of-the-art results on six real-world benchmarks.

Effectively detects anomalies in diverse fields like server monitoring and space exploration.

Enhances anomaly detection by focusing on non-adjacent regions in time series.

Abstract

In this paper, we present the Sub-Adjacent Transformer with a novel attention mechanism for unsupervised time series anomaly detection. Unlike previous approaches that rely on all the points within some neighborhood for time point reconstruction, our method restricts the attention to regions not immediately adjacent to the target points, termed sub-adjacent neighborhoods. Our key observation is that owing to the rarity of anomalies, they typically exhibit more pronounced differences from their sub-adjacent neighborhoods than from their immediate vicinities. By focusing the attention on the sub-adjacent areas, we make the reconstruction of anomalies more challenging, thereby enhancing their detectability. Technically, our approach concentrates attention on the non-diagonal areas of the attention matrix by enlarging the corresponding elements in the training stage. To facilitate the implementation of the desired attention matrix pattern, we adopt linear attention because of its flexibility and adaptability. Moreover, a learnable mapping function is proposed to improve the performance of linear attention. Empirically, the Sub-Adjacent Transformer achieves state-of-the-art performance across six real-world anomaly detection benchmarks, covering diverse fields such as server monitoring, space exploration, and water treatment.