RESTAD: REconstruction and Similarity based Transformer for time series Anomaly Detection

TL;DR

RESTAD is a novel Transformer-based model with RBF layers designed for unsupervised time series anomaly detection, combining similarity scores and reconstruction errors to improve detection sensitivity.

Contribution

Introduces RESTAD, integrating RBF neurons into Transformer architecture to enhance anomaly detection in time series by leveraging similarity measures.

Findings

RESTAD outperforms existing baselines on benchmark datasets.

The RBF layer improves sensitivity to subtle anomalies.

Combining similarity scores with reconstruction errors enhances detection accuracy.

Abstract

Anomaly detection in time series data is crucial across various domains. The scarcity of labeled data for such tasks has increased the attention towards unsupervised learning methods. These approaches, often relying solely on reconstruction error, typically fail to detect subtle anomalies in complex datasets. To address this, we introduce RESTAD, an adaptation of the Transformer model by incorporating a layer of Radial Basis Function (RBF) neurons within its architecture. This layer fits a non-parametric density in the latent representation, such that a high RBF output indicates similarity with predominantly normal training data. RESTAD integrates the RBF similarity scores with the reconstruction errors to increase sensitivity to anomalies. Our empirical evaluations demonstrate that RESTAD outperforms various established baselines across multiple benchmark datasets.