Attention and Autoencoder Hybrid Model for Unsupervised Online Anomaly Detection

TL;DR

This paper presents a novel hybrid attention and autoencoder model for unsupervised online anomaly detection in time series, combining local pattern recognition with long-term feature learning for improved accuracy.

Contribution

It introduces the first hybrid attention-autoencoder architecture for time series anomaly detection, integrating transformer-like attention with autoencoder predictions.

Findings

Effective detection on diverse real-world datasets

Improved accuracy using statistical moment analysis

Outperforms existing models in benchmark tests

Abstract

This paper introduces a hybrid attention and autoencoder (AE) model for unsupervised online anomaly detection in time series. The autoencoder captures local structural patterns in short embeddings, while the attention model learns long-term features, facilitating parallel computing with positional encoding. Unique in its approach, our proposed hybrid model combines attention and autoencoder for the first time in time series anomaly detection. It employs an attention-based mechanism, akin to the deep transformer model, with key architectural modifications for predicting the next time step window in the autoencoder's latent space. The model utilizes a threshold from the validation dataset for anomaly detection and introduces an alternative method based on analyzing the first statistical moment of error, improving accuracy without dependence on a validation dataset. Evaluation on diverse real-world benchmark datasets and comparing with other well-established models, confirms the effectiveness of our proposed model in anomaly detection.