Towards Foundation Auto-Encoders for Time-Series Anomaly Detection

TL;DR

This paper proposes Foundation Auto-Encoders (FAE), a novel pretrained generative model based on VAEs and DCNNs, designed for accurate, zero-shot anomaly detection in diverse time-series datasets, inspired by foundation models.

Contribution

It introduces FAE, a new pretrained model for time-series anomaly detection that leverages large-scale training and complex temporal pattern learning.

Findings

Preliminary results show FAE effectively detects anomalies across multiple datasets.

FAE demonstrates potential for out-of-the-box, zero-shot anomaly detection.

Initial experiments indicate promising generalization to unseen datasets.

Abstract

We investigate a novel approach to time-series modeling, inspired by the successes of large pretrained foundation models. We introduce FAE (Foundation Auto-Encoders), a foundation generative-AI model for anomaly detection in time-series data, based on Variational Auto-Encoders (VAEs). By foundation, we mean a model pretrained on massive amounts of time-series data which can learn complex temporal patterns useful for accurate modeling, forecasting, and detection of anomalies on previously unseen datasets. FAE leverages VAEs and Dilated Convolutional Neural Networks (DCNNs) to build a generic model for univariate time-series modeling, which could eventually perform properly in out-of-the-box, zero-shot anomaly detection applications. We introduce the main concepts of FAE, and present preliminary results in different multi-dimensional time-series datasets from various domains, including a real dataset from an operational mobile ISP, and the well known KDD 2021 Anomaly Detection dataset.