Lossy Compression for Robust Unsupervised Time-Series Anomaly Detection

TL;DR

This paper introduces a lossy compression-based neural network autoencoder for unsupervised time-series anomaly detection, capable of robustly identifying anomalies even with some anomalous data in training.

Contribution

It proposes a novel rate-distortion loss framework that learns robust representations tolerant to anomalies, contrasting with traditional methods requiring purely normal training data.

Findings

High accuracy in anomaly detection using distortion scores.

Robust representations tolerate some anomalous training data.

Outperforms traditional models trained only on normal data.

Abstract

A new Lossy Causal Temporal Convolutional Neural Network Autoencoder for anomaly detection is proposed in this work. Our framework uses a rate-distortion loss and an entropy bottleneck to learn a compressed latent representation for the task. The main idea of using a rate-distortion loss is to introduce representation flexibility that ignores or becomes robust to unlikely events with distinctive patterns, such as anomalies. These anomalies manifest as unique distortion features that can be accurately detected in testing conditions. This new architecture allows us to train a fully unsupervised model that has high accuracy in detecting anomalies from a distortion score despite being trained with some portion of unlabelled anomalous data. This setting is in stark contrast to many of the state-of-the-art unsupervised methodologies that require the model to be only trained on "normal data". We argue that this partially violates the concept of unsupervised training for anomaly detection as the model uses an informed decision that selects what is normal from abnormal for training. Additionally, there is evidence to suggest it also effects the models ability at generalisation. We demonstrate that models that succeed in the paradigm where they are only trained on normal data fail to be robust when anomalous data is injected into the training. In contrast, our compression-based approach converges to a robust representation that tolerates some anomalous distortion. The robust representation achieved by a model using a rate-distortion loss can be used in a more realistic unsupervised anomaly detection scheme.