Concrete Dense Network for Long-Sequence Time Series Clustering

TL;DR

LoSTer is a novel dense autoencoder architecture designed for efficient and accurate clustering of long time series, overcoming limitations of RNNs and Transformers by enabling direct optimization of k-means with Gumbel-softmax.

Contribution

The paper introduces LoSTer, a dense autoencoder that directly optimizes k-means for long time series clustering using Gumbel-softmax, improving speed and accuracy.

Findings

Outperforms state-of-the-art RNN and Transformer methods

Effective on multiple benchmark datasets

Proves fast and accurate clustering of long sequences

Abstract

Time series clustering is fundamental in data analysis for discovering temporal patterns. Despite recent advancements, learning cluster-friendly representations is still challenging, particularly with long and complex time series. Deep temporal clustering methods have been trying to integrate the canonical k-means into end-to-end training of neural networks but fall back on surrogate losses due to the non-differentiability of the hard cluster assignment, yielding sub-optimal solutions. In addition, the autoregressive strategy used in the state-of-the-art RNNs is subject to error accumulation and slow training, while recent research findings have revealed that Transformers are less effective due to time points lacking semantic meaning, to the permutation invariance of attention that discards the chronological order and high computation cost. In light of these observations, we present LoSTer which is a novel dense autoencoder architecture for the long-sequence time series clustering problem (LSTC) capable of optimizing the k-means objective via the Gumbel-softmax reparameterization trick and designed specifically for accurate and fast clustering of long time series. Extensive experiments on numerous benchmark datasets and two real-world applications prove the effectiveness of LoSTer over state-of-the-art RNNs and Transformer-based deep clustering methods.