Time Series Clustering with General State Space Models via Stochastic Variational Inference

TL;DR

This paper introduces a novel model-based time series clustering method using mixtures of general state space models estimated via stochastic variational inference, improving accuracy and interpretability.

Contribution

It is the first computationally feasible approach for clustering time series with nonlinear, non-Gaussian state space models using neural networks and variational inference.

Findings

Effective clustering and parameter estimation demonstrated on simulated data.

Accurate estimation of the number of clusters using BIC.

Enhanced interpretability of model parameters.

Abstract

In this paper, we propose a novel method of model-based time series clustering with mixtures of general state space models (MSSMs). Each component of MSSMs is associated with each cluster. An advantage of the proposed method is that it enables the use of time series models appropriate to the specific time series. This not only improves clustering and prediction accuracy but also enhances the interpretability of the estimated parameters. The parameters of the MSSMs are estimated using stochastic variational inference, a subtype of variational inference. The proposed method estimates the latent variables of an arbitrary state space model by using neural networks with a normalizing flow as a variational estimator. The number of clusters can be estimated using the Bayesian information criterion. In addition, to prevent MSSMs from converging to the local optimum, we propose several optimization tricks, including an additional penalty term called entropy annealing. To our best knowledge, the proposed method is the first computationally feasible one for time series clustering based on general (possibly nonlinear, non-Gaussian) state space models. Experiments on simulated datasets show that the proposed method is effective for clustering, parameter estimation, and estimating the number of clusters.