Variational Inference and Learning of Piecewise-linear Dynamical Systems

TL;DR

This paper introduces a variational inference approach for piecewise-linear dynamical systems, enabling efficient modeling of complex, multi-modal temporal data with switching behaviors, demonstrated on head-pose tracking.

Contribution

It develops a variational EM algorithm for intractable switching linear dynamical systems, allowing estimation of model parameters and number of modes.

Findings

Effective head-pose tracking demonstrated

Model handles multiple behavioral modes

Outperforms several state-of-the-art trackers

Abstract

Modeling the temporal behavior of data is of primordial importance in many scientific and engineering fields. Baseline methods assume that both the dynamic and observation equations follow linear-Gaussian models. However, there are many real-world processes that cannot be characterized by a single linear behavior. Alternatively, it is possible to consider a piecewise-linear model which, combined with a switching mechanism, is well suited when several modes of behavior are needed. Nevertheless, switching dynamical systems are intractable because of their computational complexity increases exponentially with time. In this paper, we propose a variational approximation of piecewise linear dynamical systems. We provide full details of the derivation of two variational expectation-maximization algorithms, a filter and a smoother. We show that the model parameters can be split into two sets, static and dynamic parameters, and that the former parameters can be estimated off-line together with the number of linear modes, or the number of states of the switching variable. We apply the proposed method to a visual tracking problem, namely head-pose tracking, and we thoroughly compare our algorithm with several state of the art trackers.