Learning Mixtures of Linear Dynamical Systems

TL;DR

This paper introduces a new algorithm for learning mixtures of linear dynamical systems from unlabeled short trajectories, providing theoretical guarantees and demonstrating effectiveness through experiments.

Contribution

The paper proposes a novel two-stage meta-algorithm with performance guarantees for learning LDS mixtures from unlabeled data, addressing key technical challenges.

Findings

Algorithm recovers LDS models with error $ ilde{O}(rac{ ext{sqrt}(d)}{T})$

Theoretical guarantees are validated through numerical experiments

Addresses challenges like latent variables and short trajectories

Abstract

We study the problem of learning a mixture of multiple linear dynamical systems (LDSs) from unlabeled short sample trajectories, each generated by one of the LDS models. Despite the wide applicability of mixture models for time-series data, learning algorithms that come with end-to-end performance guarantees are largely absent from existing literature. There are multiple sources of technical challenges, including but not limited to (1) the presence of latent variables (i.e. the unknown labels of trajectories); (2) the possibility that the sample trajectories might have lengths much smaller than the dimension $d$ of the LDS models; and (3) the complicated temporal dependence inherent to time-series data. To tackle these challenges, we develop a two-stage meta-algorithm, which is guaranteed to efficiently recover each ground-truth LDS model up to error $\tilde{O}(\sqrt{d/T})$, where $T$ is the total sample size. We validate our theoretical studies with numerical experiments, confirming the efficacy of the proposed algorithm.