An M-Estimator for Reduced-Rank High-Dimensional Linear Dynamical System Identification

TL;DR

This paper introduces MR. SID, an efficient and stable M-estimator for high-dimensional reduced-rank linear dynamical systems, enabling improved parameter estimation and prediction in complex time-series data.

Contribution

It proposes a novel M-estimator combining low-rank approximations and regularization techniques, generalizing the Kalman Filter-Smoother for high-dimensional system identification.

Findings

Effective in estimating spatial filters and connectivity graphs from fMRI data

Demonstrates computational efficiency and numerical stability

Validated through simulations and real-world examples

Abstract

High-dimensional time-series data are becoming increasingly abundant across a wide variety of domains, spanning economics, neuroscience, particle physics, and cosmology. Fitting statistical models to such data, to enable parameter estimation and time-series prediction, is an important computational primitive. Existing methods, however, are unable to cope with the high-dimensional nature of these problems, due to both computational and statistical reasons. We mitigate both kinds of issues via proposing an M-estimator for Reduced-rank System IDentification (MR. SID). A combination of low-rank approximations, L-1 and L-2 penalties, and some numerical linear algebra tricks, yields an estimator that is computationally efficient and numerically stable. Simulations and real data examples demonstrate the utility of this approach in a variety of problems. In particular, we demonstrate that MR. SID can estimate spatial filters, connectivity graphs, and time-courses from native resolution functional magnetic resonance imaging data. Other applications and extensions are immediately available, as our approach is a generalization of the classical Kalman Filter-Smoother Expectation-Maximization algorithm.