No-Regret Prediction in Marginally Stable Systems

TL;DR

This paper develops a novel analysis for online prediction in marginally stable linear systems, achieving sublinear regret bounds despite unidentifiability and polynomial state growth, with applications to autoregressive filtering.

Contribution

It introduces a refined regret analysis for marginally stable systems, enabling sublinear regret bounds for online least-squares and autoregressive filtering under challenging conditions.

Findings

Online least-squares achieves sublinear regret in marginally stable systems.

Logarithmic regret is achievable in partially observed autoregressive models.

The analysis handles polynomial state growth despite unidentifiability.

Abstract

We consider the problem of online prediction in a marginally stable linear dynamical system subject to bounded adversarial or (non-isotropic) stochastic perturbations. This poses two challenges. Firstly, the system is in general unidentifiable, so recent and classical results on parameter recovery do not apply. Secondly, because we allow the system to be marginally stable, the state can grow polynomially with time; this causes standard regret bounds in online convex optimization to be vacuous. In spite of these challenges, we show that the online least-squares algorithm achieves sublinear regret (improvable to polylogarithmic in the stochastic setting), with polynomial dependence on the system's parameters. This requires a refined regret analysis, including a structural lemma showing the current state of the system to be a small linear combination of past states, even if the state grows polynomially. By applying our techniques to learning an autoregressive filter, we also achieve logarithmic regret in the partially observed setting under Gaussian noise, with polynomial dependence on the memory of the associated Kalman filter.