The Power of Second Order Methods for Sequence Preconditioning

TL;DR

This paper demonstrates that combining universal sequence preconditioning with the Vovk-Azoury-Warmuth algorithm achieves near-optimal regret bounds for long-memory dynamical systems, even with complex spectra.

Contribution

It introduces a novel combination of USP and VAW that achieves polylogarithmic regret for marginally-stable linear systems, extending applicability beyond bounded-spectrum cases.

Findings

USP compresses sequences, reducing memory requirements for prediction.

VAW remains robust despite exponential diameter growth.

Combined USP and VAW achieve polylogarithmic regret in complex systems.

Abstract

Sequence prediction methods for dynamical systems with long memory, i.e. marginally stable systems, typically achieve regret that grows polynomially with the hidden dimension of the underlying generative model. Universal Sequence Preconditioning (USP) is a method that compresses any sequence which comes from a linear dynamical system into a "preconditioned" sequence which requires exponentially shorter memory for accurate prediction. However, the preconditioned sequence yields exponentially larger diameters and gradients, hindering USP from unlocking optimal regret bounds. Inspired by the minimum description length principle, we show that the Vovk-Azoury-Warmuth (VAW) algorithm is naturally matched to the USP regime. Indeed, it takes advantage of the memory compression while remaining robust to the exponential explosion of the diameter. We prove that combining USP with VAW achieves astoundingly strong results: for any marginally-stable linear dynamical system, this algorithm achieves polylogarithmic regret $O \left( \log^3 T \right)$ even in the presence of asymmetric hidden transition matrices. Finally, we extend the applicability of USP beyond bounded-spectrum systems by providing new complex-analytic bounds on Chebyshev polynomials, allowing for systems with constant complex arguments.