Finite-sample bounds for multi-output system identification

TL;DR

This paper derives new finite-sample bounds for multi-output linear regression with subgaussian noise, applicable to system identification and feedback systems, improving upon previous bounds in generality and tightness.

Contribution

It introduces more general and tighter finite-sample bounds for multi-output regression, accommodating unknown dependencies and applying to system identification problems.

Findings

Bounds are applicable to multi-output regression with subgaussian noise.

Results are tighter and more general than previous bounds.

Applicable to affine-in-parameter system identification, including LTI systems.

Abstract

This paper presents uniform-in-time finite-sample bounds for regularized linear regression with vector-valued outputs and conditionally zero-mean subgaussian noise. By revisiting classical self-normalized martingale arguments, we obtain bounds that apply directly to multi-output regression, unlike most of the prior work. Compared to the state of the art, the new results are more general and yield tighter bounds, even for scalar-valued outputs. The mild assumptions we use allow for unknown dependencies between regressors and past noise terms, typically induced by system dynamics or feedback mechanisms. Therefore, these novel finite-sample bounds can be applied to many affine-in-parameter system identification problems, including the identification of a linear time-invariant system from full-state measurements. These new results may lead to significant improvements in stochastic learning-based controllers for safety-critical applications.