A Tutorial on the Non-Asymptotic Theory of System Identification

TL;DR

This tutorial introduces non-asymptotic methods in linear system identification, highlighting key tools like the Hanson-Wright Inequality and self-normalized martingales, and demonstrates their use in analyzing least-squares estimators.

Contribution

It provides a comprehensive introduction to non-asymptotic techniques in system identification and illustrates their application to linear and nonlinear models.

Findings

Streamlined proofs of estimator performance

Application of non-asymptotic tools to autoregressive models

Extension potential to nonlinear identification

Abstract

This tutorial serves as an introduction to recently developed non-asymptotic methods in the theory of -- mainly linear -- system identification. We emphasize tools we deem particularly useful for a range of problems in this domain, such as the covering technique, the Hanson-Wright Inequality and the method of self-normalized martingales. We then employ these tools to give streamlined proofs of the performance of various least-squares based estimators for identifying the parameters in autoregressive models. We conclude by sketching out how the ideas presented herein can be extended to certain nonlinear identification problems.