Learning Linear Models Using Distributed Iterative Hessian Sketching

TL;DR

This paper introduces a distributed Newton algorithm utilizing Hessian sketching to efficiently learn Markov parameters of linear systems, achieving geometric convergence and parallelization.

Contribution

It presents a novel randomized distributed Newton method with Hessian sketching for scalable system identification, improving computational tractability.

Findings

Algorithm converges geometrically to $\\epsilon$-optimal solutions.

Method is parallelizable and works with various sketching matrices.

Numerical examples validate theoretical results.

Abstract

This work considers the problem of learning the Markov parameters of a linear system from observed data. Recent non-asymptotic system identification results have characterized the sample complexity of this problem in the single and multi-rollout setting. In both instances, the number of samples required in order to obtain acceptable estimates can produce optimization problems with an intractably large number of decision variables for a second-order algorithm. We show that a randomized and distributed Newton algorithm based on Hessian-sketching can produce $\epsilon$-optimal solutions and converges geometrically. Moreover, the algorithm is trivially parallelizable. Our results hold for a variety of sketching matrices and we illustrate the theory with numerical examples.