LQG Online Learning

TL;DR

This paper introduces a novel online learning algorithm based on LQG control theory that offers robustness to outliers and smooth estimates, extending to nonlinear models via kernel methods.

Contribution

It formulates online learning as an LQG control problem with regularization, providing a closed-form solution and extensions to nonlinear models.

Findings

Less sensitive to outliers than Kalman filter

Provides smoother, regularized estimates over time

Extends to infinite horizon and nonlinear models using kernel trick

Abstract

Optimal control theory and machine learning techniques are combined to formulate and solve in closed form an optimal control formulation of online learning from supervised examples with regularization of the updates. The connections with the classical Linear Quadratic Gaussian (LQG) optimal control problem, of which the proposed learning paradigm is a non-trivial variation as it involves random matrices, are investigated. The obtained optimal solutions are compared with the Kalman-filter estimate of the parameter vector to be learned. It is shown that the proposed algorithm is less sensitive to outliers with respect to the Kalman estimate (thanks to the presence of the regularization term), thus providing smoother estimates with respect to time. The basic formulation of the proposed online-learning framework refers to a discrete-time setting with a finite learning horizon and a linear model. Various extensions are investigated, including the infinite learning horizon and, via the so-called "kernel trick", the case of nonlinear models.