Online Gradient Descent for Linear Dynamical Systems

TL;DR

This paper introduces an online gradient descent-based control algorithm for linear dynamical systems that adapts to changing cost functions, providing theoretical performance guarantees and demonstrating convergence and effectiveness through simulations.

Contribution

It proposes a novel online control method for linear systems with time-varying costs and derives regret bounds showing sublinear regret and convergence properties.

Findings

Achieves sublinear regret under sublinear variation of costs

System converges to optimal equilibrium when costs stabilize

Numerical simulations validate theoretical results

Abstract

In this paper, online convex optimization is applied to the problem of controlling linear dynamical systems. An algorithm similar to online gradient descent, which can handle time-varying and unknown cost functions, is proposed. Then, performance guarantees are derived in terms of regret analysis. We show that the proposed control scheme achieves sublinear regret if the variation of the cost functions is sublinear. In addition, as a special case, the system converges to the optimal equilibrium if the cost functions are invariant after some finite time. Finally, the performance of the resulting closed loop is illustrated by numerical simulations.