Online Linear Quadratic Control

TL;DR

This paper introduces efficient online algorithms for controlling linear systems with noisy dynamics and adversarial costs, achieving sublinear regret by leveraging a novel SDP relaxation that ensures stability and rapid mixing.

Contribution

It presents the first efficient online control algorithms with $O( oot{T})$ regret guarantees using a new SDP relaxation ensuring stability and fast mixing.

Findings

Achieves $O( oot{T})$ regret in online linear control.

Introduces a novel SDP relaxation for steady-state distribution.

Ensures policies are strongly stable with exponential mixing.

Abstract

We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee $O(\sqrt{T})$ regret under mild assumptions, where $T$ is the time horizon. Our algorithms rely on a novel SDP relaxation for the steady-state distribution of the system. Crucially, and in contrast to previously proposed relaxations, the feasible solutions of our SDP all correspond to "strongly stable" policies that mix exponentially fast to a steady state.