Learning Decentralized Linear Quadratic Regulators with $\sqrt{T}$ Regret

TL;DR

This paper introduces an online learning algorithm for decentralized linear quadratic regulators that adaptively learns system control with sublinear regret, applicable to unknown models and various information patterns.

Contribution

It develops a novel algorithm combining disturbance-feedback and online convex optimization to achieve $\sqrt{T}$ regret in decentralized LQR problems with unknown models.

Findings

Achieves $\\sqrt{T}$ regret under partially nested information patterns.

Provides regret bounds relative to sub-optimal controllers for general information patterns.

Validates theoretical results through numerical experiments.

Abstract

We propose an online learning algorithm that adaptively designs a decentralized linear quadratic regulator when the system model is unknown a priori and new data samples from a single system trajectory become progressively available. The algorithm uses a disturbance-feedback representation of state-feedback controllers coupled with online convex optimization with memory and delayed feedback. Under the assumption that the system is stable or given a known stabilizing controller, we show that our controller enjoys an expected regret that scales as $\sqrt{T}$ with the time horizon $T$ for the case of partially nested information pattern. For more general information patterns, the optimal controller is unknown even if the system model is known. In this case, the regret of our controller is shown with respect to a linear sub-optimal controller. We validate our theoretical findings using numerical experiments.