Learning the Globally Optimal Distributed LQ Regulator

TL;DR

This paper develops a model-free learning approach with sample complexity bounds for globally optimal distributed output-feedback LQ control under subspace constraints, addressing a key challenge in distributed control.

Contribution

It introduces the first sample-complexity bounds for learning globally optimal distributed output-feedback LQ controllers, leveraging recent zeroth-order optimization results.

Findings

Established sample complexity bounds for distributed LQ control.

Proved that Quadratically Invariant problems have the necessary gradient dominance property.

First to provide theoretical guarantees for learning globally optimal distributed control policies.

Abstract

We study model-free learning methods for the output-feedback Linear Quadratic (LQ) control problem in finite-horizon subject to subspace constraints on the control policy. Subspace constraints naturally arise in the field of distributed control and present a significant challenge in the sense that standard model-based optimization and learning leads to intractable numerical programs in general. Building upon recent results in zeroth-order optimization, we establish model-free sample-complexity bounds for the class of distributed LQ problems where a local gradient dominance constant exists on any sublevel set of the cost function. %which admit a local gradient dominance constant valid on the sublevel set of the cost function. We prove that a fundamental class of distributed control problems - commonly referred to as Quadratically Invariant (QI) problems - as well as others possess this property. To the best of our knowledge, our result is the first sample-complexity bound guarantee on learning globally optimal distributed output-feedback control policies.