Scalar Federated Learning for Linear Quadratic Regulator

TL;DR

ScalarFedLQR introduces a communication-efficient federated algorithm for LQR control that reduces per-agent communication to a scalar, enabling scalable, fast convergence in multi-agent systems.

Contribution

It presents a novel scalar projection-based federated learning method for LQR that significantly reduces communication while maintaining convergence and stability.

Findings

Reduces communication from O(d) to O(1) per agent.

Larger agent fleets improve gradient recovery and convergence.

Achieves linear convergence with stable iterates and reduced communication.

Abstract

We propose ScalarFedLQR, a communication-efficient federated algorithm for model-free learning of a common policy in linear quadratic regulator (LQR) control of heterogeneous agents. The method builds on a decomposed projected gradient mechanism, in which each agent communicates only a scalar projection of a local zeroth-order gradient estimate. The server aggregates these scalar messages to reconstruct a global descent direction, reducing per-agent uplink communication from O(d) to O(1), independent of the policy dimension. Crucially, the projection-induced approximation error diminishes as the number of participating agents increases, yielding a favorable scaling law: larger fleets enable more accurate gradient recovery, admit larger stepsizes, and achieve faster linear convergence despite high dimensionality. Under standard regularity conditions, all iterates remain stabilizing and the average LQR cost decreases linearly fast. Numerical results demonstrate performance comparable to full-gradient federated LQR with substantially reduced communication.