Linear Quadratic Mean Field Teams: Optimal and Approximately Optimal Decentralized Solutions

TL;DR

This paper develops decentralized control strategies for large-scale linear quadratic systems with multiple sub-populations, providing optimal and approximately optimal solutions that are computationally efficient and applicable to smart grid demand response.

Contribution

It introduces a novel framework for mean-field team control with non-classical information structures, deriving explicit solutions via Riccati equations independent of agent count.

Findings

Linear control strategies are optimal or approximately optimal under the considered information structures.

The solution complexity is independent of the number of agents, depending only on the number of sub-populations.

The approach is applicable to various extensions, including major-minor agents and infinite horizon scenarios.

Abstract

We consider team optimal control of decentralized systems with linear dynamics, quadratic costs, and arbitrary disturbance that consist of multiple sub-populations with exchangeable agents (i.e., exchanging two agents within the same sub-population does not affect the dynamics or the cost). Such a system is equivalent to one where the dynamics and costs are coupled across agents through the mean-field (or empirical mean) of the states and actions (even when the primitive random variables are non-exchangeable). Two information structures are investigated. In the first, all agents observe their local state and the mean-field of all sub-populations, in the second, all agents observe their local state but the mean-field of only a subset of the sub-populations. Both information structures are non-classical and not partially nested. Nonetheless, it is shown that linear control strategies are optimal for the first and approximately optimal for the second, the approximation error is inversely proportional to the size of the sub-populations whose mean-fields are not observed. The corresponding gains are determined by the solution of K+1 decoupled standard Riccati equations, where K is the number of sub-populations. The dimensions of the Riccati equations do not depend on the size of the sub-populations, thus the solution complexity is independent of the number of agents. Generalizations to major-minor agents, tracking cost, weighted mean-field, and infinite horizon are provided. The results are illustrated using an example of demand response in smart grids.