Team Optimal Control of Coupled Subsystems with Mean-Field Sharing

TL;DR

This paper develops a dynamic programming approach for optimal control of multiple coupled stochastic subsystems with mean-field sharing, applicable to large systems like smart grids, and handles noisy observations efficiently.

Contribution

It introduces an information state and dynamic programming framework for team control with mean-field coupling, extending to infinite-horizon and noisy observation scenarios.

Findings

The approach finds globally optimal strategies for all controllers.

The information state size remains time-invariant, enabling scalability.

The method effectively handles systems with hundreds of subsystems, demonstrated on a smart grid example.

Abstract

We investigate team optimal control of stochastic subsystems that are weakly coupled in dynamics (through the mean-field of the system) and are arbitrary coupled in the cost. The controller of each subsystem observes its local state and the mean-field of the state of all subsystems. The system has a non-classical information structure. Exploiting the symmetry of the problem, we identify an information state and use that to obtain a dynamic programming decomposition. This dynamic program determines a globally optimal strategy for all controllers. Our solution approach works for arbitrary number of controllers and generalizes to the setup when the mean-field is observed with noise. The size of the information state is time-invariant; thus, the results generalize to the infinite-horizon control setups as well. In addition, when the mean-field is observed without noise, the size of the corresponding information state increases polynomially (rather than exponentially) with the number of controllers which allows us to solve problems with moderate number of controllers. We illustrate our approach by an example motivated by smart grids that consists of $100$ coupled subsystems.