Mean-Field Control Approach to Decentralized Stochastic Control with Finite-Dimensional Memories

TL;DR

This paper introduces a memory-limited decentralized stochastic control framework that simplifies multi-agent estimation problems using mean-field control theory, demonstrated through a modified Riccati equation in LQG systems.

Contribution

It proposes a novel memory-limited approach to DSC that enables solutions in more general cases via mean-field control and a modified Riccati equation.

Findings

Decentralized Riccati equation improves estimation and control.

Numerical experiments show superiority over conventional Riccati equations.

ML-DSC extends solvability of multi-agent control problems.

Abstract

Decentralized stochastic control (DSC) considers the optimal control problem of a multi-agent system. However, DSC cannot be solved except in the special cases because the estimation among the agents is generally intractable. In this work, we propose memory-limited DSC (ML-DSC), in which each agent compresses the observation history into the finite-dimensional memory. Because this compression simplifies the estimation among the agents, ML-DSC can be solved in more general cases based on the mean-field control theory. We demonstrate ML-DSC in the general LQG problem. Because estimation and control are not clearly separated in the general LQG problem, the Riccati equation is modified to the decentralized Riccati equation, which improves estimation as well as control. Our numerical experiment shows that the decentralized Riccati equation is superior to the conventional Riccati equation.