Maximum Principle of Optimal Probability Density Control

TL;DR

This paper introduces a maximum principle framework for optimal control of probability densities in large-scale multi-agent systems, combining rigorous theory with neural network-based scalable algorithms.

Contribution

It develops a maximum principle and Hamilton-Jacobi-Bellman equation for infinite-dimensional probability spaces, enabling efficient numerical solutions for multi-agent control.

Findings

Proposed a maximum principle for density control problems.

Derived Hamilton-Jacobi-Bellman equations in infinite-dimensional spaces.

Demonstrated scalable neural network algorithms on multi-agent examples.

Abstract

We develop a general theoretical framework for optimal probability density control on standard measure spaces, aimed at addressing large-scale multi-agent control problems. In particular, we establish a maximum principle (MP) for control problems posed on infinite-dimensional spaces of probability distributions and control vector fields. We further derive the Hamilton--Jacobi--Bellman equation for the associated value functional defined on the space of probability distributions. Both results are presented in a concise form and supported by rigorous mathematical analysis, enabling efficient numerical treatment of these problems. Building on the proposed MP, we introduce a scalable numerical algorithm that leverages deep neural networks to handle high-dimensional settings. The effectiveness of the approach is demonstrated through several multi-agent control examples involving domain obstacles and inter-agent interactions.