Deep Generalized Schr\"odinger Bridge

TL;DR

This paper introduces DeepGSB, a novel deep learning framework that generalizes Schr"odinger Bridge to solve complex mean-field games, including high-dimensional problems, by leveraging stochastic differential equations and reinforcement learning techniques.

Contribution

It extends Schr"odinger Bridge to mean-field games with non-differentiable preferences, enabling practical solutions for high-dimensional and complex population dynamics.

Findings

Outperforms prior methods in classical population navigation MFGs.

Capable of solving 1000-dimensional opinion depolarization.

Provides a new state-of-the-art numerical solver for high-dimensional MFGs.

Abstract

Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling the collective behavior of individual agents interacting stochastically with a large population. In this work, we aim at solving a challenging class of MFGs in which the differentiability of these interacting preferences may not be available to the solver, and the population is urged to converge exactly to some desired distribution. These setups are, despite being well-motivated for practical purposes, complicated enough to paralyze most (deep) numerical solvers. Nevertheless, we show that Schr\"odinger Bridge - as an entropy-regularized optimal transport model - can be generalized to accepting mean-field structures, hence solving these MFGs. This is achieved via the application of Forward-Backward Stochastic Differential Equations theory, which, intriguingly, leads to a computational framework with a similar structure to Temporal Difference learning. As such, it opens up novel algorithmic connections to Deep Reinforcement Learning that we leverage to facilitate practical training. We show that our proposed objective function provides necessary and sufficient conditions to the mean-field problem. Our method, named Deep Generalized Schr\"odinger Bridge (DeepGSB), not only outperforms prior methods in solving classical population navigation MFGs, but is also capable of solving 1000-dimensional opinion depolarization, setting a new state-of-the-art numerical solver for high-dimensional MFGs. Our code will be made available at https://github.com/ghliu/DeepGSB.