Large-Scale Multi-Agent Deep FBSDEs

TL;DR

This paper introduces a scalable deep learning framework for multi-agent stochastic games that efficiently finds Nash Equilibria, demonstrating superior performance and scalability up to 3000 agents in complex simulations.

Contribution

The paper presents a novel deep learning approach based on FBSDEs for multi-agent games, significantly improving scalability and efficiency over existing methods.

Findings

Outperforms state-of-the-art deep fictitious play algorithms.

Successfully scales to 3000 agents in simulation.

Applicable to robotics and autonomous racing problems.

Abstract

In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations (FBSDE) and their implementation in a deep learning setting, which is the source of our algorithm's sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.