Mean Field Games Flock! The Reinforcement Learning Way

TL;DR

This paper introduces a novel deep reinforcement learning approach combined with normalizing flows to enable large populations of agents to learn flocking behavior modeled as a mean field game, overcoming previous limitations.

Contribution

It presents a tractable method for multi-agent flocking using mean field games, deep RL, and normalizing flows, requiring minimal assumptions and handling high-dimensional scenarios.

Findings

Successfully learned multi-group flocking behaviors

Demonstrated adaptability to obstacles in high-dimensional settings

Achieved convergence to Nash Equilibrium in complex environments

Abstract

We present a method enabling a large number of agents to learn how to flock, which is a natural behavior observed in large populations of animals. This problem has drawn a lot of interest but requires many structural assumptions and is tractable only in small dimensions. We phrase this problem as a Mean Field Game (MFG), where each individual chooses its acceleration depending on the population behavior. Combining Deep Reinforcement Learning (RL) and Normalizing Flows (NF), we obtain a tractable solution requiring only very weak assumptions. Our algorithm finds a Nash Equilibrium and the agents adapt their velocity to match the neighboring flock's average one. We use Fictitious Play and alternate: (1) computing an approximate best response with Deep RL, and (2) estimating the next population distribution with NF. We show numerically that our algorithm learn multi-group or high-dimensional flocking with obstacles.