Machine Learning Methods for Large Population Games with Applications in Operations Research

TL;DR

This tutorial explores machine learning techniques for computing Nash equilibria in large population games, emphasizing their applications in operations research such as epidemic control, financial markets, and traffic management.

Contribution

It introduces a comprehensive framework combining stochastic control, mean field games, and machine learning algorithms for large-scale multi-agent game problems.

Findings

Mean field game approaches efficiently approximate Nash equilibria.

Machine learning algorithms like reinforcement learning are effective in large population games.

Numerical examples demonstrate practical applications in operations research.

Abstract

In this tutorial, we provide an introduction to machine learning methods for finding Nash equilibria in games with large number of agents. These types of problems are important for the operations research community because of their applicability to real life situations such as control of epidemics, optimal decisions in financial markets, electricity grid management, or traffic control for self-driving cars. We start the tutorial by introducing stochastic optimal control problems for a single agent, in discrete time and in continuous time. Then, we present the framework of dynamic games with finite number of agents. To tackle games with a very large number of agents, we discuss the paradigm of mean field games, which provides an efficient way to compute approximate Nash equilibria. Based on this approach, we discuss machine learning algorithms for such problems. First in the context of discrete time games, we introduce fixed point based methods and related methods based on reinforcement learning. Second, we discuss machine learning methods that are specific to continuous time problems, by building on optimality conditions phrased in terms of stochastic or partial differential equations. Several examples and numerical illustrations of problems arising in operations research are provided along the way.