Learning to Stop: Deep Learning for Mean Field Optimal Stopping

TL;DR

This paper introduces a deep learning framework for mean-field optimal stopping problems, providing scalable solutions for multi-agent systems with applications across various fields.

Contribution

It formalizes and computationally solves the mean-field optimal stopping problem using neural networks, extending optimal stopping theory to large multi-agent systems.

Findings

Proposes two neural network-based methods for mean-field optimal stopping.

Demonstrates scalability with problems up to 300 spatial dimensions.

Validates effectiveness through numerical experiments on six different problems.

Abstract

Optimal stopping is a fundamental problem in optimization with applications in risk management, finance, robotics, and machine learning. We extend the standard framework to a multi-agent setting, named multi-agent optimal stopping (MAOS), where agents cooperate to make optimal stopping decisions in a finite-space, discrete-time environment. Since solving MAOS becomes computationally prohibitive as the number of agents is very large, we study the mean-field optimal stopping (MFOS) problem, obtained as the number of agents tends to infinity. We establish that MFOS provides a good approximation to MAOS and prove a dynamic programming principle (DPP) based on mean-field control theory. We then propose two deep learning approaches: one that learns optimal stopping decisions by simulating full trajectories and another that leverages the DPP to compute the value function and to learn the optimal stopping rule using backward induction. Both methods train neural networks to approximate optimal stopping policies. We demonstrate the effectiveness and the scalability of our work through numerical experiments on 6 different problems in spatial dimension up to 300. To the best of our knowledge, this is the first work to formalize and computationally solve MFOS in discrete time and finite space, opening new directions for scalable MAOS methods.