An Effective Particle Gradient Projection Method for Solving Stochastic and Mean Field Control Problem

TL;DR

This paper introduces a mesh-free, neural network-based projection method inspired by the stochastic maximum principle for efficiently solving high-dimensional stochastic and mean field control problems, outperforming traditional deep learning approaches.

Contribution

A novel derivative-free, regression-based projection algorithm that effectively addresses high-dimensional stochastic and mean field control problems without requiring loss minimization.

Findings

Successfully solves high-dimensional control problems (100+ dimensions)

Outperforms traditional deep learning methods in accuracy and efficiency

Provides a new approach for high-dimensional HJB equations and mean field control

Abstract

This work puts forward a novel numerical approach for solving the stochastic optimal control problem (SOCP) and the mean field control (MFC) problem using projection algorithm inspired by the stochastic maximum principle (SMP) which is also powered by the randomized neural network. This approach is mesh-free, derivative free and it relies on gradually updating the underlying control via regression. It distinguishes itself from other traditional deep learning methods as it does not require minimizing the loss/cost function via direct error backward propagation to train the neural networks. The methodology designed can effectively solve stochastic optimal control problem in high dimensions ($100$ and above) and it can also be used to solve the mean field control problems. Due to the connection between the HJB equations and SOCP, the designed approach also provides a procedure for solving high dimensional HJB equations. Importantly, the infinite dimensional HJ equation related to the mean field control problem can also be solved in a point-wise sense (given the initial distribution) due to its connection with the Mean Field Control (MFC) problem. Our extensive test results show that the proposed approach typically performs better than the direct deep learning based approaches for solving control problems. We will leave the convergence proof and the extension to Mean Field Games (MFG) as future works.