On a mean-field Pontryagin minimum principle for stochastic optimal control

TL;DR

This paper introduces a deterministic mean-field extension of the Pontryagin minimum principle for stochastic optimal control, simplifying the solution process and applicable to complex systems.

Contribution

It proposes the McKean-Pontryagin minimum principle, a novel deterministic mean-field formulation that decouples equations and extends to infinite horizon problems.

Findings

Numerical tests on inverted pendulum, Lorenz-63, and Lorenz-96 systems validate the approach.

The method simplifies solving boundary value problems in stochastic control.

Extension potential to more general mean-field control problems.

Abstract

This paper outlines a novel extension of the classical Pontryagin minimum (maximum) principle to stochastic optimal control problems. Contrary to the well-known stochastic Pontryagin minimum principle involving forward-backward stochastic differential equations, the proposed formulation is deterministic and of mean-field type. We denote it by the McKean-Pontryagin minimum principle. The Hamiltonian structure of the proposed McKean-Pontryagin minimum principle is achieved via the introduction of a pair of auxiliary functions. A gauge freedom in the choice of one of these two functions can be used to decouple the forward and reverse time equations; hence simplifying the solution of the underlying boundary value problem. We also consider infinite horizon discounted cost optimal control problems. In this case, the mean-field formulation allows one to convert the computation of the desired optimal control law into solving a pair of forward mean-field ordinary differential equations. The McKean-Pontryagin minimum principle is tested numerically for a controlled inverted pendulum, a controlled Lorenz-63 system, and a controlled Lorenz-96 system. Although the focus is on linear-quadratic control problems, the proposed methodology is extendable to more general problems including mean-field type control formulations.