Near Optimality of Discrete-Time Approximations for Controlled McKean-Vlasov Diffusions and Interacting Particle Systems

TL;DR

This paper demonstrates that discrete-time approximations of controlled McKean-Vlasov diffusions and interacting particle systems are nearly optimal, providing convergence results and numerical methods for solving these complex stochastic control problems.

Contribution

It establishes the near-optimality of discrete-time policies for McKean-Vlasov control problems and their particle system approximations, with convergence rates and applicability to large systems.

Findings

Discrete-time policies are near-optimal for continuous-time McKean-Vlasov control.

Value functions of discrete approximations converge to continuous counterparts.

Discrete policies are asymptotically optimal for large interacting particle systems.

Abstract

We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Under mild assumptions, we first prove the existence of optimal relaxed controls by endowing the space of relaxed policies with a compact weak topology. (ii) Establishing continuity of the cost in control policy, we establish near-optimality of piecewise-constant strict policies, show that the discrete-time value functions (finite-horizon and discounted infinite-horizon) converge to their continuous-time counterparts as the timestep converges to zero, and that optimal discrete-time policies are near-optimal for the original continuous-time problem, where rates of convergence are also obtained. (iii) We then extend these approximation and near-optimality results to $N$-particle interacting systems under centralized or decentralized mean-field sharing information structure, proving that the discrete-time McKean-Vlasov policy is asymptotically optimal as $N\to \infty$ and the time step goes to zero. We thus develop an approximation of McKean-Vlasov optimal control problems via discrete-time McKean-Vlasov control problems (and associated numerical methods such as finite model approximation), and also show the near optimality of such approximate policy solutions for the $N$-agent interacting models under centralized and decentralized control.