Probabilistic error analysis for some approximation schemes to optimal control problems

TL;DR

This paper develops a probabilistic framework for analyzing the error of novel numerical schemes in optimal control, achieving near-optimal convergence rates under various regularity conditions.

Contribution

It introduces a new Markov chain approximation scheme combining policy, Euler-Maruyama, and Gauss-Hermite methods, with rigorous error bounds and improvement techniques.

Findings

Lower error bounds close to 1/2 in time and 1/3 in space for Lipschitz solutions

Upper bounds of order 1/4 in time and 1/5 in space

Optimal order of 1 in both time and space for regular solutions

Abstract

We introduce a class of numerical schemes for optimal control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite approximation of the Gaussian increments. We provide lower error bounds of order arbitrarily close to 1/2 in time and 1/3 in space for Lipschitz viscosity solutions, coupling probabilistic arguments with regularization techniques as introduced by Krylov. The corresponding order of the upper bounds is 1/4 in time and 1/5 in space. For sufficiently regular solutions, the order is 1 in both time and space for both bounds. Finally, we propose techniques for further improving the accuracy of the individual components of the approximation.