Reinforced optimal control

TL;DR

This paper extends a reinforced regression method for stochastic control problems, improving accuracy and efficiency by incorporating previously computed value functions, with theoretical analysis and numerical validation.

Contribution

It generalizes the reinforced regression approach from optimal stopping to broader stochastic control problems, enhancing computational efficiency and accuracy.

Findings

Significant accuracy improvements demonstrated in numerical examples

Theoretical analysis confirms efficiency gains

Method applicable to a wide class of stochastic control problems

Abstract

Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoenmakers, Spokoiny, Zharkynbay. Commun.~Math.~Sci., 18(1):109-121, 2020](arXiv:1808.02341) proposes to reinforce the basis functions in the case of optimal stopping problems by already computed value functions for later times, thereby considerably improving the accuracy with limited additional computational cost. We extend the reinforced regression method to a general class of stochastic control problems, while considerably improving the method's efficiency, as demonstrated by substantial numerical examples as well as theoretical analysis.