Solving Feynman-Kac Forward Backward SDEs Using McKean-Markov Branched Sampling

TL;DR

This paper introduces a novel McKean-Markov branched sampling approach combined with entropy-weighted least squares Monte Carlo for efficiently solving Feynman-Kac FBSDEs in stochastic control, showing improved convergence.

Contribution

It presents a new numerical method using branched sampling and entropy-weighted LSMC for solving FBSDEs in stochastic control, enhancing convergence over existing methods.

Findings

Significant convergence improvements demonstrated

Effective handling of nonlinear control problems

Method applicable to non-quadratic costs

Abstract

We propose a new method for the numerical solution of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. Using Girsanov's change of probability measures, it is demonstrated how a McKean-Markov branched sampling method can be utilized for the forward integration pass, as long as the controlled drift term is appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of a space-filling tree consisting of trajectory samples. Moreover, a local entropy-weighted least squares Monte Carlo (LSMC) method is developed to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories. The proposed methodology is numerically demonstrated on linear and nonlinear stochastic optimal control problems with non-quadratic running costs, which reveal significant convergence improvements over previous FBSDE-based numerical solution methods.