A modified MSA for stochastic control problems

TL;DR

This paper introduces a modified Method of Successive Approximations (MSA) that guarantees convergence for general stochastic control problems involving both drift and diffusion controls, overcoming limitations of the classical approach.

Contribution

It proposes a new modification to the MSA algorithm with proven convergence and convergence rate under broad conditions, without restrictions on the time horizon.

Findings

Modified MSA converges for general stochastic control problems.

Convergence rate established under additional assumptions.

Results hold for problems with arbitrary time horizons.

Abstract

The classical Method of Successive Approximations (MSA) is an iterative method for solving stochastic control problems and is derived from Pontryagin's optimality principle. It is known that the MSA may fail to converge. Using careful estimates for the backward stochastic differential equation (BSDE) this paper suggests a modification to the MSA algorithm. This modified MSA is shown to converge for general stochastic control problems with control in both the drift and diffusion coefficients. Under some additional assumptions the rate of convergence is shown. The results are valid without restrictions on the time horizon of the control problem, in contrast to iterative methods based on the theory of forward-backward stochastic differential equations.