Method of Successive Approximations for Stochastic Optimal Control: Contractivity and Convergence

TL;DR

This paper analyzes the convergence of the Method of Successive Approximations for stochastic optimal control problems, proving contractivity and stability under specific conditions on the system's coefficients.

Contribution

It provides a rigorous proof of the contractivity and convergence of MSA for a class of stochastic systems with Lipschitz continuous coefficients.

Findings

Proves stability of the state process under control variations

Establishes stability of the adjoint process

Demonstrates contractivity and convergence of MSA

Abstract

The Method of Successive Approximations (MSA) is a fixed-point iterative method used to solve stochastic optimal control problems. It is an indirect method based on the conditions derived from the Stochastic Maximum Principle (SMP), an extension of the Pontryagin Maximum Principle (PMP) to stochastic control problems. In this study, we investigate the contractivity and the convergence of MSA for a specific and interesting class of stochastic dynamical systems (when the drift coefficient is one-sided-Lipschitz with a negative constant and the diffusion coefficient is Lipschitz continuous). Our analysis unfolds in three key steps: firstly, we prove the stability of the state process with respect to the control process. Secondly, we establish the stability of the adjoint process. Finally, we present rigorous evidence to prove the contractivity and then the convergence of MSA. This study contributes to enhancing the understanding of MSA's applicability and effectiveness in addressing stochastic optimal control problems.