Maximum principle for discrete-time stochastic optimal control problem under distribution uncertainty

TL;DR

This paper develops a maximum principle for discrete-time stochastic optimal control problems considering distribution uncertainty, introducing a backward algorithm for computing optimal controls and reference probabilities.

Contribution

It presents a novel maximum principle under distribution uncertainty using weak convergence and minimax theorem, with an algorithm to compute optimal controls and reference probabilities.

Findings

Derived variational inequality for the cost functional.

Established the stochastic maximum principle under distribution uncertainty.

Proposed a backward algorithm for practical computation of optimal controls.

Abstract

In this paper, we study a discrete-time stochastic optimal control problem under distribution uncertainty with convex control domain. By weak convergence method and Sion's minimax theorem, we obtain the variational inequality for cost functional under a reference probability $P^{\ast}$. Moreover, under the square integrability condition for noise and control, we establish the discrete-time stochastic maximum principle under $P^{\ast}$. Finally, we introduce a backward algorithm to calculate the reference probability $P^{\ast}$ and the optimal control $u^{\ast}$.