Optimality Conditions in Variational Form for Non-Linear Constrained Stochastic Control Problems

TL;DR

This paper establishes variational inequality-based optimality conditions for constrained stochastic control problems with distribution-dependent costs, and proposes an augmented Lagrangian method utilizing dynamic programming for iterative solutions.

Contribution

It introduces a novel variational inequality framework for distribution-dependent stochastic control problems and develops an augmented Lagrangian method with theoretical analysis.

Findings

Optimality conditions are derived for a class of stochastic control problems.

The augmented Lagrangian method converges under certain convexity assumptions.

Numerical examples demonstrate the effectiveness of the proposed approach.

Abstract

Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochastic optimal control problem is solved by dynamic programming. Two academical examples are investigated.