Convex Relaxations for Nonlinear Stochastic Optimal Control Problems

TL;DR

This paper introduces a novel method for deriving guaranteed convex and concave relaxations of nonlinear stochastic optimal control problems, enabling rigorous bounds on the optimal value without sampling errors.

Contribution

It extends convex relaxation techniques from deterministic to stochastic control problems, addressing the challenge of intractable expected-value cost functions.

Findings

Provides rigorous lower and upper bounds on the optimal value.

Enables application of spatial branch-and-bound methods to stochastic control.

No sample-based approximation errors in the relaxations.

Abstract

This article presents a new method for computing guaranteed convex and concave relaxations of nonlinear stochastic optimal control problems with final-time expected-value cost functions. This method is motivated by similar methods for deterministic optimal control problems, which have been successfully applied within spatial branch-and-bound (B&B) techniques to obtain guaranteed global optima. Relative to those methods, a key challenge here is that the expected-value cost function cannot be expressed analytically in closed form. Nonetheless, the presented relaxations provide rigorous lower and upper bounds on the optimal objective value with no sample-based approximation error. In principle, this enables the use of spatial B&B global optimization techniques, but we leave the details of such an algorithm for future work.