Semidefinite Relaxations for Stochastic Optimal Control Policies

TL;DR

This paper introduces a novel semidefinite relaxation approach for solving stochastic optimal control problems by approximating the value function with polynomial candidates and using SOS relaxations to ensure bounds.

Contribution

It presents a new method that applies Sum of Squares relaxations to linear PDE formulations of stochastic control, providing guaranteed bounds on the value function.

Findings

Hierarchy of semidefinite relaxations improves solution accuracy

Guaranteed over- and under-approximations of the optimal value function

Effective polynomial-based approximation for stochastic control problems

Abstract

Recent results in the study of the Hamilton Jacobi Bellman (HJB) equation have led to the discovery of a formulation of the value function as a linear Partial Differential Equation (PDE) for stochastic nonlinear systems with a mild constraint on their disturbances. This has yielded promising directions for research in the planning and control of nonlinear systems. This work proposes a new method obtaining approximate solutions to these linear stochastic optimal control (SOC) problems. A candidate polynomial with variable coefficients is proposed as the solution to the SOC problem. A Sum of Squares (SOS) relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function.