Suboptimal Stabilizing Controllers for Linearly Solvable System

TL;DR

This paper introduces a method to generate approximate stochastic control Lyapunov functions for nonlinear systems by transforming the Hamilton-Jacobi-Bellman equation into a linear form and solving it via sum of squares programming, providing bounds on suboptimality.

Contribution

It proposes a novel approach to synthesize stochastic control Lyapunov functions using linear PDE relaxation and sum of squares programming, enabling approximate solutions with bounded suboptimality.

Findings

Linear PDE transformation for stochastic control systems

Sum of squares programming for polynomial solutions

A-priori bounds on trajectory suboptimality

Abstract

This paper presents a novel method to synthesize stochastic control Lyapunov functions for a class of nonlinear, stochastic control systems. In this work, the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation is transformed into a linear partial differential equation for a class of systems with a particular constraint on the stochastic disturbance. It is shown that this linear partial differential equation can be relaxed to a linear differential inclusion, allowing for approximating polynomial solutions to be generated using sum of squares programming. It is shown that the resulting solutions are stochastic control Lyapunov functions with a number of compelling properties. In particular, a-priori bounds on trajectory suboptimality are shown for these approximate value functions. The result is a technique whereby approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems.