Linearly Solvable Stochastic Control Lyapunov Functions

TL;DR

This paper introduces a novel approach to synthesize stochastic control Lyapunov functions for nonlinear systems by transforming the Hamilton-Jacobi-Bellman equation into a linear PDE and using sum of squares programming for solutions.

Contribution

It presents a new linear PDE-based method for constructing stochastic control Lyapunov functions with bounds on suboptimality, applicable to a broader class of systems.

Findings

Relaxed solutions are viscosity super/subsolutions.

Pointwise bounds provide suboptimality guarantees.

Method is demonstrated with simulated examples.

Abstract

This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear partial differential equation can then be relaxed to a linear differential inclusion, allowing for relaxed solutions to be generated using sum of squares programming. The resulting relaxed solutions are in fact viscosity super/subsolutions, and by the maximum principle are pointwise upper and lower bounds to the underlying value function, even for coarse polynomial approximations. Furthermore, the pointwise upper bound is shown to be a stochastic control Lyapunov function, yielding a method for generating nonlinear controllers with pointwise bounded distance from the optimal cost when using the optimal controller. These approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems. Finally, this paper develops a-priori bounds on trajectory suboptimality when using these approximate value functions, as well as demonstrates that these methods, and bounds, can be applied to a more general class of nonlinear systems not obeying the constraint on stochastic forcing. Simulated examples illustrate the methodology.