SOCKS: A Stochastic Optimal Control and Reachability Toolbox Using Kernel Methods

TL;DR

SOCKS is a data-driven toolbox that uses kernel methods to solve stochastic optimal control problems, including reachability, without assuming specific system models, making it suitable for complex, uncertain systems.

Contribution

The paper introduces SOCKS, a novel kernel-based, data-driven toolbox for stochastic optimal control and reachability, capable of handling nonlinear and poorly characterized systems.

Findings

Successfully applied to multiple benchmarks

Handles nonlinear dynamics and black-box systems

Provides approximate solutions without prior system assumptions

Abstract

We present SOCKS, a data-driven stochastic optimal control toolbox based in kernel methods. SOCKS is a collection of data-driven algorithms that compute approximate solutions to stochastic optimal control problems with arbitrary cost and constraint functions, including stochastic reachability, which seeks to determine the likelihood that a system will reach a desired target set while respecting a set of pre-defined safety constraints. Our approach relies upon a class of machine learning algorithms based in kernel methods, a nonparametric technique which can be used to represent probability distributions in a high-dimensional space of functions known as a reproducing kernel Hilbert space. As a nonparametric technique, kernel methods are inherently data-driven, meaning that they do not place prior assumptions on the system dynamics or the structure of the uncertainty. This makes the toolbox amenable to a wide variety of systems, including those with nonlinear dynamics, black-box elements, and poorly characterized stochastic disturbances. We present the main features of SOCKS and demonstrate its capabilities on several benchmarks.