Path Integral Control by Reproducing Kernel Hilbert Space Embedding

TL;DR

This paper introduces a kernel-based, model-free method for solving stochastic optimal control problems in the path integral framework, improving sample efficiency and reusability across tasks.

Contribution

It embeds path integral control problems into reproducing kernel Hilbert spaces, enabling non-parametric, sample-efficient solutions that separate invariant and task-specific components.

Findings

Enhanced sample efficiency over previous methods

Allows re-use of samples across different control tasks

Demonstrates effectiveness through numerical examples

Abstract

We present an embedding of stochastic optimal control problems, of the so called path integral form, into reproducing kernel Hilbert spaces. Using consistent, sample based estimates of the embedding leads to a model free, non-parametric approach for calculation of an approximate solution to the control problem. This formulation admits a decomposition of the problem into an invariant and task dependent component. Consequently, we make much more efficient use of the sample data compared to previous sample based approaches in this domain, e.g., by allowing sample re-use across tasks. Numerical examples on test problems, which illustrate the sample efficiency, are provided.