Model-based Path Integral Stochastic Control: A Bayesian Nonparametric Approach

TL;DR

This paper introduces a model-based, data-efficient stochastic optimal control framework using Gaussian processes and path integral formulation, enabling autonomous learning of optimal controls from limited data in complex tasks.

Contribution

It presents a novel Bayesian nonparametric approach that improves learning efficiency and applicability in high-dimensional control problems compared to existing RL methods.

Findings

Outperforms state-of-the-art RL in learning efficiency

Effective in high-dimensional, complex control tasks

Demonstrated success on nontrivial examples

Abstract

Over the last few years, sampling-based stochastic optimal control (SOC) frameworks have shown impressive performances in reinforcement learning (RL) with applications in robotics. However, such approaches require a large amount of samples from many interactions with the physical systems. To improve learning efficiency, we present a novel model-based and data-driven SOC framework based on path integral formulation and Gaussian processes (GPs). The proposed approach learns explicit and time-varying optimal controls autonomously from limited sampled data. Based on this framework, we propose an iterative control scheme with improved applicability in higher-dimensional and more complex control tasks. We demonstrate the effectiveness and efficiency of the proposed framework using two nontrivial examples. Compared to state-of-the-art RL methods, the proposed framework features superior control learning efficiency.