Safe Policy Search with Gaussian Process Models

TL;DR

This paper introduces a data-efficient method for safe policy optimization using Gaussian process models, enabling reliable task execution while minimizing failure risk.

Contribution

It extends the PILCO framework by incorporating safety constraints and analytic error gradient computation for safer policy search.

Findings

Effective in minimizing system failures during training and operation

Provides a closed-form solution for error gradients and safety probability estimation

Demonstrates improved safety and efficiency in policy learning

Abstract

We propose a method to optimise the parameters of a policy which will be used to safely perform a given task in a data-efficient manner. We train a Gaussian process model to capture the system dynamics, based on the PILCO framework. Our model has useful analytic properties, which allow closed form computation of error gradients and estimating the probability of violating given state space constraints. During training, as well as operation, only policies that are deemed safe are implemented on the real system, minimising the risk of failure.