Learning Safe Control via On-the-Fly Bandit Exploration

TL;DR

This paper introduces a novel safe control method that combines control barrier functions with on-the-fly data collection using Gaussian process bandit algorithms, ensuring safety without needing a backup controller even under high model uncertainty.

Contribution

It proposes the first safe learning-based control approach that guarantees safety without a backup controller by integrating exploration with safety verification using Gaussian processes.

Findings

Ensures safety in control tasks with high model uncertainty.

Achieves safety without a backup controller.

Uses Gaussian process bandit exploration for data collection.

Abstract

Control tasks with safety requirements under high levels of model uncertainty are increasingly common. Machine learning techniques are frequently used to address such tasks, typically by leveraging model error bounds to specify robust constraint-based safety filters. However, if the learned model uncertainty is very high, the corresponding filters are potentially invalid, meaning no control input satisfies the constraints imposed by the safety filter. While most works address this issue by assuming some form of safe backup controller, ours tackles it by collecting additional data on the fly using a Gaussian process bandit-type algorithm. We combine a control barrier function with a learned model to specify a robust certificate that ensures safety if feasible. Whenever infeasibility occurs, we leverage the control barrier function to guide exploration, ensuring the collected data contributes toward the closed-loop system safety. By combining a safety filter with exploration in this manner, our method provably achieves safety in a setting that allows for a zero-mean prior dynamics model, without requiring a backup controller. To the best of our knowledge, it is the first safe learning-based control method that achieves this.