SABLAS: Learning Safe Control for Black-box Dynamical Systems

TL;DR

This paper introduces a novel approach to learning safe control policies for black-box dynamical systems that does not require an accurate system model, ensuring safety and efficiency in complex, real-world applications.

Contribution

It proposes a new method that re-designs the loss function for gradient back-propagation in non-differentiable black-box systems, enabling safe control policy learning without system modeling.

Findings

Achieves nearly 100% safety and goal reaching rates in simulations.

Significantly reduces training samples needed compared to existing methods.

Generalizes well to unseen scenarios while maintaining performance.

Abstract

Control certificates based on barrier functions have been a powerful tool to generate probably safe control policies for dynamical systems. However, existing methods based on barrier certificates are normally for white-box systems with differentiable dynamics, which makes them inapplicable to many practical applications where the system is a black-box and cannot be accurately modeled. On the other side, model-free reinforcement learning (RL) methods for black-box systems suffer from lack of safety guarantees and low sampling efficiency. In this paper, we propose a novel method that can learn safe control policies and barrier certificates for black-box dynamical systems, without requiring for an accurate system model. Our method re-designs the loss function to back-propagate gradient to the control policy even when the black-box dynamical system is non-differentiable, and we show that the safety certificates hold on the black-box system. Empirical results in simulation show that our method can significantly improve the performance of the learned policies by achieving nearly 100% safety and goal reaching rates using much fewer training samples, compared to state-of-the-art black-box safe control methods. Our learned agents can also generalize to unseen scenarios while keeping the original performance. The source code can be found at https://github.com/Zengyi-Qin/bcbf.