Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey

TL;DR

This survey reviews the use of control barrier functions in safe reinforcement learning, highlighting data-driven methods for automatic synthesis to improve safety and practicality in robotic applications.

Contribution

It provides a comprehensive overview of existing literature and explores recent data-driven techniques for automatically learning control barrier functions.

Findings

Control barrier functions enhance safety in reinforcement learning.

Data-driven methods facilitate automatic synthesis of control barrier functions.

The survey identifies promising directions for future research in safe robotics.

Abstract

Reinforcement learning is a powerful technique for developing new robot behaviors. However, typical lack of safety guarantees constitutes a hurdle for its practical application on real robots. To address this issue, safe reinforcement learning aims to incorporate safety considerations, enabling faster transfer to real robots and facilitating lifelong learning. One promising approach within safe reinforcement learning is the use of control barrier functions. These functions provide a framework to ensure that the system remains in a safe state during the learning process. However, synthesizing control barrier functions is not straightforward and often requires ample domain knowledge. This challenge motivates the exploration of data-driven methods for automatically defining control barrier functions, which is highly appealing. We conduct a comprehensive review of the existing literature on safe reinforcement learning using control barrier functions. Additionally, we investigate various techniques for automatically learning the Control Barrier Functions, aiming to enhance the safety and efficacy of Reinforcement Learning in practical robot applications.