Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning

TL;DR

This paper introduces a novel method combining Control Barrier Functions with reinforcement learning to improve safety and learning efficiency in robotic systems by modeling the temporal effects of unsafe actions.

Contribution

It presents three CBF-based approaches that integrate safety guarantees into RL, addressing the limitations of traditional safety modeling in robotic learning.

Findings

CBF approaches effectively prevent catastrophic unsafe behaviors.

Enhanced learning efficiency in simulated and real-world robotic experiments.

Traditional RL struggles with temporal safety modeling, which CBFs help mitigate.

Abstract

Safety stands as the primary obstacle preventing the widespread adoption of learning-based robotic systems in our daily lives. While reinforcement learning (RL) shows promise as an effective robot learning paradigm, conventional RL frameworks often model safety by using single scalar negative rewards with immediate episode termination, failing to capture the temporal consequences of unsafe actions (e.g., sustained collision damage). In this work, we introduce a novel approach that simulates these temporal effects by applying continuous negative rewards without episode termination. Our experiments reveal that standard RL methods struggle with this model, as the accumulated negative values in unsafe zones create learning barriers. To address this challenge, we demonstrate how Control Barrier Functions (CBFs), with their proven safety guarantees, effectively help robots avoid catastrophic regions while enhancing learning outcomes. We present three CBF-based approaches, each integrating traditional RL methods with Control Barrier Functions, guiding the agent to learn safe behavior. Our empirical analysis, conducted in both simulated environments and real-world settings using a four-wheel differential drive robot, explores the possibilities of employing these approaches for safe robotic learning.