Beyond Safety Filtering: Control Barrier Function-Informed Reinforcement Learning for Connected and Automated Vehicles

TL;DR

This paper introduces a Control Barrier Function-informed reward design for Multi-Agent Reinforcement Learning, improving safety and performance in connected vehicle scenarios by explicitly guiding safe learning.

Contribution

It proposes a novel reward shaping method using CBF constraints for MARL, enhancing safety and robustness over heuristic reward baselines.

Findings

Achieves highest task performance among tested methods.

Less sensitive to reward hyperparameter tuning.

Consistently strong performance across hyperparameter ranges.

Abstract

Reinforcement Learning (RL) uses rewards to guide learning, yet reward design is typically hand-crafted using heuristics that can be difficult to tune. We propose a Control Barrier Function (CBF)-informed reward design for Multi-Agent RL (MARL) that converts CBF constraint values under joint MARL actions into a reward signal that explicitly guides safe learning. We compare against two heuristic reward baselines in a four-way multi-lane intersection with connected and automated vehicles. Results show that our method achieves the highest task performance and is less sensitive to reward hyperparameters, yielding consistently strong performance across the tested hyperparameter range. Code for reproducing the experimental results and a video demonstration are available at https://github.com/bassamlab/SigmaRL.