Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging

TL;DR

This paper introduces a safe reinforcement learning approach for autonomous highway ramp merging using probabilistic control barrier functions to handle model uncertainty, demonstrated in simulation and dataset experiments.

Contribution

It presents a novel integration of probabilistic control barrier functions into reinforcement learning for safe autonomous driving in ramp merging scenarios.

Findings

Successfully implemented in CARLA simulator

Effective handling of model uncertainty in control barrier functions

Improved safety in ramp merging tasks

Abstract

Prior work has looked at applying reinforcement learning and imitation learning approaches to autonomous driving scenarios, but either the safety or the efficiency of the algorithm is compromised. With the use of control barrier functions embedded into the reinforcement learning policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle. However, control barrier functions need a good approximation of the model of the car. We use probabilistic control barrier functions as an estimate of the model uncertainty. The algorithm is implemented as an online version in the CARLA (Dosovitskiy et al., 2017) Simulator and as an offline version on a dataset extracted from the NGSIM Database. The proposed algorithm is not just a safe ramp merging algorithm but a safe autonomous driving algorithm applied to address ramp merging on highways.