Learning Safe Autonomous Driving Policies Using Predictive Safety Representations

TL;DR

This paper evaluates the extension of the SRPL framework, which uses predictive safety representations, to real-world autonomous driving, demonstrating improved safety-performance tradeoffs and robustness across datasets.

Contribution

It shows that SRPL can enhance real-world autonomous driving safety and efficiency, with benefits influenced by the policy optimizer and dataset, and improves robustness and generalization.

Findings

SRPL improves success rate and reduces costs significantly

Predictive safety representations enhance robustness to noise

SRPL-augmented agents generalize better across datasets

Abstract

Safe reinforcement learning (SafeRL) is a prominent paradigm for autonomous driving, where agents are required to optimize performance under strict safety requirements. This dual objective creates a fundamental tension, as overly conservative policies limit driving efficiency while aggressive exploration risks safety violations. The Safety Representations for Safer Policy Learning (SRPL) framework addresses this challenge by equipping agents with a predictive model of future constraint violations and has shown promise in controlled environments. This paper investigates whether SRPL extends to real-world autonomous driving scenarios. Systematic experiments on the Waymo Open Motion Dataset (WOMD) and NuPlan demonstrate that SRPL can improve the reward-safety tradeoff, achieving statistically significant improvements in success rate (effect sizes r = 0.65-0.86) and cost reduction (effect sizes r = 0.70-0.83), with p < 0.05 for observed improvements. However, its effectiveness depends on the underlying policy optimizer and the dataset distribution. The results further show that predictive safety representations play a critical role in improving robustness to observation noise. Additionally, in zero-shot cross-dataset evaluation, SRPL-augmented agents demonstrate improved generalization compared to non-SRPL methods. These findings collectively demonstrate the potential of predictive safety representations to strengthen SafeRL for autonomous driving.