Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filter

TL;DR

The paper presents Deep QP Safety Filter, a data-driven safety layer for black-box systems that combines Hamilton-Jacobi reachability with model-free learning to improve safety and learning efficiency.

Contribution

It introduces a novel neural network-based approach to learn a Quadratic-Program safety filter without requiring system models, applicable to diverse dynamical systems.

Findings

Substantially reduces pre-convergence failures in RL tasks.

Accelerates learning towards higher returns compared to baselines.

Converges to the viscosity solution even for non-smooth values.

Abstract

We introduce Deep QP Safety Filter, a fully data-driven safety layer for black-box dynamical systems. Our method learns a Quadratic-Program (QP) safety filter without model knowledge by combining Hamilton-Jacobi (HJ) reachability with model-free learning. We construct contraction-based losses for both the safety value and its derivatives, and train two neural networks accordingly. In the exact setting, the learned critic converges to the viscosity solution (and its derivative), even for non-smooth values. Across diverse dynamical systems -- even including a hybrid system -- and multiple RL tasks, Deep QP Safety Filter substantially reduces pre-convergence failures while accelerating learning toward higher returns than strong baselines, offering a principled and practical route to safe, model-free control.