V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions

TL;DR

V-OCBF introduces a novel offline learning framework for safety controllers that learns neural control barrier functions directly from demonstrations without requiring system models, ensuring strict safety guarantees.

Contribution

It proposes a model-free, offline method to learn neural control barrier functions using value-guided updates and expectile objectives, avoiding expert-designed barriers and online queries.

Findings

V-OCBF significantly reduces safety violations compared to baselines.

The method maintains strong task performance in safety-critical scenarios.

It scales effectively for offline safety controller synthesis without online interaction.

Abstract

Ensuring safety in autonomous systems requires controllers that aim to satisfy state-wise constraints without relying on online interaction.While existing Safe Offline RL methods typically enforce soft expected-cost constraints, they struggle to ensure strict state-wise safety. Conversely, Control Barrier Functions (CBFs) offer a principled mechanism to enforce forward invariance, but often rely on expert-designed barrier functions or knowledge of the system dynamics. We introduce Value-Guided Offline Control Barrier Functions (V-OCBF), a framework that learns a neural CBF entirely from offline demonstrations. Unlike prior approaches, V-OCBF does not assume access to the dynamics model; instead, it derives a recursive finite-difference barrier update, enabling model-free learning of a barrier that propagates safety information over time. Moreover, V-OCBF incorporates an expectile-based objective that avoids querying the barrier on out-of-distribution actions and restricts updates to the dataset-supported action set. The learned barrier is then used with a Quadratic Program (QP) formulation to synthesize real-time safe control. Across multiple case studies, V-OCBF yields substantially fewer safety violations than baseline methods while maintaining strong task performance, highlighting its scalability for offline synthesis of safety-critical controllers without online interaction or hand-engineered barriers.