Contextual Safety Reasoning and Grounding for Open-World Robots

TL;DR

CORE is a novel safety framework for robots that uses vision-language models to reason about and enforce context-dependent safety rules in real-time, without prior environment knowledge, ensuring adaptable and safe robot behavior.

Contribution

This work introduces CORE, a framework that combines online contextual reasoning, grounding, and safety enforcement using vision-language models, with probabilistic safety guarantees in open-world environments.

Findings

CORE outperforms prior semantic safety methods in unseen environments.

Probabilistic safety guarantees account for perceptual uncertainty.

Ablation studies confirm the importance of reasoning and grounding components.

Abstract

Robots are increasingly operating in open-world environments where safe behavior depends on context: the same hallway may require different navigation strategies when crowded versus empty, or during an emergency versus normal operations. Traditional safety approaches enforce fixed constraints in user-specified contexts, limiting their ability to handle the open-ended contextual variability of real-world deployment. We address this gap via CORE, a safety framework that enables online contextual reasoning, grounding, and enforcement without prior knowledge of the environment (e.g., maps or safety specifications). CORE uses a vision-language model (VLM) to continuously reason about context-dependent safety rules directly from visual observations, grounds these rules in the physical environment, and enforces the resulting spatially-defined safe sets via control barrier functions. We provide probabilistic safety guarantees for CORE that account for perceptual uncertainty, and we demonstrate through simulation and real-world experiments that CORE enforces contextually appropriate behavior in unseen environments, significantly outperforming prior semantic safety methods that lack online contextual reasoning. Ablation studies validate our theoretical guarantees and underscore the importance of both VLM-based reasoning and spatial grounding for enforcing contextual safety in novel settings. We provide additional resources at https://zacravichandran.github.io/CORE.