From Words to Safety: Language-Conditioned Safety Filtering for Robot Navigation

TL;DR

This paper introduces a modular framework that uses large language models, perception, and model predictive control to interpret natural language instructions and enforce safety constraints in robot navigation, enhancing robustness and applicability.

Contribution

It presents a novel, modular approach combining language understanding, environment grounding, and real-time safety filtering for robot navigation based on natural language instructions.

Findings

Robust interpretation of diverse language safety constraints

Effective real-time enforcement of safety in simulation and hardware

Improved safety adherence in open-world environments

Abstract

As robots become increasingly integrated into open-world, human-centered environments, their ability to interpret natural language instructions and adhere to safety constraints is critical for effective and trustworthy interaction. Existing approaches often focus on mapping language to reward functions instead of safety specifications or address only narrow constraint classes (e.g., obstacle avoidance), limiting their robustness and applicability. We propose a modular framework for language-conditioned safety in robot navigation. Our framework is composed of three core components: (1) a large language model (LLM)-based module that translates free-form instructions into structured safety specifications, (2) a perception module that grounds these specifications by maintaining object-level 3D representations of the environment, and (3) a model predictive control (MPC)-based safety filter that enforces both semantic and geometric constraints in real time. We evaluate the effectiveness of the proposed framework through both simulation studies and hardware experiments, demonstrating that it robustly interprets and enforces diverse language-specified constraints across a wide range of environments and scenarios.