Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning

TL;DR

FORTRESS is a real-time framework that uses multi-modal reasoning and planning to generate safe fallback strategies for robots, preventing out-of-distribution failures without hard-coded rules.

Contribution

It introduces FORTRESS, a novel joint reasoning and planning system that anticipates failures and synthesizes safe fallback plans in real time using multi-modal foundation models.

Findings

Outperforms prompt-based reasoning in safety classification accuracy.

Enhances safety and planning success in simulation and hardware tests.

Effectively prevents safety-critical failures in diverse robotic scenarios.

Abstract

While foundation models offer promise toward improving robot safety in out-of-distribution (OOD) scenarios, how to effectively harness their generalist knowledge for real-time, dynamically feasible response remains a crucial problem. We present FORTRESS, a joint reasoning and planning framework that generates semantically safe fallback strategies to prevent safety-critical, OOD failures. At a low frequency under nominal operation, FORTRESS uses multi-modal foundation models to anticipate possible failure modes and identify safe fallback sets. When a runtime monitor triggers a fallback response, FORTRESS rapidly synthesizes plans to fallback goals while inferring and avoiding semantically unsafe regions in real time. By bridging open-world, multi-modal reasoning with dynamics-aware planning, we eliminate the need for hard-coded fallbacks and human safety interventions. FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation. Website can be found at https://milanganai.github.io/fortress.