Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress

TL;DR

Sentinel is a runtime monitoring framework for generative robot policies that combines statistical temporal consistency checks and Vision Language Models to detect diverse failure modes, improving detection accuracy in robotic manipulation tasks.

Contribution

The paper introduces Sentinel, a novel framework that unifies temporal consistency measures and VLM-based detection to identify failure modes in generative policies.

Findings

Sentinel detects 18% more failures than individual detectors.

Combining detectors significantly improves failure detection accuracy.

The approach outperforms existing baselines in robotic manipulation scenarios.

Abstract

Robot behavior policies trained via imitation learning are prone to failure under conditions that deviate from their training data. Thus, algorithms that monitor learned policies at test time and provide early warnings of failure are necessary to facilitate scalable deployment. We propose Sentinel, a runtime monitoring framework that splits the detection of failures into two complementary categories: 1) Erratic failures, which we detect using statistical measures of temporal action consistency, and 2) task progression failures, where we use Vision Language Models (VLMs) to detect when the policy confidently and consistently takes actions that do not solve the task. Our approach has two key strengths. First, because learned policies exhibit diverse failure modes, combining complementary detectors leads to significantly higher accuracy at failure detection. Second, using a statistical temporal action consistency measure ensures that we quickly detect when multimodal, generative policies exhibit erratic behavior at negligible computational cost. In contrast, we only use VLMs to detect failure modes that are less time-sensitive. We demonstrate our approach in the context of diffusion policies trained on robotic mobile manipulation domains in both simulation and the real world. By unifying temporal consistency detection and VLM runtime monitoring, Sentinel detects 18% more failures than using either of the two detectors alone and significantly outperforms baselines, thus highlighting the importance of assigning specialized detectors to complementary categories of failure. Qualitative results are made available at https://sites.google.com/stanford.edu/sentinel.