World Model Failure Classification and Anomaly Detection for Autonomous Inspection

TL;DR

This paper presents a hybrid failure classification and anomaly detection framework for autonomous inspection robots, achieving over 90% accuracy and enabling real-time, early failure detection using a world model backbone and conformal prediction.

Contribution

It introduces a novel distribution-free, policy-agnostic framework combining failure classification with anomaly detection for autonomous inspection tasks.

Findings

Achieves over 90% accuracy in classifying success, failure, and anomalies.

Detects failures earlier than human observers in real-time.

Demonstrates deployment on Boston Dynamics Spot robot.

Abstract

Autonomous inspection robots for monitoring industrial sites can reduce costs and risks associated with human-led inspection. However, accurate readings can be challenging due to occlusions, limited viewpoints, or unexpected environmental conditions. We propose a hybrid framework that combines supervised failure classification with anomaly detection, enabling classification of inspection tasks as a success, known failure, or anomaly (i.e., out-of-distribution) case. Our approach uses a world model backbone with compressed video inputs. This policy-agnostic, distribution-free framework determines classifications based on two decision functions set by conformal prediction (CP) thresholds before a human observer does. We evaluate the framework on gauge inspection feeds collected from office and industrial sites and demonstrate real-time deployment on a Boston Dynamics Spot. Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases, with classifications occurring earlier than a human observer. These results highlight the potential for robust, anticipatory failure detection in autonomous inspection tasks or as a feedback signal for model training to assess and improve the quality of training data. Project website: https://autoinspection-classification.github.io