Introspective Perception: Learning to Predict Failures in Vision Systems

TL;DR

This paper introduces a framework for perception systems to self-assess their reliability by predicting failures directly from sensor data, enhancing autonomous decision-making in complex environments.

Contribution

It proposes a generic introspective model for vision systems to predict failures, improving situational awareness in autonomous MAV flight.

Findings

Effective failure prediction in outdoor MAV vision systems

Handles uncertain and ambiguous situations well

Enhances autonomous decision reliability

Abstract

As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical. This motivates the need to build systems that have situational awareness to assess how qualified they are at that moment to make a decision. We call this self-evaluating capability as introspection. In this paper, we take a small step in this direction and propose a generic framework for introspective behavior in perception systems. Our goal is to learn a model to reliably predict failures in a given system, with respect to a task, directly from input sensor data. We present this in the context of vision-based autonomous MAV flight in outdoor natural environments, and show that it effectively handles uncertain situations.