A Metacognitive Approach to Out-of-Distribution Detection for Segmentation

TL;DR

This paper presents a lightweight, real-time metacognitive module that improves out-of-distribution detection in semantic segmentation by leveraging uncertainty measures and synthetic OOD data generation, achieving state-of-the-art results.

Contribution

It introduces a novel metacognitive approach combining entropy-based uncertainty, spatial context, and synthetic OOD data generation for enhanced segmentation OOD detection.

Findings

Achieves state-of-the-art OOD detection performance on benchmarks.

Enables real-time pixel-wise OOD detection in segmentation.

Improves model robustness without needing access to real OOD data.

Abstract

Despite outstanding semantic scene segmentation in closed-worlds, deep neural networks segment novel instances poorly, which is required for autonomous agents acting in an open world. To improve out-of-distribution (OOD) detection for segmentation, we introduce a metacognitive approach in the form of a lightweight module that leverages entropy measures, segmentation predictions, and spatial context to characterize the segmentation model's uncertainty and detect pixel-wise OOD data in real-time. Additionally, our approach incorporates a novel method of generating synthetic OOD data in context with in-distribution data, which we use to fine-tune existing segmentation models with maximum entropy training. This further improves the metacognitive module's performance without requiring access to OOD data while enabling compatibility with established pre-trained models. Our resulting approach can reliably detect OOD instances in a scene, as shown by state-of-the-art performance on OOD detection for semantic segmentation benchmarks.