Efficacy of Pixel-Level OOD Detection for Semantic Segmentation

TL;DR

This paper investigates pixel-level out-of-distribution detection methods for semantic segmentation, adapting classification-based techniques, and finds that their performance significantly drops compared to image-level detection, highlighting challenges in localising unusual objects.

Contribution

It introduces a new task for pixel-level OOD detection in segmentation, adapts existing methods, and evaluates them on new datasets with a novel metric.

Findings

Performance of classification-based OOD methods drops at pixel level

All methods perform worse than their image-level counterparts

New datasets and metrics for pixel-level OOD detection are proposed

Abstract

The detection of out of distribution samples for image classification has been widely researched. Safety critical applications, such as autonomous driving, would benefit from the ability to localise the unusual objects causing the image to be out of distribution. This paper adapts state-of-the-art methods for detecting out of distribution images for image classification to the new task of detecting out of distribution pixels, which can localise the unusual objects. It further experimentally compares the adapted methods on two new datasets derived from existing semantic segmentation datasets using PSPNet and DeeplabV3+ architectures, as well as proposing a new metric for the task. The evaluation shows that the performance ranking of the compared methods does not transfer to the new task and every method performs significantly worse than their image-level counterparts.