Multi-Scale Foreground-Background Confidence for Out-of-Distribution Segmentation

TL;DR

This paper introduces a multi-scale confidence-based method for out-of-distribution segmentation that leverages foreground-background models' confidence scores to detect unknown objects across various sizes, improving over existing benchmarks.

Contribution

The paper proposes a novel multi-scale confidence aggregation approach for OOD segmentation using foreground-background models, which are more flexible than semantic segmentation models.

Findings

Improved OOD segmentation performance on SegmentMeIfYouCan benchmark.

Effective detection of objects of various sizes.

Outperforms comparable baseline methods.

Abstract

Deep neural networks have shown outstanding performance in computer vision tasks such as semantic segmentation and have defined the state-of-the-art. However, these segmentation models are trained on a closed and predefined set of semantic classes, which leads to significant prediction failures in open-world scenarios on unknown objects. As this behavior prevents the application in safety-critical applications such as automated driving, the detection and segmentation of these objects from outside their predefined semantic space (out-of-distribution (OOD) objects) is of the utmost importance. In this work, we present a multi-scale OOD segmentation method that exploits the confidence information of a foreground-background segmentation model. While semantic segmentation models are trained on specific classes, this restriction does not apply to foreground-background methods making them suitable for OOD segmentation. We consider the per pixel confidence score of the model prediction which is close to 1 for a pixel in a foreground object. By aggregating these confidence values for different sized patches, objects of various sizes can be identified in a single image. Our experiments show improved performance of our method in OOD segmentation compared to comparable baselines in the SegmentMeIfYouCan benchmark.