On Advantages of Mask-level Recognition for Outlier-aware Segmentation

TL;DR

This paper demonstrates that mask-level recognition significantly improves outlier-aware segmentation, especially in wild scenarios, by reducing false positives and enhancing performance over existing methods.

Contribution

It introduces a novel mask-level formulation of recognition uncertainty that outperforms baselines and sets new state-of-the-art results in outlier-aware semantic segmentation.

Findings

Mask-level predictions improve outlier detection.

Proposed uncertainty formulation reduces false positives.

Method enhances performance in panoptic segmentation.

Abstract

Most dense recognition approaches bring a separate decision in each particular pixel. These approaches deliver competitive performance in usual closed-set setups. However, important applications in the wild typically require strong performance in presence of outliers. We show that this demanding setup greatly benefit from mask-level predictions, even in the case of non-finetuned baseline models. Moreover, we propose an alternative formulation of dense recognition uncertainty that effectively reduces false positive responses at semantic borders. The proposed formulation produces a further improvement over a very strong baseline and sets the new state of the art in outlier-aware semantic segmentation with and without training on negative data. Our contributions also lead to performance improvement in a recent panoptic setup. In-depth experiments confirm that our approach succeeds due to implicit aggregation of pixel-level cues into mask-level predictions.