Fool Me Once: Robust Selective Segmentation via Out-of-Distribution Detection with Contrastive Learning

TL;DR

This paper introduces a method that combines segmentation and out-of-distribution detection using contrastive learning to improve robustness in unknown scene regions, validated on real-world driving data.

Contribution

It proposes a novel contrastive learning approach with data augmentation for joint segmentation and OoD detection, enhancing robustness against unknown classes.

Findings

Increased segmentation accuracy by 0.2 IoU on real-world driving scenes.

Effective detection of unknown regions improves trustworthiness of segmentation.

Outperforms existing methods in handling unknown scene elements.

Abstract

In this work, we train a network to simultaneously perform segmentation and pixel-wise Out-of-Distribution (OoD) detection, such that the segmentation of unknown regions of scenes can be rejected. This is made possible by leveraging an OoD dataset with a novel contrastive objective and data augmentation scheme. By combining data including unknown classes in the training data, a more robust feature representation can be learned with known classes represented distinctly from those unknown. When presented with unknown classes or conditions, many current approaches for segmentation frequently exhibit high confidence in their inaccurate segmentations and cannot be trusted in many operational environments. We validate our system on a real-world dataset of unusual driving scenes, and show that by selectively segmenting scenes based on what is predicted as OoD, we can increase the segmentation accuracy by an IoU of 0.2 with respect to alternative techniques.