Leveraging Text-Driven Semantic Variation for Robust OOD Segmentation

TL;DR

This paper introduces a novel vision-language approach for out-of-distribution segmentation in autonomous driving, leveraging semantic cues to improve detection of unseen objects and enhance safety.

Contribution

It proposes a new Text-Driven OOD Segmentation model combining vision-language encoding, semantic prompts, and augmentation, achieving state-of-the-art results in diverse datasets.

Findings

Achieves state-of-the-art performance on Fishyscapes, Segment-Me-If-You-Can, and Road Anomaly datasets.

Effectively generalizes to unseen objects in complex driving scenarios.

Demonstrates robustness and reliability improvements in OOD segmentation.

Abstract

In autonomous driving and robotics, ensuring road safety and reliable decision-making critically depends on out-of-distribution (OOD) segmentation. While numerous methods have been proposed to detect anomalous objects on the road, leveraging the vision-language space-which provides rich linguistic knowledge-remains an underexplored field. We hypothesize that incorporating these linguistic cues can be especially beneficial in the complex contexts found in real-world autonomous driving scenarios. To this end, we present a novel approach that trains a Text-Driven OOD Segmentation model to learn a semantically diverse set of objects in the vision-language space. Concretely, our approach combines a vision-language model's encoder with a transformer decoder, employs Distance-Based OOD prompts located at varying semantic distances from in-distribution (ID) classes, and utilizes OOD Semantic Augmentation for OOD representations. By aligning visual and textual information, our approach effectively generalizes to unseen objects and provides robust OOD segmentation in diverse driving environments. We conduct extensive experiments on publicly available OOD segmentation datasets such as Fishyscapes, Segment-Me-If-You-Can, and Road Anomaly datasets, demonstrating that our approach achieves state-of-the-art performance across both pixel-level and object-level evaluations. This result underscores the potential of vision-language-based OOD segmentation to bolster the safety and reliability of future autonomous driving systems.