Uncertainty-Gated Region-Level Retrieval for Robust Semantic Segmentation

TL;DR

This paper introduces an uncertainty-gated region-level retrieval method that enhances the robustness and accuracy of semantic segmentation in outdoor scenes, especially under domain shifts, while significantly reducing computational costs.

Contribution

The paper presents a novel uncertainty-gated retrieval mechanism for semantic segmentation that improves accuracy and calibration, and reduces retrieval costs under challenging conditions.

Findings

11.3% increase in mean intersection-over-union

87.5% reduction in retrieval cost

retrieves only 12.5% of regions compared to baseline

Abstract

Semantic segmentation of outdoor street scenes plays a key role in applications such as autonomous driving, mobile robotics, and assistive technology for visually-impaired pedestrians. For these applications, accurately distinguishing between key surfaces and objects such as roads, sidewalks, vehicles, and pedestrians is essential for maintaining safety and minimizing risks. Semantic segmentation must be robust to different environments, lighting and weather conditions, and sensor noise, while being performed in real-time. We propose a region-level, uncertainty-gated retrieval mechanism that improves segmentation accuracy and calibration under domain shift. Our best method achieves an 11.3% increase in mean intersection-over-union while reducing retrieval cost by 87.5%, retrieving for only 12.5% of regions compared to 100% for always-on baseline.