DAWA: Dynamic Ambiguity-Wise Adaptation for Real-Time Domain Adaptive Semantic Segmentation

TL;DR

DAWA introduces a real-time test-time domain adaptation framework for semantic segmentation that dynamically detects domain shifts and mitigates ambiguities, achieving high efficiency and improved accuracy.

Contribution

It proposes novel dynamic strategies, DAP Mask and DAC Mix, to adaptively handle domain shifts and semantic ambiguities during real-time inference.

Findings

Outperforms state-of-the-art TTDA methods on benchmark datasets.

Maintains real-time inference speed of approximately 40 FPS.

Effectively reduces semantic ambiguities and error accumulation.

Abstract

Test-time domain adaption (TTDA) for semantic segmentation aims to adapt a segmentation model trained on a source domain to a target domain for inference on-the-fly, where both efficiency and effectiveness are critical. However, existing TTDA methods either rely on costly frame-wise optimization or assume unrealistic domain shifts, resulting in poor adaptation efficiency and continuous semantic ambiguities. To address these challenges, we propose a real-time framework for TTDA semantic segmentation, called Dynamic Ambiguity-Wise Adaptation (DAWA), which adaptively detects domain shifts and dynamically adjusts the learning strategies to mitigate continuous ambiguities in the test time. Specifically, we introduce the Dynamic Ambiguous Patch Mask (DAP Mask) strategy, which dynamically identifies and masks highly disturbed regions to prevent error accumulation in ambiguous classes. Furthermore, we present the Dynamic Ambiguous Class Mix (DAC Mix) strategy that leverages vision-language models to group semantically similar classes and augment the target domain with a meta-ambiguous class buffer. Extensive experiments on widely used TTDA benchmarks demonstrate that DAWA consistently outperforms state-of-the-art methods, while maintaining real-time inference speeds of approximately 40 FPS.