MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation

TL;DR

MICDrop introduces a novel complementary dropout strategy that masks image and depth features inversely to enhance domain-adaptive semantic segmentation, leveraging geometric cues for improved boundary delineation and overall accuracy.

Contribution

The paper proposes MICDrop, a new masking technique that effectively combines image and depth features for better unsupervised domain adaptation in segmentation tasks.

Findings

Consistently improves UDA segmentation performance across benchmarks.

Achieves new state-of-the-art results on standard UDA datasets.

Enhances boundary accuracy by leveraging depth discontinuities.

Abstract

Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain, e.g., synthetic data, and an unlabeled target domain. We observe that current UDA methods show inferior results on fine structures and tend to oversegment objects with ambiguous appearance. To address these shortcomings, we propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that naively incorporating depth into current UDA methods does not fully exploit the potential of this complementary information. To this end, we present MICDrop, which learns a joint feature representation by masking image encoder features while inversely masking depth encoder features. With this simple yet effective complementary masking strategy, we enforce the use of both modalities when learning the joint feature representation. To aid this process, we propose a feature fusion module to improve both global as well as local information sharing while being robust to errors in the depth predictions. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks, obtaining new state-of-the-art performances.