Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation

TL;DR

This paper introduces a novel approach that integrates self-supervised monocular depth estimation into unsupervised domain adaptation for semantic segmentation, significantly enhancing performance in autonomous driving scenarios.

Contribution

It proposes a plug-in depth component and a self-training strategy that leverage geometric cues and synthetic samples to improve UDA for semantic segmentation.

Findings

Achieves 58.8 mIoU on GTA5->CS benchmark

Enhances UDA performance with depth-based geometric cues

Utilizes depth to generate diverse training samples

Abstract

Although recent semantic segmentation methods have made remarkable progress, they still rely on large amounts of annotated training data, which are often infeasible to collect in the autonomous driving scenario. Previous works usually tackle this issue with Unsupervised Domain Adaptation (UDA), which entails training a network on synthetic images and applying the model to real ones while minimizing the discrepancy between the two domains. Yet, these techniques do not consider additional information that may be obtained from other tasks. Differently, we propose to exploit self-supervised monocular depth estimation to improve UDA for semantic segmentation. On one hand, we deploy depth to realize a plug-in component which can inject complementary geometric cues into any existing UDA method. We further rely on depth to generate a large and varied set of samples to Self-Train the final model. Our whole proposal allows for achieving state-of-the-art performance (58.8 mIoU) in the GTA5->CS benchmark benchmark. Code is available at https://github.com/CVLAB-Unibo/d4-dbst.