Co-Training for Unsupervised Domain Adaptation of Semantic Segmentation Models

TL;DR

This paper introduces a co-training method for unsupervised domain adaptation in semantic segmentation, leveraging self-training and model collaboration to improve performance on synthetic-to-real datasets without modifying loss functions.

Contribution

The proposed co-training procedure is a novel approach that enhances domain adaptation for semantic segmentation by mutual model improvement without explicit feature alignment.

Findings

Achieves 13-26 mIoU improvement over baselines.

Establishes new state-of-the-art results on synthetic-to-real datasets.

Effective without modifying loss functions or explicit feature alignment.

Abstract

Semantic image segmentation is a central and challenging task in autonomous driving, addressed by training deep models. Since this training draws to a curse of human-based image labeling, using synthetic images with automatically generated labels together with unlabeled real-world images is a promising alternative. This implies to address an unsupervised domain adaptation (UDA) problem. In this paper, we propose a new co-training procedure for synth-to-real UDA of semantic segmentation models. It consists of a self-training stage, which provides two domain-adapted models, and a model collaboration loop for the mutual improvement of these two models. These models are then used to provide the final semantic segmentation labels (pseudo-labels) for the real-world images. The overall procedure treats the deep models as black boxes and drives their collaboration at the level of pseudo-labeled target images, i.e., neither modifying loss functions is required, nor explicit feature alignment. We test our proposal on standard synthetic and real-world datasets for on-board semantic segmentation. Our procedure shows improvements ranging from ~13 to ~26 mIoU points over baselines, so establishing new state-of-the-art results.