Domain Adaptation for Semantic Segmentation with Maximum Squares Loss

TL;DR

This paper introduces the maximum squares loss for unsupervised domain adaptation in semantic segmentation, effectively balancing easy and hard target samples and addressing class imbalance, leading to improved adaptation performance.

Contribution

It proposes the maximum squares loss and image-wise weighting to enhance semi-supervised domain adaptation for semantic segmentation.

Findings

Effective in synthetic-to-real adaptation

Improves performance in cross-city adaptation

Balances gradient contributions from target samples

Abstract

Deep neural networks for semantic segmentation always require a large number of samples with pixel-level labels, which becomes the major difficulty in their real-world applications. To reduce the labeling cost, unsupervised domain adaptation (UDA) approaches are proposed to transfer knowledge from labeled synthesized datasets to unlabeled real-world datasets. Recently, some semi-supervised learning methods have been applied to UDA and achieved state-of-the-art performance. One of the most popular approaches in semi-supervised learning is the entropy minimization method. However, when applying the entropy minimization to UDA for semantic segmentation, the gradient of the entropy is biased towards samples that are easy to transfer. To balance the gradient of well-classified target samples, we propose the maximum squares loss. Our maximum squares loss prevents the training process being dominated by easy-to-transfer samples in the target domain. Besides, we introduce the image-wise weighting ratio to alleviate the class imbalance in the unlabeled target domain. Both synthetic-to-real and cross-city adaptation experiments demonstrate the effectiveness of our proposed approach. The code is released at https://github. com/ZJULearning/MaxSquareLoss.