Domain Adaptive Semantic Segmentation via Regional Contrastive Consistency Regularization

TL;DR

This paper introduces RCCR, a novel end-to-end method for unsupervised domain adaptation in semantic segmentation that emphasizes regional consistency across domains using contrastive learning, momentum heads, and a memory bank.

Contribution

It proposes a region-wise contrastive loss with sampling strategies, momentum projection heads, and a memory bank to improve robustness in domain adaptive semantic segmentation.

Findings

Outperforms state-of-the-art on GTAV to Cityscapes

Effective regional consistency improves segmentation accuracy

Robust to environmental variations

Abstract

Unsupervised domain adaptation (UDA) for semantic segmentation has been well-studied in recent years. However, most existing works largely neglect the local regional consistency across different domains and are less robust to changes in outdoor environments. In this paper, we propose a novel and fully end-to-end trainable approach, called regional contrastive consistency regularization (RCCR) for domain adaptive semantic segmentation. Our core idea is to pull the similar regional features extracted from the same location of different images, i.e., the original image and augmented image, to be closer, and meanwhile push the features from the different locations of the two images to be separated. We innovatively propose a region-wise contrastive loss with two sampling strategies to realize effective regional consistency. Besides, we present momentum projection heads, where the teacher projection head is the exponential moving average of the student. Finally, a memory bank mechanism is designed to learn more robust and stable region-wise features under varying environments. Extensive experiments on two common UDA benchmarks, i.e., GTAV to Cityscapes and SYNTHIA to Cityscapes, demonstrate that our approach outperforms the state-of-the-art methods.