Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation

TL;DR

This paper introduces a novel hierarchical domain adaptation framework with class divergence guidance, improving cross-domain semantic segmentation by aligning features and differentiating categories, achieving state-of-the-art results.

Contribution

Proposes the ISIA mechanism and AIM strategy for better class-wise feature alignment and differentiation in unsupervised domain adaptation for segmentation.

Findings

Achieves state-of-the-art accuracy on GTA5 to Cityscapes and SYNTHIA to Cityscapes tasks.

Develops new datasets for remote sensing building and road segmentation.

Demonstrates improved generalization through hierarchical alignment at multiple levels.

Abstract

Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. Present UDA models focus on alleviating the domain shift by minimizing the feature discrepancy between the source domain and the target domain but usually ignore the class confusion problem. In this work, we propose an Inter-class Separation and Intra-class Aggregation (ISIA) mechanism. It encourages the cross-domain representative consistency between the same categories and differentiation among diverse categories. In this way, the features belonging to the same categories are aligned together and the confusable categories are separated. By measuring the align complexity of each category, we design an Adaptive-weighted Instance Matching (AIM) strategy to further optimize the instance-level adaptation. Based on our proposed methods, we also raise a hierarchical unsupervised domain adaptation framework for cross-domain semantic segmentation task. Through performing the image-level, feature-level, category-level and instance-level alignment, our method achieves a stronger generalization performance of the model from the source domain to the target domain. In two typical cross-domain semantic segmentation tasks, i.e., GTA5 to Cityscapes and SYNTHIA to Cityscapes, our method achieves the state-of-the-art segmentation accuracy. We also build two cross-domain semantic segmentation datasets based on the publicly available data, i.e., remote sensing building segmentation and road segmentation, for domain adaptive segmentation.