More Separable and Easier to Segment: A Cluster Alignment Method for Cross-Domain Semantic Segmentation

TL;DR

This paper introduces a novel domain adaptation method for semantic segmentation that clusters pixels to maintain semantic associations and uses a graph-based loss to adapt classifiers, achieving state-of-the-art results.

Contribution

It proposes a cluster alignment approach with prototype clustering and a normalized cut loss to improve domain adaptation in semantic segmentation.

Findings

Achieved new state-of-the-art performance on GTA5 to Cityscapes.

Demonstrated better preservation of pixel associations during adaptation.

Enhanced classifier adaptability to target domain.

Abstract

Feature alignment between domains is one of the mainstream methods for Unsupervised Domain Adaptation (UDA) semantic segmentation. Existing feature alignment methods for semantic segmentation learn domain-invariant features by adversarial training to reduce domain discrepancy, but they have two limits: 1) associations among pixels are not maintained, 2) the classifier trained on the source domain couldn't adapted well to the target. In this paper, we propose a new UDA semantic segmentation approach based on domain closeness assumption to alleviate the above problems. Specifically, a prototype clustering strategy is applied to cluster pixels with the same semantic, which will better maintain associations among target domain pixels during the feature alignment. After clustering, to make the classifier more adaptive, a normalized cut loss based on the affinity graph of the target domain is utilized, which will make the decision boundary target-specific. Sufficient experiments conducted on GTA5 $\rightarrow$ Cityscapes and SYNTHIA $\rightarrow$ Cityscapes proved the effectiveness of our method, which illustrated that our results achieved the new state-of-the-art.