Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings

TL;DR

This paper introduces a novel unsupervised domain adaptation method for semantic segmentation that uses feature clustering, orthogonality, and sparsity losses to improve transfer from synthetic to real data, achieving state-of-the-art results.

Contribution

It proposes a new UDA approach with orthogonal and clustered embeddings, enhancing discriminative feature clustering for semantic segmentation.

Findings

Achieves state-of-the-art performance in synthetic-to-real domain adaptation.

Effective feature clustering improves semantic segmentation accuracy.

Orthogonality and sparsity losses regularize feature space structure.

Abstract

Deep learning frameworks allowed for a remarkable advancement in semantic segmentation, but the data hungry nature of convolutional networks has rapidly raised the demand for adaptation techniques able to transfer learned knowledge from label-abundant domains to unlabeled ones. In this paper we propose an effective Unsupervised Domain Adaptation (UDA) strategy, based on a feature clustering method that captures the different semantic modes of the feature distribution and groups features of the same class into tight and well-separated clusters. Furthermore, we introduce two novel learning objectives to enhance the discriminative clustering performance: an orthogonality loss forces spaced out individual representations to be orthogonal, while a sparsity loss reduces class-wise the number of active feature channels. The joint effect of these modules is to regularize the structure of the feature space. Extensive evaluations in the synthetic-to-real scenario show that we achieve state-of-the-art performance.