Maximizing Cosine Similarity Between Spatial Features for Unsupervised Domain Adaptation in Semantic Segmentation

TL;DR

This paper introduces a novel unsupervised domain adaptation method for semantic segmentation that maximizes cosine similarity between source and target features, improving domain alignment at the feature level.

Contribution

It proposes a cosine similarity maximization approach with a class-wise feature dictionary to better align source and target domain features for semantic segmentation.

Findings

Improves performance on GTA5 to Cityscapes adaptation

Enhances SYNTHIA to Cityscapes segmentation accuracy

Demonstrates effectiveness through extensive experiments

Abstract

We propose a novel method that tackles the problem of unsupervised domain adaptation for semantic segmentation by maximizing the cosine similarity between the source and the target domain at the feature level. A segmentation network mainly consists of two parts, a feature extractor and a classification head. We expect that if we can make the two domains have small domain gap at the feature level, they would also have small domain discrepancy at the classification head. Our method computes a cosine similarity matrix between the source feature map and the target feature map, then we maximize the elements exceeding a threshold to guide the target features to have high similarity with the most similar source feature. Moreover, we use a class-wise source feature dictionary which stores the latest features of the source domain to prevent the unmatching problem when computing the cosine similarity matrix and be able to compare a target feature with various source features from various images. Through extensive experiments, we verify that our method gains performance on two unsupervised domain adaptation tasks (GTA5$\to$ Cityscaspes and SYNTHIA$\to$ Cityscapes).