Unsupervised domain adaptation via coarse-to-fine feature alignment method using contrastive learning

TL;DR

This paper introduces CFContra, a novel unsupervised domain adaptation method that uses coarse-to-fine contrastive feature alignment to improve performance, especially in semantic segmentation tasks, with efficient memory usage.

Contribution

It proposes a new contrastive learning-based coarse-to-fine feature alignment method for UDA, enhancing class-wise feature alignment and efficiency in semantic segmentation.

Findings

Boosts mIOU by 3.5 on GTA5 to Cityscapes dataset

Effective class-wise feature alignment improves adaptation performance

Memory-efficient contrastive loss implementation

Abstract

Previous feature alignment methods in Unsupervised domain adaptation(UDA) mostly only align global features without considering the mismatch between class-wise features. In this work, we propose a new coarse-to-fine feature alignment method using contrastive learning called CFContra. It draws class-wise features closer than coarse feature alignment or class-wise feature alignment only, therefore improves the model's performance to a great extent. We build it upon one of the most effective methods of UDA called entropy minimization to further improve performance. In particular, to prevent excessive memory occupation when applying contrastive loss in semantic segmentation, we devise a new way to build and update the memory bank. In this way, we make the algorithm more efficient and viable with limited memory. Extensive experiments show the effectiveness of our method and model trained on the GTA5 to Cityscapes dataset has boost mIOU by 3.5 compared to the MinEnt algorithm. Our code will be publicly available.