Self-training via Metric Learning for Source-Free Domain Adaptation of Semantic Segmentation

TL;DR

This paper introduces a source-free domain adaptation method for semantic segmentation that leverages a mean-teacher model and proxy-based metric learning to improve pseudo-label reliability without thresholding, outperforming existing methods.

Contribution

It proposes a novel source-free adaptation approach using metric learning to assess pseudo-label reliability, enhancing self-training effectiveness.

Findings

Outperforms state-of-the-art methods in synthetic-to-real adaptation

Effective pseudo-label weighting improves segmentation accuracy

Demonstrates robustness in cross-city adaptation scenarios

Abstract

Unsupervised source-free domain adaptation methods aim to train a model for the target domain utilizing a pretrained source-domain model and unlabeled target-domain data, particularly when accessibility to source data is restricted due to intellectual property or privacy concerns. Traditional methods usually use self-training with pseudo-labeling, which is often subjected to thresholding based on prediction confidence. However, such thresholding limits the effectiveness of self-training due to insufficient supervision. This issue becomes more severe in a source-free setting, where supervision comes solely from the predictions of the pre-trained source model. In this study, we propose a novel approach by incorporating a mean-teacher model, wherein the student network is trained using all predictions from the teacher network. Instead of employing thresholding on predictions, we introduce a method to weight the gradients calculated from pseudo-labels based on the reliability of the teacher's predictions. To assess reliability, we introduce a novel approach using proxy-based metric learning. Our method is evaluated in synthetic-to-real and cross-city scenarios, demonstrating superior performance compared to existing state-of-the-art methods.