Unsupervised Domain Adaptation for Semantic Segmentation with Pseudo Label Self-Refinement

TL;DR

This paper introduces a pseudo-label refinement network to improve unsupervised domain adaptation for semantic segmentation, effectively reducing noise in pseudo-labels and enhancing model robustness across different domain shifts.

Contribution

The paper proposes an auxiliary pseudo-label refinement network (PRN) that online refines pseudo labels and identifies noisy labels, improving the robustness of self-training in UDA for segmentation.

Findings

Consistently outperforms previous SOTA methods on benchmark datasets.

Significantly reduces pseudo-label noise during training.

Enhances model performance across multiple domain shifts.

Abstract

Deep learning-based solutions for semantic segmentation suffer from significant performance degradation when tested on data with different characteristics than what was used during the training. Adapting the models using annotated data from the new domain is not always practical. Unsupervised Domain Adaptation (UDA) approaches are crucial in deploying these models in the actual operating conditions. Recent state-of-the-art (SOTA) UDA methods employ a teacher-student self-training approach, where a teacher model is used to generate pseudo-labels for the new data which in turn guide the training process of the student model. Though this approach has seen a lot of success, it suffers from the issue of noisy pseudo-labels being propagated in the training process. To address this issue, we propose an auxiliary pseudo-label refinement network (PRN) for online refining of the pseudo labels and also localizing the pixels whose predicted labels are likely to be noisy. Being able to improve the quality of pseudo labels and select highly reliable ones, PRN helps self-training of segmentation models to be robust against pseudo label noise propagation during different stages of adaptation. We evaluate our approach on benchmark datasets with three different domain shifts, and our approach consistently performs significantly better than the previous state-of-the-art methods.