Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation

TL;DR

This paper introduces Stable Neighbor Denoising (SND), a novel method for source-free unsupervised domain adaptation in semantic segmentation that effectively reduces pseudo-label noise by leveraging stable neighbor samples, improving adaptation accuracy.

Contribution

The paper proposes a universal denoising mechanism using stable neighbor samples and bi-level learning, addressing class, domain, and confirmation biases in SFUDA for segmentation.

Findings

SND outperforms state-of-the-art methods in various SFUDA segmentation tasks.

SND can be integrated with other approaches for further improvements.

SND is applicable without specific segmentor structures.

Abstract

We study source-free unsupervised domain adaptation (SFUDA) for semantic segmentation, which aims to adapt a source-trained model to the target domain without accessing the source data. Many works have been proposed to address this challenging problem, among which uncertainty-based self-training is a predominant approach. However, without comprehensive denoising mechanisms, they still largely fall into biased estimates when dealing with different domains and confirmation bias. In this paper, we observe that pseudo-label noise is mainly contained in unstable samples in which the predictions of most pixels undergo significant variations during self-training. Inspired by this, we propose a novel mechanism to denoise unstable samples with stable ones. Specifically, we introduce the Stable Neighbor Denoising (SND) approach, which effectively discovers highly correlated stable and unstable samples by nearest neighbor retrieval and guides the reliable optimization of unstable samples by bi-level learning. Moreover, we compensate for the stable set by object-level object paste, which can further eliminate the bias caused by less learned classes. Our SND enjoys two advantages. First, SND does not require a specific segmentor structure, endowing its universality. Second, SND simultaneously addresses the issues of class, domain, and confirmation biases during adaptation, ensuring its effectiveness. Extensive experiments show that SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings. In addition, SND can be easily integrated with other approaches, obtaining further improvements.