PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning

TL;DR

PiPa++ is a unified self-supervised learning framework that improves domain adaptive semantic segmentation by promoting discriminative, robust, and temporally consistent pixel-wise features, applicable to both image and video data.

Contribution

It introduces a unified framework that combines contrastive learning with task-specific sampling to enhance feature discrimination and temporal consistency in domain adaptation.

Findings

Effective on both image and video domain adaptation benchmarks.

Compatible with other UDA methods for improved performance.

No extra parameters needed for integration.

Abstract

Unsupervised domain adaptive segmentation aims to improve the segmentation accuracy of models on target domains without relying on labeled data from those domains. This approach is crucial when labeled target domain data is scarce or unavailable. It seeks to align the feature representations of the source domain (where labeled data is available) and the target domain (where only unlabeled data is present), thus enabling the model to generalize well to the target domain. Current image- and video-level domain adaptation have been addressed using different and specialized frameworks, training strategies and optimizations despite their underlying connections. In this paper, we propose a unified framework PiPa++, which leverages the core idea of ``comparing'' to (1) explicitly encourage learning of discriminative pixel-wise features with intraclass compactness and inter-class separability, (2) promote the robust feature learning of the identical patch against different contexts or fluctuations, and (3) enable the learning of temporal continuity under dynamic environments. With the designed task-smart contrastive sampling strategy, PiPa++ enables the mining of more informative training samples according to the task demand. Extensive experiments demonstrate the effectiveness of our method on both image-level and video-level domain adaption benchmarks. Moreover, the proposed method is compatible with other UDA approaches to further improve the performance without introducing extra parameters.