Class-Balanced Pixel-Level Self-Labeling for Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces CPSL, a novel method for domain adaptive semantic segmentation that leverages pixel clustering and class distribution alignment to improve pseudo labels and performance, especially on long-tailed categories.

Contribution

The paper proposes a class-balanced pixel-level self-labeling approach that directly explores target domain pixel distributions and rectifies pseudo labels without extra training rounds.

Findings

Significant performance improvement over state-of-the-art methods.

Effective handling of long-tailed category distributions.

Online clustering and label rectification enhance pseudo label quality.

Abstract

Domain adaptive semantic segmentation aims to learn a model with the supervision of source domain data, and produce satisfactory dense predictions on unlabeled target domain. One popular solution to this challenging task is self-training, which selects high-scoring predictions on target samples as pseudo labels for training. However, the produced pseudo labels often contain much noise because the model is biased to source domain as well as majority categories. To address the above issues, we propose to directly explore the intrinsic pixel distributions of target domain data, instead of heavily relying on the source domain. Specifically, we simultaneously cluster pixels and rectify pseudo labels with the obtained cluster assignments. This process is done in an online fashion so that pseudo labels could co-evolve with the segmentation model without extra training rounds. To overcome the class imbalance problem on long-tailed categories, we employ a distribution alignment technique to enforce the marginal class distribution of cluster assignments to be close to that of pseudo labels. The proposed method, namely Class-balanced Pixel-level Self-Labeling (CPSL), improves the segmentation performance on target domain over state-of-the-arts by a large margin, especially on long-tailed categories.