Balanced Learning for Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces BLDA, a novel method for unsupervised domain adaptation in semantic segmentation that balances class predictions by analyzing logits distributions, leading to improved performance especially on under-predicted classes.

Contribution

BLDA is a new approach that directly assesses and alleviates class bias in UDA for semantic segmentation without prior distribution knowledge.

Findings

BLDA improves segmentation performance on benchmark datasets.

BLDA effectively balances class predictions during domain adaptation.

BLDA enhances under-predicted class accuracy.

Abstract

Unsupervised domain adaptation (UDA) for semantic segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Despite the effectiveness of self-training techniques in UDA, they struggle to learn each class in a balanced manner due to inherent class imbalance and distribution shift in both data and label space between domains. To address this issue, we propose Balanced Learning for Domain Adaptation (BLDA), a novel approach to directly assess and alleviate class bias without requiring prior knowledge about the distribution shift. First, we identify over-predicted and under-predicted classes by analyzing the distribution of predicted logits. Subsequently, we introduce a post-hoc approach to align the logits distributions across different classes using shared anchor distributions. To further consider the network's need to generate unbiased pseudo-labels during self-training, we estimate logits distributions online and incorporate logits correction terms into the loss function. Moreover, we leverage the resulting cumulative density as domain-shared structural knowledge to connect the source and target domains. Extensive experiments on two standard UDA semantic segmentation benchmarks demonstrate that BLDA consistently improves performance, especially for under-predicted classes, when integrated into various existing methods. Code is available at https://github.com/Woof6/BLDA.