KRADA: Known-region-aware Domain Alignment for Open-set Domain Adaptation in Semantic Segmentation

TL;DR

KRADA introduces a novel framework for open-set domain adaptation in semantic segmentation, enabling models to identify unknown classes and align known class distributions across labeled and unlabeled open-world images.

Contribution

The paper proposes KRADA, an end-to-end learning framework that distinguishes unknown classes and aligns known class distributions in open-set domain adaptation for segmentation.

Findings

Effective on synthetic and COVID-19 segmentation tasks.

Improves unknown class detection and known class alignment.

Demonstrates robustness in open-world scenarios.

Abstract

In semantic segmentation, we aim to train a pixel-level classifier to assign category labels to all pixels in an image, where labeled training images and unlabeled test images are from the same distribution and share the same label set. However, in an open world, the unlabeled test images probably contain unknown categories and have different distributions from the labeled images. Hence, in this paper, we consider a new, more realistic, and more challenging problem setting where the pixel-level classifier has to be trained with labeled images and unlabeled open-world images -- we name it open-set domain adaptation segmentation (OSDAS). In OSDAS, the trained classifier is expected to identify unknown-class pixels and classify known-class pixels well. To solve OSDAS, we first investigate which distribution that unknown-class pixels obey. Then, motivated by the goodness-of-fit test, we use statistical measurements to show how a pixel fits the distribution of an unknown class and select highly-fitted pixels to form the unknown region in each test image. Eventually, we propose an end-to-end learning framework, known-region-aware domain alignment (KRADA), to distinguish unknown classes while aligning the distributions of known classes in labeled and unlabeled open-world images. The effectiveness of KRADA has been verified on two synthetic tasks and one COVID-19 segmentation task.