Open-Set Domain Adaptation for Semantic Segmentation

TL;DR

This paper introduces a novel open-set domain adaptation method for semantic segmentation that effectively detects unknown classes in the target domain by boundary and shape-aware techniques, outperforming previous methods.

Contribution

The paper proposes BUS, a boundary and shape-aware open-set domain adaptation approach with a contrastive loss and domain mixing augmentation for improved unknown class detection.

Findings

BUS significantly improves unknown class detection accuracy.

The method outperforms previous approaches by a large margin.

Extensive experiments validate the effectiveness of the proposed techniques.

Abstract

Unsupervised domain adaptation (UDA) for semantic segmentation aims to transfer the pixel-wise knowledge from the labeled source domain to the unlabeled target domain. However, current UDA methods typically assume a shared label space between source and target, limiting their applicability in real-world scenarios where novel categories may emerge in the target domain. In this paper, we introduce Open-Set Domain Adaptation for Semantic Segmentation (OSDA-SS) for the first time, where the target domain includes unknown classes. We identify two major problems in the OSDA-SS scenario as follows: 1) the existing UDA methods struggle to predict the exact boundary of the unknown classes, and 2) they fail to accurately predict the shape of the unknown classes. To address these issues, we propose Boundary and Unknown Shape-Aware open-set domain adaptation, coined BUS. Our BUS can accurately discern the boundaries between known and unknown classes in a contrastive manner using a novel dilation-erosion-based contrastive loss. In addition, we propose OpenReMix, a new domain mixing augmentation method that guides our model to effectively learn domain and size-invariant features for improving the shape detection of the known and unknown classes. Through extensive experiments, we demonstrate that our proposed BUS effectively detects unknown classes in the challenging OSDA-SS scenario compared to the previous methods by a large margin. The code is available at https://github.com/KHU-AGI/BUS.