SCMix: Stochastic Compound Mixing for Open Compound Domain Adaptation in Semantic Segmentation

TL;DR

SCMix introduces a stochastic augmentation strategy for open compound domain adaptation in semantic segmentation, effectively reducing distribution divergence and outperforming existing methods by leveraging theoretical insights and transformer architecture.

Contribution

The paper proposes SCMix, a novel stochastic augmentation method for OCDA, with theoretical analysis and superior empirical performance over prior approaches.

Findings

SCMix reduces empirical risk in OCDA semantic segmentation tasks.

Theoretical analysis shows SCMix's superiority over previous methods.

Combining SCMix with transformers yields state-of-the-art results.

Abstract

Open compound domain adaptation (OCDA) aims to transfer knowledge from a labeled source domain to a mix of unlabeled homogeneous compound target domains while generalizing to open unseen domains. Existing OCDA methods solve the intra-domain gaps by a divide-and-conquer strategy, which divides the problem into several individual and parallel domain adaptation (DA) tasks. Such approaches often contain multiple sub-networks or stages, which may constrain the model's performance. In this work, starting from the general DA theory, we establish the generalization bound for the setting of OCDA. Built upon this, we argue that conventional OCDA approaches may substantially underestimate the inherent variance inside the compound target domains for model generalization. We subsequently present Stochastic Compound Mixing (SCMix), an augmentation strategy with the primary objective of mitigating the divergence between source and mixed target distributions. We provide theoretical analysis to substantiate the superiority of SCMix and prove that the previous methods are sub-groups of our methods. Extensive experiments show that our method attains a lower empirical risk on OCDA semantic segmentation tasks, thus supporting our theories. Combining the transformer architecture, SCMix achieves a notable performance boost compared to the SoTA results.