IntraStyler: Exemplar-based Style Synthesis for Cross-modality Domain Adaptation

TL;DR

IntraStyler is a novel exemplar-based style synthesis method that captures diverse intra-domain styles without prior knowledge, improving cross-modality domain adaptation and segmentation performance.

Contribution

The paper introduces IntraStyler, a style synthesis approach that leverages exemplars and contrastive learning to generate diverse styles without pre-specified variations.

Findings

Effective style diversity without prior assumptions

Improved segmentation performance on CrossMoDA 2023

Controllable style synthesis demonstrated

Abstract

Image-level domain alignment is the de facto approach for unsupervised domain adaptation, where unpaired image translation is used to minimize the domain gap. Prior studies mainly focus on the domain shift between the source and target domains, whereas the intra-domain variability remains under-explored. To address the latter, an effective strategy is to diversify the styles of the synthetic target domain data during image translation. However, previous methods typically require intra-domain variations to be pre-specified for style synthesis, which may be impractical. In this paper, we propose an exemplar-based style synthesis method named IntraStyler, which can capture diverse intra-domain styles without any prior knowledge. Specifically, IntraStyler uses an exemplar image to guide the style synthesis such that the output style matches the exemplar style. To extract the style-only features, we introduce a style encoder to learn styles discriminatively based on contrastive learning. We evaluate the proposed method on the largest public dataset for cross-modality domain adaptation, CrossMoDA 2023. Our experiments show the efficacy of our method in controllable style synthesis and the benefits of diverse synthetic data for downstream segmentation. Code is available at https://github.com/han-liu/IntraStyler.