Domain Bridge for Unpaired Image-to-Image Translation and Unsupervised Domain Adaptation

TL;DR

This paper introduces the domain bridge method, leveraging web data and multimodal style sampling to improve unpaired image-to-image translation and unsupervised domain adaptation, enhancing realism and performance.

Contribution

It proposes a novel domain bridge technique, an online multimodal style-sampling strategy, and a self-supervised learning approach to advance unpaired image translation and domain adaptation.

Findings

Enhanced realism of generated images.

Achieved state-of-the-art performance in UDA.

Simplified approach compared to existing methods.

Abstract

Image-to-image translation architectures may have limited effectiveness in some circumstances. For example, while generating rainy scenarios, they may fail to model typical traits of rain as water drops, and this ultimately impacts the synthetic images realism. With our method, called domain bridge, web-crawled data are exploited to reduce the domain gap, leading to the inclusion of previously ignored elements in the generated images. We make use of a network for clear to rain translation trained with the domain bridge to extend our work to Unsupervised Domain Adaptation (UDA). In that context, we introduce an online multimodal style-sampling strategy, where image translation multimodality is exploited at training time to improve performances. Finally, a novel approach for self-supervised learning is presented, and used to further align the domains. With our contributions, we simultaneously increase the realism of the generated images, while reaching on par performances with respect to the UDA state-of-the-art, with a simpler approach.