TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images

TL;DR

TuiGAN is a novel one-shot unsupervised image-to-image translation model that effectively learns mappings between two domains using only a single unpaired image from each domain, outperforming existing methods.

Contribution

We introduce TuiGAN, the first model capable of one-shot UI2I translation, enabling effective image translation with minimal data.

Findings

Outperforms strong baselines on various UI2I tasks

Achieves comparable results to data-intensive state-of-the-art models

Operates effectively with only two unpaired images

Abstract

An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images. While existing UI2I methods usually require numerous unpaired images from different domains for training, there are many scenarios where training data is quite limited. In this paper, we argue that even if each domain contains a single image, UI2I can still be achieved. To this end, we propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning. With TuiGAN, an image is translated in a coarse-to-fine manner where the generated image is gradually refined from global structures to local details. We conduct extensive experiments to verify that our versatile method can outperform strong baselines on a wide variety of UI2I tasks. Moreover, TuiGAN is capable of achieving comparable performance with the state-of-the-art UI2I models trained with sufficient data.