Asymmetric GAN for Unpaired Image-to-image Translation

TL;DR

This paper introduces Asymmetric GAN (AsymGAN), a novel approach for unpaired image-to-image translation that effectively handles asymmetric domain complexities by incorporating an auxiliary variable, improving quality, controllability, and robustness.

Contribution

AsymGAN is the first to explicitly model asymmetric domain complexities using an auxiliary variable, enhancing translation quality and controllability over existing symmetric models.

Findings

Improves image translation quality in asymmetric domains.

Enables controllable one-to-many mappings.

Demonstrates superior performance on Cityscapes and Helen datasets.

Abstract

Unpaired image-to-image translation problem aims to model the mapping from one domain to another with unpaired training data. Current works like the well-acknowledged Cycle GAN provide a general solution for any two domains through modeling injective mappings with a symmetric structure. While in situations where two domains are asymmetric in complexity, i.e., the amount of information between two domains is different, these approaches pose problems of poor generation quality, mapping ambiguity, and model sensitivity. To address these issues, we propose Asymmetric GAN (AsymGAN) to adapt the asymmetric domains by introducing an auxiliary variable (aux) to learn the extra information for transferring from the information-poor domain to the information-rich domain, which improves the performance of state-of-the-art approaches in the following ways. First, aux better balances the information between two domains which benefits the quality of generation. Second, the imbalance of information commonly leads to mapping ambiguity, where we are able to model one-to-many mappings by tuning aux, and furthermore, our aux is controllable. Third, the training of Cycle GAN can easily make the generator pair sensitive to small disturbances and variations while our model decouples the ill-conditioned relevance of generators by injecting aux during training. We verify the effectiveness of our proposed method both qualitatively and quantitatively on asymmetric situation, label-photo task, on Cityscapes and Helen datasets, and show many applications of asymmetric image translations. In conclusion, our AsymGAN provides a better solution for unpaired image-to-image translation in asymmetric domains.