Conditional Image-to-Image Translation

TL;DR

This paper introduces a conditional image-to-image translation method using GANs that allows for controlled, diverse translations by conditioning on a target domain image, demonstrated on face and object translations.

Contribution

It proposes a novel conditional translation framework that enables diverse and controllable image translation conditioned on target domain images, using unpaired data and dual learning.

Findings

Effective control over translation results via conditional inputs

Diverse translation outputs for fixed source images

Successful applications on face and object translation tasks

Abstract

Image-to-image translation tasks have been widely investigated with Generative Adversarial Networks (GANs) and dual learning. However, existing models lack the ability to control the translated results in the target domain and their results usually lack of diversity in the sense that a fixed image usually leads to (almost) deterministic translation result. In this paper, we study a new problem, conditional image-to-image translation, which is to translate an image from the source domain to the target domain conditioned on a given image in the target domain. It requires that the generated image should inherit some domain-specific features of the conditional image from the target domain. Therefore, changing the conditional image in the target domain will lead to diverse translation results for a fixed input image from the source domain, and therefore the conditional input image helps to control the translation results. We tackle this problem with unpaired data based on GANs and dual learning. We twist two conditional translation models (one translation from A domain to B domain, and the other one from B domain to A domain) together for inputs combination and reconstruction while preserving domain independent features. We carry out experiments on men's faces from-to women's faces translation and edges to shoes&bags translations. The results demonstrate the effectiveness of our proposed method.