Unsupervised Cross-Domain Image Generation

TL;DR

This paper introduces an unsupervised method for cross-domain image transfer that maintains identity features while translating images between related visual domains, using a novel generative approach.

Contribution

It presents the Domain Transfer Network (DTN), a new unsupervised generative model combining multiple loss components for effective cross-domain image translation.

Findings

Successfully transfers images between digits and face domains

Generates convincing images of unseen entities

Preserves identity features during translation

Abstract

We study the problem of transferring a sample in one domain to an analog sample in another domain. Given two related domains, S and T, we would like to learn a generative function G that maps an input sample from S to the domain T, such that the output of a given function f, which accepts inputs in either domains, would remain unchanged. Other than the function f, the training data is unsupervised and consist of a set of samples from each domain. The Domain Transfer Network (DTN) we present employs a compound loss function that includes a multiclass GAN loss, an f-constancy component, and a regularizing component that encourages G to map samples from T to themselves. We apply our method to visual domains including digits and face images and demonstrate its ability to generate convincing novel images of previously unseen entities, while preserving their identity.