DRIT++: Diverse Image-to-Image Translation via Disentangled Representations

TL;DR

DRIT++ introduces a novel disentangled representation framework for diverse image-to-image translation that operates effectively without paired training data, enabling the generation of multiple realistic outputs from a single input.

Contribution

The paper proposes a disentangled representation approach with domain-invariant content and domain-specific attribute spaces for unpaired image translation, incorporating a cross-cycle consistency loss.

Findings

Generates diverse, realistic images across various tasks without paired data

Achieves high realism scores in user studies and Fréchet inception distance

Demonstrates significant diversity using perceptual distance and Jensen-Shannon divergence

Abstract

Image-to-image translation aims to learn the mapping between two visual domains. There are two main challenges for this task: 1) lack of aligned training pairs and 2) multiple possible outputs from a single input image. In this work, we present an approach based on disentangled representation for generating diverse outputs without paired training images. To synthesize diverse outputs, we propose to embed images onto two spaces: a domain-invariant content space capturing shared information across domains and a domain-specific attribute space. Our model takes the encoded content features extracted from a given input and attribute vectors sampled from the attribute space to synthesize diverse outputs at test time. To handle unpaired training data, we introduce a cross-cycle consistency loss based on disentangled representations. Qualitative results show that our model can generate diverse and realistic images on a wide range of tasks without paired training data. For quantitative evaluations, we measure realism with user study and Fr\'{e}chet inception distance, and measure diversity with the perceptual distance metric, Jensen-Shannon divergence, and number of statistically-different bins.