Generative Transition Mechanism to Image-to-Image Translation via Encoded Transformation

TL;DR

This paper introduces TEGAN, a novel image-to-image translation model that enforces both result and transition consistency through transition encoding, significantly improving generalization to unseen data transitions.

Contribution

The paper proposes a transition encoding mechanism that explicitly regularizes result and transition consistency, enhancing generalization in I2I translation models.

Findings

TEGAN outperforms existing models on four I2I tasks.

Explicit transition regularization improves unseen transition handling.

Unified framework for transition modeling in I2I translation.

Abstract

In this paper, we revisit the Image-to-Image (I2I) translation problem with transition consistency, namely the consistency defined on the conditional data mapping between each data pairs. Explicitly parameterizing each data mappings with a transition variable $t$, i.e., $x \overset{t(x,y)}{\mapsto}y$, we discover that existing I2I translation models mainly focus on maintaining consistency on results, e.g., image reconstruction or attribute prediction, named result consistency in our paper. This restricts their generalization ability to generate satisfactory results with unseen transitions in the test phase. Consequently, we propose to enforce both result consistency and transition consistency for I2I translation, to benefit the problem with a closer consistency between the input and output. To benefit the generalization ability of the translation model, we propose transition encoding to facilitate explicit regularization of these two {kinds} of consistencies on unseen transitions. We further generalize such explicitly regularized consistencies to distribution-level, thus facilitating a generalized overall consistency for I2I translation problems. With the above design, our proposed model, named Transition Encoding GAN (TEGAN), can poss superb generalization ability to generate realistic and semantically consistent translation results with unseen transitions in the test phase. It also provides a unified understanding of the existing GAN-based I2I transition models with our explicitly modeling of the data mapping, i.e., transition. Experiments on four different I2I translation tasks demonstrate the efficacy and generality of TEGAN.