UTSGAN: Unseen Transition Suss GAN for Transition-Aware Image-to-image Translation

TL;DR

UTSGAN introduces a transition-aware generative framework for image-to-image translation, explicitly modeling unseen attribute changes to improve translation consistency across observed and unobserved cases.

Contribution

The paper proposes a novel transition-aware approach with a stochastic transition encoder and transition consistency regularization, enhancing generalization to unseen translations in I2I tasks.

Findings

UTSGAN outperforms existing methods on multiple I2I datasets.

The transition consistency regularization improves translation stability.

The model effectively handles unseen attribute transitions.

Abstract

In the field of Image-to-Image (I2I) translation, ensuring consistency between input images and their translated results is a key requirement for producing high-quality and desirable outputs. Previous I2I methods have relied on result consistency, which enforces consistency between the translated results and the ground truth output, to achieve this goal. However, result consistency is limited in its ability to handle complex and unseen attribute changes in translation tasks. To address this issue, we introduce a transition-aware approach to I2I translation, where the data translation mapping is explicitly parameterized with a transition variable, allowing for the modelling of unobserved translations triggered by unseen transitions. Furthermore, we propose the use of transition consistency, defined on the transition variable, to enable regularization of consistency on unobserved translations, which is omitted in previous works. Based on these insights, we present Unseen Transition Suss GAN (UTSGAN), a generative framework that constructs a manifold for the transition with a stochastic transition encoder and coherently regularizes and generalizes result consistency and transition consistency on both training and unobserved translations with tailor-designed constraints. Extensive experiments on four different I2I tasks performed on five different datasets demonstrate the efficacy of our proposed UTSGAN in performing consistent translations.