CoMoGAN: continuous model-guided image-to-image translation

TL;DR

CoMoGAN introduces a novel continuous GAN framework with a functional normalization layer, enabling advanced image translation tasks like cyclic and linear translation, outperforming existing methods across datasets.

Contribution

It presents a new normalization layer and residual mechanism for disentangling content from position, enabling flexible, continuous image translation with any GAN backbone.

Findings

Outperforms existing methods on all datasets

Enables cyclic and detached linear image translation

Works with any GAN architecture

Abstract

CoMoGAN is a continuous GAN relying on the unsupervised reorganization of the target data on a functional manifold. To that matter, we introduce a new Functional Instance Normalization layer and residual mechanism, which together disentangle image content from position on target manifold. We rely on naive physics-inspired models to guide the training while allowing private model/translations features. CoMoGAN can be used with any GAN backbone and allows new types of image translation, such as cyclic image translation like timelapse generation, or detached linear translation. On all datasets, it outperforms the literature. Our code is available at http://github.com/cv-rits/CoMoGAN .