Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps

TL;DR

Polymorphic-GAN is a novel generative model that creates aligned images across multiple related domains by learning shared features and domain-specific morph layers, enabling diverse applications like segmentation transfer and cross-domain editing.

Contribution

It introduces a framework that models multiple domains with varying geometries simultaneously, surpassing previous methods in handling large geometric differences.

Findings

Produces aligned samples across diverse domains.

Outperforms previous methods in large-geometry translation tasks.

Enables applications like segmentation transfer and low-data training.

Abstract

Modern image generative models show remarkable sample quality when trained on a single domain or class of objects. In this work, we introduce a generative adversarial network that can simultaneously generate aligned image samples from multiple related domains. We leverage the fact that a variety of object classes share common attributes, with certain geometric differences. We propose Polymorphic-GAN which learns shared features across all domains and a per-domain morph layer to morph shared features according to each domain. In contrast to previous works, our framework allows simultaneous modelling of images with highly varying geometries, such as images of human faces, painted and artistic faces, as well as multiple different animal faces. We demonstrate that our model produces aligned samples for all domains and show how it can be used for applications such as segmentation transfer and cross-domain image editing, as well as training in low-data regimes. Additionally, we apply our Polymorphic-GAN on image-to-image translation tasks and show that we can greatly surpass previous approaches in cases where the geometric differences between domains are large.