MixSyn: Learning Composition and Style for Multi-Source Image Synthesis

TL;DR

MixSyn introduces a novel method for multi-source image synthesis that learns to compose and generate high-quality, mask-aware images by combining uncorrelated regions from multiple sources, enabling diverse and realistic image creation.

Contribution

The paper presents MixSyn, a new approach for learning fuzzy compositions from multiple sources to generate novel, high-quality images without mask dependency, advancing multi-source image synthesis.

Findings

Outperforms state-of-the-art methods in quality and diversity

Enables interactive synthesis and edit propagation

Produces realistic images from uncorrelated source regions

Abstract

Synthetic images created by generative models increase in quality and expressiveness as newer models utilize larger datasets and novel architectures. Although this photorealism is a positive side-effect from a creative standpoint, it becomes problematic when such generative models are used for impersonation without consent. Most of these approaches are built on the partial transfer between source and target pairs, or they generate completely new samples based on an ideal distribution, still resembling the closest real sample in the dataset. We propose MixSyn (read as " mixin' ") for learning novel fuzzy compositions from multiple sources and creating novel images as a mix of image regions corresponding to the compositions. MixSyn not only combines uncorrelated regions from multiple source masks into a coherent semantic composition, but also generates mask-aware high quality reconstructions of non-existing images. We compare MixSyn to state-of-the-art single-source sequential generation and collage generation approaches in terms of quality, diversity, realism, and expressive power; while also showcasing interactive synthesis, mix & match, and edit propagation tasks, with no mask dependency.