Conditional Adversarial Generative Flow for Controllable Image Synthesis

TL;DR

This paper introduces CAGlow, a flow-based generative model that improves controllable image synthesis by learning an adversarial encoder for condition-to-latent mapping, enabling complex multi-condition image generation.

Contribution

It proposes a novel adversarial encoder approach within flow models to enhance conditional image synthesis, especially for multi-label and unknown conditions.

Findings

CAGlow outperforms regular Glow in conditional image synthesis.

It ensures independence of different conditions during generation.

The model can synthesize images with multiple and unknown attributes.

Abstract

Flow-based generative models show great potential in image synthesis due to its reversible pipeline and exact log-likelihood target, yet it suffers from weak ability for conditional image synthesis, especially for multi-label or unaware conditions. This is because the potential distribution of image conditions is hard to measure precisely from its latent variable $z$. In this paper, based on modeling a joint probabilistic density of an image and its conditions, we propose a novel flow-based generative model named conditional adversarial generative flow (CAGlow). Instead of disentangling attributes from latent space, we blaze a new trail for learning an encoder to estimate the mapping from condition space to latent space in an adversarial manner. Given a specific condition $c$, CAGlow can encode it to a sampled $z$, and then enable robust conditional image synthesis in complex situations like combining person identity with multiple attributes. The proposed CAGlow can be implemented in both supervised and unsupervised manners, thus can synthesize images with conditional information like categories, attributes, and even some unknown properties. Extensive experiments show that CAGlow ensures the independence of different conditions and outperforms regular Glow to a significant extent.